Emulator tests: Difference between revisions

From NESdev Wiki
Jump to navigationJump to search
(add url to a large archive of collected test programs)
(Fix download links to allpads and Zap Ruder, as they are now handled as GitHub releases.)
 
(95 intermediate revisions by 11 users not shown)
Line 2: Line 2:


== Validation ROMs ==
== Validation ROMs ==
There is a substantial archive of test roms available at [http://gitorious.org/nesicide/nes-test-roms/trees/master http://gitorious.org/nesicide/nes-test-roms/trees/master]
{{mbox
=== Multiple ===
| type = warning
* [http://nesdev.parodius.com/NEStress.zip NEStress] partially tests PPU, CPU, and controller operation (old; some tests seem to always fail)
| text = '''Some of the download links below are currently dead, but many have been archived at [https://github.com/christopherpow/nes-test-roms https://github.com/christopherpow/nes-test-roms]'''.
* [http://www.slack.net/~ant/nes-tests/ Blargg's test ROMs] partially test APU, misc PPU behavior, sprite 0 hit, and MMC3 operation. Refer to [[PPU_frame_timing]] for new information that the PPU ROMs test.
}}


=== CPU ===
=== CPU Tests ===
* [http://nickmass.com/images/nestest.nes nestest] fairly thoroughly tests CPU operation. This is the best test to start with when getting a CPU emulator working for the first time. Start execution at $C000 and compare execution with a [http://nickmass.com/images/nestest.log log from Nintendulator], whose CPU works (apart from some details of the power-up state).
* [http://blargg.parodius.com/nes-tests/instr_test-v3.zip instr_test] tests official and unofficial CPU instructions and lists which ones failed. It will work even if emulator has no PPU and only supports NROM, writing a copy of output to $6000 (see readme). This more thoroughly tests instructions, but can't help you figure out what's wrong beyond what instruction(s) are failing, so it's better for testing mature CPU emulators.
* [http://blargg.parodius.com/nes-tests/nes_instr_misc.zip instr_misc] tests some miscellaneous aspects of instructions, including behavior when 16-bit address wraps around, and dummy reads.
* [http://blargg.parodius.com/nes-tests/instr_timing.zip instr_timing] tests timing of all instructions, including unofficial ones, page-crossing, etc.
* [http://blargg.parodius.com/nes-tests/cpu_interrupts_v2.zip cpu_interrupts_v2] tests the behavior and timing of CPU in the presence of interrupts, both IRQ and NMI.
* [http://blargg.parodius.com/nes-tests/cpu_reset.zip cpu_reset] tests CPU registers just after power and changes during reset, and that RAM isn't changed during reset.


=== PPU ===
{| class="wikitable sortable"
* [http://nesdev.parodius.com/bbs/viewtopic.php?t=626 Sprite 0 Hit] test ROMs
|-
* [http://nesdev.parodius.com/bbs/viewtopic.php?t=567 Misc PPU Tests]
! Name
* [http://blargg.parodius.com/nes-tests/ppu_vbl_nmi.zip ppu_vbl_nmi] tests the behavior and timing of the NTSC PPU's VBL flag, NMI enable, and NMI interrupt. Timing is tested to an accuracy of one PPU clock.
! Author
* [http://www.slack.net/~ant/nes-tests/sprite_overflow_tests.zip PPU sprite overflow flag timing tests] ($2002 bit 5), covering general operation, timing, and obscure pathological behavior ([http://nesdev.parodius.com/bbs/viewtopic.php?p=11416 discussion]).
! Description
* [http://pics.pineight.com/nes/tvpassfail.zip tvpassfail]: NTSC color and NTSC/PAL pixel aspect ratio test ROM
! Resources
|-
| [http://www.slack.net/~ant/nes-tests/branch_timing_tests.zip branch_timing_tests] || blargg || These ROMs test timing of the branch instruction, including edge cases ||
|-
| [https://github.com/christopherpow/nes-test-roms/raw/master/cpu_dummy_reads/cpu_dummy_reads.nes cpu_dummy_reads] || blargg || Tests the CPU's dummy reads || [https://forums.nesdev.org/viewtopic.php?p=31629 thread]
|-
| [http://bisqwit.iki.fi/src/nes_tests/cpu_dummy_writes.zip cpu_dummy_writes] || bisqwit || Tests the CPU's dummy writes || [https://forums.nesdev.org/viewtopic.php?t=8738 thread]
|-
| [http://bisqwit.iki.fi/src/nes_tests/cpu_exec_space.zip cpu_exec_space] || bisqwit || Verifies that the CPU can execute code from any possible memory location, even if that is mapped as I/O space || [https://forums.nesdev.org/viewtopic.php?t=8755 thread]
|-
| [http://bisqwit.iki.fi/kala/test1.zip cpu_flag_concurrency] || bisqwit || Unsure what results are meant to be, see thread for more info. || [https://forums.nesdev.org/viewtopic.php?f=3&t=8757 thread]
|-
| [http://blargg.8bitalley.com/parodius/nes-tests/cpu_interrupts_v2.zip cpu_interrupts_v2] || blargg || Tests the behavior and timing of CPU in the presence of interrupts, both IRQ and NMI; see CPU interrupts. || [https://forums.nesdev.org/viewtopic.php?f=2&t=6510 thread]
|-
| [http://blargg.8bitalley.com/parodius/nes-tests/cpu_reset.zip cpu_reset] || blargg || Tests CPU registers just after power and changes during reset, and that RAM isn't changed during reset. ||
|-
| [http://blargg.8bitalley.com/parodius/nes-tests/cpu_timing_test6.zip cpu_timing_test6] || blargg || This program tests instruction timing for all official and unofficial NES 6502 instructions except the 8 branch instructions (Bxx) and the 12 halt instructions (HLT) || [https://forums.nesdev.org/viewtopic.php?f=3&t=3831 thread]
|-
| [//forums.nesdev.org/viewtopic.php?p=157782#p157782 coredump] || jroatch || Coredump tool for displaying contents of RAM || [http://forums.nesdev.org/viewtopic.php?f=22&t=11520 thread]
|-
| [http://blargg.8bitalley.com/parodius/nes-tests/nes_instr_misc.zip instr_misc] || blargg || Tests some miscellaneous aspects of instructions, including behavior when 16-bit address wraps around, and dummy reads. ||
|-
| [http://blargg.8bitalley.com/nes-tests/instr_test-v5.zip instr_test_v5] || blargg || Tests official and unofficial CPU instructions and lists which ones failed. It will work even if emulator has no PPU and only supports NROM, writing a copy of output to $6000 (see readme). This more thoroughly tests instructions, but can't help you figure out what's wrong beyond what instruction(s) are failing, so it's better for testing mature CPU emulators. ||
|-
| [http://blargg.8bitalley.com/parodius/nes-tests/instr_timing.zip instr_timing] || blargg || Tests timing of all instructions, including unofficial ones, page-crossing, etc. ||
|-
| [http://nickmass.com/images/nestest.nes nestest] ([https://www.qmtpro.com/~nes/misc/nestest.txt doc]) || kevtris || fairly thoroughly tests CPU operation. This is the best test to start with when getting a CPU emulator working for the first time. Start execution at $C000 and compare execution with a [https://www.qmtpro.com/~nes/misc/nestest.log known good log] (created using [[Nintendulator]], an emulator chosen by the test's author because its CPU was verified to function correctly, aside from some minor details of the power-up state). ||
|-
| [//forums.nesdev.org/viewtopic.php?t=13334 ram_retain] || rainwarrior || RAM contents test, for displaying contents of RAM at power-on or after reset || [http://forums.nesdev.org/viewtopic.php?t=13334 thread]
|}


=== APU ===
=== PPU Tests ===
* [http://blargg.parodius.com/nes-tests/apu_test.zip apu_test] tests many aspects of the APU that are visible to the CPU. Really obscure things are not tested here.
* [http://blargg.parodius.com/nes-tests/apu_mixer.zip apu_mixer] verifies proper operation of the APU's sound channel mixer, including relative volumes of channels and non-linear mixing. [http://blargg.parodius.com/nes-tests/apu_mixer_recordings.zip recordings when run on NES] are available for comparison, though the tests are made so that you don't really need these.
* [http://blargg.parodius.com/nes-tests/apu_reset.zip apu_reset] tests initial APU state at power, and the effect of reset.
* [http://pics.pineight.com/nes/volume_tests.zip volume_tests] plays tones on all the APU's channels to show their relative volumes at various settings of $4011. Package includes a recording from an NES's audio output for comparison.


=== Mapper ===
{| class="wikitable sortable"
* [http://blargg.parodius.com/nes-tests/mmc3_test_2.zip mmc3_test] tests the [[MMC3]] scanline counter and IRQ generation, not much else currently
! Name
* [http://pics.pineight.com/nes/bntest.zip BNTest] tests how many PRG banks are reachable in [[BxROM]] and [[AxROM]]
! Author
! Description
! Resources
|-
| [https://forums.nesdev.org/viewtopic.php?p=155593#p155593 color_test] || rainwarrior || Simple display of any chosen color full-screen || [http://forums.nesdev.org/viewtopic.php?p=155593#p155593 thread]
|-
| [http://www.slack.net/~ant/nes-tests/blargg_ppu_tests.zip blargg_ppu_tests_2005.09.15b] || blargg || Miscellaneous PPU tests (palette ram, sprite ram, etc.) || [https://forums.nesdev.org/viewtopic.php?t=567 thread]
|-
| [http://www.slack.net/~ant/nes-hacks/full_nes_palette.nes full_nes_palette] || blargg || Displays the full palette with all emphasis states, demonstrates direct PPU color control || [https://forums.nesdev.org/viewtopic.php?p=10658 thread]
|-
| [http://www.slack.net/~ant/old/nes-code/nmi_sync.zip nmi_sync] || blargg || Verifies NMI timing by creating a specific pattern on the screen (NTSC & PAL versions) || [https://forums.nesdev.org/viewtopic.php?t=6589 thread]
|-
| [http://forums.nesdev.org/viewtopic.php?f=2&t=15080 ntsc_torture] || rainwarrior || NTSC Torture Test displays visual patterns to demonstrate NTSC signal artifacts || [http://forums.nesdev.org/viewtopic.php?f=2&t=15080 thread]
|-
| [http://blargg.8bitalley.com/parodius/nes-tests/oam_read.zip oam_read] || blargg || Tests OAM reading ($2004), being sure it reads the byte from OAM at the current address in $2003. || [https://forums.nesdev.org/viewtopic.php?t=6424 thread]
|-
| [http://blargg.8bitalley.com/parodius/nes-tests/oam_stress.zip oam_stress] || blargg || Thoroughly tests OAM address ($2003) and read/write ($2004) || [https://forums.nesdev.org/viewtopic.php?t=6424 thread]
|-
| [https://forums.nesdev.org/download/file.php?id=1537 oamtest3] || lidnariq || Utility to upload OAM data via $2003/$2004 - can be used to test for the OAMADDR bug behavior || [http://forums.nesdev.org/viewtopic.php?p=128913#p128913 thread 1] [http://forums.nesdev.org/viewtopic.php?p=128842#p128842 thread 2]
|-
| [https://forums.nesdev.org/viewtopic.php?t=13264 palette] || rainwarrior || Palette display requiring only scanline-based palette changes, intended to demonstrate the full palette even on less advanced emulators || [http://forums.nesdev.org/viewtopic.php?t=13264 thread]
|-
| [http://blargg.8bitalley.com/parodius/nes-tests/ppu_open_bus.zip ppu_open_bus] || blargg || Tests behavior when reading from open-bus PPU bits/registers ||
|-
| [http://bisqwit.iki.fi/src/nes_tests/ppu_read_buffer.zip ppu_read_buffer] || bisqwit || Mammoth test pack tests many aspects of the NES system, mostly centering around the PPU $2007 read buffer || [https://forums.nesdev.org/viewtopic.php?f=3&t=8847&start=0 thread]
|-
| [http://blargg.8bitalley.com/parodius/nes-tests/ppu_sprite_hit.zip ppu_sprite_hit] || blargg || Tests sprite 0 hit behavior and timing || [https://forums.nesdev.org/viewtopic.php?t=626 thread]
|-
| [http://www.slack.net/~ant/nes-tests/sprite_overflow_tests.zip sprite_overflow_tests] || blargg || Tests sprite overflow behavior and timing || [https://forums.nesdev.org/viewtopic.php?t=1308 thread]
|-
| [http://blargg.8bitalley.com/parodius/nes-tests/ppu_vbl_nmi.zip ppu_vbl_nmi] || blargg || Tests the behavior and timing of the NTSC PPU's VBL flag, NMI enable, and NMI interrupt. Timing is tested to an accuracy of one PPU clock. || [https://forums.nesdev.org/viewtopic.php?t=730 thread]
|-
| [http://www.qmtpro.com/~nes/demos/scanline.zip scanline] || Quietust || Displays a test screen that will contain glitches if certain portions of the emulation are not perfect. ||
|-
| [http://blargg.parodius.com/temp/sprdma_and_dmc_dma.zip sprdma_and_dmc_dma] || blargg || Tests the cycle stealing behavior of the DMC DMA while running Sprite DMAs || [https://forums.nesdev.org/viewtopic.php?f=3&t=6100 thread]
|-
| [http://www.slack.net/~ant/nes-tests/sprite_hit_tests.zip sprite_hit_tests_2005.10.05] || blargg || Generally the same as ppu_sprite_hit (older revision of the tests - ppu_sprite_hit is most likely better) || [https://forums.nesdev.org/viewtopic.php?t=626 thread]
|-
| [http://www.slack.net/~ant/nes-tests/sprite_overflow_tests.zip sprite_overflow_tests] || blargg || Generally the same as ppu_sprite_overflow (older revision of the tests - ppu_sprite_overflow is most likely better) || [https://forums.nesdev.org/viewtopic.php?t=1308 thread]
|-
| [http://pics.pineight.com/nes/tvpassfail.zip tvpassfail] || tepples || NTSC color and NTSC/PAL pixel aspect ratio test ROM (older revision of the tests - 240p Test Suite is most likely better)  || [https://forums.nesdev.org/viewtopic.php?t=3393 thread]
|-
| [http://blargg.8bitalley.com/parodius/nes-tests/ppu_vbl_nmi.zip vbl_nmi_timing] || blargg || Generally the same as ppu_vbl_nmi (older revision of the tests - ppu_vbl_nmi is most likely better) || [https://forums.nesdev.org/viewtopic.php?f=3&t=3953 thread]
|}


== Hard-to-emulate games ==
=== APU Tests ===


* ''Balloon Fight'' relies on reading the nametables through $2007 to twinkle the stars in the background. (The code is at $D603.)
{| class="wikitable sortable"
* Among the most popular [[NROM]] games, which are generally the first targets against which an emulator author tests his or her work, ''Super Mario Bros.'' is probably the hardest to emulate. It relies on JMP indirect, correct palette mirroring, sprite 0 detection, the 1-byte delay when reading from CHR ROM through $2007, and proper behavior of the nametable selection bits of $2000 and $2006.[http://nesdev.parodius.com/bbs/viewtopic.php?p=22022#22022] In addition, there are several bad dumps floating around, some of which were ripped from pirate multicarts whose cheat menus leave several key parameters in RAM.
! Name
* ''Adventures of Lolo 2'', ''Spelunker'' rely on 1 cycle NMI delay when $2002 bit 7 gets set inside vblank (if $2002 has not been read yet)
! Author
* ''Bases Loaded II'' glitches after a pitch is thrown ([[:File:Bases Loaded 2 without re-NMI.jpg|screenshot]]) if writing $00 then $80 to $2000 during vertical blank does not cause an additional [[NMI]]
! Description
* ''Battletoads'' needs precise CPU and PPU timing
! Resources
* ''Bee 52'' needs accurate DMC timing and relies on $2002 bit 5 as well
|-
* ''Cobra Triangle'', ''Iron Sword'' relies on the dummy read for the sta $4000,X instruction to acknowledge pending APU IRQs.
| [http://blargg.8bitalley.com/parodius/nes-tests/apu_mixer.zip apu_mixer] || blargg || Verifies proper operation of the APU's sound channel mixer, including relative volumes of channels and non-linear mixing. recordings when run on NES are available for comparison, though the tests are made so that you don't really need these. || [https://forums.nesdev.org/viewtopic.php?f=3&t=4911 thread]
* ''Fire Hawk'' needs accurate DMC timing and does mid-frame palette changes
|-
* ''Micro Machines'' requires correct values when reading PPU $2004 during rendering, and also relies on proper background color selection when rendering is disabled and the VRAM address points to the palette
| [https://forums.nesdev.org/download/file.php?id=7496 apu_phase_reset] || Rahsennor || Tests the correct square channel behavior when writing to $4003/4007 (reset the duty cycle sequencers but not the clock dividers) || [https://forums.nesdev.org/viewtopic.php?f=2&t=15346 thread]
* ''Ms. Pac-Man (Tengen)'' relies on being able to read $2002 bit 7 as true before NMI occurs
|-
* ''Galaxian'' requires proper handling of bit 4 of the [[CPU status flag behavior|P register]] for /IRQ.
| [http://blargg.8bitalley.com/parodius/nes-tests/apu_reset.zip apu_reset] || blargg || Tests initial APU state at power, and the effect of reset. ||
* ''Punch-Out!!'' requires fetching the 34th tile; otherwise, the ring will be glitched.
|-  
* ''Puzznic'' uses [[Programming_with_unofficial_opcodes#Watermarking_instructions|unofficial]] opcode $89, which is a two-byte NOP on 6502 and BIT #imm on 65C02. ([http://tasvideos.org/forum/viewtopic.php?p=306520#306520 tasvideos discussion])
| [http://blargg.8bitalley.com/parodius/nes-tests/apu_test.zip apu_test] || blargg || Tests many aspects of the APU that are visible to the CPU. Really obscure things are not tested here. ||
* ''Slalom'' does a JSR while the stack pointer is 0, so that half of the return address ends up at $0100 and the other half at $01FF.
|-
* ''Super Mario Bros. 3'' relies on an interaction between the [[sprite priority]] bit and the OAM index to put power-ups behind blocks
| [http://www.slack.net/~ant/nes-tests/blargg_apu_2005.07.30.zip blargg_apu_2005.07.30] || blargg || Tests APU length counters, frame counter, IRQ, etc. ||
|-  
| [http://blargg.8bitalley.com/parodius/nes-tests/dmc_dma_during_read4.zip dmc_dma_during_read4] || blargg || Tests register read/write behavior while DMA is running ||
|-
| [https://github.com/christopherpow/nes-test-roms/tree/master/dmc_tests dmc_tests] || ? || ? ||
|-
| [http://pics.pineight.com/nes/dpcmletterbox.zip dpcmletterbox] || tepples || Tests accuracy of the DMC channel's IRQ ||
|-
| [http://www.slack.net/~ant/nes-tests/pal_apu_tests.zip pal_apu_tests] || blargg || PAL version of the blargg_apu_2005.07.30 tests ||
|-
| [http://blargg.8bitalley.com/parodius/nes-tests/older/square_timer_div2.zip square_timer_div2] || blargg || Tests the square timer's period ||
|-  
| [https://forums.nesdev.org/download/file.php?id=1494 test_apu_2 (1-10)] [https://forums.nesdev.org/download/file.php?id=5699 (11)] || x0000 || 11 tests that verify a number of behaviors with the APU (including the frame counter) || [https://forums.nesdev.org/viewtopic.php?f=3&t=11174 thread]
|-
| [http://blargg.8bitalley.com/parodius/nes-tests/older/test_apu_env.zip test_apu_env] || blargg || Tests the APU envelope for correctness. ||
|-
| [http://blargg.8bitalley.com/parodius/nes-tests/older/test_apu_sweep.zip test_apu_sweep] || blargg || Tests the sweep unit's add, subtract, overflow cutoff, and minimum period behaviors. ||
|-
| [http://blargg.8bitalley.com/parodius/nes-tests/older/test_apu_timers.zip test_apu_timers] || blargg || Tests frequency timer of all 5 channels ||
|-
| [http://blargg.8bitalley.com/parodius/nes-tests/older/test_tri_lin_ctr.zip test_tri_lin_ctr] || blargg || Tests triangle's linear counter and clocking by the frame counter ||
|-
| [http://pics.pineight.com/nes/volume_tests.zip volume_tests] || tepples || Plays tones on all the APU's channels to show their relative volumes at various settings of $4011. Package includes a recording from an NES's audio output for comparison. ||
|}


=== Mapper-specific Tests ===


[[Game Bugs]] lists games that have glitches on NES hardware, so you won't go "fixing" them while breaking your emulator.
{| class="wikitable sortable"
! Name
! Author
! Description
! Resources
|-
| [https://forums.nesdev.org/viewtopic.php?f=3&t=13120 31_test] || rainwarrior || Tests for mapper 31 (see thread for ROMs in various PRG sizes) || [https://forums.nesdev.org/viewtopic.php?f=3&t=13120 thread]
|-
| [https://forums.nesdev.org/download/file.php?id=2253 BNTest] || tepples || Tests how many PRG banks are reachable in BxROM and AxROM. || [https://forums.nesdev.org/viewtopic.php?p=79826#p79826 thread] [https://github.com/pinobatch/little-things-nes/tree/master/bntest GitHub]
|-
| [https://forums.nesdev.org/viewtopic.php?f=3&t=12085 bxrom_512k_test] || rainwarrior || Similar to the BxROM test in BNTest above. || [https://forums.nesdev.org/viewtopic.php?f=3&t=12085 thread]
|-
| [http://forums.nesdev.org/download/file.php?id=10240 FdsIrqTests (v7)] || Sour || Tests various elements of the FDS' IRQ || [http://forums.nesdev.org/viewtopic.php?f=3&t=16507 thread]
|-
| [http://www.qmtpro.com/~nes/demos/mmc5exram.zip exram] || Quietust || MMC5 exram test ||
|-
| [http://forums.nesdev.org/viewtopic.php?t=8639 famicom_audio_swap_tests] || rainwarrior || Hotswap tests for Famicom expansion audio (5B, MMC5, VRC6, VRC7, N163, FDS) || [http://forums.nesdev.org/viewtopic.php?t=8639 thread]
|-
| [https://forums.nesdev.org/download/file.php?id=2759 fme7acktest-r1] || tepples || Checks some IRQ acknowledgment behiaviors of Sunsoft FME-7 that emulators were getting wrong in 2015. || [https://forums.nesdev.org/viewtopic.php?f=2&t=12436 thread]
|-
| [https://forums.nesdev.org/download/file.php?id=2800 fme7ramtest-r1] || tepples || Checks how much work RAM the Sunsoft FME-7 can access || [https://forums.nesdev.org/viewtopic.php?f=9&t=12467 thread] [https://github.com/pinobatch/little-things-nes/tree/master/fme7acktest GitHub]
|-
| [https://pineight.com/nes/holydiverbatman-bin-0.01.7z Holy Mapperel] || tepples || Detects over a dozen mappers and verifies that all PRG ROM and CHR ROM banks are reachable, that PRG RAM and CHR RAM can be written and read back without error, and that nametable mirroring, IRQ, and WRAM protection work. (Formerly Holy Diver Batman) || [https://forums.nesdev.org/viewtopic.php?f=22&t=10640 thread] [https://github.com/pinobatch/holy-mapperel GitHub]
|-
| [https://forums.nesdev.org/download/file.php?id=5166 mmc3bigchrram] || tepples || MMC3 test for large 32kb CHR RAM with NES 2.0 headers || [https://forums.nesdev.org/viewtopic.php?f=3&t=13890 thread] [https://github.com/pinobatch/little-things-nes/tree/master/mmc3bigchrram GitHub]
|-
| [http://slack.net/~ant/old/nes-tests/mmc3_test_2.zip mmc3_test_2] || blargg || MMC3 scanline counter and IRQ generation tests. ||
|-
| [https://forums.nesdev.org/viewtopic.php?p=261236#p261236 mmc3irqtest] || N-K || MMC3 scanline IRQ test and $C000 glitch investigation. || [https://forums.nesdev.org/viewtopic.php?p=261236#p261236 thread]
|-
| [https://forums.nesdev.org/viewtopic.php?p=76817#p76817 mmc5test] || Drag || MMC5 scanline counter || [https://forums.nesdev.org/viewtopic.php?t=7653 thread]
|-
| [http://forums.nesdev.org/download/file.php?id=8609 mmc5test_v2] || AWJ || MMC5 tests || [http://forums.nesdev.org/viewtopic.php?f=2&t=15788 thread]
|-
| [https://forums.nesdev.org/download/file.php?id=3753 serom] || lidnariq || Tests the constraints of SEROM / SHROM / SH1ROM variations of the MMC1 boards. || [https://forums.nesdev.org/viewtopic.php?f=3&t=9350&start=90#p153298 thread]
|-
| rowspan="4" | NES 2.0 Submapper Tests || rainwarrior || [https://forums.nesdev.org/viewtopic.php?f=3&t=9350&start=90#p157804 2_test] - Mapper 2, Submappers 0, 1 and 2 || [https://forums.nesdev.org/viewtopic.php?f=3&t=9350&start=90#p157804 thread]
|-
| rainwarrior || [https://forums.nesdev.org/viewtopic.php?f=3&t=9350&start=90#p154555 3_test] - Mapper 3, Submappers 0, 1 and 2 || [https://forums.nesdev.org/viewtopic.php?f=3&t=9350&start=90#p154555 thread]
|-
| rainwarrior || [https://forums.nesdev.org/viewtopic.php?f=3&t=9350&start=90#p157804 7_test] - Mapper 7, Submappers 0, 1 and 2 || [https://forums.nesdev.org/viewtopic.php?f=3&t=9350&start=90#p157804 thread]
|-
| rainwarrior || [https://forums.nesdev.org/viewtopic.php?f=3&t=9350&start=90#p153334 34_test] - Mapper 34, Submappers 1 and 2 || [https://forums.nesdev.org/viewtopic.php?f=3&t=9350&start=90#p153334 thread]
|-
| [https://forums.nesdev.org/download/file.php?id=12185 test28] || tepples || Tests for mapper 28 || [https://forums.nesdev.org/viewtopic.php?p=215345#p215345 thread] [https://github.com/pinobatch/little-things-nes/tree/master/test28 GitHub]
|-
| [http://forums.nesdev.org/download/file.php?id=10017 vrc24test] || AWJ || Detects and tests all VRC 2/4 variants || [http://forums.nesdev.org/viewtopic.php?f=3&t=16009 thread]
|-
| [http://www.mediafire.com/download/6hvuj53omv8y3fn/vrc6test.zip vrc6test] || natt || VRC6 mirroring tests || [https://forums.nesdev.org/viewtopic.php?t=11028 thread]
|-
| [https://forums.nesdev.org/viewtopic.php?p=244062#p244062 mmc5ramsize] || rainwarrior || MMC5 large PRG-RAM support tests || [https://forums.nesdev.org/viewtopic.php?p=244062#p244062 thread]
|-
| [https://forums.nesdev.org/viewtopic.php?t=23619 mmc1atest] || tepples || Characterizes behavior of [[iNES Mapper 155|MMC1A]] vs. [[iNES Mapper 001|MMC1B]] || [https://forums.nesdev.org/viewtopic.php?t=23619 thread] [https://github.com/pinobatch/little-things-nes/tree/master/mmc1a GitHub]
|-
| [https://forums.nesdev.org/viewtopic.php?p=284414#p284414 n163_soundram] || rainwarrior || Test for [[Namco 163 audio]] sound RAM read-back. || [https://forums.nesdev.org/viewtopic.php?p=284414#p284414 thread]
|-
| [https://forums.nesdev.org/viewtopic.php?p=285135#p285135 n163_soundram_init] || rainwarrior || Test for [[Namco 163 audio]] sound RAM contents at power-on. || [https://forums.nesdev.org/viewtopic.php?p=285135#p285135 thread]
|}


== Troubleshooting games ==
=== Input Tests ===
{| class="wikitable sortable"
! Name
! Author
! Description
! Resources
|-
| [https://github.com/pinobatch/allpads-nes/releases/latest/download/allpads.nes allpads] || tepples || Multiple controller test supporting NES controller, Super NES controller, Famicom microphone, Four Score, Zapper, NES Arkanoid controller, and Super NES Mouse; also has raw 32-bit report mode || [https://forums.nesdev.org/viewtopic.php?f=2&t=12549 thread] [https://github.com/pinobatch/allpads-nes GitHub]
|-
| [https://forums.nesdev.org/download/file.php?id=5905 dma_sync_test_v2] || Rahsennor || Tests DMC DMA read corruption || [https://forums.nesdev.org/viewtopic.php?f=2&t=14319 thread]
|-
| [https://github.com/christopherpow/nes-test-roms/raw/master/PaddleTest3/PaddleTest.nes PaddleTest3] || 3gengames || Test for the Arkanoid controller || [https://forums.nesdev.org/viewtopic.php?t=7929 thread]
|-
| [https://forums.nesdev.org/download/file.php?id=21972 vaus] || lidnariq || Arkanoid controller 9-bit result test || [https://forums.nesdev.org/viewtopic.php?t=23801 thread]
|-
| [http://blargg.8bitalley.com/parodius/nes-code/read_joy3.zip read_joy3] || blargg || Various NES controllers tests, including read corruption due to DMC DMA || [https://forums.nesdev.org/viewtopic.php?f=2&t=4124&start=0 thread]
|-
| [https://github.com/pinobatch/zap-ruder/releases/latest/download/ruder.nes Zap Ruder] || tepples || Test for the Zapper, including dual wield but not the serial Vs. variant || [https://forums.nesdev.org/viewtopic.php?t=8108 thread] [https://github.com/pinobatch/zap-ruder GitHub]
|-
| [https://forums.nesdev.org/download/file.php?id=5356 spadtest-nes] || tepples || Super Nintendo controller test (when connected to a NES) || [https://forums.nesdev.org/viewtopic.php?p=167288 thread]
|-
| [http://pics.pineight.com/nes/vaus-test-0.02.zip vaus_test] || tepples || Another test for the Arkanoid controller || [https://forums.nesdev.org/viewtopic.php?p=120455 thread] [https://github.com/pinobatch/little-things-nes/tree/master/vaus-test GitHub]
|-
| [https://forums.nesdev.org/viewtopic.php?p=231608#p231608 mset] || rainwarrior || SNES mouse test || [https://forums.nesdev.org/viewtopic.php?p=231608#p231608 thread]
|-
| [https://forums.nesdev.org/viewtopic.php?f=3&t=18314&p=232358#p232358 mict] || rainwarrior || Famicom microphone test || [https://forums.nesdev.org/viewtopic.php?f=3&t=18314 thread]
|-
| [https://forums.nesdev.org/viewtopic.php?f=22&t=18998 Telling LYs?] || tepples || Tests whether input can change on any scanline || [https://forums.nesdev.org/viewtopic.php?f=22&t=18998 thread] [https://github.com/pinobatch/little-things-nes/tree/master/tellinglys GitHub]
|-
| [https://forums.nesdev.org/viewtopic.php?f=2&t=19752 ctrltest] || rainwarrior || Generic log of 16-bit report on all 5 input data lines. || [https://forums.nesdev.org/viewtopic.php?f=2&t=19752 thread]
|-
| [https://forums.nesdev.org/viewtopic.php?p=106954#p106954 raw]
[https://forums.nesdev.org/viewtopic.php?t=23745 pack2]
|| lidnariq || Immediate state of 32-bit report on all 5x2 input data lines.
Immediate state of 64-bit report on all 5x2 input data lines.
|| [https://forums.nesdev.org/viewtopic.php?p=106954#p106954 thread]
|-
| [https://forums.nesdev.org/viewtopic.php?f=9&t=13021 zapper tests] || rainwarrior || Simple tests for displaying output of zapper reads. || [https://forums.nesdev.org/viewtopic.php?f=9&t=13021 thread]
|-
| [https://forums.nesdev.org/viewtopic.php?t=24192 powerpadgesture] || tepples || Gesture test for [[Power Pad]] displaying log of presses and releases. || [https://forums.nesdev.org/viewtopic.php?t=24192 thread]
|-
| [https://www.nesdev.org/powerpd.zip POWERPAD.NES] || Tennessee Carmel-Veilleux || Old simple test for Power Pad. || [https://www.nesdev.org/powerpad.txt powerpad.txt]
|-
| [https://forums.nesdev.org/viewtopic.php?p=294558#p294558 d34test] || rainwarrior || Generic log of 32-bit report on D3 and D4 data lines, with 16-bit report on D0. || [https://forums.nesdev.org/viewtopic.php?p=294558#p294558 thread]
|-
| [https://forums.nesdev.org/viewtopic.php?p=294558#p294558 kmbtest] || rainwarrior || Input test for a proposed USB Keyboard and Mouse interface. || [https://forums.nesdev.org/viewtopic.php?p=294558#p294558 thread]
|}


If a scroll split doesn't work, and a garbage sprite shows up around the intended split point, then the game is probably trying to use a sprite 0 hit, but either the wrong tile data is loaded or the background is scrolled to a position that doesn't overlap correctly.
=== Misc Tests ===
This could be a problem with nametable [[mirroring]], with CHR bankswitching in [[mapper]]s that support it, or with the CPU and PPU timing of whatever happened above the split. Battletoads, for one, uses 1-screen mirroring and requires exact timing to get the background scroll position dead-on.
{| class="wikitable sortable"
! Name
! Author
! Description
! Resources
|-
| [https://forums.nesdev.org/download/file.php?id=19130 240pee-0.22] || tepples || 240p Test Suite (an NES version of the [http://junkerhq.net/xrgb/index.php/240p_test_suite 240p Test Suite] by Artemio Urbina), including an MDFourier tone generator || [https://forums.nesdev.org/viewtopic.php?t=13394 thread] [https://github.com/pinobatch/240p-test-mini/tree/master/nes GitHub]
|-
| [https://forums.nesdev.org/download/file.php?id=1415 characterize-vs] || lidnariq || VS System tests || [https://forums.nesdev.org/viewtopic.php?f=3&t=10276&start=15#p127592 thread]
|-
| [http://nesdev.org/NEStress.zip NEStress] || Flubba || Old test - some of the tests are supposed to fail on real hardware. ||
|-
| [https://forums.nesdev.org/download/file.php?id=2826 oc-r1a] || tepples || Detects and displays accurate clock rate of the NES (since incorporated into 240p Test Suite) || [https://forums.nesdev.org/viewtopic.php?f=22&t=12499 thread]
|-
| [https://github.com/bbbradsmith/nes-audio-tests nes-audio-tests] || rainwarrior || NSF and NES ROM tests for expansion audio sound, NSF behaviour, and other various audio related things. ||
|}


== Automated testing ==
== Automated testing ==
Line 62: Line 266:
It's best if your emulator can automatically run a suite of tests at the press of a button. This allows you to re-run them every time you make a change, without any effort. Automation can be difficult, because the emulator must be able to determine success/failure without your help.
It's best if your emulator can automatically run a suite of tests at the press of a button. This allows you to re-run them every time you make a change, without any effort. Automation can be difficult, because the emulator must be able to determine success/failure without your help.


Automating test ROMs that don't require any button presses is simplest. An emulator merely needs to run the ROM for sufficient time (at with an uncapped frame rate), then take a screenshot. If the screenshot differs from what it was the last time the test was run, it should make a note of this in the log. Later you can re-run any tests whose screenshots changed and examine the result yourself. Determining a changed screenshot doesn't even require any image files; the emulator can simply make a checksum of the picture, and compare with the previous.
The first part of automated testing is support for a "movie" or "demo", or a list of what buttons were pressed when.
An emulator makes a movie by recording presses while the user is playing, and then it plays the movie by feeding the recorded presses back through the input system.
This not only helps automated testing but also makes your emulator attractive to [[wikipedia:Speedrun|speedrunners]].


Automating test ROMs that do require button presses is more complex. In many cases, the emulator can simply feed a series of button presses, with a fixed delay between each, perhaps one second. If the emulator has a movie feature which merely records controller input, you can simply record a movie of your pressing the appropriate buttons, then have the emulator play that back normally.
To create a test case, record a movie of the player activating all tests in a ROM, take a screenshot of each result screen, and log the time and a [[wikipedia:Hash function|hash]] of each screenshot.
The simplest test ROMs won't require any button presses.
ROMs that test more than one thing are more likely to require them, and an [[Tricky-to-emulate games|actual game]] will require a playthrough.
Then to run a test case, play the movie in fast-forward (no delay between frames) and take screenshots at the same times.
If a screenshot's hash differs from that of the corresponding screenshot from when the test case was created, make a note of this difference in the log.
Then you can compare the emulator's output frame-by-frame to that of the previous release of your emulator running the same test case.


Automating game testing is the most complex. As with test ROMs that require button presses, playing back a movie might work. One difference is that the result won't be a simple screenshot at the end; it will potentially need to monitor images/sounds throughout the movie. This could still be a simple checksum. If a game's behavior changes, you can go back and compare it frame-by-frame with what played back on the previous version of your emulator.
== See also ==
* [[Emulators]]

Latest revision as of 00:06, 23 September 2024

There are many ROMs available that test an emulator for inaccuracies.

Validation ROMs

CPU Tests

Name Author Description Resources
branch_timing_tests blargg These ROMs test timing of the branch instruction, including edge cases
cpu_dummy_reads blargg Tests the CPU's dummy reads thread
cpu_dummy_writes bisqwit Tests the CPU's dummy writes thread
cpu_exec_space bisqwit Verifies that the CPU can execute code from any possible memory location, even if that is mapped as I/O space thread
cpu_flag_concurrency bisqwit Unsure what results are meant to be, see thread for more info. thread
cpu_interrupts_v2 blargg Tests the behavior and timing of CPU in the presence of interrupts, both IRQ and NMI; see CPU interrupts. thread
cpu_reset blargg Tests CPU registers just after power and changes during reset, and that RAM isn't changed during reset.
cpu_timing_test6 blargg This program tests instruction timing for all official and unofficial NES 6502 instructions except the 8 branch instructions (Bxx) and the 12 halt instructions (HLT) thread
coredump jroatch Coredump tool for displaying contents of RAM thread
instr_misc blargg Tests some miscellaneous aspects of instructions, including behavior when 16-bit address wraps around, and dummy reads.
instr_test_v5 blargg Tests official and unofficial CPU instructions and lists which ones failed. It will work even if emulator has no PPU and only supports NROM, writing a copy of output to $6000 (see readme). This more thoroughly tests instructions, but can't help you figure out what's wrong beyond what instruction(s) are failing, so it's better for testing mature CPU emulators.
instr_timing blargg Tests timing of all instructions, including unofficial ones, page-crossing, etc.
nestest (doc) kevtris fairly thoroughly tests CPU operation. This is the best test to start with when getting a CPU emulator working for the first time. Start execution at $C000 and compare execution with a known good log (created using Nintendulator, an emulator chosen by the test's author because its CPU was verified to function correctly, aside from some minor details of the power-up state).
ram_retain rainwarrior RAM contents test, for displaying contents of RAM at power-on or after reset thread

PPU Tests

Name Author Description Resources
color_test rainwarrior Simple display of any chosen color full-screen thread
blargg_ppu_tests_2005.09.15b blargg Miscellaneous PPU tests (palette ram, sprite ram, etc.) thread
full_nes_palette blargg Displays the full palette with all emphasis states, demonstrates direct PPU color control thread
nmi_sync blargg Verifies NMI timing by creating a specific pattern on the screen (NTSC & PAL versions) thread
ntsc_torture rainwarrior NTSC Torture Test displays visual patterns to demonstrate NTSC signal artifacts thread
oam_read blargg Tests OAM reading ($2004), being sure it reads the byte from OAM at the current address in $2003. thread
oam_stress blargg Thoroughly tests OAM address ($2003) and read/write ($2004) thread
oamtest3 lidnariq Utility to upload OAM data via $2003/$2004 - can be used to test for the OAMADDR bug behavior thread 1 thread 2
palette rainwarrior Palette display requiring only scanline-based palette changes, intended to demonstrate the full palette even on less advanced emulators thread
ppu_open_bus blargg Tests behavior when reading from open-bus PPU bits/registers
ppu_read_buffer bisqwit Mammoth test pack tests many aspects of the NES system, mostly centering around the PPU $2007 read buffer thread
ppu_sprite_hit blargg Tests sprite 0 hit behavior and timing thread
sprite_overflow_tests blargg Tests sprite overflow behavior and timing thread
ppu_vbl_nmi blargg Tests the behavior and timing of the NTSC PPU's VBL flag, NMI enable, and NMI interrupt. Timing is tested to an accuracy of one PPU clock. thread
scanline Quietust Displays a test screen that will contain glitches if certain portions of the emulation are not perfect.
sprdma_and_dmc_dma blargg Tests the cycle stealing behavior of the DMC DMA while running Sprite DMAs thread
sprite_hit_tests_2005.10.05 blargg Generally the same as ppu_sprite_hit (older revision of the tests - ppu_sprite_hit is most likely better) thread
sprite_overflow_tests blargg Generally the same as ppu_sprite_overflow (older revision of the tests - ppu_sprite_overflow is most likely better) thread
tvpassfail tepples NTSC color and NTSC/PAL pixel aspect ratio test ROM (older revision of the tests - 240p Test Suite is most likely better) thread
vbl_nmi_timing blargg Generally the same as ppu_vbl_nmi (older revision of the tests - ppu_vbl_nmi is most likely better) thread

APU Tests

Name Author Description Resources
apu_mixer blargg Verifies proper operation of the APU's sound channel mixer, including relative volumes of channels and non-linear mixing. recordings when run on NES are available for comparison, though the tests are made so that you don't really need these. thread
apu_phase_reset Rahsennor Tests the correct square channel behavior when writing to $4003/4007 (reset the duty cycle sequencers but not the clock dividers) thread
apu_reset blargg Tests initial APU state at power, and the effect of reset.
apu_test blargg Tests many aspects of the APU that are visible to the CPU. Really obscure things are not tested here.
blargg_apu_2005.07.30 blargg Tests APU length counters, frame counter, IRQ, etc.
dmc_dma_during_read4 blargg Tests register read/write behavior while DMA is running
dmc_tests ? ?
dpcmletterbox tepples Tests accuracy of the DMC channel's IRQ
pal_apu_tests blargg PAL version of the blargg_apu_2005.07.30 tests
square_timer_div2 blargg Tests the square timer's period
test_apu_2 (1-10) (11) x0000 11 tests that verify a number of behaviors with the APU (including the frame counter) thread
test_apu_env blargg Tests the APU envelope for correctness.
test_apu_sweep blargg Tests the sweep unit's add, subtract, overflow cutoff, and minimum period behaviors.
test_apu_timers blargg Tests frequency timer of all 5 channels
test_tri_lin_ctr blargg Tests triangle's linear counter and clocking by the frame counter
volume_tests tepples Plays tones on all the APU's channels to show their relative volumes at various settings of $4011. Package includes a recording from an NES's audio output for comparison.

Mapper-specific Tests

Name Author Description Resources
31_test rainwarrior Tests for mapper 31 (see thread for ROMs in various PRG sizes) thread
BNTest tepples Tests how many PRG banks are reachable in BxROM and AxROM. thread GitHub
bxrom_512k_test rainwarrior Similar to the BxROM test in BNTest above. thread
FdsIrqTests (v7) Sour Tests various elements of the FDS' IRQ thread
exram Quietust MMC5 exram test
famicom_audio_swap_tests rainwarrior Hotswap tests for Famicom expansion audio (5B, MMC5, VRC6, VRC7, N163, FDS) thread
fme7acktest-r1 tepples Checks some IRQ acknowledgment behiaviors of Sunsoft FME-7 that emulators were getting wrong in 2015. thread
fme7ramtest-r1 tepples Checks how much work RAM the Sunsoft FME-7 can access thread GitHub
Holy Mapperel tepples Detects over a dozen mappers and verifies that all PRG ROM and CHR ROM banks are reachable, that PRG RAM and CHR RAM can be written and read back without error, and that nametable mirroring, IRQ, and WRAM protection work. (Formerly Holy Diver Batman) thread GitHub
mmc3bigchrram tepples MMC3 test for large 32kb CHR RAM with NES 2.0 headers thread GitHub
mmc3_test_2 blargg MMC3 scanline counter and IRQ generation tests.
mmc3irqtest N-K MMC3 scanline IRQ test and $C000 glitch investigation. thread
mmc5test Drag MMC5 scanline counter thread
mmc5test_v2 AWJ MMC5 tests thread
serom lidnariq Tests the constraints of SEROM / SHROM / SH1ROM variations of the MMC1 boards. thread
NES 2.0 Submapper Tests rainwarrior 2_test - Mapper 2, Submappers 0, 1 and 2 thread
rainwarrior 3_test - Mapper 3, Submappers 0, 1 and 2 thread
rainwarrior 7_test - Mapper 7, Submappers 0, 1 and 2 thread
rainwarrior 34_test - Mapper 34, Submappers 1 and 2 thread
test28 tepples Tests for mapper 28 thread GitHub
vrc24test AWJ Detects and tests all VRC 2/4 variants thread
vrc6test natt VRC6 mirroring tests thread
mmc5ramsize rainwarrior MMC5 large PRG-RAM support tests thread
mmc1atest tepples Characterizes behavior of MMC1A vs. MMC1B thread GitHub
n163_soundram rainwarrior Test for Namco 163 audio sound RAM read-back. thread
n163_soundram_init rainwarrior Test for Namco 163 audio sound RAM contents at power-on. thread

Input Tests

Name Author Description Resources
allpads tepples Multiple controller test supporting NES controller, Super NES controller, Famicom microphone, Four Score, Zapper, NES Arkanoid controller, and Super NES Mouse; also has raw 32-bit report mode thread GitHub
dma_sync_test_v2 Rahsennor Tests DMC DMA read corruption thread
PaddleTest3 3gengames Test for the Arkanoid controller thread
vaus lidnariq Arkanoid controller 9-bit result test thread
read_joy3 blargg Various NES controllers tests, including read corruption due to DMC DMA thread
Zap Ruder tepples Test for the Zapper, including dual wield but not the serial Vs. variant thread GitHub
spadtest-nes tepples Super Nintendo controller test (when connected to a NES) thread
vaus_test tepples Another test for the Arkanoid controller thread GitHub
mset rainwarrior SNES mouse test thread
mict rainwarrior Famicom microphone test thread
Telling LYs? tepples Tests whether input can change on any scanline thread GitHub
ctrltest rainwarrior Generic log of 16-bit report on all 5 input data lines. thread
raw

pack2

lidnariq Immediate state of 32-bit report on all 5x2 input data lines.

Immediate state of 64-bit report on all 5x2 input data lines.

thread
zapper tests rainwarrior Simple tests for displaying output of zapper reads. thread
powerpadgesture tepples Gesture test for Power Pad displaying log of presses and releases. thread
POWERPAD.NES Tennessee Carmel-Veilleux Old simple test for Power Pad. powerpad.txt
d34test rainwarrior Generic log of 32-bit report on D3 and D4 data lines, with 16-bit report on D0. thread
kmbtest rainwarrior Input test for a proposed USB Keyboard and Mouse interface. thread

Misc Tests

Name Author Description Resources
240pee-0.22 tepples 240p Test Suite (an NES version of the 240p Test Suite by Artemio Urbina), including an MDFourier tone generator thread GitHub
characterize-vs lidnariq VS System tests thread
NEStress Flubba Old test - some of the tests are supposed to fail on real hardware.
oc-r1a tepples Detects and displays accurate clock rate of the NES (since incorporated into 240p Test Suite) thread
nes-audio-tests rainwarrior NSF and NES ROM tests for expansion audio sound, NSF behaviour, and other various audio related things.

Automated testing

It's best if your emulator can automatically run a suite of tests at the press of a button. This allows you to re-run them every time you make a change, without any effort. Automation can be difficult, because the emulator must be able to determine success/failure without your help.

The first part of automated testing is support for a "movie" or "demo", or a list of what buttons were pressed when. An emulator makes a movie by recording presses while the user is playing, and then it plays the movie by feeding the recorded presses back through the input system. This not only helps automated testing but also makes your emulator attractive to speedrunners.

To create a test case, record a movie of the player activating all tests in a ROM, take a screenshot of each result screen, and log the time and a hash of each screenshot. The simplest test ROMs won't require any button presses. ROMs that test more than one thing are more likely to require them, and an actual game will require a playthrough. Then to run a test case, play the movie in fast-forward (no delay between frames) and take screenshots at the same times. If a screenshot's hash differs from that of the corresponding screenshot from when the test case was created, make a note of this difference in the log. Then you can compare the emulator's output frame-by-frame to that of the previous release of your emulator running the same test case.

See also