PPU sprite evaluation

During all visible scanlines, the PPU scans through OAM to determine which sprites to render on the next scanline. Sprites found to be within range are copied into the secondary OAM, which is then used to initialize eight internal sprite output units.

OAM[n][m] below refers to the byte at offset 4*n + m within OAM, i.e. OAM byte m (0-3) of sprite n (0-63).

During each pixel clock (341 total per scanline), the PPU accesses OAM in the following pattern:

1. Cycles 1-64: Secondary OAM (32-byte buffer for current sprites on scanline) is initialized to $FF - attempting to read $2004 will return $FF
2. Cycles 65-256: Sprite evaluation
   • On odd cycles, data is read from (primary) OAM
   • On even cycles, data is written to secondary OAM (unless writes are inhibited, in which case it will read the value in secondary OAM instead)
   • 1. Starting at n = 0, read a sprite's Y-coordinate (OAM[n][0], copying it to the next open slot in secondary OAM (unless 8 sprites have been found, in which case the write is ignored).
     • 1a. If Y-coordinate is in range, copy remaining bytes of sprite data (OAM[n][1] thru OAM[n][3]) into secondary OAM.
   • 2. Increment n
     • 2a. If n has overflowed back to zero (all 64 sprites evaluated), go to 4
     • 2b. If less than 8 sprites have been found, go to 1
     • 2c. If exactly 8 sprites have been found, disable writes to secondary OAM. This causes sprites in back to drop out.
   • 3. Starting at m = 0, evaluate OAM[n][m] as a Y-coordinate.
     • 3a. If the value is in range, set the sprite overflow flag in $2002 and read the next 3 entries of OAM (incrementing 'm' after each byte and incrementing 'n' when 'm' overflows); if m = 3, increment n
     • 3b. If the value is not in range, increment n and m (without carry). If n overflows to 0, go to 4; otherwise go to 3
       • The m increment is probably a hardware bug. If only n was incremented, the algorithm would set the overflow flag iff more than eight lines were in range for the scanline, as expected.
   • 4. Attempt (and fail) to copy OAM[n][0] into the next free slot in secondary OAM, and increment n (repeat until HBLANK is reached)
3. Cycles 257-320: Sprite fetches (8 sprites total, 8 cycles per sprite)
   • 1-4: Read the Y-coordinate, tile number, attributes, and X-coordinate of the selected sprite from secondary OAM
   • 5-8: Read the X-coordinate of the selected sprite from secondary OAM 4 times (while the PPU fetches the sprite tile data)
   • For the first empty sprite slot, this will consist of sprite #63's Y-coordinate followed by 3 $FF bytes; for subsequent empty sprite slots, this will be four $FF bytes
4. Cycles 321-340+0: Background render pipeline initialization
   • Read the first byte in secondary OAM (while the PPU fetches the first two background tiles for the next scanline)

This pattern was determined by doing carefully timed reads from $2004 using various sets of sprites, and simulation in Visual 2C02 seems to confirm this behavior.

Sprite overflow bug

During sprite evaluation, if eight in-range sprites have been found so far, the sprite evaluation logic continues to scan the primary OAM looking for one more in-range sprite to determine whether to set the sprite overflow flag. The first such check correctly checks the y coordinate of the next OAM entry, but after that the logic breaks and starts evaluating the tile number/attributes/X-coordinates of subsequent OAM entries as Y-coordinates (due to incorrectly bumping m when moving to the next sprite). This results in inconsistent sprite overflow behavior showing both false positives and false negatives.

The sprite priority system has a quirk when the background, a front-priority sprite, and a back-priority sprite are in the same area. Games such as Super Mario Bros. 3 take advantage of this.