PPU sprite evaluation

During all visible scanlines, the PPU scans through OAM to determine which sprites to render on the next scanline. During each pixel clock (341 total per scanline), the PPU accesses OAM in the following pattern:

1. Cycles 0-63: Secondary OAM (32-byte buffer for current sprites on scanline) is initialized to $FF - attempting to read $2004 will return $FF
2. Cycles 64-255: Sprite evaluation
   • On even cycles, data is read from (primary) OAM
   • On odd cycles, data is written to secondary OAM (unless writes are inhibited, in which case it will read the value in secondary OAM instead)
   • 1. Starting at n = 0, read a sprite's Y-coordinate (OAM[n][0], copying it to the next open slot in secondary OAM (unless 8 sprites have been found, in which case the write is ignored).
   • 1a. If Y-coordinate is in range, copy remaining bytes of sprite data (OAM[n][1] thru OAM[n][3]) into secondary OAM.
   • 2. Increment n
   • 2a. If n has overflowed back to zero (all 64 sprites evaluated), go to 4
   • 2b. If less than 8 sprites have been found, go to 1
   • 2c. If exactly 8 sprites have been found, disable writes to secondary OAM. This causes sprites in back to drop out.
   • 3. Starting at m = 0, evaluate OAM[n][m] as a Y-coordinate.
   • 3a. If the value is in range, set the sprite overflow flag in $2002 and read the next 3 entries of OAM (incrementing 'm' after each byte and incrementing 'n' when 'm' overflows); if m = 3, increment n
   • 3b. If the value is not in range, increment n AND m (without carry). If n overflows to 0, go to 4; otherwise go to 3
   • 4. Attempt (and fail) to copy OAM[n][0] into the next free slot in secondary OAM, and increment n (repeat until HBLANK is reached)
3. Cycles 256-319: Sprite fetches (8 sprites total, 8 cycles per sprite)
   • 1-4: Read the Y-coordinate, tile number, attributes, and X-coordinate of the selected sprite
   • 5-8: Read the X-coordinate of the selected sprite 4 times.
   • On the first empty sprite slot, read the Y-coordinate of sprite #63 followed by $FF for the remaining 7 cycles
   • On all subsequent empty sprite slots, read $FF for all 8 reads
4. Cycles 320-340: Background render pipeline initialization
   • Read the first byte in secondary OAM (the Y-coordinate of the first sprite found, sprite #63 if no sprites were found)

This pattern was determined by doing carefully timed reads from $2004 using various sets of sprites. In the case where there are 8 sprites on a scanline, the sprite evaluation logic effectively breaks and starts evaluating the tile number/attributes/X-coordinates of other sprites as Y-coordinates, resulting in rather inconsistent sprite overflow behavior (showing both false positives and false negatives).

The sprite priority system has a quirk when the background, a front-priority sprite, and a back-priority sprite are in the same area. Games such as Super Mario Bros. 3 take advantage of this.