Improve the outdoor sprite system - pret/pokecrystal GitHub Wiki

In the tutorial to add a new map, we covered the concept of outdoor sprite sets. Outdoor maps—those with a TOWN or ROUTE environment—can only use sprites from their map group's set of usable sprites.

The outdoor sprite sets are defined in data/maps/outdoor_sprites.asm. For example, here's the one for Olivine City's map group:

 OlivineGroupSprites:
	db SPRITE_SUICUNE
	db SPRITE_SILVER_TROPHY
	db SPRITE_FAMICOM
	db SPRITE_POKEDEX
	db SPRITE_WILL
	db SPRITE_KAREN
	db SPRITE_NURSE
	db SPRITE_OLD_LINK_RECEPTIONIST
	db SPRITE_STANDING_YOUNGSTER
	db SPRITE_BIG_ONIX
	db SPRITE_SUDOWOODO
	db SPRITE_BIG_SNORLAX
	db SPRITE_OLIVINE_RIVAL
	db SPRITE_POKEFAN_M
	db SPRITE_LASS
	db SPRITE_BEAUTY
	db SPRITE_SWIMMER_GIRL
	db SPRITE_SAILOR
	db SPRITE_POKEFAN_F
	db SPRITE_SUPER_NERD
	db SPRITE_TAUROS
	db SPRITE_FRUIT_TREE
	db SPRITE_ROCK

We can see how those get loaded with BGB's VRAM viewer:

Screenshot

VRAM is divided into six areas, each 128 tiles large. The top two areas are for sprites' standing frames. The middle-right area is for the walking frames of the sprites in the top-right. But the middle-left area is for font tiles, so the top-left sprites can't walk. (If they do, they'll appear as text tiles.)

As they're currently implemented, these sprite sets are hard to edit. Every set has 23 sprites, and it's hard to tell which ones are needed for which map. It's also not clear which sprites get placed in VRAM bank 1 (the one on the right in the VRAM viewer). There's only enough room for nine sprites to have walking frames, but those nine are not in any particular order, nor does the order of the list correspond to the order in VRAM.

This tutorial will improve the outdoor sprite sets by making them variable-length lists ending with 0, and with the first nine sprites being the ones to get walking frames. The existing sprite sets for pokecrystal's maps will be optimized to work with this new format.

Make outdoor sprite sets variable-length, ending with 0
Don't automatically sort outdoor sprite sets
Update the outdoor sprite sets
Remove the now-redundant non-walking sprite versions
Remove the now-redundant variable sprites

1. Make outdoor sprite sets variable-length, ending with 0

Edit constants/map_data_constants.asm:

-DEF MAX_OUTDOOR_SPRITES EQU 23 ; see engine/overworld/overworld.asm

We won't need MAX_OUTDOOR_SPRITES any more.

Next, edit engine/overworld/overworld.asm:

 AddOutdoorSprites:
 	ld a, [wMapGroup]
 	dec a
 	ld c, a
 	ld b, 0
 	ld hl, OutdoorSprites
 	add hl, bc
 	add hl, bc
 	ld a, [hli]
 	ld h, [hl]
 	ld l, a
-	ld c, MAX_OUTDOOR_SPRITES
 .loop
-	push bc
 	ld a, [hli]
+	and a
+	ret z
 	call AddSpriteGFX
-	pop bc
-	dec c
-	jr nz, .loop
-	ret
+	jr .loop

Instead of counting c down from MAX_OUTDOOR_SPRITES to 0, now AddOutdoorSprites will continue until it finds a 0 list entry. (So don't forget to add them! We'll do so later.)

Now that outdoor sprite lists can be arbitrarily long, it's more important to enforce the limit of how much VRAM is even available. However, there happens to be a bug with the LoadSpriteGFX routine that ignores the SPRITE_GFX_LIST_CAPACITY limit. So be sure to fix that.

2. Don't automatically sort outdoor sprite sets

As we saw in earlier, the current sprite lists are not in any particular order. It turns out that the LoadAndSortSprites routine sorts the lists before loading their graphics, in order of how many tiles each one has, from most to least. Most NPC sprites have 12 tiles (four each for the front, back, and side views), so they get sorted first, and then come the still sprites like SPRITE_POKE_BALL, SPRITE_FRUIT_TREE, etc.

(Each outdoor sprite list gets padded to 23 entries with a bunch of still sprites like SPRITE_SILVER_TROPHY or SPRITE_OLD_LINK_RECEPTIONIST. They're not used in any outdoor map, but they have to be there so the walking sprites get sorted first.)

Anyway, edit engine/overworld/overworld.asm again:

 LoadAndSortSprites:
 	call LoadSpriteGFX
-	call SortUsedSprites
 	call ArrangeUsedSprites
 	ret

 ...

-SortUsedSprites:
-; Bubble-sort sprites by type.
-
-	...
-
-.quit
-	ret

Now sprites will be loaded into VRAM in whatever order the list specifies. So if we want a certain nine sprites to have walking frames available, we'll have to put them first.

3. Update the outdoor sprite sets

If you're replacing all of Crystal's maps with your own, you won't need these exact sets; but they're a good reference anyway for how the new sets work.

Edit data/maps/outdoor_sprites.asm:

-PalletGroupSprites:
-	...
-
-...
-
-CableClubGroupSprites:
-	...
+; Route1 and ViridianCity are connected
+; Route2 and PewterCity are connected
+; PalletTown and Route21 are connected
+PalletGroupSprites:
+; Route1, PalletTown
+ViridianGroupSprites:
+; Route2, Route22, ViridianCity
+PewterGroupSprites:
+; Route3, PewterCity
+CinnabarGroupSprites:
+; Route19, Route20, Route21, CinnabarIsland
+	db SPRITE_TEACHER
+	db SPRITE_FISHER
+	db SPRITE_YOUNGSTER
+	db SPRITE_BLUE
+	db SPRITE_GRAMPS
+	db SPRITE_BUG_CATCHER
+	db SPRITE_COOLTRAINER_F
+	db SPRITE_SWIMMER_GIRL
+	db SPRITE_SWIMMER_GUY
+	; max 9 of 9 walking sprites
+	db SPRITE_POKE_BALL
+	db SPRITE_FRUIT_TREE
+	db 0 ; end
+
+; CeruleanCity and Route5 are connected
+CeruleanGroupSprites:
+; Route4, Route9, Route10North, Route24, Route25, CeruleanCity
+SaffronGroupSprites:
+; Route5, SaffronCity
+	db SPRITE_COOLTRAINER_M
+	db SPRITE_SUPER_NERD
+	db SPRITE_COOLTRAINER_F
+	db SPRITE_FISHER
+	db SPRITE_YOUNGSTER
+	db SPRITE_LASS
+	db SPRITE_POKEFAN_M
+	db SPRITE_ROCKET
+	db SPRITE_MISTY
+	; max 9 of 9 walking sprites
+	db SPRITE_POKE_BALL
+	db SPRITE_SLOWPOKE
+	db 0 ; end
+
+CeladonGroupSprites:
+; Route7, Route16, Route17, CeladonCity
+	db SPRITE_FISHER
+	db SPRITE_TEACHER
+	db SPRITE_GRAMPS
+	db SPRITE_YOUNGSTER
+	db SPRITE_LASS
+	db SPRITE_BIKER
+	; 6 of max 9 walking sprites
+	db SPRITE_POLIWAG
+	db SPRITE_POKE_BALL
+	db SPRITE_FRUIT_TREE
+	db 0 ; end
+
+; Route11, Route12 and Route13 are connected
+VermilionGroupSprites:
+; Route6, Route11, VermilionCity
+LavenderGroupSprites:
+; Route8, Route12, Route10South, LavenderTown
+FuchsiaGroupSprites:
+; Route13, Route14, Route15, Route18, FuchsiaCity
+	db SPRITE_POKEFAN_M
+	db SPRITE_GRAMPS
+	db SPRITE_YOUNGSTER
+	db SPRITE_FISHER
+	db SPRITE_TEACHER
+	db SPRITE_SUPER_NERD
+	db SPRITE_BIKER
+	; 7 of max 9 walking sprites
+	db SPRITE_BIG_SNORLAX
+	db SPRITE_MACHOP
+	db SPRITE_POKE_BALL
+	db SPRITE_FRUIT_TREE
+	db 0 ; end
+
+IndigoGroupSprites:
+; Route23
+	; 0 of max 9 walking sprites
+	db 0 ; end
+
+; Route29 and CherrygroveCity are connected
+NewBarkGroupSprites:
+; Route26, Route27, Route29, NewBarkTown
+CherrygroveGroupSprites:
+; Route30, Route31, CherrygroveCity
+	db SPRITE_RIVAL
+	db SPRITE_TEACHER
+	db SPRITE_FISHER
+	db SPRITE_COOLTRAINER_M
+	db SPRITE_YOUNGSTER
+	db SPRITE_MONSTER
+	db SPRITE_GRAMPS
+	db SPRITE_BUG_CATCHER
+	db SPRITE_COOLTRAINER_F
+	; max 9 of 9 walking sprites
+	db SPRITE_POKE_BALL
+	db SPRITE_FRUIT_TREE
+	db 0 ; end
+
+; Route37 and EcruteakCity are connected
+VioletGroupSprites:
+; Route32, Route35, Route36, Route37, VioletCity
+EcruteakGroupSprites:
+; EcruteakCity
+	db SPRITE_FISHER
+	db SPRITE_LASS
+	db SPRITE_OFFICER
+	db SPRITE_GRAMPS
+	db SPRITE_YOUNGSTER
+	db SPRITE_COOLTRAINER_M
+	db SPRITE_BUG_CATCHER
+	db SPRITE_SUPER_NERD
+	; 8 of max 9 walking sprites
+	db SPRITE_WEIRD_TREE ; variable sprite: becomes SPRITE_SUDOWOODO and SPRITE_TWIN
+	db SPRITE_POKE_BALL
+	db SPRITE_FRUIT_TREE
+	db SPRITE_SUICUNE
+	db 0 ; end
+
+AzaleaGroupSprites:
+; Route33, AzaleaTown
+	db SPRITE_GRAMPS
+	db SPRITE_YOUNGSTER
+	db SPRITE_POKEFAN_M
+	db SPRITE_TEACHER
+	db SPRITE_AZALEA_ROCKET ; variable sprite: becomes SPRITE_ROCKET and SPRITE_RIVAL
+	db SPRITE_LASS
+	; 6 of max 9 walking sprites
+	db SPRITE_FRUIT_TREE
+	db SPRITE_SLOWPOKE
+	db SPRITE_KURT_OUTSIDE ; non-walking version of SPRITE_KURT
+	db 0 ; end
+
+GoldenrodGroupSprites:
+; Route34, GoldenrodCity
+	db SPRITE_GRAMPS
+	db SPRITE_YOUNGSTER
+	db SPRITE_OFFICER
+	db SPRITE_POKEFAN_M
+	db SPRITE_COOLTRAINER_F
+	db SPRITE_ROCKET
+	db SPRITE_LASS
+	; 7 of max 9 walking sprites
+	db SPRITE_DAY_CARE_MON_1
+	db SPRITE_DAY_CARE_MON_2
+	db SPRITE_POKE_BALL
+	db 0 ; end
+
+; OlivineCity and Route40 are connected
+OlivineGroupSprites:
+; Route38, Route39, OlivineCity
+CianwoodGroupSprites:
+; Route40, Route41, CianwoodCity, BattleTowerOutside
+	db SPRITE_OLIVINE_RIVAL ; variable sprite: becomes SPRITE_RIVAL and SPRITE_SWIMMER_GUY
+	db SPRITE_POKEFAN_M
+	db SPRITE_LASS
+	db SPRITE_BEAUTY
+	db SPRITE_SWIMMER_GIRL
+	db SPRITE_SAILOR
+	db SPRITE_POKEFAN_F
+	db SPRITE_SUPER_NERD
+	; 8 of max 9 walking sprites
+	db SPRITE_TAUROS
+	db SPRITE_FRUIT_TREE
+	db SPRITE_ROCK
+	db SPRITE_STANDING_YOUNGSTER ; non-walking version of SPRITE_YOUNGSTER
+	db SPRITE_SUICUNE
+	db 0 ; end
+
+MahoganyGroupSprites:
+; Route42, Route44, MahoganyTown
+	db SPRITE_GRAMPS
+	db SPRITE_YOUNGSTER
+	db SPRITE_LASS
+	db SPRITE_SUPER_NERD
+	db SPRITE_COOLTRAINER_M
+	db SPRITE_POKEFAN_M
+	db SPRITE_COOLTRAINER_F
+	db SPRITE_FISHER
+	; 8 of max 9 walking sprites
+	db SPRITE_FRUIT_TREE
+	db SPRITE_POKE_BALL
+	db SPRITE_SUICUNE
+	db 0 ; end
+
+LakeOfRageGroupSprites:
+; Route43, LakeOfRage
+	db SPRITE_LANCE
+	db SPRITE_GRAMPS
+	db SPRITE_SUPER_NERD
+	db SPRITE_COOLTRAINER_F
+	db SPRITE_FISHER
+	db SPRITE_COOLTRAINER_M
+	db SPRITE_LASS
+	db SPRITE_YOUNGSTER
+	; 8 of max 9 walking sprites
+	db SPRITE_GYARADOS
+	db SPRITE_FRUIT_TREE
+	db SPRITE_POKE_BALL
+	db 0 ; end
+
+BlackthornGroupSprites:
+; Route45, Route46, BlackthornCity
+	db SPRITE_GRAMPS
+	db SPRITE_YOUNGSTER
+	db SPRITE_LASS
+	db SPRITE_SUPER_NERD
+	db SPRITE_COOLTRAINER_M
+	db SPRITE_POKEFAN_M
+	db SPRITE_BLACK_BELT
+	db SPRITE_COOLTRAINER_F
+	; 8 of max 9 walking sprites
+	db SPRITE_FRUIT_TREE
+	db SPRITE_POKE_BALL
+	db 0 ; end
+
+SilverGroupSprites:
+; Route28, SilverCaveOutside
+	; 0 of max 9 walking sprites
+	db 0 ; end
+
+DungeonsGroupSprites:
+; NationalPark, NationalParkBugContest, RuinsOfAlphOutside
+	db SPRITE_LASS
+	db SPRITE_POKEFAN_F
+	db SPRITE_TEACHER
+	db SPRITE_YOUNGSTER
+	db SPRITE_POKEFAN_M
+	db SPRITE_ROCKER
+	db SPRITE_FISHER
+	db SPRITE_SCIENTIST
+	; 8 of max 9 walking sprites
+	db SPRITE_GAMEBOY_KID
+	db SPRITE_GROWLITHE
+	db SPRITE_POKE_BALL
+	db 0 ; end
+
+FastShipGroupSprites:
+; OlivinePort, VermilionPort, MountMoonSquare, TinTowerRoof
+	db SPRITE_SAILOR
+	db SPRITE_FISHING_GURU
+	db SPRITE_SUPER_NERD
+	db SPRITE_COOLTRAINER_F
+	db SPRITE_YOUNGSTER
+	db SPRITE_FAIRY
+	; 6 of max 9 walking sprites
+	db SPRITE_HO_OH
+	db SPRITE_ROCK
+	db 0 ; end
+
+CableClubGroupSprites:
+; (no outdoor maps)
+	; 0 of max 9 walking sprites
+	db 0 ; end

Now it works! These new sets are easier to define and debug than before. For example, here's the one for Olivine City's map group:

 ; OlivineCity and Route40 are connected
 OlivineGroupSprites:
 ; Route38, Route39, OlivineCity
 CianwoodGroupSprites:
 ; Route40, Route41, CianwoodCity, BattleTowerOutside
	db SPRITE_OLIVINE_RIVAL; variable sprite: becomes SPRITE_RIVAL and SPRITE_SWIMMER_GUY
	db SPRITE_POKEFAN_M
	db SPRITE_LASS
	db SPRITE_BEAUTY
	db SPRITE_SWIMMER_GIRL
	db SPRITE_SAILOR
	db SPRITE_POKEFAN_F
	db SPRITE_SUPER_NERD
	; 8 of max 9 walking sprites
	db SPRITE_TAUROS
	db SPRITE_FRUIT_TREE
	db SPRITE_ROCK
	db SPRITE_STANDING_YOUNGSTER ; non-walking version of SPRITE_YOUNGSTER
	db SPRITE_SUICUNE
	db 0 ; end

And here's how they get loaded into VRAM:

Screenshot

The comments make it clear which maps the set applies to; and the order matches their order in VRAM, from top to bottom, right and then left.

Some things to note about the new system:

If you can walk across a map connection from one map group to another, those groups now share an outdoor sprite set. Sprites and tilesets are only reloaded when you warp to a different map, not when you cross a connection, so this is necessary. Previously, connected sets like OlivineGroupSprites and CianwoodGroupSprites used separate lists which had to be kept in sync.
Removing the LoadAndSortSprites also affects indoor maps. If a map's object_events use so many sprites that some get loaded in VRAM0, make sure that the walking ones come first and have their walking frames in VRAM1.

We're technically done at this point, but the new system makes a few sprites redundant. Let's see how that works.

4. Remove the now-redundant non-walking sprite versions

Two sprites are just copies of other sprites, but with the walking frames removed:

SPRITE_STANDING_YOUNGSTER is a non-walking version of SPRITE_YOUNGSTER used by OlivineGroupSprites and CianwoodGroupSprites.
SPRITE_KURT_OUTSIDE is a non-walking version of SPRITE_KURT used by AzaleaGroupSprites.

This used to be necessary because if there were ten or more walking sprites, you couldn't control which nine would be loaded first and have their walking frames available; it was up to LoadAndSortSprites.

Now, though, these alternate non-walking sprites are redundant, so you can delete them:

Remove the SPRITE_STANDING_YOUNGSTER and SPRITE_KURT_OUTSIDE definitions and data from constants/sprite_constants.asm, data/sprites/sprites.asm, gfx/sprites.asm, and gfx/sprites/*.png.
Replace SPRITE_STANDING_YOUNGSTER and SPRITE_KURT_OUTSIDE with SPRITE_YOUNGSTER and SPRITE_KURT respectively in maps/*.asm and data/maps/outdoor_sprites.asm. Be sure to put them after all the walking sprites in their respective outdoor sprite sets, since their walking frames aren't needed.

For example, when you replace SPRITE_STANDING_YOUNGSTER with SPRITE_YOUNGSTER in OlivineGroupSprites, its standing frames will get loaded in VRAM bank 0, right where SPRITE_STANDING_YOUNGSTER in the previous screenshot; and its walking frames won't interfere with the font graphics.

5. Remove the now-redundant variable sprites

Three sprites are actually variable sprites:

SPRITE_WEIRD_TREE is used by VioletGroupSprites and EcruteakGroupSprites. It starts out looking like SPRITE_SUDOWOODO, and becomes SPRITE_TWIN after you battle Sudowoodo. This works because the only Twins in those map groups are encountered after Sudowoodo disappears.
SPRITE_AZALEA_ROCKET is used by AzaleaGroupSprites. It starts out looking like SPRITE_ROCKET, and becomes SPRITE_RIVAL after you save the Slowpoke. This works because the rival encounter occurs after all the Rockets disappear.
SPRITE_OLIVINE_RIVAL is used by OlivineGroupSprites and CianwoodGroupSprites. It starts out looking like SPRITE_RIVAL, and becomes SPRITE_SWIMMER_GUY after you run into your rival. This works because the only male Swimmers in those map groups are encountered after your rival disappears.

At first glance, it makes sense why these variable sprites exist. They're a neat way to have ten or more walking sprites: if you know that only nine will be needed at a time, then a variable sprite can look like one now and another later.

...Except, all three of those are used in outdoor sprite sets with enough room for more walking sprites! Out of a maximum 9 walking sprites, VioletGroupSprites and EcruteakGroupSprites only use 8; AzaleaGroupSprites uses 6; and OlivineGroupSprites and CianwoodGroupSprites use 8.

It turns out that Crystal doesn't need these variable sprites, but Gold and Silver did. Crystal is exclusive to the GameBoy Color, so it has twice as much VRAM, and it's able to load walking frames for every sprite in VRAM1 while still having room for standing-still sprites in VRAM0. But Gold and Silver supported the Super GameBoy, which only had one VRAM bank; so still sprites like SPRITE_POKE_BALL and SPRITE_SLOWPOKE used up the same space budget as walking sprites.

Anyway, the point is that we don't need them any more. So you can completely delete those three variable sprites:

Remove the SPRITE_WEIRD_TREE, SPRITE_AZALEA_ROCKET, and SPRITE_OLIVINE_RIVAL definitions from constants/sprite_constants.asm.
Remove their variablesprite commands from maps/*.asm and InitializeEventsScript in engine/events/std_scripts.asm. You can also remove special LoadUsedSpritesGFX when it occurs in a map script right after a variablesprite command.
Replace SPRITE_WEIRD_TREE, SPRITE_AZALEA_ROCKET, and SPRITE_OLIVINE_RIVAL with the actual sprites they're supposed to use in maps/*.asm and data/maps/outdoor_sprites.asm. Be sure to put the actual sprites among the first nine list entries, since they all need to walk (except for the SPRITE_SUDOWOODO and SPRITE_TWIN which replace SPRITE_WEIRD_TREE).

For more information on variable sprites, see the tutorial to add a new sprite.