S3K Priority Manager - RetroKoH/S1Fixed GitHub Wiki
(Based on a Sonic 2 guide by redhotsonic)
(Adapted and built upon by RetroKoH and Devon)
S2 Guide Source: Sonic 2 SCHG Page
Initial S1 Guide Source: Sonic Retro thread
Commits:
NOTE: The initial commit incorrectly labels this as the Sonic 2 Priority Manager. Please disregard any such incorrect labeling on my part.
Preface
Please back-up your disassembly before attempting to use this guide for your own projects. If you're reading this on GitHub, then this shouldn't be an issue. Also, this guide requires shifting OSTs, so every OST must be equated. If you're using an old disassembly base, you likely won't be able to follow this guide. Also, for any objects you've made yourself or have ported, make sure that it's fully equated (instead of $18(a0), it should be priority(a0)). Anything that has not been equated may start to cause problems; you have been warned!
What is "Priority"?
In all classic Sonic games, every object has an OST value for sprite priority. The higher the priority value, the lower priority it has. Objects are displayed on-screen in accordance with their priority value. For example, if Sonic has a priority of 2, and Dr. Eggman's ship has a priority of 3, then when both sprites overlap each other, Sonic will ALWAYS be displayed in front of him. It is the equivalent of what other game development frameworks (such as Game Maker) would call "depth".
Difference between managers
Sonic 1/2
The priority OST is a single byte. Its value ranges from #0 (highest) to #7 (lowest). When each object jumps to the DisplaySprite subroutine, it first converts the priority byte into a word, then uses the resulting word value to access the correct section of the sprite queue. It then checks whether or not the sprite can be displayed (based on whether or not that section of the sprite queue full or not. It has to do this every frame for every object with a sprite (or sprites) to be displayed.
Sonic 3 & Knuckles
Priority is a word-length value. its value ranges from #0 (highest) #$380 (lowest), incrementing by $80 for each level (#0, $80, $100, $180, etc.). Because it's already set as a word, the DisplaySprite subroutine doesn't have to do those calculations. It simply accesses the corresponding sprite queue level, and checks whether or not the sprite can be displayed.
S1Fixed
S1Fixed takes this one step further. Both iterations of the priority manager noted above have to take a word length value and add it to the RAM address of the sprite queue (v_spritequeue). The S1Fixed Priority Manager stores the the exact address of the desired section of the sprite queue as a word length value in the object's priority OST. This modification removes the final bit of calculation from the DisplaySprite subroutine, instead loading the word directly as an address.
This table shows how each manager handles priority as an OST value (For S1Fixed, v_spritequeue is assumed to be at RAM address $FFAC00:
As you can see, we will be shaving off a few commands from DisplaySprite, due to not having to convert a byte to a word value, as well as not having to calculate to find the exact queue location, as it's already loaded into the object's OST value. Removing these calculations saves a lot of time, and when you have many sprites on-screen at once, this can make a huge difference in terms of efficiency.
Porting the S3K+ Priority Manager into your engine can help get rid of some of the lag your hack may be experiencing. If you follow this guide step-by-step carefully, everything should go fine. Please be aware, once you've started this, you cannot rebuild your ROM until you've finished. Otherwise, you'll get errors and crashes. As this is a time-consuming mod, make sure you are willing to commit the time to making all necessary changes.
Step 1: Freeing up a universal OST byte
The biggest problem is freeing a universal OST, as they are all seemingly being used. Luckily, redhotsonic showed us how to free two universal SST's in Sonic 2 here. I'll walk you through what I did in Sonic 1 to achieve the same result. Go to Constants.asm and look for a list of values starting with this:
; Object variables
obID: equ 0 ; object ID number
obRender: equ 1 ; bitfield for x/y flip, display mode
obGfx: equ 2 ; palette line & VRAM setting (2 bytes)
; ...
You'll see obInertia at $14. We are going to move this to $20, where obColType and obColProp are also located. obInertia is used almost exclusively by Sonic, while obColType and obColProp are used exclusively by enemies, so we let them overlap. This does cause one conflict, but we can address that later. Now, $14 and $15 are free, and we have two options. We can either move obPriority to $14, or we can move obActWid to $14 which frees up $19, allowing obPriority to use it for word-length values. I went with the latter. Here is the result:
; Object variables
obID: equ 0 ; object ID number
obRender: equ 1 ; bitfield for x/y flip, display mode
obGfx: equ 2 ; palette line & VRAM setting (2 bytes)
obMap: equ 4 ; mappings address (4 bytes)
obX: equ 8 ; x-axis position (2-4 bytes)
obScreenY: equ $A ; y-axis position for screen-fixed items (2 bytes)
obY: equ $C ; y-axis position (2-4 bytes)
obVelX: equ $10 ; x-axis velocity (2 bytes)
obVelY: equ $12 ; y-axis velocity (2 bytes)
obActWid: equ $14 ; action width (formerly obInertia)
; $15 is now free
obHeight: equ $16 ; height/2
obWidth: equ $17 ; width/2
obPriority: equ $18 ; sprite stack priority (2 bytes)
; $19 is now free (can be used by obPriority)
obFrame: equ $1A ; current frame displayed
obAniFrame: equ $1B ; current frame in animation script
obAnim: equ $1C ; current animation
obPrevAni: equ $1D ; previous animation
obTimeFrame: equ $1E ; time to next frame
obDelayAni: equ $1F ; time to delay animation
obInertia: equ $20 ; potential speed (2 bytes) -- Exclusive to Sonic
obColType: equ $20 ; collision response type
obColProp: equ $21 ; collision extra property
obStatus: equ $22 ; orientation or mode
; ...
Step 2: Setting up address constants
As mentioned before, S1Fixed's Priority Manager stores word-length addresses directly into object RAM for use in DisplaySprite, ridding the subroutine of all calculation. Place this list of constants somewhere in the Constants.asm file you should already have open:
; priority address variables -- RetroKoH/Devon S3K+ Priority Manager
priority0: equ v_spritequeue
priority1: equ v_spritequeue+$80
priority2: equ v_spritequeue+$100
priority3: equ v_spritequeue+$180
priority4: equ v_spritequeue+$200
priority5: equ v_spritequeue+$280
priority6: equ v_spritequeue+$300
priority7: equ v_spritequeue+$380
Before continuing, I want to address something. RAM addresses are 32-bit, right? Why is it that we can place this within a 16-bit OST variable in Object RAM? Devon explained it to me as such:
Luckily, you can keep [the address] as 16-bit, because with the M68k, it can interpret 16-bit addresses by sign extending it (0x8000 -> 0xFFFF8000). Because the M68k has a 24-bit address bus, it becomes 0xFF8000 (this is why 0xFFFF8000-0xFFFFFFFF are "valid" RAM addresses, and why they have ".w" after them).
Step 3: Modifying DisplaySprite
Sonic 1 has two separate subroutines for DisplaySprite, and both of them are nearly identical apart from the address register used, so changing both is quite simple. All we are doing is truncating the first two instructions down to one, and removing the subsequent calculations:
DisplaySprite:
- lea (v_spritequeue).w,a1
- move.w obPriority(a0),d0 ; get sprite priority
- lsr.w #1,d0
- andi.w #$380,d0
- adda.w d0,a1 ; jump to position in queue
+ movea.w obPriority(a0),a1 ; get sprite queue pointer
cmpi.w #$7E,(a1) ; is this part of the queue full?
bcc.s DSpr_Full ; if yes, branch
addq.w #2,(a1) ; increment sprite count
adda.w (a1),a1 ; jump to empty position
move.w a0,(a1) ; insert RAM address for object
DSpr_Full:
rts
We can copy these instructions for DisplaySprite1: as well:
DisplaySprite1:
movea.w obPriority(a1),a2 ; get sprite queue pointer
cmpi.w #$7E,(a2) ; is this part of the queue full?
bcc.s DSpr1_Full ; if yes, branch
addq.w #2,(a2) ; increment sprite count
adda.w (a2),a2 ; jump to empty position
move.w a1,(a2) ; insert RAM address for object
DSpr1_Full:
rts
If you have applied Devon's Subsprites mod, you can simply change the first instruction of that subroutine to match the other two.
Part 4: Replacing obPriority values with pointers
Now, you can modify every instance where a byte is loaded to obPriority with one of our pointers. Because of the names of the pointers, this is incredibly easy. To save time, you can do file-by-file search and replace (Ctrl+Shift+F in Notepad++). From there, use regular expressions to search.
Regex string to find:
move.b #([0-7]),obPriority
Regex string to replace with:
move.w #priority$1,obPriority
Part 5: Setting obPriority pointers for objects w/ priority of 0
In the old system, some specific objects had a priority value of 0, and in many cases, their code simply didn't load a value to their obPriority variable. That won't work in this case, because having an address of 0000 means that DisplaySprite will treat it as a pointer to RAM address $FFFF0000. At best, they simply won't display. At worst, DisplaySprite could muck something else up. We need to find all of these outliers and fix them with move.w #priority0,obPriority(a0). Here is every object with a priority of #0 that needs to be fixed:
- 09 Sonic in Special Stage.asm, Obj09_Main:,
move.w #priority0,obPriority(a0) - 0F Press Start and TM.asm, PSB_Main:,
move.w #priority0,obPriority(a0) - 1B Water Surface.asm, Surf_Main:,
move.w #priority0,obPriority(a0) - 34 Title Cards.asm, Card_MakeSprite:,
move.w #priority0,obPriority(a1) - 39 Game Over.asm, Over_1stWord:,
move.w #priority0,obPriority(a0) - 3A Got Through Card.asm, loc_C5CA:,
move.w #priority0,obPriority(a1) - 4C & 4D Lava Geyser Maker.asm, .activate:,
move.w #priority0,obPriority(a1) - 5C Pylon.asm, Pyl_Main:,
move.w #priority0,obPriority(a0) - 65 Waterfalls.asm, .under80:,
move.w #priority0,obPriority(a0) - 71 Invisible Barriers.asm, Invis_Main:,
move.w #priority0,obPriority(a0) - 7C Ring Flash.asm, Flash_Main:,
move.w #priority0,obPriority(a0) - 7D Hidden Bonuses.asm, Bonus_Main:,
move.w #priority0,obPriority(a0) - 7E Special Stage Results.asm, SSR_Loop:,
move.w #priority0,obPriority(a1) - 7F SS Result Chaos Emeralds.asm, .loop:,
move.w #priority0,obPriority(a1) - 80 Continue Screen Elements.asm, CSI_Main:,
move.w #priority0,obPriority(a0) - 80 Continue Screen Elements.asm, CSI_Even:,
move.w #priority0,obPriority(a1) - 89 Ending Sequence STH.asm, ESth_Main:,
move.w #priority0,obPriority(a0) - 8A Credits.asm, Cred_Main:,
move.w #priority0,obPriority(a0)
All files noted above are located in the _incObj folder.
Part 6: Make sure copiers are copying correctly
There are a few instances where you'll see a copy instruction like this:
move.b priority(a0),priority(a1)
Simply change it to this:
move.w priority(a0),priority(a1)
Part 7: Modify objects that use tables to set obPriority
There are multiple objects that use data tables to set multiple OSTs based on subtype. This pertains to a few bosses, the animal prison, and the scenery object. The bosses and prison capsule are actually comprised of multiple related instances of the same object, each with a different subtype and operating routine. We need to take steps to modify all of these accordingly.
_incObj/1C Scenery.asm
Remove this line from the end of Scen_Main:
move.b (a1)+,obColType(a0)
We don't need this OST, as the Scenery object doesn't have collision, and every subtype loads 00 to this variable.
The line before that loads a byte to obPriority. Change the instruction from .b to .w.
Now, (a1) points to Scen_Values:. There is data for 4 types of object. For each one, we are going to remove the last 0, and replace the priority byte with a corresponding pointer. Here is an example:
Scen_Values: dc.l Map_Scen ; mappings address
dc.w make_art_tile(ArtTile_SLZ_Fireball_Launcher,2,0) ; VRAM setting
- dc.b 0, 8, 2, 0 ; frame, width, priority, collision response
+ dc.b 0, 8 ; frame, width
+ dc.w priority2 ; priority
Do this for the other 3 sets of data. Note that each set of data starts with either Map_Scen or Map_Bri.
_incObj/3E Prison Capsule.asm
First, look towards the end of Pri_Main: and replace this:
move.b (a1)+,obPriority(a0)
move.b (a1)+,obFrame(a0)
with this:
move.b (a1)+,obFrame(a0)
lea 1(a1),a1 ; increment a1 to skip 00
move.w (a1)+,obPriority(a0)
So, as you see, we reversed the move instructions and added an increment to skip an empty byte. We are going to modify the table slightly. Replace the Pri_Var: table with this:
Pri_Var:
; routine, width, frame, priority
dc.b 2, $20, 0,0 ; (subtype 0: body)
dc.w priority4
dc.b 4, $C, 1,0 ; (subtype 1: button)
dc.w priority5
dc.b 6, $10, 3,0 ; (subtype 2: button 2)
dc.w priority4
dc.b 8, $10, 5,0 ; (subtype 3: ???)
dc.w priority3
So after the routine, width and mapping frame are loaded, the next byte is skipped, and it then loads the priority pointer. Next!?
_incObj/73 Boss - Marble.asm
This one is far less complicated. First, let's modify the BossMarble_ObjData: table. All we're doing is adding an extra byte, since priority is now a word-length value. Replace the table with this:
BossMarble_ObjData:
; routine, anim, priority
dc.b 2, 0
dc.w priority4
dc.b 4, 1
dc.w priority4
dc.b 6, 7
dc.w priority4
dc.b 8, 0
dc.w priority3
Then, go to the instruction that loads from (a2) to obPriority and change the instruction to .w:
move.w (a2)+,obPriority(a1)
_incObj/75 Boss - Spring Yard.asm
This one is even easier to fix. With BossSpringYard_ObjData:, we don't even need to load obPriority from this table, as every priority value is #5. Remove the 5 from every entry in the table. Then, go to the instruction that loads from (a2) to obPriority and change it to this:
move.w #priority5,obPriority(a1)
_incObj/7A Boss - Star Light.asm
We literally do the same exact thing with this object as we did with the Marble Zone boss. Literally the only difference is the name of the data table for this object.
_incObj/82 Eggman - Scrap Brain 2
This one isn't too hard, actually. First off, we can get rid of SEgg_ObjData: entirely. Why? Well, two of the values are identical for both entries, and the routine value being different doesn't matter, because both entries are loaded by two separate piece of code. The table is redundant! So, with the table gone, we need to slightly alter how we initialize data. First, remove the first line at the start of SEgg_Main: that loads the data table to a2. Next, if we look down a few lines, we will see two instances of the same code, one with a0, and one with a1:
move.b (a2)+,obRoutine(a0)
move.b (a2)+,obAnim(a0)
move.b (a2)+,obPriority(a0)
;...
move.b (a2)+,obRoutine(a1)
move.b (a2)+,obAnim(a1)
move.b (a2)+,obPriority(a1)
Replace them with these, respectively:
move.b #2,obRoutine(a1)
move.w #priority3,obPriority(a1)
;...
move.b #4,obRoutine(a1)
move.w #priority3,obPriority(a1)
We don't even need to load anything to obAnim, as those are meant to be 0. This object is finished! We're done now, right? Well...
Part 8: Caterkiller OST Conflict Fix
Remember how I said earlier that there was one OST conflict caused by overlapping obColType/obColProp with obInertia? Well, it happens with the Caterkiller object. Caterkiller is the only object in the entire game (other than Sonic) that uses obInertia. Because we overlapped these OST variables, Caterkiller will not register any collision with Sonic. Turns out fixing this is extremely easy! Simply open _incObj/78 Caterkiller.asm and replace every instance of obInertia(a0) with obVelY(a0). It doesn't use obVelY during normal function, and only uses this SST after it's broken apart (at which point obInertia isn't used anyway). This will cause no issues with the object.