Alternative Fade Effects - RetroKoH/S1Fixed GitHub Wiki

(Original guides by RetroKoH and MarkeyJester)
Source: Original Work
Original Commit: aa946bc
Refactored Commit: 7c599c1
This mod can be toggled in S1Fixed by setting PaletteFadeSetting to any value from 0-6.

So, there's a backstory for this one. Who remembers Sonic 2: Pink Edition? So, back in 2013, I made a modified version of the trademark colored fade that faded with a pink tint instead of blue. While working on S1Fixed, I tried recreating this, only to realize that I could make it far easier to use and edit. Given the nature of this project, that's exactly what I'm doing now. Also, I never made any tweaks to the Special Stage fades. That's been done here too! So here we are!

If you are working with the original s1disasm, everything we'll be editing is located in sonic.asm, and you'll be modifying code at the following labels (We're only modifying what's in parentheses).:

So when we are finished, we will have 6 functional fade modes, and we'll add a 7th one for good measure, courtesy of myself and MarkeyJester.

Mode sequences

First, here are the fading sequences for each mode (You can alternate the order of colors for slightly different effects):

00 Blue

Fade In from Black:	Blue Add > Green Add > Red Add
Fade Out to Black:	Red Dec > Green Dec > Blue Dec
Fade In from White:	Blue Dec > Green Dec > Red Dec
Fade Out to White:	Red Add > Green Add > Blue Add

01 Green

Fade In from Black:	Green Add > Red Add > Blue Add
Fade Out to Black:	Blue Dec > Red Dec > Green Dec
Fade In from White:	Green Dec > Red Dec > Blue Dec
Fade Out to White:	Blue Add > Red Add > Green Add

02 Red

Fade In from Black:	Red Add > Blue Add > Green Add
Fade Out to Black:	Green Dec > Blue Dec > Red Dec
Fade In from White:	Red Dec > Blue Dec > Green Dec
Fade Out to White:	Green Add > Blue Add > Red Add

03 Blue+Green (Cyan)

Fade In from Black:	Blue/Green Add > Red Add
Fade Out to Black:	Red Dec > Blue/Green Dec
Fade In from White:	Blue/Green Dec > Red Dec
Fade Out to White:	Red Add > Blue/Green Add

04 Blue+Red (Magenta)

Fade In from Black:	Blue/Red Add > Green Add
Fade Out to Black:	Green Dec > Blue/Red Dec
Fade In from White:	Blue/Red Dec > Green Dec
Fade Out to White:	Green Add > Blue/Red Add

05 Green+Red

Fade In from Black:	Green/Red Add > Blue Add
Fade Out to Black:	Blue Dec > Green/Red Dec
Fade In from White:	Green/Red Dec > Blue Dec
Fade Out to White:	Blue Add > Green/Red Add

One thing you'll notice is that the in/out white fades display complementary colors for each mode. Feel free to mix and match them around to make things even more consistent, if you feel like it (FUTURE ME: Please make a note of which ones to match together for proper effect). Now time to get modding. Let's redo Mode 00 as an example:

FadeIn_AddColour

Here is what stock Sonic 1's engine does to fade in the colors on each frame that the fade effect is active.

FadeIn_AddColour:
.addblue:
	move.w	(a1)+,d2
	move.w	(a0),d3
	cmp.w	d2,d3		; is colour already at threshold level?
	beq.s	.next		; if yes, branch
	move.w	d3,d1
	addi.w	#$200,d1	; increase blue	value
	cmp.w	d2,d1		; has blue reached threshold level?
	bhi.s	.addgreen	; if yes, branch
	move.w	d1,(a0)+	; update palette
	rts	
; ===========================================================================

.addgreen:
	move.w	d3,d1
	addi.w	#$20,d1		; increase green value
	cmp.w	d2,d1
	bhi.s	.addred
	move.w	d1,(a0)+	; update palette
	rts	
; ===========================================================================

.addred:
	addq.w	#2,(a0)+	; increase red value
	rts	
; ===========================================================================

.next:
	addq.w	#2,a0		; next colour
	rts	
; ===========================================================================

In case you aren't clear on what's happening, it checks to see if the current color has faded into its target color. If it hasn't, it increments the blue value first. This is done safely in register d1 where the actual color value is not affected. It then checks the result to see if we've gone over target. If not, we update the palette and move on to the next color. If yes, it effectively discards the change and repeats the process with the green color value. Again, it updates green in register d1, then checks the result to see if we've gone over target. If yes, then this also gets discarded, and since we weren't at the target color yet, it just updates red and exits. This is a horribly inefficient process, especially towards the end of the fade, because the engine will increment blue and green each time, even if it's not necessary, only to then load red until the target color is met. The current system, as is, also makes it fairly difficult (though not impossible) for newer users to edit. I can't imagine Sonic Team cared about that during development over 30 years ago, but I care now, so let's hit two birds with one stone and make something that is A. more efficient, and B. more user-friendly:

FadeIn_AddColour:
	tst.w	(a1)		; Does this colour entry need to be faded in?
	beq.s	.next		; if not, branch and jump to the next palette entry

.checkcolors:
	move.b	(a1),d5		; d5 = target blue
	move.b	1(a1),d1	; d1 = GR byte
	move.b	d1,d2		; copy to d2
	lsr.b	#4,d1		; d1 = target green
	andi.b	#$E,d2		; d2 = target red
	; Check components
	tst.b	d5		; Does blue need adding (yes if d5 is non-zero)
	bne.s	.addblue	; if yes, branch and add blue to current colour
	tst.b	d1		; Does green need adding (yes if d5 is non-zero)
	bne.s	.addgreen	; if yes, branch and add green to current colour
; fallthrough

.addred:
	addq.b	#2,1(a0)	; increase red value
	subq.b	#2,1(a1)	; decrease target red value

.next:
	addq.w	#2,a0		; next colour
	addq.w	#2,a1		; next target colour
	rts
; ===========================================================================

.addblue:
	addq.b	#2,(a0)		; increase blue	value
	subq.b	#2,(a1)		; decrease target blue value
	bra.s	.next		; advance color values and exit	
; ===========================================================================

.addgreen:
	addi.b	#$20,1(a0)	; increase green value
	subi.b	#$20,1(a1)	; decrease target green value
	bra.s	.next		; advance color values and exit	
; ===========================================================================

So, the first major difference between the old method, and my method, is that my method doesn't constantly compare the active palette (loaded into a0) with the target palette (loaded into a1). Instead, I decrement a target color while I simultaneously increment an active color. Since the active color is starting at 0 as the target color is slated to drop to 0, this method works just fine. It wouldn't work quite as well if I wanted to fade from one color to another. Anyway, we run a simple zero/non-zero check on the target color, and if it's 0, this means the active color must be faded in, and we proceed. From there, each color component of the target color (green, red, blue) is stored in a separate data register (d1, d2, and d5, respectively). We check the target blue value first. If it's nonzero, we decrement it while incrementing the active blue value. If it's zero, we simply do the same check to green. If that one is zero, then we just increment the active color's red while decrementing the target color. No unnecessary changes to discard with this process. What's more is that the structure allows us to order the three functions in any manner we want, with ease! This is how I was able to make so many color variations. Now, let's move on to the next part.

NOTE: Credit to MarkeyJester for the portion of code where each color component is loaded into separate registers.

FadeOut_DecColour

Here is the original routine in Sonic 1:

FadeOut_DecColour:
.dered:
	move.w	(a0),d2
	beq.s	.next
	move.w	d2,d1
	andi.w	#$E,d1
	beq.s	.degreen
	subq.w	#2,(a0)+	; decrease red value
	rts	
; ===========================================================================

.degreen:
	move.w	d2,d1
	andi.w	#$E0,d1
	beq.s	.deblue
	subi.w	#$20,(a0)+	; decrease green value
	rts	
; ===========================================================================

.deblue:
	move.w	d2,d1
	andi.w	#$E00,d1
	beq.s	.next
	subi.w	#$200,(a0)+	; decrease blue	value
	rts	
; ===========================================================================

.next:
	addq.w	#2,a0
	rts	
; End of function FadeOut_DecColour

And here is mine:

FadeOut_DecColour:
	tst.w	(a0)		; Does this colour entry need to be faded out?
	beq.s	.next		; if not, branch and jump to the next palette entry

.dered:
	move.b	1(a0),d1	; d1 = GR byte
	andi.b	#$E,d1		; d1 = current red
	beq.s	.degreen	; if red is already faded out, check green
	subq.b	#2,1(a0)	; decrease red value
	addq.w	#2,a0		; next colour
	rts	
; ===========================================================================

.degreen:
	move.b	1(a0),d1	; d1 = GR byte
	andi.b	#$E0,d1		; d1 = current green
	beq.s	.deblue		; if green is already faded out, that means only blue remains
	subi.b	#$20,1(a0)	; decrease green value
	addq.w	#2,a0		; next colour
	rts	
; ===========================================================================

.deblue:
	subq.b	#2,(a0)		; decrease blue	value

.next:
	addq.w	#2,a0		; next colour
	rts
; End of function FadeOut_DecColour

This one didn't require too much changing compared to the original. You could probably even argue that I didn't even need to change much, if anything. In any case, FadeOut_DecColour is far more flexible in terms of reorganizing blocks of code to fade out colors in different manners, so there isn't much to cover here. Moving on!

WhiteIn_DecColour

So, WhiteIn_DecColour is basically the inverse of FadeIn_AddColour. In the original game, it has the same exact pitfalls and inefficiencies that FadeIn_AddColour had in the original engine. With that said, to keep things simple, I'm just going to show my implementation here:

WhiteIn_DecColour:
	cmpi.w	#cWhite,(a1)	; Does this colour entry need to be faded in?
	beq.s	.next		; if not, branch and jump to the next palette entry

.checkcolors:
	move.b	(a1),d5		; d5 = target blue
	move.b	1(a1),d1	; d1 = GR byte
	move.b	d1,d2		; copy to d2
	lsr.b	#4,d1		; d1 = target green
	andi.b	#$E,d2		; d2 = target red
	; Check components
	cmpi.b	#$E,d5		; Does blue need removal (yes if d5 is not $E)
	bne.s	.deblue		; if yes, branch and decrement blue to current colour
	cmpi.b	#$E,d1		; Does green need removal (yes if d1 is not $E)
	bne.s	.degreen	; if yes, branch and decrement green to current colour
; fallthrough

.dered:
	subq.b	#2,1(a0)	; decrease red value
	addq.b	#2,1(a1)	; increase target red value

.next:
	addq.w	#2,a0		; next colour
	addq.w	#2,a1		; next target colour
	rts
; ===========================================================================

.deblue:
	subq.b	#2,(a0)		; decrease blue	value
	addq.b	#2,(a1)		; increase target blue value
	bra.s	.next		; advance color values and exit	
; ===========================================================================

.degreen:
	subi.b	#$20,1(a0)	; decrease green value
	addi.b	#$20,1(a1)	; increase target green value
	bra.s	.next		; advance color values and exit	
; ===========================================================================

This runs much the same as FadeIn_AddColour, only we are incrementing the target color components toward $E as we decrement the active components, which start at $E (because cWhite = $0EEE). Again, no discarded changes, and structured in a way where rearranging parts of the function for different effects is easier. Onto the final part.

WhiteOut_AddColour

Just like with FadeOut_DecColour, there isn't too much that's changing here. Here is the implementation as seen in S1Fixed:

WhiteOut_AddColour:
	cmpi.w	#cWhite,(a0)	; Does this colour entry need to be faded out?
	beq.s	.next			; if not, branch and jump to the next palette entry

.addred:
	move.b	1(a0),d1	; d1 = GR byte
	andi.b	#$E,d1		; d1 = current red
	cmpi.b	#$E,d1
	beq.s	.addgreen	; if red is already whited out, check green
	addq.b	#2,1(a0)	; increase red value
	addq.w	#2,a0		; next colour
	rts	
; ===========================================================================

.addgreen:
	move.b	1(a0),d1	; d1 = GR byte
	andi.b	#$E0,d1		; d1 = current green
	cmpi.b	#$E0,d1
	beq.s	.addblue	; if green is already whited out, that means only blue remains
	addi.b	#$20,1(a0)	; increase green value
	addq.w	#2,a0		; next colour
	rts		
; ===========================================================================

.addblue:
	addq.b	#2,(a0)		; increase blue	value	

.next:
	addq.w	#2,a0		; next colour
	rts	
; End of function WhiteOut_AddColour

I'll add a continuation where we explore Modes 03, 04 and 05, since those are slightly more complex (but not much so).