asm3
Category: Reverse Engineering
Points: 300 pts
What does asm3(0xaeed09cb,0xb7acde91,0xb7facecd) return? Submit the flag as a hexadecimal value (starting with ‘0x’). NOTE: Your submission for this question will NOT be in the normal flag format. Source located in the directory at /problems/asm3_0_9cdf5fc9325b2a6276fb8e5908f0b5df.
Since machines are little endian, the parameters will be stored like this on the stack (addresses are relative to the base pointer):
| Address | 8 | 9 | a | b | c | d | e | f | 10 | 11 | 12 | 13 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Value | cb | 09 | ed | ae | 91 | de | ac | b7 | cd | ce | fa | b7 |
1 | <+3>: xor eax,eax |
eax is set to 0.
1 | <+5>: mov ah,BYTE PTR [ebp+0xb] |
ah is set to 0xae.
1 | <+8>: shl ax,0x10 |
ax becomes 0 again.
1 | <+12>: sub al,BYTE PTR [ebp+0xe] |
al becomes 0x54 (equivalent to ~0xac+1).
1 | <+15>: add ah,BYTE PTR [ebp+0xd] |
ah becomes 0xde.
1 | <+18>: xor ax,WORD PTR [ebp+0x12] |
ax becomes 0x69ae (0xde54^0xb7fa).
The flag is 0x69ae.