This fall, FireEye’s FLARE team hosted its third annual FLARE On Challenge. It was a capture-the-flag (CTF) challenge that encouraged security researchers, malware analysts and reverse engineers of all skill levels to try their hand at finding flags in ten unique and intricate binaries. The challenge binaries this year contained puzzles which ran the gamut of cryptography, memory forensics, anti-analysis and program obfuscation. These puzzles manifested themselves in an even wider variety of target platforms, including 32-bit and 16-bit x86 binaries (both PE and ELF formats), obfuscated .NET binaries, network packet captures, JavaScript, ActionScript and Python.
Part of the fun of completing CTF challenges, such as the FireEye FLARE On challenge, is sharing your own and reading others’ solutions to the most difficult challenges. In CTF competitions and in real-world scenarios, there are often multiple ways to approach a reverse engineering task. In this two-part blog post, I will share my in-depth solutions to the challenges that I thought were the most interesting (and fun) – specifically, challenges 4, 8, 9 and 10. I hope to walk through my thought process as I completed the challenges, while also providing the technical solutions. This post will focus on challenges 4 and 8, while part two will focus on challenges 9 and 10. If you would like to play along, you can download the challenges from the FLARE On web page here (password for the zip file is “flare”).
Challenge 4 – flareon2016challenge.dll
Tools Used: IDA Pro, Python, OllyDbg
Challenge 4 of the FLARE On challenge was a 32-bit Windows dynamically-loaded library (DLL). Let’s throw the DLL into IDA Pro and have a look at the exported functions, shall we?
The DLL has 51 exported functions, in addition to the DllMain function. This seems to be a lot of functions to delve into, so we examine the binary’s strings to maybe give us some hints as to where to look.
The string “Play me a song, have me play along…” seems interesting. Following the cross-reference to it, we see it is printed out in export ordinal 51. Walking backward through the export ordinals, we see that export 50 (pictured below) takes two arguments, which are passed to the Kernel32.Beep function.
After being run 18 (0x12) times, it calls export 49 (pictured below), which seems to print a value out.
We can assume this printed value is some sort of decrypted string (probably the flag for the challenge). Exports 1-48 all have very similar looking structure, at this point we don’t know what they do, but they all seem to perform some sort of cryptographic operation, with a different seed value at the beginning. An example of one of these exports can be seen below.
Another thing to notice is that all functions within this encryption loop return an integer, so we can use the Python ctypes library to write a script to call every export and print the return value.
from ctypes import *
mydll = cdll.LoadLibrary("C:\Users\user\Desktop\flareon2016challenge.dll")
for i in range(1, 49):
mydll[i]()
The values are shown in the figure below.
There are some interesting things to note about these values, namely, values 30, 50 and 49 are never returned. Earlier , we assumed that exports 1-48 are some sort of cryptographic operation – so a good hypothesis as to the meaning of these values is that each one is the next ordinal export to be called to complete the cryptographic operation. Following this hypothesis, and assuming exports 50 and 49 are not part of the cryptographic operation, we can assume that the first export to call is export 30. Following each ordinal returned from each function, we see that the last ordinal to be called is ordinal 51 – which prints the “Play me a song, have me play along…” string. Perhaps after the decryption process, there is some hint in memory as to how to play this “song.” To do this, we can attach OllyDbg to a python terminal running the script below and set a breakpoint right before the return instruction of export ordinal 51.
from ctypes import *
mydll = cdll.LoadLibrary("C:\Users\user\Desktop\flareon2016challenge.dll")
index = [30, 15, 42, 18, 12, 9, 4, 17, 33, 31, 44, 34, 19, 38, 40, 48, 37, 10, 25, 3, 26, 32, 22, 2, 8, 13, 28, 45, 5, 47, 20, 36, 7, 16, 29, 39, 24, 41, 43, 46, 14, 11, 21, 27, 23, 6, 35, 1, 51]
for i in index:
mydll[i]()
Now, when I was completing the challenge, I simply navigated to the .data section of the DLL in OllyDbg and scanned memory until I found something interesting. One could also look in IDA Pro to pinpoint the exact location of where decrypted data would appear. However, being a massive Star Wars fan, I became very excited when first thing I noticed was the string “usetheforceluke!”
Looking even closer, I saw the letters “MZ” after it – a PE header! I dumped memory starting at the “MZ” and opened it in IDA Pro.
Eureka! It’s a program that calls Kernel32.Beep 18 times. Export 50 also takes two arguments that are passed to Kernel32.Beep, so this must be the “song” that the program is asking to be played! We can modify our Python script to call export 50 18 times with the correct values from this PE file.
from ctypes import *
mydll = cdll.LoadLibrary("C:\Users\user\Desktop\flareon2016challenge.dll")
index = [30, 15, 42, 18, 12, 9, 4, 17, 33, 31, 44, 34, 19, 38, 40, 48, 37, 10, 25, 3, 26, 32, 22, 2, 8, 13, 28, 45, 5, 47, 20, 36, 7, 16, 29, 39, 24, 41, 43, 46, 14, 11, 21, 27, 23, 6, 35, 1, 51]
for i in index:
mydll[i]()
mydll[50](0x1b8, 0x1f4)
mydll[50](0x1b8, 0x1f4)
mydll[50](0x1b8, 0x1f4)
mydll[50](0x15d, 0x15e)
mydll[50](0x20b, 0x96)
mydll[50](0x1b8, 0x1f4)
mydll[50](0x15d, 0x15e)
mydll[50](0x20b, 0x96)
mydll[50](0x1b8, 0x3e8)
mydll[50](0x293, 0x1f4)
mydll[50](0x293, 0x1f4)
mydll[50](0x293, 0x1f4)
mydll[50](0x2ba, 0x15e)
mydll[50](0x20b, 0x96)
mydll[50](0x19f, 0x1f4)
mydll[50](0x15d, 0x15e)
mydll[50](0x20b, 0x96)
mydll[50](0x1b8, 0x3e8)
The flag [email protected] is printed to the tune of the Star Wars Imperial March :]
Challenge 8 – CHIMERA
Tools Used: IDA Pro, DOSBox, Python, Hex Editor
This was my personal favorite challenge of the 2016 FLARE On challenges. Challenge 8 of the FLARE On challenge was a 32-bit Windows Portable Executable file – at least on the surface. Opening the challenge file up in IDA yields what looks like a simple single-byte XOR decryption loop – a deceptively simple challenge.
Reversing the single-byte XOR loop yields the key “this is the wrong password.” Obviously, this is not the code path we are looking for, but what is the correct one? Initially, I began looking through the binary for anything that may resemble code or a base-64 encoded string that could be decoded into executable code, but I came up short. Then I reconsidered the challenge name – CHIMERA. In Greek mythology, the Chimera was a monster composed of different animals’ body parts. Additionally, there was an interesting string in the binary that could be a clue – “This one’s for the geezers…” After reflecting on this for a while, I remembered the legacy support of Windows PE files – namely that PE headers contain a DOS stub with a 16-bit executable – this is because all PE files can be run in 16-bit DOS mode. Typically, this 16-bit executable usually just prints a message to the screen such as “This program cannot be run in DOS mode,” as most modern 32-bit PE files are not built with full DOS legacy compatibility on. However, given the clues presented, I decided it would be worth disassembling the executable in IDA pro as a 16-bit binary. Sure enough, I found something interesting.
The first thing to note that I had initially overlooked is a slight variation on the standard “This program cannot be run in DOS mode” string that is found in the default DOS stub in most modern PE headers – here it is “This program cannot not be run in DOS mode.” The double negative in this sentence only verifies that we are on the right track. Additionally, after printing this string, there is a jump instruction that jumps to the following code snippet.
This code simply works backwards from 0x70 words ahead and adds a counter variable starting at 0x70 and counting down to 0 to each word. This effectively decrypts the code. In order to disassemble the decrypted code, I replicated the above code in Python and patched the CHIMERA binary with the decrypted code. Loading it into IDA yielded code which had a variety of anti-disassembly measures in it (namely jumping into the middle of instructions. Once the code was cleaned up, I discovered that the code checks to see if the date is before 1990 before proceeding. In order to circumvent this check, I patched the binary again by overwriting the check with NOP instructions (0x90).
After NOP-ing out the date check, I reversed the password verification algorithm by using a combination of static analysis with IDA and dynamic analysis with DOSBOX. The code begins its verification algorithm by reading in the input data and looping over the length of it with the following code (note that the dl register is initialized to 0x97 before this loop):
This code alters the input data to the program according the the algorithm below. Implementation of the ror and roloperations can be found on Github here.
import binascii
rol = lambda val, r_bits, max_bits:
(val << r_bits%max_bits) & (2**max_bits-1) |
((val & (2**max_bits-1)) >> (max_bits-(r_bits%max_bits)))
ror = lambda val, r_bits, max_bits:
((val & (2**max_bits-1)) >> r_bits%max_bits) |
(val << (max_bits-(r_bits%max_bits)) & (2**max_bits-1))
max_bits = 8
pool = "FF157420400089EC5DC34246C063862AAB08BF8C4C25193192B0AD14A2B667DD39D85F3F7B5CC2B2F62E759B6194CFCE6A9850F25BF045300E38EB3B6C667F243DDF8897B9B3F1CB83991A0DEFB103559E9A7A10E036E8D3E432C17807B76BC770C92CA091356DFE735EF4A4D9DB4369F58DEE447D48B5DC4B02A1E3D2A6213E2FA3D7BB845AFB8F121C4128C576599CF73306270A0BAF71164AE99F4F6FE20FBE2BE756D553792D641795A7BD7C1D5893A565F81813EABCE5F3370496A81E012982513C681F8EDA8A05227249FA87A95462C6AA09B4FDD6D1AC8511473A9DE64D1BCC528023FCED8B7E60CD6E57BADEAECAC4770C4ED4D0C8E1B8F92690813400000000000000000000000000000000"
pool = binascii.unhexlify(pool)
for i in range(len(input_data)):
if i == 0:
prev = 0x97
else:
prev = input_buffer[len(input_buffer) - i]
input_buffer[len(input_buffer) - (i + 1)] ^= ord(pool[ord(pool[rol(prev, 3, 8)])])
Essentially, it encrypts the input data according to a static table of byte values (referenced in the code as pool). After the data is encrypted, it is finalized using a simple XOR loop in the next block of assembly.
This block of assembly can be translated to Python as follows:
for i in range(len(input_buffer)):
if i == 0:
prev = 0xC5
else:
prev = input_buffer[i-1]
input_buffer[i] ^= prev
Finally, the program verifies the modified input by XORing each element of the resultant input_buffer with an element in this array:
end_compare= [0x38, 0xe1, 0x4a, 0x1b, 0xc, 0x1a, 0x46, 0x46, 0xa, 0x96, 0x29, 0x73, 0x73, 0xa4, 0x69, 0x3, 0x0, 0x1b, 0xa8, 0xf8, 0xb8, 0x24, 0x16, 0xd6, 0x9, 0xcb]
If the result of each XOR operation is 0, the password is accepted. We can reverse all of these operations with the following Python code:
import binascii
rol = lambda val, r_bits, max_bits:
(val << r_bits%max_bits) & (2**max_bits-1) |
((val & (2**max_bits-1)) >> (max_bits-(r_bits%max_bits)))
ror = lambda val, r_bits, max_bits:
((val & (2**max_bits-1)) >> r_bits%max_bits) |
(val << (max_bits-(r_bits%max_bits)) & (2**max_bits-1))
max_bits = 8
end_compare= [0x38, 0xe1, 0x4a, 0x1b, 0xc, 0x1a, 0x46, 0x46, 0xa, 0x96, 0x29, 0x73, 0x73, 0xa4, 0x69, 0x3, 0x0, 0x1b, 0xa8, 0xf8, 0xb8, 0x24, 0x16, 0xd6, 0x9, 0xcb]
pool = "FF157420400089EC5DC34246C063862AAB08BF8C4C25193192B0AD14A2B667DD39D85F3F7B5CC2B2F62E759B6194CFCE6A9850F25BF045300E38EB3B6C667F243DDF8897B9B3F1CB83991A0DEFB103559E9A7A10E036E8D3E432C17807B76BC770C92CA091356DFE735EF4A4D9DB4369F58DEE447D48B5DC4B02A1E3D2A6213E2FA3D7BB845AFB8F121C4128C576599CF73306270A0BAF71164AE99F4F6FE20FBE2BE756D553792D641795A7BD7C1D5893A565F81813EABCE5F3370496A81E012982513C681F8EDA8A05227249FA87A95462C6AA09B4FDD6D1AC8511473A9DE64D1BCC528023FCED8B7E60CD6E57BADEAECAC4770C4ED4D0C8E1B8F92690813400000000000000000000000000000000"
pool = binascii.unhexlify(pool)
# Reverse final encryption loop
for i in range(len(end_compare)):
if i == len(end_compare) - 1:
prev = 0xC5
else:
prev = end_compare[len(end_compare) - (i + 2)]
end_compare[len(end_compare) - (i + 1)] ^= prev
# Reverse static lookup encryption
for i in range(len(end_compare)):
if i == len(end_compare) - 1:
next = 0x97
else:
next = end_compare[i + 1]
end_compare[i] ^= ord(pool[ord(pool[rol(next, 3, 8)])])
out = ''
for i in end_compare:
out += chr(i)
print out
This code reveals the flag [email protected], and sure enough, putting this into the program by running it in DOSBOX yield a success message!
This concludes part one of my FLARE On Challenge write up. Be on the lookout for part two, which will cover challenges 9 and 10 (Challenge 10 is particularly interesting :])!
Bridging the gap between the impossible and the practical with innovative technology solutions in cyber, data science, mobile, microelectronics and information operations and a full spectrum of products and capabilities to advance national security missions.