Babysc

This challenge looked innocent at first, just a basic shellcode injection problem. But of course, like any good CTF, there’s always a catch. Here's how we tackled it, the thought process behind each move, and how a random YouTube video ended up being the hero of the day.

We started off by checking the files in the challenge directory:

ls -l

Cool. We’ve got:

· A binary (babysc)

· Its source code (babysc.c)

· A Dockerfile for deployment

Next, we probed the binary to see what kind of beast we were dealing with:

Okay, good news: it’s a PIE binary, but it’s not stripped, which means symbols are still intact. Makes life a whole lot easier if we need to reverse it.

After doing basic recon, we cracked open babysc.c to see what this baby was really doing. Our eyes locked onto the vuln() function, and this is where the magic (and the trap) happens.

Here’s the critical section:

So basically:

· The binary allocates 0x1000 bytes (4096 bytes) at a fixed address 0x26e45000

· The memory is marked RWX — which screams “inject your shellcode here”

· Then it reads our input (stdin) straight into that memory

And then it hits us with the classic:

Which means… it literally executes our shellcode. Directly. No filters, no sanitization. At this point, we were like:

"Okay... this is either super easy, or there's a trap waiting." Spoiler: There was a trap.

Right after the read, we found this sneaky little loop:

Let’s break that down:

· It scans our input two bytes at a time using a uint16_t pointer.

· If it detects any of these:

o 0x80cd → int 0x80 (legacy syscall)

o 0x340fsysenter (also old school)

o 0x050fsyscall (modern 64-bit syscall)

So essentially, we’re allowed to write and execute shellcode… as long as it doesn’t contain any syscallinstructions.

Yeah. The binary gave us full RWX memory and then said, “you can cook, but no knives allowed.”

So now we knew what we were up against:

· We can run shellcode.

· We just can’t use syscall, int 0x80, or sysenter.

· Which is… kind of a problem because shellcode literally survives on syscalls.

Our first instinct was:

"Okay maybe we can find some obscure way to do a syscall without those bytes." But then we reminded ourselves: “work smart, not hard.”

One of us remembered watching a video a while back (shoutout this gemarrow-up-right) where the guy solved a similar syscall-restricted shellcoding challenge using Metasploit's msfvenom.

So, we figured: why not try the same? Here's what we ran:

Let’s break that down:

· -p linux/x64/exec → generate a shell-spawning payload for x64 Linux

· CMD="/bin/sh" → tells it to run /bin/sh

· -f raw → output as raw shellcode (not in ELF or C form)

· -e x64/xor → use the XOR encoder to hide the bad bytes

· -o bismillahflag.bin → save it to this file

The x64/xor encoder does two things for us:

1. Obfuscates the actual syscall bytes (0x0f 05) so they’re no longer detectable

2. Includes a decoder stub that runs first, decodes the payload in memory, and then jumps to the clean version

Think of it like smuggling forbidden items past airport security but wrapped in bubble wrap and a hoodie.

To be sure the encoder did its job, we ran:

We got no results, which meant the forbidden instructions were no longer present in raw form. Our payload was stealthy enough to survive the filter check.

So now, all we had to do… was send it.

So now that we had our XOR-encoded shellcode sitting in bismillahflag.bin, the next question was:

“How do we actually deliver it to the target?”

Lucky for us, the Dockerfile made that very clear:

This tells us:

· The binary is running inside a container

· It’s listening on port 10001

· When a client connects, socat spawns a fresh instance of the binary

· Whatever we type gets piped straight into the binary’s stdin

Sounds like the perfect opportunity to do a little shellcode delivery over TCP. We used this classic trick:

Explanation:

· cat bismillahflag.bin sends our raw shellcode as input

· The second cat keeps stdin open, so if the shellcode spawns a shell, we can interact with it

· nc connects to the challenge server on port 10001 And just like that, we saw:

At this point, we knew the payload executed. The syscall filter didn’t catch us.

The shell spawned.

We had officially hacked into a container that begged us not to syscall. We said "no problem", and called /bin/sh anyway

Once our shellcode ran and we saw:

Enter 0x1000

Executing shellcode!

…we knew we had a shell and a real one, not a fake prompt.

We did a quick test just to confirm we could actually run commands:

We were inside. Naturally, we ran:

Ah, flag.txt? Tempting… but this looked too easy. We opened it anyway because, well, curiosity:

A decoy flag :(

So, we did what any desperate shell user does:

Flag: umcs{shellcoding_78b18b51641a3d8ea260e91d7d05295a}

Last updated