Damaged Zip archive (video)

In this video we can see how to inspect a damaged Zip archive using the Profiler in a real-world scenario. Although soon the automatic recovery of damaged Zip archives will be available and it will be possible to perform this sort of task programmatically, it’s still useful to see how to do this kind of thing manually.

Filters with range parameters

The upcoming 0.8.9 release improves filters and introduces range parameters. If you don’t know what filters are you can take a look at the original introductory post.

What is now possible to do is to specify an optional range for the filter to be applied to. This is extremely useful, since many times we need to modify only a portion of the input data. Take, for instance, a file in which a portion of data is encrypted (or compressed) and we want to keep the outer parts not affected by the filter. This would be the layout of the input data:

Data layout

To handle the outer parts (before and after) we have an extra parameter called ‘trim’. This parameter can be set to one of the following values:

  • no: the outer parts are not trimmed and kept in the output data
  • left: the left part is trimmed while the right one is kept
  • right: the right part is trimmed while the left one is kept
  • both: both outer parts are trimmed and the output data will contain only the output of the filter

Let’s try to understand this better with a practical example. Let’s consider a compressed SWF file. The Profiler handles the decompression automatically, but just for the sake of this post we’ll decompress it manually. In case you don’t know, the format of a compressed SWF is as following:

3 bytes: CWS signature (would be FSW uncompressed files)
4 bytes: uncompressed file size
1 byte: version

To dig deeper into the file format you may want to take a look at a new web page we are preparing at fileanalysis.net. But for the sake of our example you need only to know the initial fields of the header. To decompress a SWF we need to decompress the data following the header and then to fix the first signature byte and we’ll do this manually and graphically with filters.

Let’s start by selecting the compressed input data and then by trying to add the unpack/zlib filter.

ZLib filter

As you can see we are prompted with a dialog asking us if we want to use the range defined by the selection of the hex editor on the left (input data). We accept and specify that we want to keep the data on the left (although in this case we could have just disabled the trimming completely, let’s just pretend there’s some data on the right we want to discard).

Now we need to fix the signature byte. To do this we use the misc/basic filter (set operation, offset 0, size 1 and no trimming).

By running the preview we’ll have the decompressed SWF.

The exported filters will look like this:


    
    

While this was a very simple case, much more complicated cases can be handled. Also remember that filters can be used to specify how to load embedded files, which means that, for instance, it’s easy to decrypt a file contained into another file and inspect it without ever having to save the decrypted data to disk.

I hope this post offers some understanding of an advanced use of filters.

Python SDK improvements

The upcoming 0.8.9 release of the Profiler improves integration with Python and the SDK exposes new functionality. Moreover, it lays down the groundwork needed to expand the SDK in the next releases. Documentation of the SDK has been included in the docs directory and also a Python command line has been added to the workspace.

To offer a glimpse of the capabilities here is a small code snippet to create a custom plot which could be used, for instance, to display entropy.

import random

ctx = proContext()
v = ctx.createView(ProView.Type_Plot, "Test") 
v.setPlotTitle("Random")
v.setAxisScale(ProPlotView.xBottom, 0, 100)
v.setAxisScale(ProPlotView.yLeft, 0, 50)
x = NTDoubleVector()
y = NTDoubleVector()
i = 0
while i < 100:
    x.append(i)
    y.append(random.randrange(50))
    i = i + 1
v.addCurve(x, y)
ctx.addView(v)

And the screenshot of the created plot.

Custom plot

While this doesn't sound too exciting at first, the on-going SDK expansion will allow to do some very interesting things. Stay tuned as some more technical posts are coming.

News for version 0.8.8

While we talked about some of the news of this version, there are some more which are worth mentioning.

introduced new multi-file report and project technology with compression and encryption
introduced new UI for workspace mode
added Windows Lnk support
– added file extensions scan option
– added directory scan to command line
added PNG CRC validation
added new filters: misc/replace and dev/array
– several UI improvements
– hex editor improvements
– increased memory limit

New workspace UI

The workspace features a new dock-based UI. It is easy to get used to the new UI as it is completely intuitive.

Workspace UI

Now our users can completely costumize their analysis workspace.

PNG CRC validation

This feature was requested by one of our customers and it can come handy during forensic analysis. The CRC of each PNG chunk is verified and those which don’t match are signaled in the format view (highlighted in red).

PNG CRC

New filters: misc/replace and dev/array

Two new filters have been added. misc/replace is self explanatory: it replaces bytes and strings. While dev/array is a small addition which can come handy all those times we need to copy some bytes to an array in a programming language. We can specify the language, the radix and the number of columns and we get something like:

unsigned char data[64] =
{
    0x4D, 0x5A, 0x90, 0x00, 0x03, 0x00, 0x00, 0x00,
    0x04, 0x00, 0x00, 0x00, 0xFF, 0xFF, 0x00, 0x00,
    0xB8, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x40, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0xF8, 0x00, 0x00, 0x00
};

We hope you enjoy this new version.

Windows Link Support

The next Profiler update, along with several new features, will also include preliminary support for the Windows Link files (also known as Shell Links). This format has been introduced with Windows 95 and is frequently taken into consideration in forensic analysis procedures to discover usage information about files and folders.

In the image below, a link file that opens the Cerbero homepage using Internet Explorer is being shown.

Windows Link Format

Next generation reports

The upcoming 0.8.8 version of the Profiler features a new way of generating reports. In particular it will be possible to scan multiple files and save a single report for all of them. Moreover, the report can contain the original files (either plain or compressed and/or encrypted).

But let’s proceed in order. While it is certainly easy to use the new features, reading this article will help you understanding of the under the hood details.

Let’s begin with a custom scan of the SysWOW64 directory.

When the scan has finished, we get the list of scanned files.

At this point we can click on “Save report”.

In the options above I decided to include all files in the saved project and use the password “test” to encrypt them. This comes handy if we want to move our analysis to another system or send it to a colleague.

What we also can do is to save the unpacked database files instead of the project.

This will create a directory with the extension “cprodb” containing two files. A main.db and an objects.sqlite file. Both are SQLite databases: main.db contains all kinds of information while objects.sqlite is reserved to store data about scanned files. Plugins will have access to main.db to store their own data (to be documented in some future post), keeping in mind that tables which begin their name with “pro” are reserved for internal use.

When a report is saved, not as a project, but in its unpacked state, it can be saved back to a project by clicking “Save project”. If no project is associated with the report, the user will be asked to select an existing project. Unpacked reports are modified directly and don’t need to be saved.

Most of the work for this feature went into testing database technologies suitable to contain a large number of elements and to stress-test them. In the end I decided to implement an abstract interface so that in the future other kind of databases can be added and the user will be in the position to decide whether he prefers to use SQLite or some other database to store information of the scanned files.

However, even by using the current SQLite implementation there should be no problem in generating reports with tens of millions of scanned files. I personally did tests up to 100 million of entries.

Here’s a screenshot with a saved report of an entire Virtual Machine: the project includes all of the files.

The new reporting technology is going to enable other important features (soon to be documented). Please be patient, because it will still take some more weeks to deliver the 0.8.8 version, which will include other significant improvements and features.

Profiler 0.8.7 Demo

The upcoming version 0.8.8 of the Profiler is feature-packed (so stay tuned). In the meanwhile we found some time to prepare a new evaluation version based on the previous 0.8.7 version.

SHA1: EB208B9CB9FB3B7ADAE5A0FD24C4AA08E591752F

This new demo version is no longer subject to the DOC only limitation, which means it is able to open every CFBF file (including XLS and PPT files).

Please note that this new demo is subject to the following limitations:

  • only CFBF files are supported
  • embedded and referenced files can’t be inspected
  • only single file analysis is available
  • although the action system is working, some actions are not available
  • filters are not available

Enjoy!

An introduction to Filters

While filters have always been a part of the Profiler, they are now partly exposed only with the upcoming 0.8.7 version. Shortly explained: filters are algorithms of different nature which can be applied to portions of data. The starting point to interact with filters is the filter view. This view is triggered with the Ctrl+T shortcut. Let’s start with some step by step screenshots.

This is a Zip file and the gray area marks a compressed file. Let’s select this area with Ctrl+Alt+A.

Zipped file

Now let’s press Ctrl+T and take a look at the filter view.

Filter View

The tree at the top-left displays the available filters. The property editor next to it contains the available parameters for the currently selected filter and the list beneath the filters is the stack of filters which need to be applied to the input data. The hex view on the left contains the data which we have previously selected in the hex view (meaning the compressed file) and represents the input data.

Now if we want to decompress the data, let’s add the unpack/zlib filter (with the raw parameter checked) to the stack by clicking on “Add”.

Filter stack

And at this point we can click on the “Preview” button.

Unzipped file

But what if we want to retrieve the SHA1 of the decompressed data? Just add the hash/sha1 filter to the stack.

Filter stack

This time we’ll have only the SHA1 hash in the output view.

SHA1

The filter stack can be imported and exported. Just right-click inside the stack list. The exported filters of the current stack will look like this:

If we had opened the filter view without a selection we would have had an editable hex view on the left, which we could fill with our custom data. Also, the hex views in the filter view have all the benefits of a regular hex view (that is if the filter view is not triggered while loading a file, as we’ll see later); this means that it’s possible to trigger a new filter view by selecting something in the hex view containing the filtered output data.

What I’ve showed until now isn’t an every-day task, but it serves the purpose of introducing the topic. We’ll see some real-word cases after I have introduced the misc/basic filter. But before of that let me shortly mention text output filters.

Right now the Profiler features two filters with text output: disasm/x86 and disasm/x64. When a filter outputs text, we’ll be shown by default a text view instead of the hex view on the right.

x86 filter

This can come handy when decrypting shellcode. What we also most probably need is the misc/basic filter, which I think will be one of the most commonly used filters. Let’s look at its parameters.

basic filter params

operation specifies what is going to be performed on the input data. Available operations are: set(fill), add, sub, mul, div, mod, and, or, xor, shl, shr, rol, ror. bits specifies the size in bits of each element the operation has to be performed on (values up to 64 bit are supported). endianness applies only to elements greater than 8 bits. radix is related to the input format of value. value contains the data used for the operation.

If I now inserted “FF” as value, then I would be xoring every byte of the input data with that 1-byte value. If, however, I inserted “FF 00”, then I would be xoring every even byte with “FF” and every uneven one with “00”. I may even only modify every third byte by using wildcards and leaving the first two unchanged, I only have to insert “* * FF”. Wildcards and arrays can be used even in the context of values bigger than 1 byte of course.

Now that I’ve introduced the basic filter, let’s see our first real-world case. What you see is a PDF containing an encrypted malware.

Malware PDF

At the caret position an encrypted Portable Executable starts, it’s quite easy to figure out because of the repetitive sequence of bytes (a PE header is full of zeroes). I select the data with the go to dialog (Ctrl+G).

Select encrypted malware

Now instead of triggering the filter view directly, I’m going to load the embedded file with Ctrl+E. This triggers the load file dialog.

Load malware

As you can see I specified the format of the file to load (PE) and inserted an optional name for it, but now it’s necessary to tell the Profiler how to decrypt the file, otherwise it won’t be able to load it. So I’ll just click on the “Filters” button at the bottom-right and a filter view dialog will pop up.

Malware filters

The decryption sequence is “EB FF FD FC EB FF 23 95”. It can be expressed like I did as an array.

But since it amounts to a 64-bit value, it can also be expressed like this.

Ok, back to the malware. After having added the decryption filter to the stack I just close the dialog and confirm the load file dialog. We’re now able to navigate the decrypted PE.

Decrypted malware

Now I can just save the project (which is under 1KB) and send it to a colleague who will be able to inspect the decrypted file.

The last real-world case I’m going to present I made it myself, because it was easier than to start looking for one. I just took a cursor resource from a PE and appended another RC4-encrypted PE to it. Then replaced the original resource and opened the host file with the Profiler.

Fake resource

The Profiler tells us that the are problems in the resource and also shows us the file. In fact, we can see where the foreign data begins. I just select the foreign data (I can do this both from the resource directory or from the DIBCUR embedded file), trigger the load file dialog and apply the decryption filter.

In upcoming updates there will be many improvements to filters: many more will be added and the concept will be extended and made customizable. Clearly this post doesn’t cover all of that, but I think for now this is enough to keep the introduction simple.

Microsoft Authenticode

Based on RSA’s PKCS7 standard, Authenticode is the technology developed by Microsoft to digitally certify programs and drivers on Windows. Trusted signatures guarantee that the certificate owner is indeed the author of the signed executable, and also that the data itself has not been tampered with by anyone else.

In a default configuration scenario, the operating system considers these signatures during all but three events:

  1. When kernel-mode drivers are loaded
  2. When executable images that derive directly from content downloaded using Internet Explorer (or any other third-party browser which supports it) are written to disk.
  3. When an application requires admin privileges.

Point two is a weak security measure for the following reasons:

  1. Even if the right browser is used, there’s still no guarantee that the verification request (because it is by no means mandatory) is honored by the operating system unless the UAC privilege elevation dialog is invoked (either via manifest or using the “Run as Administrator” menu item). It’s important to note that if the Authenticode signature can’t be verified, the dialog being shown to the user is the same as the one for unsigned executables. At this point, there is no way to distinguish from unsigned (no code signature) and untrusted programs (invalid code signature) using the UAC dialog alone.
  2. Once an application has been authorized by a user and his certificate store, no checks are performed when it is moved to another system (even if the new certificate store can’t validate the Authenticode signature)
  3. The whole mechanism heavily relies on the file system being used; copying unauthorized files to a non-NTFS file system (which happens quite a lot, considering the vast majority of USB drives are using FAT32) doesn’t preserve the alternate data streams created by browser.

Also the verification is not self evident if done manually using Windows Explorer, as the properties dialog doesn’t show the validity of the certificate until the user clicks the “Details” button. This is highly misleading, because the user might get a false sense of security by just checking whether the executable contains a digital signature.

Windows certificate dialog

The upcoming version 0.8.6 of the Profiler introduces support for this technology, allowing users to very quickly access and verify code signing information.

The following screenshot shows a perfectly valid digital signature; all the certificates taken from the Authenticode data have been successfully used to build a trust chain that validates the PKCS7 using the system store, which means that those at the root of the tree have been directly validated by the Windows Certificate Store.

Microsoft Authenticode – A valid digital signature

As you can see, the risk factor is set to zero, since the validity of the publisher has been determined. This behavior can be changed from the Risk panel of the options: it is not on by default!

For comparison, the following screenshot shows how an invalid digital certificate is displayed:

Microsoft Authenticode – An invalid digital signature

In this case the hash is no longer what expected, issuing both a digest and an invalid certificate errors.
Countersignatures are of course supported and I think you’ll be pleased with how fast our implementation is.

Additional resources:

  1. Windows NTFS Alternate Data Streams, from Symantec
  2. Mark of the Web, from MSDN

Validation of Portable Executable resources

One of the new features of the upcoming 0.8.6 version of the Profiler is the validation of resources. This means the Profiler verifies the integrity of resources and lets the user inspect problems, making it easy to discover things like appended files or fake resources. This feature comes handy since very often malware is hidden in resources and droppers often use resources to store their payload.

All the most important resource types are supported:

  • Version info
  • Bitmaps
  • Icons
  • Cursors
  • Icon groups
  • Cursor groups
  • Configuration files
  • Accelerators
  • Menus
  • Dialogs
  • String tables
  • Message tables
  • Any other supported file format

So let’s see a simple test case. What I did is to append a DLL to a bitmap and then replace one of the bitmaps in explorer.exe with my modified one.

I could’ve used any other resource type, or even a PNG or GIF, it wouldn’t have mattered.

The simplified resource tree highlights problems with their risk color, while unsupported types are highlighted in gray.

One can jump to problems with the F2 shortcut, no need to scroll the tree ourselves in search of problems.

In the screenshot above the analysis shown is for my fake bitmap. As it’s possible to see, the bitmap ends where the red-marked data begins.

In this context it is very easy to just load the embedded PE with the “Load as…” (Ctrl+W) command.

And this gives me the opportunity to mention briefly another nice improvement to the hierarchy view.

As you can see files are now grouped according to their type. This makes it much easier to go over the files or to look for specific types. This behavior is optional and can be changed from the settings.

I think I could’ve presented this new feature with a more interesting real-world case. However, there are still some things to do in order for the new version to come out and there wasn’t enough time. I hope, nevertheless, that you enjoyed the post. 🙂