PySide support

This is really a small addition which took just a couple of hours of work, but since it can come very handy, it’s worth dedicating a post to it. The upcoming 0.9.1 version of the Profiler adds explicit support for PySide. Thus, it will be possible to create Qt widgets and add them to the workspace.

Installing PySide

First of all, let’s install PySide. There are 3 ways to do this.

1) Install it from the qt-project page.

Make sure you select the package matching the current Python version used by the Profiler.

2) Install the package we compiled for you. It’s vanilla, directly from the original sources, but it has the advantage that it is guaranteed to work. In fact, at the time of writing the official package contains a bug (missing shiboken Python module) and so the first one is not really an option until it is not fixed.

Download
SHA1: 2024348E79890A167BB231098A6A16FC8BB02C9E

3) You can compile PySide yourself following the instructions at qt-project. At the end, use the installer created inside ‘c:\pyside-setup\dist’.

A code sample

Using it is even easier than the setup process. Basically ProContext has a new method called createViewFromWidget which takes as a parameter a widget created by PySide and returns a ProView which in turn can be added to the workspace.

Adding a widget to the workspace only takes the following line:

ctx.addView(ctx.createViewFromWidget(widget))

Therefore using an existing widget and adding it to the workspace is very easy. Let’s see a real-world widget like an official PySide sample: PySide/examples/effects/lighting.py. It’s sufficient to remove:

if __name__ == '__main__':

    import sys

    app = QtGui.QApplication(sys.argv)

    lighting = Lighting()
    lighting.setWindowTitle("Lighting and Shadows")
    lighting.resize(640, 480)
    lighting.show()

    sys.exit(app.exec_())

And add:

lighting = Lighting()
lighting.setWindowTitle("PySide widget")

ctx = proContext()
ctx.addView(ctx.createViewFromWidget(lighting))

Now we can add an action to execute the code or just insert it in the custom script box (Ctrl+Alt+R) and the view will be shown like this:

To set a custom icon for the view use setWindowIcon.

As usual stay tuned as the upcoming version is going to include some major additions and significant changes.

News for version 0.9.0

We’d like to wish everybody merry Christmas and a happy new year!

We’re just in time to place under the Christmas tree a new version of the Profiler with the following news:

added Java Class support including byte code disassembler and layout ranges
added .NET support including byte code disassembler and layout ranges
added DEX support including byte code disassembler and layout ranges
added dedicated view to display data like raw PDF objects
added PE MUI resources validation
– added Adler32 to filters
– updated jsbeautifier
separated malicious threats from intrinsic ones in the report view
– fixed update with unprivileged user account on Windows
– fixed several bugs

The main addition in this new release as seen previously is the managed trio Java, DEX and .NET.

Dedicated view to display raw data

Previously PDFs had 3 views to display objects: one for the dictionary, one for the decoded stream and one for the decoded stream shown as text. Now there’s also a ‘raw data’ view to show the object unmodified just as it is in the file.

Highlighted with different colors you can see the dictionary/value and the stream part. The same applies to child objects which are highlighted in the stream of their parent.

PE MUI resources validation

Following a short post about MUI resources, validation for them has been added. Also some bug fixes related to resource validation.

Separated malicious threats from intrinsic ones

In order to better separate intrinsic risk factors from malicious threats, they are now shown separately in the report view.

Have some nice holidays and stay tuned as we’ll try to add even more features the upcoming year.

Christmas hat image from freevector.com

DEX support

Support for Android’s DEX format is the last major feature of the upcoming 0.9.0 release of the Profiler. The support includes format, layout ranges and a Dalvik disassembler. Support for APK is implicit, since support for Zip archives has been added long ago.

All sections of the format are accessible. The central point for parsing a DEX file is the Classes view. Hence it’s also the most complex view of the format.

Also accurate layout ranges can help to analyze the format.

And finally a disassembler to quickly inspect the Dalvik bytecode.

Here’s a disassembled function from the Android SDK NotePad sample:

  private final void cancelNote()
  {
    const/4 v3, #int 0 // #0
    iget-object v1, v4, android.database.Cursor mCursor
    if-eqz v1, loc_74 // +34
    iget v1, v4, int mState
    if-nez v1, loc_90 // +38
    iget-object v1, v4, android.database.Cursor mCursor
    invoke-interface {v1}, void android.database.Cursor.close()
    iput-object v3, v4, android.database.Cursor mCursor
    new-instance v0, android.content.ContentValues
    invoke-direct {v0}, void android.content.ContentValues.()
    const-string v1, "note"
    iget-object v2, v4, java.lang.String mOriginalContent
    invoke-virtual {v0, v1, v2}, void android.content.ContentValues.put(java.lang.String, java.lang.String)
    invoke-virtual {v4}, android.content.ContentResolver getContentResolver()
    move-result-object v1
    iget-object v2, v4, android.net.Uri mUri
    invoke-virtual {v1, v2, v0, v3, v3}, int android.content.ContentResolver.update(android.net.Uri, android.content.ContentValues, java.lang.String, java.lang.String[])
loc_74:
    const/4 v1, #int 0 // #0
    invoke-virtual {v4, v1}, void setResult(int)
    invoke-virtual {v4}, void finish()
    return-void
loc_90:
    iget v1, v4, int mState
    const/4 v2, #int 1 // #1
    if-ne v1, v2, loc_74 // -11
    invoke-direct {v4}, void deleteNote()
    goto loc_74 // -16
  }

And the original source for comparison:

    private final void cancelNote() {
        if (mCursor != null) {
            if (mState == STATE_EDIT) {
                // Put the original note text back into the database
                mCursor.close();
                mCursor = null;
                ContentValues values = new ContentValues();
                values.put(NotePad.Notes.COLUMN_NAME_NOTE, mOriginalContent);
                getContentResolver().update(mUri, values, null, null);
            } else if (mState == STATE_INSERT) {
                // We inserted an empty note, make sure to delete it
                deleteNote();
            }
        }
        setResult(RESULT_CANCELED);
        finish();
    }

Next week will be dedicated to fixing reported bugs and adding some small improvements. After that and some testing the new version should be ready, so keep tuned as the new version should be deployed soon!

.NET support

Although there haven’t been customer requests for this, the upcoming 0.9.0 version of the Profiler adds support for .NET, which includes format, layout ranges and an MSIL disassembler.

As usual, let’s begin with the format itself. Since some users probably have used CFF Explorer to inspect the .NET format in the past, I have kept the same format view in the Profiler as well.

So how is it better than CFF Explorer? First of all, it’s very fast. There’s absolutely no wait time in opening a large metadata set as the following.

And then one handy feature which was not available in CFF Explorer, is the capability to display a second set of metadata. For instance, .NET native images (meaning those files in the assembly cache created with ngen.exe) contain two sets of metadata. The Profiler lets you inspect both sets.

Layout ranges cover all PE parts, so it was normal to add them for .NET as well. These are the ranges available for .NET:

Let’s see them in the hex editor.

If you’re asking yourself why it’s all gray, it’s because more ranges are blended together, meaning that all .NET metadata is contained in the .text section of the PE and that section is marked as executable even if the assembly contains only MSIL code. While it doesn’t make much sense to mark as executable a region of data containing strings and tables, my best guess is that old versions of Windows had no in-built support for .NET in the loader, which is why assemblies contain a single ‘mscoree.dll’ import descriptor and the entry point is just a jmp to the only IAT thunk (_CorExeMain for executables, _CorDllMain for dlls). That API then loads the .NET framework if necessary. Since that single native jmp instruction requires a section in the PE marked as executable and because the granularity of sections is the same as virtual memory pages, it was probably considered a waste to use an entire memory page just for a jmp instruction and so the result is that everything is contained into a single executable section. This could probably be changed as new versions of the framework do not even run on older systems.

However, for our inspection purposes it suffice to get rid of the Code range by pressing Ctrl+Alt+F.

We can now inspect .NET layout ranges without the conflict. One nice aspect about it is that the code of IL methods is easy to distinguish between its header and extra sections.

Included is also an IL disassembler. Its purpose is to let our customers quickly browse the contents of an assembly. As such readability was a priority and the output has been grouped for classes: I have always found it cumbersome in ILDasm to open every single method to inspect an assembly.

Here’s an output example:

  private static void Main(string [] args)
  {
    locals: int local_0,
            int local_1

    ldc_i4_2
    stloc_0 // int local_0
    ldloc_0 // int local_0
    stloc_1 // int local_1
    ldloc_1 // int local_1
    ldc_i4_1
    sub
    switch
      goto loc_22
      goto loc_60
    br_s loc_71
loc_22:
    try
    {
      ldstr "h"
      call System.Console::WriteLine(string) // returns void
      leave_s loc_81
    }
    catch (System.ArgumentNullException)
    {
      pop
      ldstr "null"
      call System.Console::WriteLine(string) // returns void
      leave_s loc_81
    }
    catch (System.ArgumentException)
    {
      pop
      ldstr "error"
      call System.Console::WriteLine(string) // returns void
      leave_s loc_81
    }
loc_60:
    ldstr "k"
    call System.Console::WriteLine(string) // returns void
    ret
loc_71:
    ldstr "c"
    call System.Console::WriteLine(string) // returns void
loc_81:
    ret
  }

And the original code:

        static void Main(string[] args)
        {
            int a = 2;
            switch (a)
            {
                case 1:
                    try
                    {
                        System.Console.WriteLine("h");
                    }
                    catch (ArgumentNullException)
                    {
                        System.Console.WriteLine("null");
                    }
                    catch (ArgumentException)
                    {
                        System.Console.WriteLine("error");
                    }
                    break;
                case 2:
                    System.Console.WriteLine("k");
                    break;
                default:
                    System.Console.WriteLine("c");
                    break;
            }
        }

There’s one last thing worth mentioning. .NET manifest resources are displayed as sub-files.

However, the parsing of ‘.resources’ files was still too partial and thus won’t be included in 0.9.0.

From the last two posts you may have guessed the topic of the upcoming release. So stay tuned as there’s yet more to come.

Java Class support

The upcoming 0.9.0 version of the Profiler adds support for Java class files. This will be especially useful for malware analysis. The support includes: disassembler, format views and layout ranges. Let’s see some screeshots of the format itself first.

Here’s a view of the constants:

Note: Utf8 strings are highlighted in orange just to distinguish them.

Here’s a view of the methods with their attributes:

And here’s the layout ranges view:

Again strings are in orange, while the actual code of a method is in a slightly lighter green than the method itself.

Since the format of class files is contiguous, it’s extremely easy to use layout ranges to create a new custom class file using the hex editor.

And finally, the disassembler:

The output shown in the screenshot above:

// SourceFile: HelloWorld.java

super class HelloWorld extends java.lang.Object
{

  static float f1;
  public static java.lang.String hello;

  HelloWorld()
  {
    // max_stack = 1  max_locals = 1
// line 1
    aload_0
    invokespecial java.lang.Object.() // returns void
    return
  }

  public static void main(java.lang.String[])
  {
    // max_stack = 7  max_locals = 6
// line 7
    ldc2_w 454.546
    dstore_1
// line 8
    ldc2_w 552441554577111995
    lstore_3
// line 9
    getstatic java.lang.System.out // java.io.PrintStream
    ldc "The value of i is: %f and %d"
    iconst_2
    anewarray java.lang.Object
    dup
    iconst_0
    getstatic HelloWorld.f1 // float
    invokestatic java.lang.Float.valueOf(float) // returns java.lang.Float
    aastore
    dup
    iconst_1
    lload_3
    invokestatic java.lang.Long.valueOf(long) // returns java.lang.Long
    aastore
    invokevirtual java.io.PrintStream.format(java.lang.Stringjava.lang.Object[]) // returns java.io.PrintStream
    pop
// line 10
    getstatic java.lang.System.out // java.io.PrintStream
    getstatic HelloWorld.hello // java.lang.String
    invokevirtual java.io.PrintStream.println(java.lang.String) // returns void
// line 13
    try
    {
      getstatic java.lang.System.out // java.io.PrintStream
      ldc "test2"
      invokevirtual java.io.PrintStream.println(java.lang.String) // returns void
    }
    catch (java.lang.Exception)  goto loc_57
// line 18
    goto loc_67
// line 15
loc_57:
    astore 5
// line 17
    getstatic java.lang.System.out // java.io.PrintStream
    ldc "error"
    invokevirtual java.io.PrintStream.println(java.lang.String) // returns void
// line 19
loc_67:
    return
  }

  static void ()
  {
    // max_stack = 1  max_locals = 0
// line 3
    ldc 43
    putstatic HelloWorld.f1 // float
// line 4
    ldc "Hello world!"
    putstatic HelloWorld.hello // java.lang.String
    return
  }

}

And the original source file:

class HelloWorld
{
  static float f1 = 43;
  public static String hello = "Hello world!";
  public static void main(String[] args)
  {
    double d1 = 454.545774;
    long l1 = 552441554577111995L;
    System.out.format("The value of i is: %f and %d", f1, l1);
    System.out.println(hello);
    try
    {
      System.out.println("test2");
    }
    catch (Exception e)
    {
      System.out.println("error");
    }
  }
}

There’s yet more to come. Stay tuned. 🙂

News for version 0.8.9

The new version is out. 🙂 Here’s a recap of the latest improvements:

increased python integration and exposed more parts of the SDK
– added SDK documentation to the docs directory
– added Python command line
– added global and individual file notes
improved filters and added range parameters
– introduced fullscreen modality in workspace (F11)

This has been mostly a transition release and what took most of the time were structural changes in how the SDK is exposed to Python so that future releases will benefit from it. The main advantage for the user may be the addition of range parameters in the filters and the introduction of file notes, which is a small feature, but very useful in the context of analysis.

The next release will bring some new file formats and interesting improvements.

Damaged Zip archive (video)

In this video we can see how to inspect a damaged Zip archive using the Profiler in a real-world scenario. Although soon the automatic recovery of damaged Zip archives will be available and it will be possible to perform this sort of task programmatically, it’s still useful to see how to do this kind of thing manually.

Filters with range parameters

The upcoming 0.8.9 release improves filters and introduces range parameters. If you don’t know what filters are you can take a look at the original introductory post.

What is now possible to do is to specify an optional range for the filter to be applied to. This is extremely useful, since many times we need to modify only a portion of the input data. Take, for instance, a file in which a portion of data is encrypted (or compressed) and we want to keep the outer parts not affected by the filter. This would be the layout of the input data:

Data layout

To handle the outer parts (before and after) we have an extra parameter called ‘trim’. This parameter can be set to one of the following values:

  • no: the outer parts are not trimmed and kept in the output data
  • left: the left part is trimmed while the right one is kept
  • right: the right part is trimmed while the left one is kept
  • both: both outer parts are trimmed and the output data will contain only the output of the filter

Let’s try to understand this better with a practical example. Let’s consider a compressed SWF file. The Profiler handles the decompression automatically, but just for the sake of this post we’ll decompress it manually. In case you don’t know, the format of a compressed SWF is as following:

3 bytes: CWS signature (would be FSW uncompressed files)
4 bytes: uncompressed file size
1 byte: version

To dig deeper into the file format you may want to take a look at a new web page we are preparing at fileanalysis.net. But for the sake of our example you need only to know the initial fields of the header. To decompress a SWF we need to decompress the data following the header and then to fix the first signature byte and we’ll do this manually and graphically with filters.

Let’s start by selecting the compressed input data and then by trying to add the unpack/zlib filter.

ZLib filter

As you can see we are prompted with a dialog asking us if we want to use the range defined by the selection of the hex editor on the left (input data). We accept and specify that we want to keep the data on the left (although in this case we could have just disabled the trimming completely, let’s just pretend there’s some data on the right we want to discard).

Now we need to fix the signature byte. To do this we use the misc/basic filter (set operation, offset 0, size 1 and no trimming).

By running the preview we’ll have the decompressed SWF.

The exported filters will look like this:


    
    

While this was a very simple case, much more complicated cases can be handled. Also remember that filters can be used to specify how to load embedded files, which means that, for instance, it’s easy to decrypt a file contained into another file and inspect it without ever having to save the decrypted data to disk.

I hope this post offers some understanding of an advanced use of filters.

Python SDK improvements

The upcoming 0.8.9 release of the Profiler improves integration with Python and the SDK exposes new functionality. Moreover, it lays down the groundwork needed to expand the SDK in the next releases. Documentation of the SDK has been included in the docs directory and also a Python command line has been added to the workspace.

To offer a glimpse of the capabilities here is a small code snippet to create a custom plot which could be used, for instance, to display entropy.

import random

ctx = proContext()
v = ctx.createView(ProView.Type_Plot, "Test") 
v.setPlotTitle("Random")
v.setAxisScale(ProPlotView.xBottom, 0, 100)
v.setAxisScale(ProPlotView.yLeft, 0, 50)
x = NTDoubleVector()
y = NTDoubleVector()
i = 0
while i < 100:
    x.append(i)
    y.append(random.randrange(50))
    i = i + 1
v.addCurve(x, y)
ctx.addView(v)

And the screenshot of the created plot.

Custom plot

While this doesn't sound too exciting at first, the on-going SDK expansion will allow to do some very interesting things. Stay tuned as some more technical posts are coming.

News for version 0.8.8

While we talked about some of the news of this version, there are some more which are worth mentioning.

introduced new multi-file report and project technology with compression and encryption
introduced new UI for workspace mode
added Windows Lnk support
– added file extensions scan option
– added directory scan to command line
added PNG CRC validation
added new filters: misc/replace and dev/array
– several UI improvements
– hex editor improvements
– increased memory limit

New workspace UI

The workspace features a new dock-based UI. It is easy to get used to the new UI as it is completely intuitive.

Workspace UI

Now our users can completely costumize their analysis workspace.

PNG CRC validation

This feature was requested by one of our customers and it can come handy during forensic analysis. The CRC of each PNG chunk is verified and those which don’t match are signaled in the format view (highlighted in red).

PNG CRC

New filters: misc/replace and dev/array

Two new filters have been added. misc/replace is self explanatory: it replaces bytes and strings. While dev/array is a small addition which can come handy all those times we need to copy some bytes to an array in a programming language. We can specify the language, the radix and the number of columns and we get something like:

unsigned char data[64] =
{
    0x4D, 0x5A, 0x90, 0x00, 0x03, 0x00, 0x00, 0x00,
    0x04, 0x00, 0x00, 0x00, 0xFF, 0xFF, 0x00, 0x00,
    0xB8, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x40, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0xF8, 0x00, 0x00, 0x00
};

We hope you enjoy this new version.