Previews

The upcoming version 0.9.2 of the Profiler adds previews for various things: images (all supported formats), several Portable Executable resources and Office Word Documents (text-only).

PE resources preview

Since media elements are rendered through third-party code, the Profiler displays a warning box before actually rendering a media element.

Preview warning

The ‘Allow all’ button allows media elements for the current session only. If the Profiler is running in a safe environment (like a VM), the user can decide to permanently disable the warning box and allow all media elements.

Preview settings

Last but not least, text-only preview of Office Word Documents has been introduced. This allows users to safely inspect the text content of a document without processing the file with an official viewer which could be the target of exploits.

Office document preview

While there are already enough new features to release, some smaller additions will be squeezed into 0.9.2 during the next days. Stay tuned!

Zip bomb revisited

The upcoming 0.9.2 release of the Profiler removes the virtual memory constraint, meaning that it is now able to open and process files of any size (the hex editor can edit large files as well). This feature has actually been in the TODO list from day 1 and I’ll write about the internals of it in some other post, in order to better demonstrate the capabilities gained by these changes to the core.

Also, because of the increased functionality, it made sense to add detection for Zip (decompression) bombs. Almost a year ago we’ve talked about Zip bombs, but it was limited to the safe exploration of such files. Let’s start with the new ‘Limits’ page in the setup.

Limits

  • What had been once the maximum size of a file is now the size of virtual memory the memory pool is allowed to use. Of course, the more virtual memory is granted, the faster it becomes to analyze large files. But it’s only a matter of speed, you can choose to give to the pool the bare minimum, it’ll work just as well.
  • The nesting option shouldn’t need any explanation since it hasn’t changed. But just for completeness: it specifies the maximum level of scanning into a root object. If more levels are available, it is signaled. For instance, level 0 specifies that children objects should not be scanned automatically (though they might still be opened manually by the user).
  • The maximum file size can be used to discard files larger than the specified size during batch scan operations. The default is 0, which stands for infinite.
  • The decompression bomb threshold is the limit we’re interested in for this post. It represents a cumulative size which can’t be exceeded. In other words if an archive contains 1 file of 100+ GBs (let’s use this number for the purpose of this example) it’s the same as whether one sub-archive contains 100 files of 1 GB each (plus a single byte in excess). If the threshold is exceeded, it will be reported as a threat. While 100 GBs is the default, you can speed up scanning by specifying a lower limit.
  • The maximum number of children files is cumulative as well. This constraint depends on virtual memory limitations (as all children are shown in a tree). 100.000 (the default) is a safe choice. If more children than the imposed limit are present, it is signaled.

So let’s again take the famous 42.zip as an example of Zip bomb and let’s scan it. We’ll get this in the summary:

Decompression bomb threat

Please note that the threat may not be reported in the summary of the root object itself, but in one of its children objects (once the bomb threshold has indeed been exceeded). But since we know there’s a threat (as reported by the risk factor) we can just jump to it by pressing F2 in the hierarchy view.

Widgets and Views

The last release of the Profiler featured some significant improvements. So while it also included initial PySide support, there wasn’t much time to make it really nice. One of the missing things was the ability to mix internal Profiler views (such as the hex editor) with PySide widgets. With the upcoming 0.9.2 release it will be possible to create a view and obtain a PySide widget with just one method:

widget = view.toWidget()

This way one can make use of advanced internal views of the Profiler and combine them with other custom controls. Let’s see a practical example.

Mixed widget

The widget in the screenshot combines a QTreeView with a directory model and a hex view. When a file is activated in the tree, it is opened by the hex editor. To try it out, just press Ctrl+Alt+R and enter the following code:

from Pro import *
from PySide import QtCore, QtGui

class MixedWidget(QtGui.QSplitter):
    def __init__(self, parent=None):
        super(MixedWidget, self).__init__(parent)

        self.setWindowTitle("Mixed widget")
        self.setOrientation(QtCore.Qt.Vertical)

        self.model = QtGui.QDirModel()
        tree = QtGui.QTreeView()
        tree.setModel(self.model)
        self.addWidget(tree)

        ctx = proContext()
        self.hex = ctx.createView(ProView.Type_Hex, "")
        self.addWidget(self.hex.toWidget())

        tree.activated.connect(self.updateFile)

    def updateFile(self, idx):
        if self.model.isDir(idx) == True:
            self.hex.clear()
        else:
            name = self.model.filePath(idx)
            self.hex.setFileName(name)


ctx = proContext()
w = MixedWidget()
v = ctx.createViewFromWidget(w)
ctx.addView(v)

Amazingly little code snippet, right? Please note that the ProHexView setFileName method is also a new addition to the SDK.

News for version 0.9.1

The new version of the Profiler is out with the following news:

added capability of opening multiple analysis views
added capability of switching root object in the workspace
added navigation in analysis views
added bookmarks
added PySide integration
– added user application data folder support
– added history for the Python command line and script dialog
– added save option to the keys input dialog
– improved notes: the toolbar now signals their presence
– updated Qt to 4.8.4

Also a new Demo version has been released, which as usual can be found on the product page.

UI Improvements & Bookmarks

The upcoming 0.9.1 version of the Profiler features some important UI improvements and the introduction of bookmarks. Among the UI improvements there’s:

  • the ability to switch root in the workspace
  • multiple analysis views displaying data of different roots
  • navigation

In this case a video is probably more worth than a thousand words.

These new features lay down the groundwork for some more interesting capabilities which will be added soon. Stay tuned! 🙂

PySide support

This is really a small addition which took just a couple of hours of work, but since it can come very handy, it’s worth dedicating a post to it. The upcoming 0.9.1 version of the Profiler adds explicit support for PySide. Thus, it will be possible to create Qt widgets and add them to the workspace.

Installing PySide

First of all, let’s install PySide. There are 3 ways to do this.

1) Install it from the qt-project page.

Make sure you select the package matching the current Python version used by the Profiler.

2) Install the package we compiled for you. It’s vanilla, directly from the original sources, but it has the advantage that it is guaranteed to work. In fact, at the time of writing the official package contains a bug (missing shiboken Python module) and so the first one is not really an option until it is not fixed.

Download
SHA1: 2024348E79890A167BB231098A6A16FC8BB02C9E

3) You can compile PySide yourself following the instructions at qt-project. At the end, use the installer created inside ‘c:\pyside-setup\dist’.

A code sample

Using it is even easier than the setup process. Basically ProContext has a new method called createViewFromWidget which takes as a parameter a widget created by PySide and returns a ProView which in turn can be added to the workspace.

Adding a widget to the workspace only takes the following line:

ctx.addView(ctx.createViewFromWidget(widget))

Therefore using an existing widget and adding it to the workspace is very easy. Let’s see a real-world widget like an official PySide sample: PySide/examples/effects/lighting.py. It’s sufficient to remove:

if __name__ == '__main__':

    import sys

    app = QtGui.QApplication(sys.argv)

    lighting = Lighting()
    lighting.setWindowTitle("Lighting and Shadows")
    lighting.resize(640, 480)
    lighting.show()

    sys.exit(app.exec_())

And add:

lighting = Lighting()
lighting.setWindowTitle("PySide widget")

ctx = proContext()
ctx.addView(ctx.createViewFromWidget(lighting))

Now we can add an action to execute the code or just insert it in the custom script box (Ctrl+Alt+R) and the view will be shown like this:

To set a custom icon for the view use setWindowIcon.

As usual stay tuned as the upcoming version is going to include some major additions and significant changes.

News for version 0.9.0

We’d like to wish everybody merry Christmas and a happy new year!

We’re just in time to place under the Christmas tree a new version of the Profiler with the following news:

added Java Class support including byte code disassembler and layout ranges
added .NET support including byte code disassembler and layout ranges
added DEX support including byte code disassembler and layout ranges
added dedicated view to display data like raw PDF objects
added PE MUI resources validation
– added Adler32 to filters
– updated jsbeautifier
separated malicious threats from intrinsic ones in the report view
– fixed update with unprivileged user account on Windows
– fixed several bugs

The main addition in this new release as seen previously is the managed trio Java, DEX and .NET.

Dedicated view to display raw data

Previously PDFs had 3 views to display objects: one for the dictionary, one for the decoded stream and one for the decoded stream shown as text. Now there’s also a ‘raw data’ view to show the object unmodified just as it is in the file.

Highlighted with different colors you can see the dictionary/value and the stream part. The same applies to child objects which are highlighted in the stream of their parent.

PE MUI resources validation

Following a short post about MUI resources, validation for them has been added. Also some bug fixes related to resource validation.

Separated malicious threats from intrinsic ones

In order to better separate intrinsic risk factors from malicious threats, they are now shown separately in the report view.

Have some nice holidays and stay tuned as we’ll try to add even more features the upcoming year.

Christmas hat image from freevector.com

DEX support

Support for Android’s DEX format is the last major feature of the upcoming 0.9.0 release of the Profiler. The support includes format, layout ranges and a Dalvik disassembler. Support for APK is implicit, since support for Zip archives has been added long ago.

All sections of the format are accessible. The central point for parsing a DEX file is the Classes view. Hence it’s also the most complex view of the format.

Also accurate layout ranges can help to analyze the format.

And finally a disassembler to quickly inspect the Dalvik bytecode.

Here’s a disassembled function from the Android SDK NotePad sample:

  private final void cancelNote()
  {
    const/4 v3, #int 0 // #0
    iget-object v1, v4, android.database.Cursor mCursor
    if-eqz v1, loc_74 // +34
    iget v1, v4, int mState
    if-nez v1, loc_90 // +38
    iget-object v1, v4, android.database.Cursor mCursor
    invoke-interface {v1}, void android.database.Cursor.close()
    iput-object v3, v4, android.database.Cursor mCursor
    new-instance v0, android.content.ContentValues
    invoke-direct {v0}, void android.content.ContentValues.()
    const-string v1, "note"
    iget-object v2, v4, java.lang.String mOriginalContent
    invoke-virtual {v0, v1, v2}, void android.content.ContentValues.put(java.lang.String, java.lang.String)
    invoke-virtual {v4}, android.content.ContentResolver getContentResolver()
    move-result-object v1
    iget-object v2, v4, android.net.Uri mUri
    invoke-virtual {v1, v2, v0, v3, v3}, int android.content.ContentResolver.update(android.net.Uri, android.content.ContentValues, java.lang.String, java.lang.String[])
loc_74:
    const/4 v1, #int 0 // #0
    invoke-virtual {v4, v1}, void setResult(int)
    invoke-virtual {v4}, void finish()
    return-void
loc_90:
    iget v1, v4, int mState
    const/4 v2, #int 1 // #1
    if-ne v1, v2, loc_74 // -11
    invoke-direct {v4}, void deleteNote()
    goto loc_74 // -16
  }

And the original source for comparison:

    private final void cancelNote() {
        if (mCursor != null) {
            if (mState == STATE_EDIT) {
                // Put the original note text back into the database
                mCursor.close();
                mCursor = null;
                ContentValues values = new ContentValues();
                values.put(NotePad.Notes.COLUMN_NAME_NOTE, mOriginalContent);
                getContentResolver().update(mUri, values, null, null);
            } else if (mState == STATE_INSERT) {
                // We inserted an empty note, make sure to delete it
                deleteNote();
            }
        }
        setResult(RESULT_CANCELED);
        finish();
    }

Next week will be dedicated to fixing reported bugs and adding some small improvements. After that and some testing the new version should be ready, so keep tuned as the new version should be deployed soon!

.NET support

Although there haven’t been customer requests for this, the upcoming 0.9.0 version of the Profiler adds support for .NET, which includes format, layout ranges and an MSIL disassembler.

As usual, let’s begin with the format itself. Since some users probably have used CFF Explorer to inspect the .NET format in the past, I have kept the same format view in the Profiler as well.

So how is it better than CFF Explorer? First of all, it’s very fast. There’s absolutely no wait time in opening a large metadata set as the following.

And then one handy feature which was not available in CFF Explorer, is the capability to display a second set of metadata. For instance, .NET native images (meaning those files in the assembly cache created with ngen.exe) contain two sets of metadata. The Profiler lets you inspect both sets.

Layout ranges cover all PE parts, so it was normal to add them for .NET as well. These are the ranges available for .NET:

Let’s see them in the hex editor.

If you’re asking yourself why it’s all gray, it’s because more ranges are blended together, meaning that all .NET metadata is contained in the .text section of the PE and that section is marked as executable even if the assembly contains only MSIL code. While it doesn’t make much sense to mark as executable a region of data containing strings and tables, my best guess is that old versions of Windows had no in-built support for .NET in the loader, which is why assemblies contain a single ‘mscoree.dll’ import descriptor and the entry point is just a jmp to the only IAT thunk (_CorExeMain for executables, _CorDllMain for dlls). That API then loads the .NET framework if necessary. Since that single native jmp instruction requires a section in the PE marked as executable and because the granularity of sections is the same as virtual memory pages, it was probably considered a waste to use an entire memory page just for a jmp instruction and so the result is that everything is contained into a single executable section. This could probably be changed as new versions of the framework do not even run on older systems.

However, for our inspection purposes it suffice to get rid of the Code range by pressing Ctrl+Alt+F.

We can now inspect .NET layout ranges without the conflict. One nice aspect about it is that the code of IL methods is easy to distinguish between its header and extra sections.

Included is also an IL disassembler. Its purpose is to let our customers quickly browse the contents of an assembly. As such readability was a priority and the output has been grouped for classes: I have always found it cumbersome in ILDasm to open every single method to inspect an assembly.

Here’s an output example:

  private static void Main(string [] args)
  {
    locals: int local_0,
            int local_1

    ldc_i4_2
    stloc_0 // int local_0
    ldloc_0 // int local_0
    stloc_1 // int local_1
    ldloc_1 // int local_1
    ldc_i4_1
    sub
    switch
      goto loc_22
      goto loc_60
    br_s loc_71
loc_22:
    try
    {
      ldstr "h"
      call System.Console::WriteLine(string) // returns void
      leave_s loc_81
    }
    catch (System.ArgumentNullException)
    {
      pop
      ldstr "null"
      call System.Console::WriteLine(string) // returns void
      leave_s loc_81
    }
    catch (System.ArgumentException)
    {
      pop
      ldstr "error"
      call System.Console::WriteLine(string) // returns void
      leave_s loc_81
    }
loc_60:
    ldstr "k"
    call System.Console::WriteLine(string) // returns void
    ret
loc_71:
    ldstr "c"
    call System.Console::WriteLine(string) // returns void
loc_81:
    ret
  }

And the original code:

        static void Main(string[] args)
        {
            int a = 2;
            switch (a)
            {
                case 1:
                    try
                    {
                        System.Console.WriteLine("h");
                    }
                    catch (ArgumentNullException)
                    {
                        System.Console.WriteLine("null");
                    }
                    catch (ArgumentException)
                    {
                        System.Console.WriteLine("error");
                    }
                    break;
                case 2:
                    System.Console.WriteLine("k");
                    break;
                default:
                    System.Console.WriteLine("c");
                    break;
            }
        }

There’s one last thing worth mentioning. .NET manifest resources are displayed as sub-files.

However, the parsing of ‘.resources’ files was still too partial and thus won’t be included in 0.9.0.

From the last two posts you may have guessed the topic of the upcoming release. So stay tuned as there’s yet more to come.

Java Class support

The upcoming 0.9.0 version of the Profiler adds support for Java class files. This will be especially useful for malware analysis. The support includes: disassembler, format views and layout ranges. Let’s see some screeshots of the format itself first.

Here’s a view of the constants:

Note: Utf8 strings are highlighted in orange just to distinguish them.

Here’s a view of the methods with their attributes:

And here’s the layout ranges view:

Again strings are in orange, while the actual code of a method is in a slightly lighter green than the method itself.

Since the format of class files is contiguous, it’s extremely easy to use layout ranges to create a new custom class file using the hex editor.

And finally, the disassembler:

The output shown in the screenshot above:

// SourceFile: HelloWorld.java

super class HelloWorld extends java.lang.Object
{

  static float f1;
  public static java.lang.String hello;

  HelloWorld()
  {
    // max_stack = 1  max_locals = 1
// line 1
    aload_0
    invokespecial java.lang.Object.() // returns void
    return
  }

  public static void main(java.lang.String[])
  {
    // max_stack = 7  max_locals = 6
// line 7
    ldc2_w 454.546
    dstore_1
// line 8
    ldc2_w 552441554577111995
    lstore_3
// line 9
    getstatic java.lang.System.out // java.io.PrintStream
    ldc "The value of i is: %f and %d"
    iconst_2
    anewarray java.lang.Object
    dup
    iconst_0
    getstatic HelloWorld.f1 // float
    invokestatic java.lang.Float.valueOf(float) // returns java.lang.Float
    aastore
    dup
    iconst_1
    lload_3
    invokestatic java.lang.Long.valueOf(long) // returns java.lang.Long
    aastore
    invokevirtual java.io.PrintStream.format(java.lang.Stringjava.lang.Object[]) // returns java.io.PrintStream
    pop
// line 10
    getstatic java.lang.System.out // java.io.PrintStream
    getstatic HelloWorld.hello // java.lang.String
    invokevirtual java.io.PrintStream.println(java.lang.String) // returns void
// line 13
    try
    {
      getstatic java.lang.System.out // java.io.PrintStream
      ldc "test2"
      invokevirtual java.io.PrintStream.println(java.lang.String) // returns void
    }
    catch (java.lang.Exception)  goto loc_57
// line 18
    goto loc_67
// line 15
loc_57:
    astore 5
// line 17
    getstatic java.lang.System.out // java.io.PrintStream
    ldc "error"
    invokevirtual java.io.PrintStream.println(java.lang.String) // returns void
// line 19
loc_67:
    return
  }

  static void ()
  {
    // max_stack = 1  max_locals = 0
// line 3
    ldc 43
    putstatic HelloWorld.f1 // float
// line 4
    ldc "Hello world!"
    putstatic HelloWorld.hello // java.lang.String
    return
  }

}

And the original source file:

class HelloWorld
{
  static float f1 = 43;
  public static String hello = "Hello world!";
  public static void main(String[] args)
  {
    double d1 = 454.545774;
    long l1 = 552441554577111995L;
    System.out.format("The value of i is: %f and %d", f1, l1);
    System.out.println(hello);
    try
    {
      System.out.println("test2");
    }
    catch (Exception e)
    {
      System.out.println("error");
    }
  }
}

There’s yet more to come. Stay tuned. 🙂