libmagic Support

While Profiler offers an API to identify file formats, it does so only for those which are supported. The list of supported files is vast, but there will be always unrecognized formats.

It’s certainly a good idea to introduce a file signature identification API. This API might be useful for several purposes, not all foreseeable right away. That’s why the upcoming version introduces support for libmagic (it comes with the latest 5.11 version). The library is exposed to Python in the ‘Pro.magic’ module. Here are the functions:

magic_buffer(magic_t m, NTByteArray const & buf) -> char const *
magic_builtin_db_name() -> NTString
magic_check(magic_t m, char const * fname) -> int
magic_close(magic_t m)
magic_compile(magic_t m, char const * fname) -> int
magic_descriptor(magic_t m, int fd) -> char const *
magic_errno(magic_t m) -> int
magic_error(magic_t m) -> char const *
magic_file(magic_t m, char const * fname) -> char const *
magic_getpath(char const * fname, int action) -> char const *
magic_list(magic_t m, char const * fname) -> int
magic_load(magic_t m, char const * fname) -> int
magic_open(int flags) -> magic_t
magic_setflags(magic_t m, int flags) -> int

Just as a note: magic_file just calls magic_buffer internally.

Let’s create a small hook to demonstrate the use of the library, although it’s quite intuitive. Here’s the cfg entry:

[MagicInfo]
label = Magic: information provided by libmagic
file = magicinfo.py
init = init
end = end
scanning = scanning

The Python code:

from Pro.magic import *

def init():
    m = magic_open(MAGIC_CONTINUE)
    magic_load(m, magic_builtin_db_name())
    return m

def end(m):
    magic_close(m)

def scanning(sp, m):
    s = sp.getObjectStream()
    buf = s.read(0, min(0x1000, s.size()))
    info = magic_buffer(m, buf)
    if info != None:
        sp.addMetaDataString("Magic", info)

The addMetaDataString in ScanProvider adds a string in the individual file report, which is visible from the file stats page in the workspace.

So if we open a file in the workspace, we’ll get the following extra information:

The script above will be included in the update.