IRhelper (work in progress)

A play POC tool for first quick analysis of memory images for fun and learning!

IRHelper is meant to be a simple tool for automating as much as possible the tasks an analyst would perform when acquiring a memory dump. It was inspired by the SANS Windows Forensic Analysis poster which provides steps to perform during DFIR assuming both disk and memory dumps are available.

https://digital-forensics.sans.org/community/cheat-sheets

https://digital-forensics.sans.org/media/poster-windows-forensics-2016.pdf

  • Step 01: Prep Evidence/Data Reduction
  • Step 02: Anti-Virus Checks
  • Step 03: Indicators of compromise search
  • Step 04: Automated memory analysis
  • Step 05: Evidence of persistence
  • Step 06: Packing/Entropy check
  • Step 07: Review event logs
  • Step 08: Super timeline examination
  • Step 09: By-Hand memory analysis
  • Step 10: By-Hand 3rd Party Hash lookups
  • Step 11: MFT Anomalies
  • Step 12: File time anomalies
  • Step 13: If you got malware then Hurrey !!! If not look deeper!

IRHelper will cover for you the following steps which normally would be run By-Hand!

  • Step 09: By-Hand memory analysis
  • Step 10: By-Hand 3rd Party Hash lookups
  • Bonus: Try to extract other information which would normally be found on the disk

The Bonus part is only best effort as data might be paged so we wont have enough information to extract what we want.

So the high level objectives of IRHelper would be:

  • To extract as much information as possible from a memory dump and present it to the user in well presented/readable format
  • To enable even novice users to be able to use it and extend it
  • Learn while you play with it (Python/Memory analysis/Writing Volatility plugins)
  • Integrate with other tools which can help in your decision making progress
  • Focus on what matters
  • Experiment with new techniques of detecting suspicious patterns
  • Utilize as much as possible the existing volatility plugins
  • Have fun running memory analysis!

Volatility is an amazing and very powerful tool for performing memory analysis. However for the novice user it does come with some drawbacks.

  • Has to decide which plugins to run and what information is valuable for further analysis
  • Requires scripting or development/coding skills to take full advantage of it
  • Difficult to keep track of the different information and where to go next
  • To detect some obvious patterns or leads can take long time
  • Different OS versions require from the user to know by heart many OS internals

Similar tools

  • Volutility
  • VolatilityBot
  • Evolve
  • DAMM

DAMM and VolatilityBot have similar objectives to irhelper. However DAMM is not easy to extend unless the user knows how to write volatility code and does not provide standard report at the end of the analysis and integrations with 3rd party tools. VolatilityBot was discovered after started writing irhelper :) although they have quite different objectives. IRHelper is also not meant to be very production code !

IRHelper concepts

  1. Information extraction: IRHelper modules used to extract as much information as possible from the memory image and store them for later processing. Usually sqlite is best option.
  2. Analysers: Analyser modules or code is used to run data analysis on the extracted information. Analysers can combine multiple information and logic to provide some results or indicators which are not readily available to the original plugin or would take multiple steps by hand to construct

Prerequisites

There are some 3rd party tools which are required to run the different modules. One basic one is Volatilty. Volatility has to be available in your path otherwise it will not be found from the modules and and you have to specify the full path. Currently volatility 2.5 was used for the development of the current code.

Other tools are used such as:

  • Exiftool
  • ClamAV
  • RegRipper

Note: You might encounter problems with matplotlib. In which case disable from the settings file.

Install

Installation is quick and easy:

git clone https://github.com/etz69/irhelper.git
cd irhelper
virtualenv venv
source dev/bin/activate
pip install -r requirements.txt

Directory structure

The directory structure of the project is as follows:

irhelper.py
This is the main program to execute

vol_plugins
Contains custom or contrib unofficial Volatilty plugins

templates
Contains the report template

dump
Directory to dump code or artifacts from memory for further analysis

modules
Contains all irhelper modules

export
This is where you will find your shiny report !

docs
This amazing documentation !

Usage

(venv)fsck:irhelper dxl$ python irhelper.py -h

usage: irhelper.py [-h] [-p [PROFILE]] [-r [RISK]] [--cache] [--debug]
                   [--initdb] [--hash] [--vt] [--osint] [-v]
                   reportTemplate memoryImageFile

  ;)( ;
 :----:
C|====|
 |    |
 `----

The IR helper python tool!

positional arguments:
  reportTemplate        Report template to use
  memoryImageFile       The memory image file you want to analyse

optional arguments:
  -h, --help            show this help message and exit
  -p [PROFILE], --profile [PROFILE]
                        Volatility profile (Optional)
  -r [RISK], --risk [RISK]
                        Risk level to show processes (default 2)
  --cache               Enable cache
  --debug               Run in debug
  --initdb              Initialise local DB
  --hash                Generate hashes
  --vt                  Check VirusTotal for suspicious hash (API KEY
                        required)
  --osint               Check C1fApp for OSINT of ip/domain (API KEY required)
  -v, --version         show program's version number and exit

Run

To run irhelper just point to the image you want to analyse:

python irhelper.py --initdb --debug templates/report.html image_samples/conficker.img
DEBUG: Cleaning DB file
No cache, normal run

-------------------------------------
Gathering image initial information
-------------------------------------

-------------------------------------
KDBG: 0xf80002e400a0
DTB: 0x187000
KUSER_SHARED_DATA: 18446734727860715520
id: 1
Number of Processors: 1
KPCR for CPU 0: 18446735277665033472
AS Layer1: AMD64PagedMemory (Kernel AS)
Image date and time: 2012-04-06 21:28:39 UTC+0000
Image local date and time: 2012-04-06 17:28:39 -0400
PAE type: No PAE
Image Type (Service Pack): 1
Suggested Profile(s): WinXPSP2x86, WinXPSP3x86 (Instantiated with WinXPSP2x86)
AS Layer2: FileAddressSpace (irhelper/image_samples/conficker.img)
-------------------------------------

0) Win7SP0x64
1) Win7SP1x64
2) Win2008R2SP0x64
3) Win2008R2SP1x64

Final Report

report_screenshot

Contributing - We need you !

There are different ways you can contribute

  • Write documentation
  • Write code
  • Report bugs

Development

You can also run each module on its own while testing:

python modules/cmds/vol_imageinfo_module.py run image_samples/conficker.img WinXPSP3x86

Python version: 2.7.10 (default, Oct 23 2015, 19:19:21)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.59.5)]

DEBUG: _cache: True
DEBUG: PLUGIN_DIR: /tmp/irhelper/vol_plugins/
DEBUG: _VOLATILITY_PROFILE: WinXPSP3x86

-------------------------------------
Gathering image initial information
-------------------------------------

DEBUG: ['vol.py', '--cache', '-f', 'image_samples/conficker.img', 'imageinfo', '--output=sqlite', '--output-file=results.db']
DEBUG: Child process pid: 38001
Volatility Foundation Volatility Framework 2.5
{
    "status": true,
    "message": "",
    "cmd_results": {
        "KDBG": "0xf80002e400a0",
        "DTB": "0x187000",
        "KUSER_SHARED_DATA": "18446734727860715520",
        "id": 1,
        "Number of Processors": "1",
        "KPCR for CPU 0": "18446735277665033472",
        "AS Layer1": "AMD64PagedMemory (Kernel AS)",
        "Image date and time": "2012-04-06 21:28:39 UTC+0000",
        "Image local date and time": "2012-04-06 17:28:39 -0400",
        "PAE type": "No PAE",
        "Image Type (Service Pack)": "1",
        "Suggested Profile(s)": "WinXPSP2x86, WinXPSP3x86 (Instantiated with WinXPSP2x86)",
        "AS Layer2": "FileAddressSpace (/tmp/irhelper/image_samples/conficker.img)"
    }
}

Logging

For logging purposes there are three methods used:

debug()
err()
print_header()

And the standard print!

Database

Various DB (sqlite) utils can be found in the modules.db.DBops

New module development

Edit cmd_processor.py and add your module as a method in the Modules() class. For example you can follow the skeleton module. All you have to do is return the standard result dict and create the appropriate section in jinja style (and bootstrap) in the templates/report.html file.

For example we want to create a new module to capture the command line executed by the user. This will be done by running volatility with the cmdscan plugin, store the results in the sqlite and finally return a dictionary with the cmds executed.

We will name our module vol_cmdline_module.py and place it inside modules/cmds/

Add our module class name in the cmd_processor.py:

def vol_cmdscan(self, **kwargs):
    '''
    Run cmdscan and record the command execution output

    Args:
        project (project): the project

    Returns:
        dict: Returns standard module response dict
    '''

    if 'project' in kwargs:
        _project = kwargs['project']

        #The module method to run
        ircmd.vol_cmdline_module.vol_cmdscan(_project)
        #Retrieve the results
        return ircmd.vol_cmdline_module.get_result()
    else:
        raise ValueError("Project info is missing")

Finally copy the skeleton_module.py in the new file and adjust!

Research

Step 1: Prep evidence and data reduction

action: Hash lists from NSRL description: Download known MD5 hashes from NSRL for minimizing the false positives references:

feature:

Step 02: AV Checks

action: Run AV scans description: Run AV scan on extracted executables and dlls. Download yara rules and search on the different memory artifacts. ClamAv also supports yara references:

feature:

Step 04: Automated memory analysis

action: Automate daunting tasks for memory analysis description: Currently this is work in progress references:

feature:

Step 06: Packing/Entropy check

action: Calculate the density (entropy) of specific filetypes (exe and dll)

description: Files with low entropy than normal (what is normal?) may be packed executables which may lead you to potential malware on the system. The tool we select to carry out the scan is DensityScout! We will also try standard entropy with python implementation (slower) and slightly different than DensityScout and other approaches to detect packing Most likely files with “entropy” less than 0.1 (DensityScout) we can bring to the attention of the analyst. However in a default Windows installation we can see that there several legitimate files below 0.1 . This technique is likely to produce false positives.Here we can use outliers

references:

feature: Packing entropy information of extracted files

Indices and tables

Module documentation

class modules.utils.helper.Project(settings_path)

Project class for all related data and methods of the project. This is the main class we have to load at the start of the project. It contains the necessary values for most of the project details such as the profile , directory locations, flags for features. It also provides several methods to provide access globally to the standard vars

clean_db()

Deletes the DB, cache and all files from dump dir

get_plugins_dir()

Returns the plugin directory for our custom plugins

get_root()

Return the root directory of the project. This is defined in the settings.py file and it is mandatory

init_db(db_name)

Set the DB name to be used from now on

@db_name (str): the db name

static load_properties()

Load the settings.py file

class modules.cmd_processor.CommandProcessor
class modules.cmd_processor.Modules

Simple command processor for adding new modules and retrieving results All modules return a dict which is of the following format:

result = {'status': True, 'message': '', 'cmd_results': ''}

status: If the module completely fails set this to False

message: A descriptive message , usually to show why it failed

cmd_results: This is usually a dict containing all the data which will be put in the report template

vol_cmdscan(**kwargs)

Run cmdscan and record the command execution output

Args:
project (project): the project
Returns:
dict: Returns standard module response dict
vol_getosversion(**kwargs)

Reads registry keys and tries to identify OS version information

Args:
project (project): the project
Returns:
dict: Returns standard module response dict
vol_imageinfo(**kwargs)

Retrieves basic image info such as the type, profiles, KDBG etc..

Args:
project (project): the project
Returns:
dict: Returns standard module response dict
vol_malfind_extend(**kwargs)

Run malfind and analyses the output. ToDo ML for asm

Args:
project (project): the project
Returns:
dict: Returns standard module response dict
vol_netscan(**kwargs)

Runs different modules to discover network connectivity

Args:
project (project): the project
Returns:
dict: Returns standard module response dict
vol_pslist(**kwargs)

Get as much as possible process information and dump pslist binaries to disk. This module will also run exiftool

Args:
project (project): the project
Returns:
dict: Returns standard module response dict
vol_regdump(**kwargs)

Dumps SAM registry and tries to extract user information

Args:
project (project): the project
Returns:
dict: Returns standard module response dict
class modules.db.DBops.DBOps(db)
clean_db(db)

Deletes the sqlite file (cleans the db)

@db: Target db name (file)

get_all_rows(table_name)

Retrieve all rows from a table

@table_name: the table name

insert_into_table(table_name, row)

Insert data into a table

@table_name: the table name

@data: Data is an array containing list of dictionary items in the form of columnName:value

new_table(table_name, table_fields)

Create a new db table

@table_name: the table name

@table_fields: Table fields is a dict containing the name of the column as the key and the data type as the value {‘id’:’integer’,’name’:’text’,’path’:’text’}

new_table_from_keys(table_name, table_keys)

Create a new db table based on table keys with default type text

@table_name: the table name

@table_fields: Table fields is a dict containing the name of the column as the key and the data type as the value {‘id’:’integer’,’name’:’text’,’path’:’text’}

patch_table(table_name, column_name, column_type)

Add a column to an exisitng table

@table_name: the table name

@column_name: the new column name

@column_type: The type of the new column

sqlite_query_to_json(query)

Execute a query and return all results in json format

@query (str): A string which describes the query for sqlite. Complex queries with filters do not work always

table_exists(table_name)

Check if a table exists

@table_name: the table name

update_value(table_name, column_name, value, key_name, key)

Update a value in a table

@table_name: The table name

@column_name: The column name

@value: The new value

@key_name: The key name you want to filter on

@key: The key value you want to filter on