Bof.

No theme, no regular posting.

© 2014-2017. Raphaël Rigo CC-BY-SA 4.0

About.

PC engines APU2, Debian Stretch and watchdog

I bought a very cool APU2 from PC engines.

Installing Debian Stretch

Very easy:

Wachtdog tricks

Once the APU crashed for some reason while I was away, which is very annoying as it is my main router. Thankfully, the APU2 has a hardware watchdog. Unfortunately, it's a bit buggy so we need to blacklist the i2c-piix4 module and load spi5100_tco ASAP.

So, do the following:

# apt install watchdog
# vim /etc/watchdog.conf # uncomment watchdog-device
# echo blacklist i2c_piix4 > /etc/modprobe.d/blacklist_piix.conf
# echo spi5100_tco >> /etc/initramfs-tools/modules
# update-initramfs -k all -u 

Adding spi5100_tco seems to be necessary to have the module reliably find the MMIO.

Enjoy.

Thoughts on IDA and disassemblers

Foreword

This post is the result of some thinking about reverse engineering tools. I have been reverse engineering for more than 15 years but it has been only very recently that I have begun feeling disappointed by the current tools. Of course, I am not the first, and as Halvar said: "I am regularly infuriated about the state of reverse engineering tools, and have only myself to blame." source

That being said, as most of the reverse work I do is static reversing on "exotic" platforms or operating systems, your perception may quite differ, particularly if your focus is automated analysis, which is not my case. As I almost exclusively reverse interactively, I think it's very important for the tools to be easily integrated in the analyst's workflow: some tools are really awesome but only usable for automated analysis.

And of course, this is my own ranting :)

Reverse engineering techniques

10 years ago most tools were pure disassemblers with nonexistant to poor advanced static analysis capabilities. But recently, techniques for binary code analysis have improved greatly and are getting practical. I will cover them quickly, describing how I understand them and how they can be useful.

Static analysis techniques

I will not discuss here the merits of symbolic execution, abstract interpretation or any other technique, as my point is about the practical tools available to the reverser. Which underlying technique is or could be used is out of scope.

Type propagation and reconstruction

Type propagation is quite simple to understand: knowing some types, either from external APIs, FLIRT or from the analyst, use data flow analysis to propagate types to arguments and relevant data. The challenge here is to do it both ways:

  • forward, for example with argument types inside in a function, or return values
  • backward, when calling an function with known argument types.

IDA has been doing it, in a limited way, for a long time (more than 10 years).

Type reconstruction is more advanced: using both type propagation and access patterns, reconstruct complex types such as structures or vtables.

The only two practical tools that I know of are:

both using HexRays's decompiler SDK to analyse the decompiled output and create the advanced structures.

Note that this is a research topic with several academic papers covering the subject, but I don't know of any tool with IDA integration.

Also, interesting approaches have been proposed for dynamic analysis, for example Trace Surfing by A. Gianni.

Taint analysis

Taint analysis is also very useful for the reverser as it can help pinpoint interesting parts of a binary or function, depending on the source of taint.

Ponce is very interesting as it uses Triton to provide taint analysis directly in IDA, with an easy to use GUI. I think it is a good way to provide advanced analytics, too bad it is limited to dynamic analysis.

Data slicing

Data slicing could be described as a kind of backward taint analysis, where the goal is to find which instructions and data inputs are used for a given resulting register or memory space.

miasm's blog gives a very good example and a practical tool ;)

Decompilation

Of course the holy grail of reversers is a good decompiler, which is currently Hex-Rays.

Some academic papers such as this one claim interesting results, but are not available.

Tools

Automated and scripting tools

Lots of very interesting tools have appeared in the last years, covering part of the techniques I mentioned before. For example:

While they all provide powerful features, they certainly do not cover the use case I covered in my introduction. Most of them could be (and have been) used as external helpers to add output to IDA but they do not provide a platform to build interactive tools upon.

Disassemblers

In addition to the previous tools, the two main challengers to IDA are:

While I did not try them extensively (Relyze does not work on Linux, Binary Ninja was slow when I tried the beta version), they look promising.

In particular, Binary Ninja's IL and API seem to cover much of the points I will cover in the next section. Be a Binary Rockstar by Sophia d’Antoine, Peter LaFosse and Rusty Wagner is a good showcase.

IDA

IDA is, like it or not, the only real tool you can use for serious reverse engineering work, particularly on exotic platforms.

Why is IDA still reigning ?

Based on the features I outlined before, IDA is not really good on most of them. So why is it still the default RE tool ?

For several reasons:

  • its GUI works, really, sometimes it's painful but it works :)
  • it supports so many architectures that it's very rare to have something not supported.
  • its plugin system allows to extend it and compensate for missing features (if one bears the pain of using the SDK).
  • its included library of information: type infos and flirt signatures.
  • its very reactive and knowledgable support.

The decompiler is of course a killer feature for efficient reverse engineering, particularly of C code.

Missing features

While this part may seem like throwing stones at IDA, I really think it's a great tool. Read it has an extended wishlist :)

Collaborative work

Clearly, one missing, essential, feature of IDA is the ability to do collaborative work. One just needs to look at the various attempts to create plugins attempting to fix the problem: collabREate, SolIDArity, polichombr, YaCO, etc.

One basic aspect of a collaboration feature would be to be able to simultaneously work on the same IDB, synchronising information like a git repository.

But, while a life changing enhancement, that wouldn't be enough. Hopefully, one could also share structures through a server. For example, someone working on a client and server could share the structures for the protocol while working on a different binary.

Also, several attempts have been made over the years to create plugins to integrate analyst's knowledge in IDA, by recognizing functions already reversed in the past. polichombr, crowdRE, IDA toolbag, etc.

The common point with (almost) all those tools is that they die slowly as their authors move on to other things. Which is definetly not helped by IDA's internals which are not suited for such low level integration.

Multiple files handling

Another painful aspect of IDA is the inability to work on several files at the same time. One trivial example is a binary that uses a shared library. One has to switch all the time between two IDB, copy pasting info (typing for example) to "synchronise" information.

This gets particularly painful when working on a more than 2 binaries at the same time.

Semantics

This is where IDA lags behind most other recent tools: instructions semantics and intermediate representation.

Currently the only way to search for instructions is syntaxic, which is definitely not enough if we need to search for a changing pattern. A trivial example is argument lookup for functions parameters, which is basically impossible.

Having an IR would also help tremendously writing scripts independently of the underlying architecture. Some would argue that the Hex-Rays decompiler provides such IR, but it is expensive and, most importantly, it is quite often wrong.

Others

Some other points:

  • the SDK is a pain, inconsistenly named, poorly documented, with only partial Python support. But it is powerful.
  • C++ support is nonexistent.
  • Porting information (typing, names) from one IDB to another can be painful.

Future ?

warning: personal feelings here

I think one of the main reasons IDA has not evolved much in 15 years is because there was simply no competition. The market was a niche but it feels like more and more people are doing RE, expanding the market somehow.

Considering that HexRays made several millions of euros of result in the past years for 5-6 full time employees, I am surprised that they did not start a new project to replace the definitely outdated base that is the IDA core.

"Just" porting the app to 64 bits seems a major pain. So with all their experience, their market share and money, I think Hex-Rays could start IDA-ng from scratch, and be very successful ! :)

Hopefully, the appearance of real competition like Binary Ninja or Relyze may stir the field a bit and force Hex-Rays to fix the fundamental problems :)

Let's Encrypt cron with acme-tiny

So, Let's Encrypt is awesome, even if the official client is a terrifying beast. I chose to use acme-tiny and this post is a quick HOWTO.

  • First, create a directory for challenges in your web root: mkdir -p /var/www/.well-known/acme-challenge/
  • Then, create a letsencrypt user : adduser --home /var/www/.well-known/acme-challenge/ --shell /bin/sh --no-create-home --disabled-password --disabled-login letsencrypt
  • Change ownership : chown letsencrypt acme-challenge
  • Create /etc/letsencrypt and setup ACL : mkdir /etc/letsencrypt ; setfacl -m u:letsencrypt:rx /etc/letsencrypt
  • Put your CSR and user key in /etc/letsencrypt/ as site.csr and user.key, set ACL : setfacl -m u:letsencrypt:r /etc/letsencrypt/*
  • Put acme-tiny somewhere, make it world readable
  • Create /etc/cron.d/letsencrypt :
MAILTO=root

1 1 21 * *  letsencrypt umask 033; python /usr/local/acme-tiny/acme_tiny.py --account-key /etc/letsencrypt/user.key --csr /etc/letsencrypt/site.csr --acme-dir /var/www/.well-known/acme-challenge/ > /tmp/site.crt && cat /tmp/site.crt > /etc/ssl/certs/site.crt
10 1 21 * * root service apache2 reload
  • test with su -c 'umask 033; python /usr/local/acme-tiny/acme_tiny.py --account-key /etc/letsencrypt/user.key --csr /etc/letsencrypt/site.csr --acme-dir /var/www/.well-known/acme-challenge/ > /tmp/site.crt && cat /tmp/site.crt > /etc/ssl/certs/site.crt' letsencrypt

Just a caveat, Let's Encrypt does not easily support challenges over HTTPS, so configure a redirect from http to https :

<VirtualHost *:80>
        ServerName syscall.eu
        Redirect permanent / https://syscall.eu/
</VirtualHost>

Lesser known tricks for IDA configuration

This post is mainly for reference but it can be useful.

Being tired of copy/pasting my IDA config and plugins after each update, I decided to check what I could do to centralize my config. As I'm running Linux, I expect everything to be configurable from ~/.idapro.

ida.cfg

You can override configuration options for IDA in ~/.idapro/idauser.cfg. For example, the classic :

#define DEMNAM_CMNT  0                  // comments
#define DEMNAM_NAME  1                  // regular names
#define DEMNAM_NONE  2                  // don't display
DemangleNames   = DEMNAM_NAME           // Show demangled names as comments

IDAPython

IDApython will load ~/.idapro/idapythonrc.py which can then be used to specify additionnal paths to python, for example :

import sys
sys.path.append('/home/raph/.idapro/python')

You can now add Python libraries that will be available in all your IDA versions. For example :

python/
├── miasm2 -> /home/raph/bin/python/lib/python2.7/site-packages/miasm2
└── pyparsing.py -> /usr/lib/python2.7/dist-packages/pyparsing.py

Plugins

Unfortunately there's no easy way right now to handle a custom user directory for plugins. While discussing the issue with Ilfak, he offered the following workaround:

User plugins can be handled the following way: defined IDAPLG envvar 
that points to the user plugins directory. Create symlinks to all IDA 
plugins from this directory.

Which is not exactly the same but may help if the user has no write access to the IDA directory. Maybe a future version will offer this feature :) Update : IDA 6.9 supports the IDAUSR environment variable. See doc for awesomness :)

High DPI screen on Linux.

I recently received a high-DPI screen at work (24" ultra HD). While the quality is excellent, a few problems arise as everything is very small by default.

As I use it on my laptop, I had to find an easy way to switch between different settings :

  • high DPI for my UHD screen
  • medium DPI when using the laptop screen (12" Full HD)
  • normal DPI when using a normal external screen

First, to handle the display configuration I'm using autorandr which is awesome.

Then, I use a little bash script that will change the needed settings :

  • cursor size
  • font size in awesome
  • layout.css.devPixelsPerPx setting in Thunderbird and Firefox

But some things still won't work :

  • Chrome
  • Flash
  • Java
  • Windows XP (in my virtual machine)

You can find my ugly shell script here.