Welcome to SuStorID’s documentation!

_images/logo_sustorid.png
Contents:

About SuStorID

SuStorID is an advanced Intrusion Detection System (IDS) for web services, based on machine learning. Its name comes from the term “Su Stori”, which in Sardinian language means “The Falcon”. It’s version is experimental, but demonstrates a number of interesting features, that can be readily exploited to detect and act against web attacks:

  • Autonomous Learning Given a sample of web requests towards your web server, it can learn autonomously the profile of legitimate traffic, exploiting a number of machine learning algorithms.
  • Anomaly-based Approach SuStorID detect known as well as unknown (never-before-seen) web attacks by highlighting anomalous traffic.
  • Multi-model Architecture A multi-model architecture is employed to characterize different categories of anomalies and provide a human-readable representation of anomalous events.
  • Real-time Counteractions Depending on the set of raised anomalies and their reliability, a different counteraction can be taken.
  • Easy integration with modsecurity SuStorID can be coupled with modsecurity, the well known web application firewall, to gather training data and provide for real-time counteractions. So, SuStorID is a host-based Intrusion Detection System, and by means of modsecurity can access internal web server’s data (i.e. request and response message fields) exactly as Apache does.
  • Inspection of Encrypted traffic Since SuStorID can directly access Apache’s internal data, it can monitor both HTTP and HTTPS traffic.
  • User-friendly Interface The IDS interface is very important to investigate security issues, so SuStorID itself has been implemented as a web application with a user-friendly interface. SuStorID is written in Python exploiting the amazing Django framework, and some machine learning algorithms have been implemented in C language.

Last, but not least, SuStorID is free software, released under the GNU General Public License version 3!

Author: Igino Corona, Pattern Recognition and Applications Group, Dept. of Electrical and Electronic Engineering, University of Cagliari, Italy.

Installation How-to

SuStorID is multi-platform software, built upon the Django framework. Here follows a quick guide to its installation, but for any problem you should refer to Django documentation, version 1.3.

  1. Make sure that Python v.2.6 or higher (and python numpy) and Django v. 1.3 are installed on your system. On Ubuntu:
    $ apt-get install python django python-numpy
  2. Download and extract SuStorID’s source code from the official repository.

  3. Open a Python Interpreter and run:
    from random import choice
    ''.join([choice('abcdefghijklmnopqrstuvwxyz0123456789!@#$%^&*(-_=+)') for i in range(50)])
    

    paste the result for your SECRET_KEY in settings.py (initial value is empty).

  4. Inside the main directory of the project digit

    $ python manage.py syncdb

    This will create all tables needed by SuStorID. Specify a username and a password for administration: these credentials will be necessary to manage SuStorID and visualize detection results. By default, SuStorID uses the SQLite database, however you may use any database supported by Django to enhance performances, or employ a client-server db infrastructure (e.g. through Postgres).

    $ python manage.py runserver

    This will run the default Django server on the loopback interface (IP 127.0.0.1 and TCP port 8000). As any Django application, settings can be modified on settings.py. You may also change the web server to anyone supported by Django (e.g. Apache), to increase reliability and performances.

  1. Point your browser (make sure that javascript is ON) to http://127.0.0.1:8000, and insert your credentials. The SuStorID interface will guide you to its setup.

Optimization of Machine Learning algorithms

Currently SuStorID employs the Hidden Markov Model implementation by Alexandre Fayolle, Logilab, Paris, France. In order to speed up the training process it is suggested to build the related C extensions. SuStorID comes with shared libraries built from C extensions on Ubuntu 10.04 (32-bit Intel processor). If you are using another OS / Processor, it is suggested to re-build C extensions:

  • Install Python Dev tools
    $ sudo apt-get install python-dev
  • Inside SuStorID/base/raw_models/hmm/ digit:
    $ python setup.py config
    $ python setup.py build

    If necessary, install missing fortran/c compilers and headers (e.g. in Ubuntu this can be done by means of Synaptic).

  • A new folder will be created, containing the dynamic libraries:
    cd build/lib.linux<platform_details>/logilab/hmm/
  • Copy all dynamic libraries inside SuStorID/base/raw_models/hmm/

That’s it!

Installation Modsecurity for SuStorID

SuStorID may be easily coupled with modsecurity for:

  • gathering training data (i.e., web request samples)
  • providing real-time counteractions against suspicious web traffic

NOTE: This guide assumes Ubuntu OS 10.04 (some steps may be valid, some other may change depending on the OS).

  1. Download modsecurity for SuStorID source code from the official repository.

  2. Install the required packages to build this code:

    $ sudo apt-get install libc6-dev libc6 apache2 apache2-threaded-dev libxml2 libxml2-dev gcc libpcre3-dev
  3. Within the main directory of the project digit
    $ ./configure

    This command will highlight any other missing dependency: correct it by installing the related headers/libraries.

  4. Get into the Apache2 folder, and digit

    $ make
    $ sudo make install

    If you don’t encounter errors, the dynamic loadable library /usr/lib/apache2/modules/mod_security2.so should be installed. It is necessary to load the mod_unique_id module of apache (thus we create a symbolic link inside the mods-enabled folder)

    $ sudo ln -s /etc/apache2/mods-available/unique_id.load /etc/apache2/mods-enabled/

    Inside the sustorid_config directory of this project, digit

    $ sudo cp security2.load /etc/apache2/mods-available/
    $ sudo ln -s /etc/apache2/mods-available/security2.load /etc/apache2/mods-enabled/
    $ sudo mkdir /etc/apache2/modsecurity_conf
    $ sudo cp modsecurity_conf/mod_security.conf /etc/apache2/modsecurity_conf/

    Make sure that the apache2 user (e.g. www-data) can read such files (check access permissions for such files).

  5. Add the following line to /etc/apache2/apache2.conf
    Include /etc/apache2/modsecurity_conf/

    This tells Apache to load the SuStorID configuration /etc/apache2/modsecurity_conf/mod_security.conf. Read the documentation in the Configuration of modsecurity for SuStorID section to correctly configure modsecurity for SuStorID. The default configuration enables the logging module only, that is, without active Intrusion Detection/Protection: the IDS has not been trained yet and needs some training data.

  6. Restart Apache2
    $ sudo /etc/init.d/apache2 restart

    As soon as Apache restarts modsecurity for SuStorID will be activated.

Configuration of modsecurity for SuStorID

modsecurity for SuStorID comes with three additional options (with respect to modsecurity options):

  • IDServer: you should specify (space-separated) IP address and TCP port where SuStorID is listening to.
  • IntrusionDetection: set to On to enable Intrusion Detection by SuStorID (in conjunction with modsecurity rules). Please note that this only works if SecRuleEngine On, because this tells modsecurity to actively counteract against detected threats.
  • PostRequests: set to On to enable request forwarding, that is, to automatically send requests and responses to SuStorID (it is necessary to get sample requests for learning).

That’s it. Enjoy!

Get Help and Send Suggestions

Have you got any problem? Did you find any bug? Would you like to send suggestions? Send me an email.

Contribute

Are you a researcher working on web security? Feel free to contact me by mail igino <dot> corona <at> diee.unica.it to join the SuStorID development team. There are a number of improvements and additional features that we plan to develop in the future, towards a safer World Wide Web.

Acknoledgements

This project has been supported by a grant from Regione Autonoma della Sardegna awarded to Igino Corona, PO Sardegna FSE 2007-2013, L.R.7/2007 “Promotion of the scientific research and technological innovation in Sardinia”.

Snapshots

_images/status.png _images/learn.png _images/counteraction.png _images/detection.png _images/settings.png