ARAKIS

1.	What is ARAKIS?
2.	What sources does ARAKIS use?
3.	How does ARAKIS detect new threats?
4.	What alarms does ARAKIS signal?
5.	Can the snort rules proposed by the system be used for snort in a production environment?
6.	Where can I learn more about ARAKIS?

1.	What is ARAKIS?
ARAKIS is a CERT Polska (NASK) project that aims to create an early warning and information system concerning novel network threats. The system developed as part of the project focuses on detection and characterization of new automated threats with a focus primarily, though not only, on exploits used in the wild, not malware. Currently the system detects threats that propagate actively through scanning. The public dashboard of the project shows a snapshot of network activity observed by the system.
2.	What sources does ARAKIS use?
ARAKIS uses four types of sources, a distributed network of honeypots, firewalls, antivirus systems and darknets. Each of these sources gives a different perspective on what is happening on the network.
3.	How does ARAKIS detect new threats?
The most important role in new threat detection is performed by honeypots. Every flow (its payload) observed to the honeypots is subjected to a sliding window mechanism, over which Rabin hashes are computed. All flows are then grouped based on their Rabin similarity. In this way, each honeypot searches for similar flows that exceed a certain activity threshold. Once a group of flows exceeds the threshold, the flows are handed over to a process that computes longest common substrings (LCS) across the flows. These LCS (computed per packet) become signatures of a potential threat. The signatures are then sent to a central repository where they are clustered by a clustering algorithm, which uses edit distance as a metric. Each cluster then has a signature computed over all the cluster members (the "super signature") which identifies some characteristic of a threat. This method is based on the assumption that a new attack will use a payload that is different to that which has been seen so far. Thus, a new payload will be signaled by the system as a new cluster of activity, and potentially a new threat. It must be stressed that the system performs only string analysis - at no point does the system know what an attack is, it just looks for similar recurring patterns of activity. Firewalls and darknets are used primarily as anomaly detectors - they signal increases and decreases of port activity. Antivirus systems on the other hand give information about known threats on the network.
4.	What alarms does ARAKIS signal?
ARAKIS has a wide scope of alarms that signal new threats, of which four alarms are available to the public. The NCLUS signals new payload on a honeypot network. This new payload may, or may not be linked to a new threat on the network. All the NCLUS alarms for the last 24 hours are presented on the first page, with the time of the alarms rounded to the full hour on which they were generated. Each NCLUS alarm consists of a signature that describes the payload in a format suitable for use in snort. Clicking on "Cluster Statistics" in the "Statistics" menu gives access to a list of top clusters (not just new clusters as signaled by NCLUS) observed by the system in the last 24 hours. The NPORT alarm on the other hand signals ports on which payloads have been detected for the first time. The system registers all ports on which payloads have been detected and keeps them in memory for a week. If a port is not updated for a week, it falls out of the list. If subsequently payload is detected again on such a port, the alarm is signaled. The NSNORT alarm works in a similar manner, but for snort rules, not ports. The SWEEP alarm registers portsweeps performed from a small number of sources (ie. likely not worm or botnet traffic). These are handled in a similar manner to NPORT and NSNORT alarms.
5.	Can the snort rules proposed by the system be used for snort in a production environment?
The quality of the snort signatures is only as good as the payloads that are observed. It is possible that the payloads monitored are not real attacks or just benign protocol headers, which may result in false alarms if such signatures are used in a production environment. Additionally, there is a potentially for false negatives, as the signatures computed may be too exploit specific, meaning that a small modification may go through undetected. NASK and CERT Polska take no responsibility for the way these snort signatures are used.
6.	Where can I learn more about ARAKIS?
Listed below are some links to articles about the project: 1. Piotr Kijewski, Automated Extraction of Threat Signatures from Network Flows, 18th Annual FIRST Conference, Baltimore, Maryland, June 2006 2. Piotr Kijewski, Metody automatycznego wytwarzania sygnatur zagrożeń sieciowych, CERT Polska SECURE Conference, October 2005 [in Polish] 3. Poster outlining the project concept [in Polish] 4. Tomasz Grudziecki, Projekt ARAKIS - doświadczenia z obserwacji zagrożeń w sieci, CERT Polska SECURE 2008 Conference, October 2008 [in Polish]