FLAIM (Framework for Log Anonymization and Information Management) is a modular tool designed to allow computer and network log sharing through application of complex data sanitization policies.
FLAIM is aimed at 3 different user communities. First, FLAIM can be used by the security engineer who is investigating a broad incident spanning multiple organizations. Because of the sensitivity inherent in security relevant logs, many organizations are reluctant to share them. However, this reluctance inhibits the sharing necessary to investigate intrusions that commonly span organizational boundaries. Second, anyone designing log analysis or
computer forensicsComputer forensics is a branch of forensic science pertaining to legal evidence found in computers and digital storage media. Computer forensics is also known as digital forensics....
tools needs data with which they can test their tools. The larger and more diverse the data set, the more robust they can make their tools. For many, this means they must gather many logs from outside sources, not just what they can generate in-house. Again, this requires log sharing. Third, researchers in many
computer scienceComputer science is the study of the theoretical foundations of information and computation, and of practical techniques for their implementation and application in computer systems. It is frequently described as the systematic study of algorithmic processes that create, describe and transform...
disciplines (e.g.,
network measurementsIn computer networks, network traffic measurement is the process of measuring the amount and type of traffic on a particular network. This is especially important with regard to effective bandwidth management.- Tools :...
,
computer securityComputer security is a branch of technology known as information security as applied to computers and networks. The objective of computer security includes protection of information and property from theft, corruption, or natural disaster, while allowing the information and property to remain...
, etc) need large and diverse data sets to study. Having data sanitization tools available makes organizations more willing to share with these researchers their own logs.
FLAIM is available under the
Open Source InitiativeThe Open Source Initiative is an organization dedicated to promoting open source software.The organization was founded in February 1998, by Bruce Perens and Eric S. Raymond, prompted by Netscape Communications Corporation publishing the source code for its flagship Netscape Communicator product...
approved
University of Illinois/NCSA Open Source License. This is BSD-style license. It runs on
UnixUnix is a computer operating system originally developed in 1969 by a group of AT&T employees at Bell Labs, including Ken Thompson, Dennis Ritchie, Brian Kernighan, Douglas McIlroy, and Joe Ossanna...
and
Unix-likeA Unix-like operating system is one that behaves in a manner similar to a Unix system, while not necessarily conforming to or being certified to any version of the Single UNIX Specification....
systems, including
LinuxLinux is a generic term referring to Unix-like computer operating systems based on the Linux kernel. Their development is one of the most prominent examples of free and open source software collaboration; typically all the underlying source code can be used, freely modified, and redistributed,...
,
FreeBSDFreeBSD is a free Unix-like operating system descended from AT&T UNIX via the Berkeley Software Distribution . It has been characterized as "the unknown giant among free operating systems". It is not a clone of UNIX, but works like UNIX, with UNIX-compliant internals and system APIs. FreeBSD is...
,
NetBSDNetBSD is a freely redistributable, open source version of the Unix-derivative Berkeley Software Distribution computer operating system. It was the second open source BSD descendant to be formally released, after 386BSD, and continues to be actively developed...
,
OpenBSDOpenBSD is a Unix-like computer operating system descended from Berkeley Software Distribution , a Unix derivative developed at the University of California, Berkeley. It was forked from NetBSD by project leader Theo de Raadt in late 1995...
and
Mac OS XMac OS X is a line of computer operating systems developed, marketed, and sold by Apple Inc., and since 2002 has been included with all new Macintosh computer systems...
.
While FLAIM is not the only
log anonymizer, it is unique in its flexibility to create complex
XMLXML is a set of rules for encoding documents electronically. It is defined in the produced by the W3C and several other related specifications; all are fee-free open standards....
policies and its support for multiple log types. More specifically, it is the only such tool to meet the following 4 goals. (1) FLAIM provides a diverse set of anonymization primitives. (2) FLAIM supports multiple log type, including linux process accounting logs, netfilter alerts,
tcpdumptcpdump is a common packet analyzer that runs under the command line. It allows the user to intercept and display TCP/IP and other packets being transmitted or received over a network to which the computer is attached...
traces and
NFDUMP NetFlowsNetFlow is a network protocol developed by Cisco Systems to run on Cisco IOS-enabled equipment for collecting IP traffic information. It is proprietary and supported by platforms other than IOS, such as Juniper routers, Linux or FreeBSD and OpenBSD....
. (3) With a flexible anonymization policy language, complex policies that make trade-offs between information loss and security can be made. (4) FLAIM is modular and easily extensible to new types of logs and data. The anonymization engine is agnostic to the syntax of the actual log.
History
Work on
log anonymization began in 2004 at the
NCSAThe National Center for Supercomputing Applications is a state-federal partnership to develop and deploy national-scale cyberinfrastructure that advances science and engineering. NCSA operates as a unit of the University of Illinois at Urbana-Champaign but it provides high-performance computing...
. At first this was for anonymizing logs in-house to share with the
SIFT group. Soon there was a need for more powerful anonymization and anonymization of different types of logs.
CANINE was created to anonymize and convert between multiple formats of
NetFlowsNetFlow is a network protocol developed by Cisco Systems to run on Cisco IOS-enabled equipment for collecting IP traffic information. It is proprietary and supported by platforms other than IOS, such as Juniper routers, Linux or FreeBSD and OpenBSD....
. This was a Java GUI based tool. Later,
Scrub-PA was created to anonymize
Process Accounting logs.
Scrub-PA was based on the Java code used for
CANINE. The development of both of these tools were funded under the
Office of Naval ResearchThe Office of Naval Research , headquartered in Arlington, Virginia , is the office within the United States Department of the Navy that coordinates, executes, and promotes the science and technology programs of the U.S...
NCASSR research center through the SLAGEL project.
It was quickly realized that building one-off tools for each new log format was not the way to go. Also, the earlier tools were limited in that they could not be scripted from the command line. It was decided that a new, modular command line based
UNIXUnix is a computer operating system originally developed in 1969 by a group of AT&T employees at Bell Labs, including Ken Thompson, Dennis Ritchie, Brian Kernighan, Douglas McIlroy, and Joe Ossanna...
tool was needed. Because speed was also a concern, this tool need to be written in
C++C++ is a statically typed, free-form, multi-paradigm, compiled, general-purpose programming language. It is regarded as a middle-level language, as it comprises a combination of both high-level and low-level language features...
. With the successful acquisition of a
Cyber Trust grant from the
National Science FoundationThe National Science Foundation is a United States government agency that supports fundamental research and education in all the non-medical fields of science and engineering. Its medical counterpart is the National Institutes of Health...
, the
LAIMThe LAIM Working Group is a NSF and ONR funded research group at the National Center for Supercomputing Applications under the direction of . Work from this group focuses upon log anonymization and Internet privacy. The LAIM group, established in 2005, has released 3 different log anonymization...
Working Group was formed at the
NCSAThe National Center for Supercomputing Applications is a state-federal partnership to develop and deploy national-scale cyberinfrastructure that advances science and engineering. NCSA operates as a unit of the University of Illinois at Urbana-Champaign but it provides high-performance computing...
. From this project headed by the PI, Adam Slagell, FLAIM was developed to overcome these limitations of
CANINE and
Scrub-PA. The first public version of FLAIM, 0.4., was released on July 23 2006.
Features
- Flexible XML
XML is a set of rules for encoding documents electronically. It is defined in the produced by the W3C and several other related specifications; all are fee-free open standards....
policy language
- Modular to support simple plugins for new log types
- Support for major UNIX-like
A Unix-like operating system is one that behaves in a manner similar to a Unix system, while not necessarily conforming to or being certified to any version of the Single UNIX Specification....
Operating Systems
- Built-in support for several anonymization primitives
- Plugin for NFDUMP format NetFlows
NetFlow is a network protocol developed by Cisco Systems to run on Cisco IOS-enabled equipment for collecting IP traffic information. It is proprietary and supported by platforms other than IOS, such as Juniper routers, Linux or FreeBSD and OpenBSD....
- Plugin for netfilter firewall logs
- Plugin for pcap
In the field of computer network administration, pcap consists of an application programming interface for capturing network traffic...
traces form tcpdumptcpdump is a common packet analyzer that runs under the command line. It allows the user to intercept and display TCP/IP and other packets being transmitted or received over a network to which the computer is attached...
- Plugin for linux process accounting logs
External links