ThreatSTOP Blog

ThreatSTOP open sources CAPS anonymization code

Written by francisturner | March 14, 2011

One of the ways that ThreatSTOP keeps our IP reputation feeds up to date is that we process the firewall logs of our subscribers to see what attacks they are currently experiencing. We also want to feed the data back to other security researchers because we only mine the logs for certain information and others will find other useful information from them if they can analyze them. However there is a problem. Customers are usually unwilling to see their internal data distributed all over the place. Hence we've been looking for a way to reliably anonymize the data so that

  1. There is no way to track the original log source
  2. The internal addresses are uniquely hashed so that the same anonymized number is used for each particular ip address
  3. There is no conflict between addresses from customer 1 and those from customer 2 (even if - as is likely - both use the same private address space

There is, fortunately, a solution to this: the CYBER-TA Anonymous Alert Publication System (aka CAPS), originally developed by SRI, has all these features but has not been maintained for some years. ThreatSTOP has reached agreement with SRI to take over maintenance of the code and to that end we have set up a project at sourceforge - http://capscode.sourceforge.net/ - managed by me (Francis Turner) on behalf of ThreatSTOP.

The code is open source under the GPL and the initial tarball is available for download. I have setup a subversion repository as well; anyone is welcome to check the code out and I will be liberal in granting write access to those interested in taking part in development. ThreatSTOP wants to use, and develop the code, that’s why we made the effort to get it, but all we are doing with it is being the project owners/managers. We will certainly commit out own changes but we hope that other interested parties will also get involved too.

So please sign up and get the word out as this will undoubtedly be of use to many other projects.