tl;dr - analyze & visualize Blue Coat censorship logs from Syria
A short video. This would be a good place to start.
About these logs...
This data was recovered from public FTP servers in Syria over a period of six weeks in late 2011. The logs are from Blue Coat SG-9000 filtering proxies (aka "deep packet inspection") installed by Syrian ISPs and used to censor and surveil the Internet. Individual log lines are data about HTTP request/response from web browsers.
The logs were eventually leaked, with client IP redacted. The data is now public domain.
The data set is ~55GB in total compressed, and almost 1/2 TB uncompressed. That's a whole lotta lines. They provide unique, first hand evidence of what censorship looks like from the censor's perspective.
Mirrored from http://sec.tl/syrian-censorship-logs/ at 2011-12-12 06:44 Two versions of the same data. Blocks are easier to work with.
Blocks: are a series of tab-separated lines, with several columns. Each file is gzipped & about ~10MB. They are numbered like block-0000.log.gz up to 5274. The fields are described here. Unparseable lines removed, etc..
Raw logs: are the original, unaltered mirror of the leaked logs.
no there are no versions with the client IP intact, please stop asking.
This mirror is hosted on Amazon Web Services. You can be certain that they keep logs too.
PLEASE USE BITTORRENT THIS THING CO$T MONEY$
kthx. If you have never used bittorrent (wat), μTorrent is a simple client.
Amazon S3 & AWS
Bandwidth is free inside AWS
Mirror of the original raw logs, as individual files. If you want the whole set, please use the torrent.
Each block torrent is
- Blocks Torrent 0 - 5.5GB
- Blocks Torrent 1 - 5.5GB
- Blocks Torrent 2 - 5.5GB
- Blocks Torrent 3 - 5.5GB
- Blocks Torrent 4 - 5.5GB
- Blocks Torrent 5 - 5.5GB
- Blocks Torrent 6 - 5.5GB
- Blocks Torrent 7 - 5.5GB
- Blocks Torrent 8 - 5.5GB
- Blocks Torrent 9 - 5.5GB
- Raw Logs Torrent - 54GB
Code & Results
We like cute cats.
The Telecomix Network | No copyright | Please copy us and make datalove