web

The Pirate Bay un-SSL

Theory

Recently, the world saw The Pirate Bay offering SSL encryption on their server. This means that your ISP won't know anymore which torrent you are downloading, right? Wrong.
HTTPS is quite useless for protecting static and public content. By static, I do mean the .torrent file itself. It is always the same. By public, I do mean than one doesn't need any kind of authentication to pick up the content. It's always the same, for everyone. For crawlers, too.
So, one could easily index (a portion of) The Pirate Bay torrent database by the Content-Length. Then, one could intercept some encrypted traffic between some machine(s) within his/her network and the torrents.thepiratebay.org server. Knowing both (encrypted) request and response lengths, it is possible to get a quite reliable list of matches from the previously indexed torrent list.

Practice

Don't try this at work, or you might hurt yourself Eye-wink

  1. Use Wireshark to capture some torrent downloads. Torrents are hosted on a separate server, which makes the task easier yet. Just use the following capture filter: "tcp and port 443 and host torrents.thepiratebay.org"
  2. Now, just go with the stream Smiling ("Follow TCP Stream" for the packet you suspect belongs to the torrent download. This will create another filter, just like "(ip.addr eq 192.168.0.10 and ip.addr eq 83.140.176.156) and (tcp.port eq 2157 and tcp.port eq 443)")
  3. Just save the displayed stream anywhere else (pcap1.pcap sounds nice)
  4. Now, use my quick&dirty TPB-TLSlen.pl Perl script to get the request/response lengths:
    perl TPB-TLSlen.pl pcap1.pcap
    Yeah, I know, it is nasty. It only supports the TLS cypher. And it simply calls the tshark (the command line version of Wireshark) to parse it's output.
  5. Now, just paste the REQ and RES values below Laughing out loud
    (note that the REQ value is optional, setting it to 0 simply ignores the request size for matching)
Note that you are able to fine-tune the maximum and minimum header sizes. For the response, the headers are almost the same all the time. The only thing that varies is the decimal representation of the file length and age. (Un)fortuately, the request headers do vary for different browsers and referring pages. However, knowing the request size still helps a bit, specially if the torrent's filename was huge Smiling

Precision

The following size distribution chart was generated using the database with ~165K torrents:

torrent size distribution

The most common torrent size is ~14 KB, and it's easy to figure out that such torrents represent the shared 700 MB files Smiling
There's also a major peak for the 454 bytes torrents. However, bigger torrents are less common, thus, the size detection technique becomes more precise. Now, the average "distance" between torrent sizes is ~44 bytes (at least for the sample I've collected). So, adding a cookie with the random size up to 128 bytes will disrupt the size matching detection a lot. The request size disruption is even easier: the largest torrent URI I've found was 150 bytes-wide. Thus, padding every request URI to match 150 characters is enough to make the requests completely indistinguishable. Joining the pieces (the padding add-on strings are bold):
GET /4319199/[a4e]Ghost_in_the_Shell_TV_01-26.4319199.TPB.torrent?nVM2UGfcG533un4ym70eT2
9r0WwBLYdmFCNN+UTV/hiJ7EAXdFU5KfdWHpkB5lXaCmITsACKOPVyjmpbaOB+CrI5
HTTP/1.1 Host: torrents.thepiratebay.org User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.1) Gecko/2008070208
Firefox/3.0.1 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive Referer: https://thepiratebay.org/recent Cookie: language=pt_BR; country=BR; PHPSESSID=ad6cb7e414c8dc88e0c2444f6215165a
HTTP/1.1 200 OK Content-Type: application/x-bittorrent Etag: "2198642509" Last-Modified: Mon, 28 Jul 2008 22:28:59 GMT Server: lighttpd Content-Length: 91601 Date: Mon, 28 Jul 2008 22:37:56 GMT X-Varnish: 108010229 107999438 Age: 253 Via: 1.1 varnish Connection: keep-alive Set-Cookie: p=68eOfxOC7JwBYcMe1RJWC4Z5PV/lJzqJORW8KROPMH9zQhszSjFnRp2tsNWEoyabWAloneUaoz
MxYtx4hoM9MZUKE/7wGzC3ZKLEZdppG4og3W; expires=Mon, 28-Jul-2008 22:37:56 GMT; path=/;
domain=torrents.thepiratebay.org
(binary torrent data)

Solution

  1. Use a constant padding in the .torrent files. This messes things a bit, but stills ineffective. The only advantage is not messing up with the server Sad
  2. Patch the lighttpd server so it sends a non-lasting cookie with a random size.

Thanks

Share/Save/Bookmark

stas's picture
stas » July 31, 2008 » 11:05
database » hack » music » network » perl » php » video » web

Formulário CEP/CPF/CNPJ


(tente preencher os campos acima; a informação é atualizada instantaneamente)


Este projeto é o sucessor do CEP-2-City. É um formulário online que:

  • Verifica a validade do número CPF
  • Verifica a validade do número CNPJ
  • A partir do CEP, deduz o endereço completo (Cidade/Estado/Bairro/Rua)
  • A partir do CEP, deduz o código DDD da região Eye-wink

O banco de dados utilizado é compilado a partir de diversas fontes. Se não constar a informação da rua, então somente a cidade é retornada. A interface com o banco de dados foi implementada usando Perl e PHP, e pode ser acessada via CGI, Flash ou AJAX. O sistema de busca é extremamente eficiente, e não necessita de MySQL. O tamanho do banco de dados é cerca de 60 MB, e a performance chega a milhares de consultas por segundo.

Portanto, eis uma solução bastante símples, flexível e eficaz para o cadastro de clientes. Já utilizei este sistema numa pesquisa que conduzi, e asseguro que me poupou bastante tempo. Para realizar as consultas, basta acessar a seguinte URL (com a devida substituição do CEP): http://sysd.org/brloc/brloc.php?cep=05437000, e processar a string retornada com a função parse_str() (em PHP).

Se tiver interesse no banco de dados em si, entre em contato!

Share/Save/Bookmark

stas's picture
stas » August 15, 2007 » 20:31

Geolizer HTTP stats

Sample Geolizer output (fragment)

About Geolizer

This is an enhanced version of the popular Webalizer HTTP server statistics generator. It's main feature is an ability to discover visitor's country by his/her IP address. Default Webalizer method is to extract host suffix from the reversal DNS query (obtained directly from log files, or by webazolver program if HTTP server doesn't reverses client IPs), which is slow and imprecise (for example, Brazilian host could be reversed as .com). Geolizer relies on the GeoIP library API to do the same thing. Thus, no more DNS queries are required, and results are much more precise. Geolizer also has some additional features: it displays file sizes in a human-readable form (bytes/KB/MB/GB/TB) instead of default kilobytes. It also compiles under MinGW/MSYS now, so you can process your UN*X log files on your Windows box. And, finally, Geolizer features a nice eye-candy: country flags! Smiling

Beware as Geolizer also has some bad features (read "bugs"): for example, webazolver won't work anymore, and already resolved hosts aren't handled well. Want to see how it looks like, at all? Take a look at some sample statistics! Or see who else uses Geolizer to produce their server stats.

Tips

  • The country flag pictures can be downloaded at http://flags.blogpotato.de/. Just download and unzip world.small.zip & special.small.zip to the flags/ subdirectory in your HTML output path.
  • You may enhance your Webalizer further (allowing it to identify more user agents, referrers and search engines than normal) using extended configuration files, provided by Enric Naval and available at http://griho.udl.es/webalizer/.
  • It is possible to use multiple configuration files on Webalizer. Just specify them at the command line:
    webalizer -c common.conf -c user_stas.conf
  • Why don't you try also AWStats & WebDruid?!


It is easy to located an internet service which provides low prices for voip. Simply by downloading the software the service of the voip can be utilized, though for this a fast wireless internet is also required. This is especially benefical for small website hosting companies to contact their clients at a low cost.

Share/Save/Bookmark

stas's picture
stas » January 16, 2007 » 14:00

tiny HTTP proxy


Main screen

This is a tiny and highly experimental HTTP/1.0 proxy software that I have written to debug HTTP protocol & it's clients. It is very small and simple, yet useful to reverse-engineering purposes. It's interface is quite obvious. The Server frame controls the IP, port and connection limit of the proxy server. It also shows how many connections are active at moment. The Data Traffic frame shows in/out packets & bytes. Service frame allows you to stop, start and quit the proxy. These are the very minimalist controls for the very minimalist proxy server.

The interesting stuff begins at the Plugin frame. All the packet passed through this proxy server are forwarded to the selectable plugin module. By default, it is logger.dll. It simply saves every single packet into separate file, which uses the following name scheme: from_IP.from_port-to_IP.to_port.log (for example, 127.0.0.1.4322-127.0.0.1.21.log). The files can be ordered by their modification date in your file explorer, so you can track the entire session:

Packets list

The logger.dll can be set up to include a sequence counter at the beginning of each packet and to output saved packets into some specific directory:

logger.dll setup

Plugin module is also capable of injecting packets. Load the replicator.dll file and check the setup screen:

replicator.dll setup

When you click the Capture button and then make some action in your proxied web client, the replicator plugin will prompt you if it got a corresponding packet. This packet may be resent automatically, at the period specified in the Period box. You can capture & replicate several packets, and manipulate their resend period. A very interesting application of the replicator plugin is to flood web chats and to spin up web counters. Of course, the right way is to use logger.dll and to make a clone that imitates the "real" web client.

The final note: this is, and always will be, an alpha-state code. I do not develop this proxy application anymore. It is useful to me the way it is. But you can grab the source and make a whatever plugin you like, or even rewrite the code entirely. I don't care. Just give me the proper credits!

Share/Save/Bookmark

stas's picture
stas » January 3, 2007 » 21:50

I.E. 'hidden image' maker

sample masked image

If you are using Internet Explorer (any version able to load this page Smiling, just select the above image (or go to "Edit => Select All", or press Ctrl-A). You may see a watermark with white letters forming a word "TEST". This only happens in IE, due to the specific way it masks the selected images to show their selected state. Firefox, for example, simply tints the image with bluish color. There are lots of jokes one can make using this simple principle. For example, all images on your homepage may have some "subliminal" messages. Adobe PhotoShop is perfectly suitable to make such 'Hidden Images' by hand; in fact, my inspiration was a tutorial made by Matt Kment to do this. But if you are too lazy and/or do not have PhotoShop installed (UN*X users, for example), you can use this small program I wrote in Perl to automate the "steganographic" process. I wrote it using ImageMagickTM library (so it's required to successfully run the program). It is a command line program, but it is simple enough to use, and it has an advantage to be integrable into your own scripts/programs (you can even setup your server to embed your company logo into all graphics of your site, but this will overload the server's CPU and RAM too quickly). I called my program "Internet Explorer Mask'O'Matic". Take a look at it's startup screen:
Internet Explorer Mask'O'Matic v1.0 by Stas 
Grab yours at http://sysdlabs.hypermart.net/ Eye-wink

Inspirated by some mad stuff by Lem0nHead
Based on Photoshop 'Hidden Image' Guide (http://www.atomicwienerdog.com/ot/)
made by Matt Kment & suggested to me by xfalmp


ERROR: please give us --visible


* Usage: MaskOMatic.pl
* Options (note that you can use syntaxes like --vis= --hid -out -p):
--visible filename of image normally seen (required)
--hidden filename of image seen when selected in IE (required)
--output filename to write out, format is selected automatically
using suffix provided (required)
--percentage float value between 0 & 100; how much of hidden appears
(optional, defaults to 50%)
--quality JPEG/MIFF/PNG compression level (optional, DON'T USE!)
--contrast flag, internal contrast reduction (optional)
--test filename to dump preview of selected image (optional)

* Notes:
# Visible & hidden images doesn't need to have same size, when size
doesn't matches then hidden image is rescaled using Lanczos filter
# A huge set of image formats is supported (JPG, GIF, PNG for example)
but I strongly advice you to write output in loseless format *only*
(BMP, PNG, TGA) and *then* fine-tune contrast/brightness & save
compressed in your favourite image editor (like GIMP)

* Example:
MaskOMatic.pl --vis bush.jpg --hid death.jpg --out sublim.bmp
Only 3 options are obligatory: --visible, --hidden & --output. You can abbreviate them as -v, -h & -o, respectively. "Visible" is an image that user sees by default. "Hidden" is what user sees when he/she selects the image. And "output" is the resulting image you can embed into your pages. It is also possible to use the "--test" option to preview the selected state of the output image without starting Internet Explorer. If it looks bad, you may play around with "--percentage" option, which specifies the 'weight' of the hidden image. The higher is this value, the greater is the hidden image visibility. If that's not enough to hide your image well (some things are pretty difficult to hide!), you can try to tune the output image with "--contrast" option, but I suggest you to use some GUI program to do that as you get the visual feedback instantly. I also suggest you to always save your output in the non-compressed lossless format (like BMP, TGA, TIFF), and then fine-tune the compression in some program specialized in it (GIMP has a nice compressor with visual feedback).

Alternatives:

Share/Save/Bookmark

stas's picture
stas » May 5, 2006 » 17:47

GibCounter QW stats

GibCounter is a tiny yet quite useful game statistics generator for QuakeWorld games. It operates parsing frag*.log files generated by the QuakeWorld game server. If your server doesn't generate such a log files by default, you may enable this feature executing the server as follows:
qw-server +set fraglogfile 1
Of course you can also edit your server's .cfg files to enable frag logging. You may run GibCounter on the same machine (and, on UN*X systems, as the same user) that runs the game server. If it's a QuakeForge server, then GibCounter will locate log files automatically at the path $HOME/.quakeforge/qw. For the different ports of the QuakeWorld server, or a QuakeForge server operating as a different user, you may specify the location of the qw directory manually. To do that, simply pass the new directory as an argument to GibCounter (using your favorite command line shell):
perl gibcounter.pl /home/qserver/.quake/qw
GibCounter outputs the generated HTML code directly to STDOUT. So, if you're going to add it into your server's crontab, don't forget to pipe the output to some file! For example, the following crontab line will regenerate GibCounter game stats every 30 min and make it available through the URL http://yourserver.com/~youruser/gibcounter.html (file paths & crontab format may be different on your system; so ask your system administrator if unsure):
0,30 * * * * perl $HOME/gibcounter.pl > $HOME/www/gibcounter.html
GibCounter is also highly theme-able: CSS style of almost every element may be changed within gibcounter.css file. Of course, you can also edit the HTML template which is contained inside the Perl source itself Smiling
The game statistics page generated by GibCounter is self-explanatory. On the top, it will show players ranking, sorted by (guess what?!) the frags they scored. Frags are computed as: kills minus suicides. GibCounter also computes how many times each player was killed by others. And, on the bottom of the generated page, some computed totals are shown. This includes the period of time for which the stats were made, the top fragger (the best) & the top fragged (the worst) players. Please note that players with default nicknames, such as "unnamed" or "user-#", are automatically excluded from being processed, simply to avoid useless information bloat (as many different players would be rated as one, with very high kill/death number)!
By the way, GibCounter preserves the colorization of the graphical font in players' nicknames (Quake console is able to print some ASCII characters in white, orange, gold & brown colors), and translates all symbols into readable ASCII.
So, after all, how does GibCounter-generated page looks like?! Look for yourself, here are some example stats. Also, feel free to modify the program itself to fit your own needs!

Share/Save/Bookmark

stas's picture
stas » April 20, 2006 » 01:48
XML feed