Question: tool to act as a sniffer to providing statistics on Social Network traffic?
New here? Learn about Bountify and follow @bountify to get notified of new bounties! x

I have an opinion based question. I'll give $25 for each unique solution well defined solution.

  1. Is it possible to build a tool that will act as a sniffer and provide statistics on
  • types of Social Network Traffic (e.g. facebook, yammer) used by developers
  • amounts of social network traffic (inbound/outbound)
  • number of unique social network users
  1. What languages, libraries, or tools would be best for this? Assume Linux.

Here's some options I thought about, if you could write a solution (ups/downs) out of it, I'll give $25 for it.

Option #1

Write custom snort rules to capture facebook/hipchat traffic and send it off to a parser or even Snorby. You'd need a snort box configured with a port mirroring.

Option #2

Use node_pcap, figure out how to filter on social network traffic, and parse it with JavaScript and display the report in Meteor.

Option #3

Use libpcap and write C to make an application like tcpdump. Still need to figure out how to filter for facebook traffic. Could be called sndump for the command line.

I'll tip $25 for more solutions/ideas like the below.
akshatpradhan over 5 years ago
awarded to ochi

Crowdsource coding tasks.

3 Solutions


You could setup up a transparent http-proxy with Polipo or Privoxy and check the log file :)

  • Set up the proxy
  • Install and configure the proxy, that he listen for example on the address 127.0.0.1:8080 and enable logging.
  • Set up the firewall rule
  • Write a firewall rule, which redirect all outgoing traffic to the port 80 to the address of proxy 127.0.0.1:8080, which then forwards the traffic to requested domain.
  • Parse the log file
  • Set up a parser to extract all the accessed domains from the log file.

This solution has it's up and downs:

UP's:
    your users don't have to change something on their system, browser, ...
    you gain some capabilities, like caching, filtering, ... depending on the used proxy
    you can expand it to log other protocols like https, ftp ... when the proxy supports it

.

DOWN's:
    it's another system you to have manage and that's maybe failing
    it's raises some legal/privacy issues, when you keep extensive logs on your users browsing habits
@ochi Could you analyze my 3 options in the updated question and write up a report of how to implement those options; along with ups/downs of the options? I'll tip you $25 for an analysis of each option.
akshatpradhan over 5 years ago
I'd consider throwing the output to logstash and kibana. That way you can drill down to discover patterns
elwood over 5 years ago
@elwood Oh wow, logstash really came through. I remember looking at them as an alternative to Splunk in 2010 and they were just so brand new that I decided not to look into them any further. Thank you so much for commenting and letting me know. I love anything written in Ruby.
akshatpradhan over 5 years ago

Option #1

I used it on windows only but it should be pretty same on linux

rule (could be 2 separate rules)
alert tcp any any -> any any (msg:"socials"; flow:established; content:"Host\:"; pcre:"/(Host\:)(\s[a-zA-Z0-9.-]+\.|\s)(facebook.com|hipchat.com)\r\n/"; sid:1000001; ver:1)

.

C:\Snort\bin>snort –c c:\snort\etc\snort.conf –l c:\snort\log\ -A console -i 1

-c tells snort to load the specified conf file and

-l log the packets on log folder

-A output to standard console

-i run on the specified network interface (1 is some interface listed by C:\Snort\bin>snort –W)

Winning solution

Option #2

var pcap = require("pcap"),
    pcap_session = pcap.createSession("", "tcp"),
    matcher = /(facebook|hipchat)/i;

console.log("Listening on " + pcap_session.device_name);

pcap_session.on('packet', function (raw_packet) {
    var packet = pcap.decode.packet(raw_packet),
        data = packet.link.ip.tcp.data;

    if (data && matcher.test(data.toString())) {
        console.log(pcap.print.packet(packet));
        console.log(data.toString());
    }
});

but don't know how to put it in Meteor

@ochi these are actually really interesting. Can you update your solution with Ups and Downs of these solutions please?
akshatpradhan over 5 years ago
@ochi you don't have to do option 3 if it looks too difficult btw. Just an Advantage/Disadvantage analysis for Option 3 would suffice.
akshatpradhan over 5 years ago
pros/cons won't be changing - for any of that options You will need another machine (disadvantage) that you put your traffic through (so no one in your network don't have to change anything - another words no one will know - it's advantage and disadvantage in the same time)
ochi over 5 years ago
@ochi Thanks! that makes sense. Out of the 3 solutions, which one is your favorite? Do you think there are any other ways to do this?
akshatpradhan over 5 years ago
@ochi btw, I'm leaving this bounty open to tip other people's opinion as well, and at the end I'll just award the final bounty to you. Thank you for your help btw, it the sample code you provided was very interesting.
akshatpradhan over 5 years ago
Actually, I'm going to just close this bounty out. If you want to answer some of the questions in my comments, that would be great. Either way, the solutions you provided were very interesting. Thank you! I hope $75 was worth your time and I do hope to hear from you in the future on Bountify!
akshatpradhan over 5 years ago
View Timeline