Transcript Sniffing network traffic in Python

Sniffing network traffic in Python
Jose Nazario, Ph.D. <[email protected]>
Why Python?
• Interpreted language
– Bound to be slower than C
•
•
•
•
•
Rapid development
Easy data structure use
Fewer LoC per tool
Easy to manipulate strings
http://www.python.org/
Marrying Python and Sniffing
• Librares in C
– Often SWIGged, exported to Python
– pcap, dnet, nids …
• Modules
– pypcap/pcappy – pcap for python
– dpkt – packet deconstruction library
– libdnet – packet construction library (has python
bindings in the distribution)
– pynids – connection reassembly tool
libnids – reassemble IP streams
NIDS “E” box (event generation box)
Userland TCP/IP stack
Based on Linux 2.0.36 IP stack
Uses libpcap, libnet internally
IP fragment reassembly
Userland
Kernel
IP stack
Userland
Kernel
Libnids
IP stack
IP stack
libnids Basics
• Initialize
– nids_init()
• Register callbacks
– nids_register_tcp()
– nids_regster_ip()
– nids_regiser_udp()
• Run!
– nids_run()
• React
– nids_kill_tcp()
nids_run()
TCP callback
TCP stream object:
- TCP state
- client data
- server data
- source IP, port
- dest IP, port
- seq, ack, etc …
UDP callback
IP callback
UDP packet:
- source IP, port
- dest IP, port
- UDP payload
IP packet
- struct IP packet
- contains upper
layers
libnids TCP states
• NIDS_JUST_ESTABLISHED
– New TCP connected state (3WHS)
– Must set stream->{client,server}.collect=1
to get stream payload collected
• NIDS_DATA
– Data within a known, established TCP connection
• NIDS_RESET, NIDS_CLOSE,
NIDS_TIMED_OUT
– TCP connection is reset, closed gracefully, or was lost
libnids doesn’t expose SYN_SENT, FIN_WAIT, etc …
pynids Basics
• Event driven interface (nids_run(),
nids_next())
– TCP stream reassembly
– TCP state exposure
– Creates a TCP object
• Holds addresses, data, etc
– UDP and IP packet reassembly
Basic pynids Steps
• Initialize
– nids_init()
• Establish parameters
– nids.param(“attribute”, value)
• Register callbacks
– nids.register_tcp(handleTcp)
– def handleTcp(tcp): …
• Go!
– nids_run()
– while 1: nids_next()
pynids Order of Operations
• Packets come in
• TCP?
– State exist? Create state or reuse state
– Append data
– Process based on state in callback
• UDP or IP?
– Use handler, pass packet in
– You process in callback
Code Example (Python)
import nids
<handleTcpStream>
def main():
nids.param("scan_num_hosts", 0)
if not nids.init():
print "error -", nids.errbuf()
sys.exit(1)
nids.register_tcp(handleTcpStream)
try:
nids.run() # loop forever
except KeyboardInterrupt:
sys.exit(1)
Code Example (Python) cont
def handleTcpStream(tcp):
if tcp.nids_state == nids.NIDS_JUST_EST:
if dport in (80, 8000, 8080):
tcp.client.collect = 1
tcp.server.collect = 1
elif tcp.nids_state == nids.NIDS_DATA:
tcp.discard(0)
elif tcp.nids_state in end_states:
print "addr:", tcp.addr
# may be binary
print "To server:“, tcp.server.data
print "To client:“, tcp.client.data
Code Example (C)
int main(int argv, char *argv[])
{
if (nids_init() == 0)
err(1, “error, %s”, nids_errbuf);
nids_register_tcp(handleTcp);
nids_run();
exit(0);
}
Code Example (C), cont
int handleTcp(struct tcp_stream *tcp)
{
switch (tcp->nids_state) {
case ‘NIDS_JUST_EST’:
if ((tcp->addr.dest == 80) ||
(tcp->addr.dest == 8000) ||
(tcp->addr.dest == 8080) {
tcp.server.collect = 1;
tcp.client.collect = 1;
}
break;
case ‘NIDS_DATA’:
nids_discard(tcp, 0);
break;
case ‘NIDS_CLOSE’:
case ‘NIDS_RESET’:
case ‘NIDS_TIMED_OUT’:
printf(“((%s, %d), (%s, %d))\n”, inet_ntoa(tcp->saddr), tcp.srce,
inet_ntoa(tcp->daddr), tcp.dest);
printf(“%s\n”, tcp->server.data);
printf(“%s\n”, tcp->client.data);
break;
}
}
About the same LoC, until we start string manipulation
VersionDetect
• Small python tool
• Reports on headers
• Fully passive
– Support for: SSH (client, server), WWW
(client, server), and SMTP clients
• Motivation: coordinate data collection with
TCP stack fingerprinting
63.236.16.161
SymbianOS 6048 (on Nokia 7650?)
63.236.16.161: 80: Microsoft-IIS/6.0
www
80/tcp
VersionDetect Output
192.168.1.7: 22: SSH-2.0-OpenSSH_3.5
192.168.1.101:http: Mozilla/5.0 (X11; U; OpenBSD i386; enUS; rv:1.5a) Gecko/20031030 Mozilla Firebird/0.6.1
168.75.65.85: 80: Microsoft-IIS/5.0
165.1.76.60: 80: Netscape-Enterprise/3.6 SP2
168.75.65.69: 80: Microsoft-IIS/5.0
168.75.65.87: 80: Microsoft-IIS/5.0
69.28.159.7: 80: ZEDO 3G
198.65.148.234: 80: Apache/1.3.29 (Unix) PHP/4.3.3
216.150.209.231: 80: Apache/1.3.31 (Unix)
212.187.153.30: 80: Apache/1.3.31 (Unix)
212.187.153.37: 80: Apache/1.3.31 (Unix)
212.187.153.32: 80: thttpd/2.25b 29dec2003
64.209.232.207: 80: Apache/1.3.27 (Unix)
mod_perl/1.27
216.239.39.99: 80: CAFE/1.0
http-graph
• Small, passive python tool
• Examines HTTP request header:
GET /blog/styles-site.css HTTP/1.1
Host: www.jackcheng.com
User-Agent: Mozilla/5.0 (X11; U; OpenBSD i386; en-US;
rv:1.5a) Gecko/20031030 Mozilla Firebird/0.6.1Accept:
text/css,*/*;q=0.1
Referer:
http://www.jackcheng.com/blog/archives/2004/12/ipod_rumo
rs.html
http-graph
• Directed graph history of browsing
• Reconstructs graph from referrer and URL
in the header:
Referrer Request
• Lets you view your history as you took it
• Shows natural “hubs” of information
• See also:
http://www.uiweb.com.nyud.net:8090/issues/issue37.htm
Displaying http-graph Output
• Writes a small “dot” file
– “dot” part of “graphviz” tool
– Use “neato” to graph
– Output formats: SVG, PS, PDF, image map
– Can make fully interactive!
Example http-graph Output
Grabbing Data with pynids
• tcp.{server, client}.data and just strings
• Any string operations will work
– Searching
if “HTTP/1.0” in tcp.client.data:
– Regular Expression searches
if re.search(“HTTP/1.[10]”, tcp.client.data):
– Rewriting
string.replace(req, “GET HTTP/1.0”, “”, 1)
More Fun!
• Privacy invasion
– Snarf mail
• Log conversations
– IRC, AIM, etc …
• Steal files
– FTP, P2P apps, HTTP downloads …
• Disrupt sessions
tcp.kill()
New dsniff is written in Python …
flowgrep
• Marries sniffing with regular expressions
• A lot like ngrep, tcpkill, and dsniff
– Logs the whole connection, not just a packet
• Look for data in streams using regular
expressions
• Log or kill selected streams
• Dirt cheap IDS or IPS
– Under 400 lines of code
Resources
•
•
•
•
•
•
•
http://www.tcpdump.org/
http://www.packetfactory.net/projects/libnids/
http://monkey.org/~provos/libevent/
http://monkey.org/~dugsong/{dpkt, pycap}
http://oss.coresecurity.com/projects/pcapy.html
http://monkey.org/~jose/software/flowgrep/
http://pilcrow.madison.wi.us/pynids/
Additional Resources
• Stevens, TCP/IP Illustrated vols 1 and 2
• Schiffman, Building Open Source Network
Security Tools
• RFCs from the IETF