Decode Your HTTP Traffic with sysdig

By Loris Degioanni - SEPTEMBER 10, 2015


Sysdig goes application aware! Recently we’ve been adding little useful features to sysdig at a good pace, and one that is worth mentioning is HTTP decoding.

Starting with sysdig 0.1.103, we have added two chisels, httplog and httptop, which provide insights into all HTTP traffic flowing on your server. They list the requests sent and received, and they can be used to find out what’s exactly hitting a web server. This blog post presents them and show a couple of usage tricks to get the best out of them.

But first, let’s take a step back and talk about echoing network connections in sysdig.

The Old Way: The echo_fds Chisel

For quite a bit of time, sysdig has offered a chisel to inspect the activity of a given set of file descriptors. It’s called echo_fds, and you can easily use it to troubleshoot network connections using the appropriate filter, for example:

# sysdig -A -pc -c echo_fds fd.port=80

------ Read 81B from  [http_server_1] [70d6e2d0d30c]> (ruby)

GET /slow/1 HTTP/1.1
Host: server:4567
User-Agent: curl/7.42.1
Accept: */*

------ Write 282B to  [http_server_1] [70d6e2d0d30c]> (ruby)

HTTP/1.1 200 OK
Content-Type: text/html;charset=utf-8
Content-Length: 12

Echo_fds has the nice benefit of being able to display network connections established by any container. And, of course, its functionality is nicely integrated into the csysdig curses interface.

The New Way: the httplog and httptop Chisels

Httplog and httptop go one step further by implementing some basic decoding of the raw data that echo_fds shows. In particular, if the connection is carrying HTTP data, they extract information like the URL and the response time. You can think about this as a simple version of ngxtop, but web server agnostic and able to see inside containers. Httplog will print information about every request, in a way similar to a log:

# sysdig -pc -c httplog 
2015-08-10 09:15:54.159173634 http_server_1 < method=GET url=server:4567/slow/3 response_code=200 latency=328ms size=12B
2015-08-10 09:15:54.166230840 http_server_1 < method=GET url=server:4567/slow/1 response_code=200 latency=130ms size=12B
2015-08-10 09:15:54.166256264 http_client_5 > method=GET url=server:4567/slow/1 response_code=200 latency=158ms size=12B
2015-08-10 09:15:54.166969191 http_client_1 > method=GET url=server:4567/slow/1 response_code=200 latency=159ms size=12B
2015-08-10 09:15:54.167024171 http_server_1 < method=GET url=server:4567/slow/1 response_code=200 latency=130ms size=12B

Httptop, on the other hand, offers a `top-like` view of all the HTTP transactions:

# sysdig -pc -c httptop 
ncalls          method          url
182             GET             server:4567/slow/2
182             GET             server:4567/slow/3
171             GET             server:4567/slow/1

Filtering Fun

As usual, leveraging sysdig’s filtering engine in conjunction to chisels can make your experience more fun and rewarding.

For example, you can observe the web requests of a particular container:

# sysdig -pc -c httplog

or the ones of that were not served by an expected process:

# sysdig -pc -c httplog “!=httpd”

or that are coming from a specific client:

# sysdig -pc -c httplog fd.cip=

Wrapping Up

The httplog and httptop chisels expand sysdig’s swiss army knife capabilities, and are great to keep an eye on your containerized web servers. And if you need this functionality (and much more!) but for all the machines and containers in your infrastructure, try Sysdig Cloud for free for 14 days.

Subscribe and get the latest updates