Extract JSON from HTTP captured traffic with Wireshark
Motivation
Sometimes you need to develop software with libraries, frameworks or software not so well documented.
In much of these cases, the software involves a kind of client/server communication that uses an API REST.
If for example we haven’t a client library for our programming language, sometimes it is preferable to study the behaviour capturing and analyzing the JSON or XML exchanged by the two parts.
Just the case that concerns us today.
1. Capturing the HTTP traffic
We first need to get the HTTP data traffic.
I like to capture the data and save it in a file, and later process it with Wireshark GUI.
This is useful if the software is running under a container, for example.
We’ll use the old “tcpdump” CLI tool. You can easily install it executing “apt install tcpdump” if you are under a Debian based system.
We execute as root user the following shell order:
tcpdump -i any -s 0 'port 8080 and (((ip[2:2] — ((ip[0]&0xf)<<2)) — ((tcp[12]&0xf0)>>2)) != 0)' -w http-traffic.capThis order starts to capturing the HTTP traffic on any NIC device, and saves it in the file “http-traffic.cap”. The flag “-s 0” indicates that we want the maximum captured packet size. Respect to the capture rule, we can say that we seek to capture the packets that have TCP payload and that have the port of our http server (the 8080 in the example) as the source or destination.
We leave tcpdump working as we proceed to interact with our server. For example, by running tests or samples.
When we finish, we press Ctrl+C or Cmd+C in the tcpdump terminal to stop the capture process.
2. Exporting JSON with WireShark
Open WireShark and go to “File → Open”. Select the file “http-traffic.cap” and click “Open”.
You can see all the packets captured.
Apply the following filter expression to reduce the list to the “http” packets with the URL path prefix “/api” and method “POST”, for example.
http.request.method == POST && http.request.uri contains "/api"
To export the JSON payload, select the “JavaScript Object Notation”, right-click and press “Export packet bytes”.
For a batch process, you can move around packets using the arrows keys and the shortcut Ctrl + Shift + X (Cmd + Shift +X) to export the bytes directly.
This process can be automated using some LUA script.
It is also possible to use an HTTP proxy that allows us to export this content.
Please, if you liked it, give it a round of applause. And if you want to know more about DevOps, Kubernetes, Docker, etc … follow me! KR
