Packet-analysis and timestamping

It's not enough to just know your tools; a great engineer knows their tools inside out.

You're going to eventually have to analyze some packets in the cloud, or on any other headless system, or generally this may just be the fastest way to get a job done.  You might also not have access to the man or info pages--for example if you are working with a minimal system, whether that is on an Industrial Control System (ICS) or any other type of "IOT" embedded device system--or if you've even installed just the "core," "cli-extra," or similar version of a distribution on purpose. Either way you better have read the manual.

One thing everyone notices on their first spin through the tcpdump and tshark man pages are the huge amounts of options for timestamping, and how they are completely different in the two often-used-together programs.  I'll show you how to easily remember them.

Tcpdump is on most unix systems, or can be quickly installed.  To get more application-layer visibility, if you didn't already realize it you can install just tshark in most unix distributions without installing the very large, entire Wireshark package.  After installation 'tshark -V -r [file]' will give you just about all the information that you get with a quick skim of the entire Wireshark GUI, and include also option '-x' to add the hexadecimal view that many of us know and love (this adds the packet in hexadecimal at the end of both the quick line-display mode, or the -V and -O [protocol] verbose output modes).  The "expert info" from the Wireshark menus is even there (it's fun to grep out!), and as well the -z [statistics] option will provide you with countless hours or more fun (of endless commands to learn).

Where do timestamps come from?

Short answer, the kernel.  For the real answer, as usual, look at the source.  For example, the file util-print.c has some of the code where timestamp precision is called, and various options are parsed, if you'd like to take a look, you can appreciate the complexity of it, but see a tiny bit of what is going on. 

While browsing the source, I also noticed that the folder also had some short awk programs intended as examples of ways to reduce tcpdump data when you're tracking particular network problems.  Nanosecond resolution is definitely supported.

The timestamp is taken from the kernel, and the resolution can be as accurate as the kernel allows, and your libpcap configuration.  This is beyond the scope of this post, but for neat discussion about nanoseconds and the current limitations of this on Golang see google's github page.  Python seems to be able to handle it.

For a more in-depth discussion, which is really relevant to this post, I'd send you to read this post on the evidence.au forensics blog.

They note that "The accuracy of a timestamp depends on many factors including the performance of the system used to capture packets, its operating system, configuration, and more. On a Microsoft Windows computer the WinPCAP library is synced with the computer clock only at the beginning of the packet capture."

Command line tips

Here's how to remember the two different types of timestamp commands:
TCPDUMP "All the t's" and ...TSHARK "tame the shark, with a single't'"

I've not given any terminal standard output here, save one example.  You have to do that yourself, now grab a few seconds of traffic with 'tcpdump -w [output file.pcap]' and then try each as you read along; it will enhance your memory of the topic.

TCPDUMP "All the t's"

The only real trick to tcpdump, which is easy to remember once you've done it a few times, is that a single t remove the timestamp.  In other words, 'tcpdump -t -n -r [packets.pcap]' gives you output with no timestamps, and no dns resolution [-n].  Both programs also use all lower-case t's for time options.

         tcpdump timestamps
-t      Don't print a timestamp on each dump line.
-tt     Print the timestamp, as seconds since January 1, 1970, 00:00:00, UTC, and fractions of a second since that time, on each dump line.
-ttt    delta (micro-second resolution)
-tttt   date + hours, minutes, seconds, and fractions of a second
-ttttt  delta (micro-second resolution) between current and first line

TSHARK "Tame the shark, with a single 't' "

So basically you type "-t" add add "r" (relative), "a" (absolute), and "u" (for UTC, similar to the older Greewich mean time), and everything else really makes sense.  If you need help with relative and absolute time in packet captures, give it a quick Google search.

You write them just like I've typed out belw, with no spaces, as if each were a different option.  As a tcpdump fan, this will confuse you at first, but the after you realize they are additive, then it makes even more sense the tcpdump and "all the t's," above.  You can get a very handy list by simply typing only "-t," or getting one wrong, which I do all the time!

          tshark timestamps
 -t       get the funny help reminder
 -ta      absolute time
 -tad     absolute including date [YYYY-MM-DD] -tadoy   absolute date but with special YYYY/[day of year]
 -td      delta [change of time between packets]
 -tdd     delta [change of time between *displayed only* packets]
 -te      epoch date [seconds elapsed from January 1, 1970, ...look out 19-Jan 2038!]
 -tu      absolute UTC
 -tuud    absolute UTC including date [YYYY-MM-DD]
 -udoy    absolute UTC date but with special YYYY/[day of year]

So other than remembering that you start with the single "t" (tame the shark with a single 't') and then add options that you want, which make sense, the only one hard to remember--at least for me--is the y, so just remember that "y" gives you the strange year-plus-day-of-year view, which is a more of a special request.

Excersize: 

Perform one example of each of these commands, and note the differences.  Which is your favorite?  Mine's below.

This is my favorite command of the last few days:
#   #By greping the expert mode you can quickly find all sorts of interesting info & anomalous
tshark -V -x -tad -n -r [path to your file.pcap] | grep -i expert
#
# the -V option adds the details, as discussed above, and -x includes hex
# -tad gives a timestamp including a human-readable date, yyyy-mm-dd
# -n leaves out name resolution, to prevent among other things mistakes or adversary surveillance
# grep -i is  great habit to develop on your fingers, it ignores case in the searched term

Grep for a regular expression, but display entire "frame" entry, only

Here's a bonus tip:  to get exactly the full display of one packet (referred to as "frames" in TShark), pipe the command through "grep -A 18 -B 50 [regex]" or "grep -A 13 -B 50 [regex]" if you are not using the -x hexadecimal option.



I've been wanting to practice writing more directly, and where possible more briefly.  So I'd better end it here on this one.  I hope that it's helped you better remember the command-line syntax for working with packets, and depended both our knowledge on the topic and where timestamps come from, namely the kernel.

Thanks to a new friend for sharing this great post as well, from the philosophical questions every day blog, about self-evaluation of creative content, as a means of defeating so-called "imposter syndrome."  Here's a few bonus links about so-called threat-hunting.  Happy hunting!