Maybe you've read the latest headlines about the adoption of HTTP/2 features from
NGINX or earlier news from their own
Nathan Moore.
While it's fresh in your memory; here is a
quick summary, then a State of the Union of HTTP version 2 for you, and an
update about HTTP/2 support in various Blue Team tools:
Take Nginx's adoption of the server push feature, as a sign of how the protocol is rapidly, but only gradually changing. Such are things in Intrusion detection and active defense, but I'm not quite sure if we're paying enough attention to the evolution of HTTP/2.
Here's what I mean: a lot of teams are actively protecting against the top web vulnerabilities like SQL injection and cross-site-scripting, but what happens when it all becomes one big encrypted blob? As well, the leading open source tools still need help to add or improve HTTP/2 decoding support. Let's team together, especially if you are a developer, to help catch these tools up to speed. A special shoutout goes to the Bropy development team, a tool that provides application-layer agnostic, anomalous intrusion detection using Bro and Python.
I'll start with a somewhat brief overview of what HTTP/2 is, and where its coverage is at for Bro, Snort, and Wireshark.
HTTP/2 basics
I started off my hunt for knowledge of the latest and greatest things, like I always do... by asking my grandma...
Although she might have had AOL once, for a few minutes; she happily admitted she had no idea about HTTP/2.
So then naturally I turned to
RFC 7540 from may 2015--authored by none other than Google and Mozilla themselves and a handful of others--and I realized that version 2 was sounding pretty next-level. It was sincerely time for an update from v1: HTTP 1.1 was standardized in 1997, so although 20 years is not long at all in human hindsight, in technology terms that is several epochs.
In my own words, HTTP/2 has 5 types of new features which I'll list below. While it keeps the
http basics like PUSH and GET, it brings features 'up the stack,' using "
streams" which are named very similarly to the TCP flags feature from layer 3 of the TCP/IP model, as well as an identification type number
(stream identifier) even similar to a tcp id number, and a
CONTINUATION stream, similar to fragmentation theory from the IP layer. The packet has its own complete structure, called a "frame" which you might think of as layer 3.5. Inside the frame are these variuos streams which can be prioritized and sent simultaneously. You'd think the RFC team could use some fresh names that would be less likely to be confused with other existing networking technologies, but they did not. See as well the neat FAQ from the
http/2 team.
 |
| image from Good Barber (See https://tr.goodbarber.com/pwa/technology/) highlighting their product utilizing http/2 |
- Headers are now compressed, but rather than gzip or other common forms of compression, it uses a technology called hpack (support was recently added to snort to decode this). Within the application-layer itself (in the probably-encrypted "frame"/payload, something new called "pseudo-headers" within the header stream are begun and essentially delimited with a colon, These are values, some of which are like the "headers" in the http 1.1 code, but don't be confused, these just give information about more streams to come. That's why they are caleld pseudo-headers. See 2.
- Data comes in tcp-flag-like-named units called "streams," as shown above, more accurately these are interwoven process tasks which can be communicated simultaneously by the client and server. These are basically the payloads inside a totally new packet-layer, as I show below. They can also be prioritized. Streams are numbered in a novel way though, luckily: streams from the client have odd numbers, and sever streams have even numbers. For example, RST_STEAM, GOAWAY (a 0x07 value, like TCP FIN), and a HEADER frame (0x01 like a TCP SYN).
- The "body" in HTTP 1 terms, is binary
- The Server pushes data in anticipation of client need (the feature recently added to Nninx)
- TLS exchange, "hello," can now happen more quickly using ALPN (Application layer protocol negotiation, that is the "speeding up of things")
You can see already that this could be potentially mind-blowing. Are the days of simple, text-based hypertext documents, that almost anyone could create, finished?
As you'll see below as we wrap up, I was surprised that so many commercial IDS's (Intrusion Detection Systems) are all-ready to go with the protocol. I even had to revise my initial title of this article from just "HTTP/2, Where are you ?.. " ...because HTTP/2 is already right here in our face!
The packet looks absolutely nothing like the line-based HTTP 1.1 text-fun-goodness that you are used to. See the RFC for details, here is the structure from it. In my own words that not including the payload this is a 9 byte "frame" structure (bytes at offets 0x0-8), and the high-level bit at 0x05 even has a "not yet assigned" evil-bit type bit, waiting to cause adventures in minimal c2:
All frames begin with a fixed 9-octet header followed by a variable-
length payload.
+-----------------------------------------------+
| Length (24) |
+---------------+---------------+---------------+
| Type (8) | Flags (8) |
+-+-------------+---------------+-------------------------------+
|R| Stream Identifier (31) |
+=+=============================================================+
| Frame Payload (0...) ...
+---------------------------------------------------------------+
Figure 1: (FROM RFC 7530) "HTTP/2 Frame Layout"
It's cited on Wikipedia that "as of March 2018, 24.3% of the top 10 million websites supported HTTP/2," so actually that is way more than I had expected.
I've compiled an update of the major open-source projects first here.
Bro, Snort, and Wireshark
Wikipedia has a pretty nice start at listing
Severs and CDNs that support HTTP/2 (most leading servers do), as mentioned earlier. But what about defense products? I also asked a team of great friends & security professionals at the slack channel for the
Braking Down Security podcast [sic] to help compile a list of support for their favorite proprietary tools (you'll see it at the bottom of this post).
Keep in mind, that of course, since HTTP/2 is encrypted in most implementations (but not by default in the protocol), inspection will be limited with out setting up proxies, and other techniques which are used to do so-called "deep packet inspection" with other forms of TLS already.
So what we mean here, is that you'd be able to open a tool, and get a clear dissection of what was happening in an HTTP/2 stream, and have an explanation of the protocol right in front of you, and crypto-text, if it was encrypted and you had no proxy.
I couldn't quickly find information about
bro, but it was mentioned on feature wish lists, along with other progressive protocols like QUIC, that
were being actively considered. I've heard before solidly, that it is "on the horizon."
Snort does have support now in BETA, which is not recent news, and was not implemented by default. It takes some pretty serious tweaking. This is an update from other news I had heard just months ago.
Wireshark--which
supports USB debugging now (yes, actually sniffing for things like bad usb aka rubber ducky, not usb network adaptors or the like)--
has included http/2 for awhile in recent versions, and even has a very clear webpage about
http/2 support in wireshark. To get full support of header decompression (hpack as mentioned earlier), you'll need to manually add library nghttp2, which was also one of the complexities I mentioned in Snort http/2 support.
My search continued on. I had to review my favorite
503 course materials; as well as skim over a few pages of google search results, and the very-non-hashtaggy twitter channel
#http/2, which may have also marked the first time I hashtagged a forward slash. For a deeper dive, you can even go right into the
SANS Reading Room and read a nice gold paper by Stefan Winkel, advised by Chris Walker. It's now outdated, but provides great insight. Ironically
HTTP/2 already has its own twitter handle, which has over 10 times as many followers as the creator's main handle. In other words, HTTP/2 is rockin' already, and doesn't need us to blog about it.
Let's compare the digging I just did, to how easy it was to find claims about http/2 speed and benefits.
Security & Marketing
It seems that this is already said and done, so where is the security response? It's also steady and growing, there was even a
Blackhat talk already; and as usual, as soon as research is born, we all must work harder, and patch the next hole, and move on to the next... which you know will come. So a lot of great work is out there; that is out of the scope of today's post. I especially enjoyed the Merlin project's research, which I highlight at the end, below, as well as
Mark Nottingham & Martin Thompson (from Mozilla)'s RFC 8164 "Opportunistic Security for HTTP/2!"
HTTP/2 allows the server to "push" content, that is, to respond with data for more queries than the client requested. Doesn't that sound a lot like javascript on your e-mail webclient? It sounds scary. So we're taking one by one things that are in other layers, and incorporating them directly into http. So in theory, eventually that might mean more security, we must learn to defend against it first. Similar concerns were already brought up by the FreeBSD team, for example on how the protocol was put together, as many things end up, by focusing only on a few key players, here with Google and Mozilla at the forefront.
Web-dev or even marketing research was a lot easier to find. Apparently HTTP/2 makes you rich and blows up your SEO presence. Well, not quite, but it was easy to find
Another
How can HTTP/2 improve results for online retailers? The updated version of the ubiquitous HTTP protocol makes sites perform faster and can improve search engine optimization.
Materials for further reading seem not far away at all from the
developers' point of view, and even from the
security perspective. In fact that last link here Russel Van Ruyl aka @ne0nd0g was exactly what I was hoping to find to dig deeper into this topic, in terms of my own current core emphasis area, of network intrusion detection.
Merlin
In ne0nd0g's post, he discusses introduces Merlin — A cross-platform post-exploitation HTTP/2 Command & Control Tool. This was actually the first and only research that quickly came up in more than one of my daily infosec research feeds, I had a
copy of the same post shared from
Lydecker Black's website
kitploit, back when Merlin had been released a few months ago.
Merlin works with powershell, and has also some python control. I can see a lot of people getting excited about this. You've probably already read up on Merlin, or even used it, so I'm just giving respect where it is due.
In short, this is the climax of the blue team call to action, because it provides an easy way to do C2 over HTTP/2, so you'd better be able to read it.
These guy's work can speak for itself, but basically the perfect forward secrecy of HTTP/2 makes it very difficult to inspect with traditional IDS/IPS and as well, Russel gives yet another shout out to the Go programming language--which seems to me without a doubt the next language security pros should get their feet wet in--if already deeply familiar with Bash, Python, and (if you're working with Windows) Power Shell--so 'go' to Go.
"Evade network detection during a penetration test/red team exercise by using a protocol that existing tools aren’t equipped to understand or inspect. Merlin is post-exploitation tool that is easily cross-compiled to run on any platform to achieve command and control of a host.... The magic of Merlin is found in its HTTP/2 protocol coupled with the use of the Go programming language and its easy to use cross-compiling capabilities"
Sounds awesome. More information about Merlin can be found on its GitHub page at
https://github.com/Ne0nd0g/merlin.
Fellow defenders, get your thinking caps on--more to come on this for years to come.
HTTP/2 we're getting ready for you.
~
_------------------------------_
Status of IDS/IPS and related products for HTTP/2 support:
Keep in mind that all will need constantly updated as the protocol and its implementations evolve.
Wireshark - YES
Bro - NO,
contribute here
Snort - YES, experimental
Bropy - YES (ignores application layer)
Surricata - tbd
Bricata - tbd
Symantec IPS
(protection agent) - YES
OSSEC (free HIDS) - NO, at least not mentioned
Cisco Firepower / umbrella extension - YES
Palo Alto Firewall products - YES
TrustKeep - YES
Fireye - YES
F5 - YES
Juniper - YES
"most firewall-based IDSs' support"
Riverbed - YES
Stellcentral/steelhead - YES
-______________________-
Vendors or product development volunteers are welcome to comment or dm me on twitter (@cherokeejb_) with information about your products, or corrections.
Huge thanks to
@waves, @SAJChurchey, @JK-47, @SeriesofTubes, and @hashtagcyber for providing information about more IDS, IPS, and networking products on the
Brake Sec Slack Channel.
