[pcap-ng-format] Cleanup of the PCAP-NG spec

[Erik Hjelmvik]

I have been a lurker on this list for a long time, but I feel it's time to
speak up. I love the addition of support for multiple interface types and
annotations in PCAP-NG. However apart from the Interface Description Block
and opt_comment I find that most of the extra block types in PCAP-NG will
prevent this new format from being as widely accepted/implemented as the
legacy PCAP format is.

The legacy PCAP format adhered to the KISS principle (Keep it simple
stupid), which was one of the reasons for why this file format prevailed
and the others didn't. However, PCAP-NG unfortunately doesn't adhere to
KISS. It has lots of block types and most of these block types will only be
used by a very small clique of users/vendors. I also see requests for
additional block types being discussed on this mailing list every now and
then. Unfortunately there seems to be no discussion about removing unneeded
block types!

I would love to see a great cleanup of the PCAN-NG specification, if
possible. I'd love to see those rarely used block types removed, including
all the optional block types. In fact declaring a block type as optional
doesn't solve anything. It's better to have a simple standard that can be
fully implemented by all, you otherwise risk running into compatibility and
interoperability problems later on. In fact designing a file type
specification is very much like defining a protocol, which is why I
recommend that you watch a presentation by Daniel Stenberg (developer of
cURL) about the process behind designing the HTTP2 specification. In this
presentation Daniel says “you should never ever have optional parts”:
https://youtu.be/3Bg5NZGsgTg?t=17m06s

I am also not very fond of the optional little/big endian part of the
specification. Converting fields from one endlessness to the other is not a
lot of work for the sniffing application, the bottlenecks of the sniffers
are in completely different areas. I claim that forcing sniffers to write
field values in well-defined endian will not affect their performance. But
it will, on the other hand, make the code in the parsers much more
readable. In the current PCAP-NG spec the Section Header Block actually has
a 4 byte length field, which the parser doesn't know how to interpret since
the endianness indicator (Byte-Order Magic) hasn't been read yet.

Robert Graham actually wrote a very good blog post about handling
endianness in software yesterday:
http://blog.erratasec.com/2016/11/how-to-teach-endian.html

Robert also mentions that aligning values isn't really important for
performance. My take on this is that it would have been better to try get a
simple and clear PCAP-NG file specification than to force everything to be
aligned to a 32-bit boundary.

Sorry for the rant, but I'd really like to see a discussion about cleaning
up the PCAP-NG spec rather than adding even more block types!

Best regards,
Erik Hjelmvik
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.winpcap.org/pipermail/pcap-ng-format/attachments/20161115/bceb3b54/attachment.html>