[pcap-ng-format] draft-tuexen-opsawg-pcapng-01a - PCAP Next Generation (pcapng) Capture File Format

[Guy Harris]

On Sep 1, 2015, at 11:13 PM, Jim Young <jyoung at gsu.edu> wrote:

> I have a few initial comments regarding draft-tuexen-opsawg-pcapng-01a - PCAP Next Generation (pcapng) Capture File Format.

Just "draft-tuexen-opsawg-pcapng" would probably suffice, given that it's a moving target - it's not as if the "01a" reflects a version that gets updated any time somebody checks a change in.

> First it is great to see this document finally get the respect it deserves.  Really nice work all of you!

That's the thing about specifications - trying to implement them finds a lot of the places where they weren't sufficiently detailed, and Hadriel's been fixing a bunch of problems with Wireshark's implementation.

> Throughout the document the term bytes is used.  Would it be better to use the term octets to represent 8 bits?

Probably not a bad idea, although, these days, processing data from outside your system (whether it's something transmitted over a network or read in from some removable medium) is much more of a pain if your system's memory can't be divided into chunks of 1 or more octets.

> I have first-hand (historical) experience will issues associated with assuming a byte always represents 8 bits.  I'm sure more than one person here has had to wrestle with the notion of 9 bit bytes on a Unisys OS1100 or OS2200 system? ;-)

No, but, ages ago, I dealt with two other 36-bit systems (PDP-10 running TOPS-10 and GE 645 running Multics).

> In section 3.3. Logical Block Hierarchy
> 
> The tree view example shows one possible hierarchy under an Interface Description Block (IDB).  Is this tree view supposed to represent a chronological view of the block structure or a parent-child relationship?

The latter, in that:

	packet blocks refer to a particular interface, as described by an Interface Description Block, via their interface ID fields;

	Interface Description Blocks are valid only within a given section, as marked by a Section Header Block.

> In the current tree example we have a Simple Packet Block (SPB) in addition to the EPB.   Section 4.4 Simple Block Block paragraph three states "... it MUST be assumed that all the Simple Packet Blocks have been captured on the interface previously specified in the first Interface Description Block."  This implies that only one interface in a multi-interface pcapng trace file can have SPBs; if there is more than one IDB, the second and subsequent IDFs can only use EPBs.  

That is correct.

> In section 3.4. Physical File Layout
> 
> The second paragraph states "... all blocks MUST have the fields Type and Length at the beginning."  What about the value of the trailing Block Total Length field?

Its value must be the same as the leading Block Total Length field's value.

>   Based on Sections 3.1. General Block structure this trailing Length field appears to be mandatory but is it required to be non-zero?  I have at least one trace file that was generated with the EPB's trailing Length field set to zero.   Wireshark (actually the wiretap library) catches this and flags it as an error and aborts reading the file.

That's because it's not a valid pcapng file.  The trailing block length is there to allow the file to be scanned backwards:

	http://xml2rfc.tools.ietf.org/cgi-bin/xml2rfc.cgi?url=https://raw.githubusercontent.com/pcapng/pcapng/master/draft-tuexen-opsawg-pcapng.xml&modeAsFormat=html/ascii&type=ascii#rfc.section.3.1

"Block Total Length: total size of this block, in bytes. This field is duplicated to permit backward file navigation."

so that if you're scanning the file backwards, you could read the trailing block length and, from that, find the beginning of the block and, therefore, the block type.

> Tcpdump on the other hand appears to successfully read this file even through a EPB's trailing Total Block Length is zero.  I can't imagine any good reason NOT to set the trailing Block Total Length to the starting Block Total Length, but i can imagine a lazy pcapng reader not caring to verify the trailing Total Block Length value.

Yeah, I was lazy, and didn't put that check into libpcap (which is what tcpdump uses to read pcap and pcapng files).  I'll fix that....