[ntar-workers] Generic Comments on NTAR format

[Guy Harris]

On Jun 30, 2005, at 4:17 PM, Jose M. Gonzalez wrote:

> If you're referring to the block type in an SHB block header, I'm  
> lost.

So am I, because, according to

     http://www.winpcap.org/ntar/draft/PCAP-DumpFileFormat.html

the block type in an SHB block header is "\r\n\n\r", so that if a  
pcap-NG file is FTPed in ASCII mode between OSes that have different  
line ending conventions, the SHB block header gets mangled in such a  
way that NTAR can detect and report that as a "file transferred in  
the wrong mode" error, and the application can report it as such.   
Given the number of Ethereal "problems" reported on the Ethereal  
lists that turn out to be the result of somebody moving files between  
UN*X and Windows systems in ASCII mode, I eagerly look forward to  
that capability.

That block type value is palindromic, so the byte order doesn't  
matter.  The SHB *itself* contains, as part of the block contents, a  
"byte-order magic number", 0x1A2B3C4D, which

     1) can be used to check whether the file really is a pcap-NG  
file, as opposed to some other type of file that happens to start  
with "\r\n\n\r" (if a byte-order magic value appears "sufficiently  
close" to the beginning of the file, and the SHB block type looks  
like the results of "\r\n\n\r" being run through an ASCII-mode FTP,  
the file's probably a pcap-NG file run through an ASCII-mode FTP)

and

     2) can be used to determine the byte order of multi-byte  
integral (and floating-point) quantities in the block headers, etc..

The best value for the block type in an SHB block is "\r\n\n\r", not  
"ENDI" or "SHB " or any other string that doesn't have lots of \rs  
and \ns in it - we want a value that will get mangled in a fairly  
standard way by ASCII FTP transfers (or plain-text HTTP content type  
transfers, etc.) between systems with different line-ending  
conventions (mainly UN*X and Windows, although I guess there might be  
some crufty old FTP apps for Mac OS that run in the Classic box - or  
even natively as Carbon apps - that mistakenly think the right line  
ending for OS X is "\r").

> I'm not sure whether I understand what difference endianness makes  
> here.
> Either if we use 4-byte integers or 4 1-byte ASCII chars, a parser  
> can't
> guess the block type until it knows the dumper endianness.

...unless the block type for the first block in a section, i.e. the  
SHB, has a palindromic block type.

> BTW, if you assume the first 4 bytes in the file correspond always  
> to an
> SHB block type, then you can also get the endianness from there.

Not if it's palindromic.

The first block in the file must be an SHB - see 2.3 "Block Hierarchy  
and Precedence".  The type field in the block is palindromic, so the  
block can be recognized as an SHB without knowing the byte order.   
The next 4 bytes are the length of the block; that can't be decoded  
without knowing the byte order, *but* the next 4 bytes *after* that  
are the byte-order magic number.  From that the byte order of the  
second *can* be determined, and from that the length can be  
determined and checked.

Note, of course, that a big-endian and little-endian capture can be  
concatenated, so the byte order applies only to the section, not  
necessarily to the entire file.