[ntar-workers] PacketBlock, PacketsBlock, GoPBlock

[Jose M. Gonzalez]

Hi, 

Here's my input on the PacketBlock and PacketsBlock issue: 

- I think the having each packet encapsulated in a block is a good idea, 
	as it permits mixing up packets and information on the capture 
	process. For example, the dumper may realize that it has dropped 
	some packets, or that some network event has happened that changes 
	the meaning of the capture process. We want that information in 
	sequential order with the packet capture process. I'd nevertheless 
	change the name from PacketBlock to something like ExtendedPacketBlock, 
	or FullPacketBlock. 

- Once this has been said, I understand the 12 byte overhead may be too 
	much in some scenarios. I don't think that's the case when dumping 
	non-compressed packets, though. You must anyway add 5 words (see 
	Figure 5), the packet header, and the packet data. 3 extra bytes 
	shouldn't kill you.

	IMO, the overhead is important when using compression. To take into 
	consideration this case, I'd use Ronnie's idea of a block of packets. 
	I'd suggest the (MPEG-inspired) terminology GoPBlock, for "Group of 
	Packets," which somebody could find less confusing than PacketsBlock 
	(I'd avoid using the word "compressed," as some uses of GoPs w/o 
	compression may also make sense).

	A GoPBlock body will follow the following structure: 

       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                       GoPBlock type                           /
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      / packet header/packet data/packet header/packet data...        /
      /                                                               /
      /          /* variable length, aligned to 32 bits */            /
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

- The GoPBlock type will determine the packet header/packet data structure.

- I'd make mandatory at least one GoPBlock type, "old-pcap", that would 
	correspond to the old 4-word pcap header format (ts_sec, ts_usec, len, 
	caplen), followed by the captured data. This will allow fast conversion 
	of old pcap traces to the new format. 

- Vern and I added packet compression to pktd, a packet capture and injection 
	daemon (there's a paper in PAM 2003). The compression was based on 
	Van Jacobson's CSLIP, and it managed to squeeze traces up to less than 
	10 bytes per packet. I'll try to provide the details tomorrow (I have 
	to take the dust off my notes).

- Some notes on this GopBlock idea:
	- If the trace reader knows about the GoPBlock type, it can easily 
		get the TSofFirstPacket and TSofLastPacket that Ronnie wanted. For 
		the TSofFirstPacket, read the first packet. For the TSofLastPacket, 
		get the next GoPBlock, ExtendedPacketBlock, or SimplePacketBlock, 
		and the (first) packet timestamp shouldn't be that far from 
		TSofLastPacket.
	- If an ntar trace reader doesn't know about the GoPBlock type, it can 
		only infer that it skipped some packets, not how many, or anything 
		about the packets themselves. 
	- Packets in a GoPBlock would be browsable only in a sequential 
		fashion (excepting tcpslice-like hacks when possible) . That's a fair 
		cost for the space savings. 

Regards. 
-Chema