[ntar-workers] PacketBlock, PacketsBlock, GoPBlock

[Gianluca Varenni]

----- Original Message ----- 
From: "Jose M. Gonzalez" <chema at cs.berkeley.edu>
To: "NTAR workers" <ntar-workers at winpcap.org>
Sent: Thursday, June 30, 2005 12:50 AM
Subject: [ntar-workers] PacketBlock, PacketsBlock, GoPBlock


> Hi,
>
> Here's my input on the PacketBlock and PacketsBlock issue:
>
> - I think the having each packet encapsulated in a block is a good idea,
> as it permits mixing up packets and information on the capture
> process. For example, the dumper may realize that it has dropped
> some packets, or that some network event has happened that changes
> the meaning of the capture process. We want that information in
> sequential order with the packet capture process. I'd nevertheless
> change the name from PacketBlock to something like ExtendedPacketBlock,
> or FullPacketBlock.
>
> - Once this has been said, I understand the 12 byte overhead may be too
> much in some scenarios. I don't think that's the case when dumping
> non-compressed packets, though. You must anyway add 5 words (see
> Figure 5), the packet header, and the packet data. 3 extra bytes
> shouldn't kill you.
>
> IMO, the overhead is important when using compression. To take into
> consideration this case, I'd use Ronnie's idea of a block of packets.
> I'd suggest the (MPEG-inspired) terminology GoPBlock, for "Group of
> Packets," which somebody could find less confusing than PacketsBlock
> (I'd avoid using the word "compressed," as some uses of GoPs w/o
> compression may also make sense).
>
> A GoPBlock body will follow the following structure:
>
>       0                   1                   2                   3
>       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
>      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>      |                       GoPBlock type                           /
>      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>      / packet header/packet data/packet header/packet data...        /
>      /                                                               /
>      /          /* variable length, aligned to 32 bits */            /
>      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>
> - The GoPBlock type will determine the packet header/packet data 
> structure.
>
> - I'd make mandatory at least one GoPBlock type, "old-pcap", that would
> correspond to the old 4-word pcap header format (ts_sec, ts_usec, len,
> caplen), followed by the captured data. This will allow fast conversion
> of old pcap traces to the new format.

Uhm, why adding such backwards compatibility? I don't see a big issue in 
converting the old pcap format in the new one, converting every single 
old-pcap header in the pcap-ng blocks (it can be done offline). Instead, 
having another type of packet block adds unneeded complexity to either to 
ntar, or to the app sitting on top of it.

In general, as Loris pointed out, the idea of pcap-ng is to not 
hierarchical, in general (and at the moment I've not seen a clear advantage 
in these proposal of "groups of packets blocks"). As I already said, I'm ok 
for the markers and/or indexing blocks, but I think that the sections are 
maybe enough to group the packets.


>
> - Vern and I added packet compression to pktd, a packet capture and 
> injection
> daemon (there's a paper in PAM 2003). The compression was based on
> Van Jacobson's CSLIP, and it managed to squeeze traces up to less than
> 10 bytes per packet. I'll try to provide the details tomorrow (I have
> to take the dust off my notes).
>
> - Some notes on this GopBlock idea:
> - If the trace reader knows about the GoPBlock type, it can easily
> get the TSofFirstPacket and TSofLastPacket that Ronnie wanted. For
> the TSofFirstPacket, read the first packet. For the TSofLastPacket,
> get the next GoPBlock, ExtendedPacketBlock, or SimplePacketBlock,
> and the (first) packet timestamp shouldn't be that far from
> TSofLastPacket.
> - If an ntar trace reader doesn't know about the GoPBlock type, it can
> only infer that it skipped some packets, not how many, or anything
> about the packets themselves.

Well, I don't thiink this is a great idea, independently of the fact that I 
like (or not) the groups of packets concept. The objective of ntar (or 
better, of the pcap-ng format) is to store packets. If a trace reader is not 
able to deal with all the possible representation of packets (excluding 
custom packet blocks created by a third party), it's not a real trace 
reader....

Have a nice day
GV


> - Packets in a GoPBlock would be browsable only in a sequential
> fashion (excepting tcpslice-like hacks when possible) . That's a fair
> cost for the space savings.
>
> Regards.
> -Chema
>
> _______________________________________________
> ntar-workers mailing list
> ntar-workers at winpcap.org
> https://www.winpcap.org/mailman/listinfo/ntar-workers