[pcap-ng-format] Proposed new Alternative Packet Block
Serge Aleynikov
serge at aleynikov.org
Sun Aug 14 03:57:04 UTC 2016
Thank you for your comments.
First, I forgot to mention that the reason Timestamp is included in this
proposed block type is that if an application is not interested in
timestamps it would use Simple Block types, but very often timestamps are
of great interest for the purpose of analysis and replay. Moreover,
obtaining timestamps is very fast (i.e. nanoseconds latency) on modern
hardware, so it's worthwhile to have an "intermediate" block format between
Simple and Enhanced Packet Block, that is timestamped, but doesn't have
overhead of the enhanced packet.
See inline.
On Sat, Aug 13, 2016 at 10:56 PM, Guy Harris <guy at alum.mit.edu> wrote:
> > This proposal introduces the format that extends the option
> > definition to be able to store values in place of the option length
> > field, therefore reducing the size of the option code/value pair to
> > four octets.
>
> So this different option format would apply *only* to this block, and this
> block would *not* support "local use" options, as the pcapng spec already
> says, in section 3.5 "Options":
>
> Option Type (2 octets): it contains the code that specifies the
> type of the current TLV record. Option types whose Most Significant Bit is
> equal to one are reserved for local use; therefore, there is no guarantee
> that the code used is unique among all capture files (generated by other
> applications), and is most certainly not portable.
>
> so
>
> 1) in other blocks, if the option type's high-order bit is set, it
> means "this is a local-use option", not "this is an option with a 24-bit
> value";
>
> 2) in a block that uses these alternative options, if the
> high-order bit is set, it's a "short" option with the value in the
> remaining 24 bits, not a "local-use" option.
>
> If you want to have a new option type that applies to *all* blocks, and to
> support local-use options in the Alternative Packet Block, you would have
> to pick another bit to indicate a "short" option.
>
> The bit below the high-order bit isn't available, unless you treat the "do
> not copy this to an output file" custom options as special cases, as the
> option codes for those are 19372 and 19373, which have the bit below the
> high-order bit set. No option code currently has the bit below the bit
> below the high-order bit (the
>
> 0x2000 bit) set.
>
You have a good suggestion and it would make sense to make this option type
portable to other blocks.
> > 3.5 Options
> > Proposed extention to Options includes a special option value setting
> > the highest bit of the "Option Code" to 1 and leaving remaining 7 bits
> > of space for 127 possible option codes. The remaining 24 bits will be
> > used for storing the option's value:
> >
> > 0 1 2 3
> > 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
> > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> > |1| Option Code | Option Value |
> > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> >
> > This layout saves 4 bytes for options with compact values, such as
> > the ones described in the following packet block.
>
> The following packet block would be the *only* one that can use it, unless
> you pick a different bit.
>
So, with this revision, let's modify the layout of this option type, and
also for the purpose of preserving alignment, let's keep the code/value 2
octets each:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0 0 1| Option Code | Option Value |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Hence, revised option codes become (with added apb_iface_id option):
Options:
Name Code Length Multiple? Description
apb_opt_size 0x2001 - no Total byte size of options
apb_orig_len 0x2002 - no Packet's Original Length
apb_iface_id 0x2003 - no Interface ID
apt_flags 0x2004 - no Alternative Packet Block Flags
> > 5.1 Alternative Packet Block
> > 0 1 2 3
> > 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
> > +---------------------------------------------------------------+
> > 0 | Block Type = 0x00000010 |
> > +---------------------------------------------------------------+
> > 4 | Block Total Length |
> > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> > 8 | Timestamp (High) |
> > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> > 12 | Timestamp (Low) |
> > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> > 16 / /
> > / Options (variable) /
> > / /
> > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> > / /
> > / Packet Data /
> > / variable length, padded to 32 bits /
> > / /
> > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> > | Block Total Length |
> > +---------------------------------------------------------------+
> >
> > Captured Packet Length - Length of the packet
> >
> >
> > Options:
> > Name Code Length Multiple? Description
> > apb_opt_size 0x81 - no Total byte size of options
>
> If that field is present, does the option list need to end with an
> opt_endofopt option? There *shouldn't* be a need for it (in fact, there
> isn't a need for an opt_endofopt option in the current pcapng spec, as, for
> all the other blocks, the number of bytes of option can be calculated from
> the size of the block and the size - no need for a redundant terminator).
>
If this field is present there's no need for opt_endofopt.
>
> > apb_capt_len 0x82 - no Packet's Captured Length
>
> Wouldn't that be redundant, just as it is for the Simple Packet Block?
> You have to figure out how many bytes of option there are before you can
> find the beginning of the packet data and, at that point, the original size
> of the packet is min(snapshot length, bytes left in the block).
>
Fair point. This option can be dropped.
>
> BTW, presumably all these packets have arrived on interface 0, as there's
> no interface ID field.
>
Added apb_iface_id option above to signify the interface.
> > 4-11 Compression type
> > 0 = uncompressed
> > 1 = lzw
> > 2 = gzip
> > 3 = bzip2
> > 4 = zip
> > 5 = 7z
> > 6 = lzo
> > 7 = ucl
> > 8 = snappy
> > ...
>
> So that indicates the form of compression used on the packet data?
>
Right, to conserve space, pcap-ng writing application could choose to
compress packets. One thought here is that perhaps that should be a
file-level setting in the Interface Description Block rather than on the
per-packet basis as proposed here?
There's no
>
> FCS length field; do these packets have an FCS iff the "FCS length" field
> for interface 0's IDB is non-zero (so that all packets for that interface,
> whether they were transmitted by this host or received by this host, either
> have an FCS or don't have an FCS)?
>
Wouldn't that frame check sequence be part of the packet data?
Regards,
Serge
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.winpcap.org/pipermail/pcap-ng-format/attachments/20160813/f397f3a1/attachment.html>
More information about the pcap-ng-format
mailing list