[pcap-ng-format] Addition of new content

[Hadriel Kaplan]

On Wed, Sep 2, 2015 at 6:27 PM, Jasper Bongertz <jasper at packet-foo.com> wrote:
>
> so far I had been under the impression that we're going not going to
> add things to the specs but try to trim/streamline them to a state
> where we can push it to be an official "1.0" version.

Agreed, though fixing bugs in the spec might require new stuff, and we
shouldn't publish a 1.0 if we believe it has bugs for its originally
intended use-case: network packet captures.


> If we're going to add new block types like an "Interface Event Block"
> (IEB) (which makes sense to me) this seems not to be the way to go
> anymore, and I'm fine with it as long as it doesn't turn into a
> never-ending story (by keeping adding new "cool" stuff).

I don't personally care if IEB makes it into a 1.0 or later.

I did want to flesh it out to understand if it would affect the IDB
and EPBs - for example if the time resolution or offset stuff could be
changed by an IEB, then that's a big deal to our 1.0 doc... because
pcapng readers that don't understand this IEB block would ignore it,
but in so doing would have incorrect information about the subsequent
EPBs in the file. (i.e., you wouldn't be able to simply ignore the IEB
and still show the file correctly for the packet blocks you do know
about)


> There are a number of other block types currently not in the specs,
> e.g. the crypto blocks by Michael Haney, the blocks used by the Hone
> Project (who seem to have now a Windows port as well), and the SysDig
> block types. Question is, do we want to add those to the spec first,
> or keep what block types we have and aim for a "1.0" with them? That
> may also mean not adding a "Interface Event Block" as of now, if we're
> fair.

I think the crypto blocks/options, the Hone blocks/options, and the
SysDig blocks should each be documented separately. We should reserve
their numbers now, in the current doc, and add references where to
find more info.

The reason I think that is (1) to make this 1.0 doc simpler to read
and understand, and (2) to get this 1.0 doc done in reasonable time.

The 1.0 doc is really more about defining the pcapng format itself,
and its main use-case, than it is about "all known use-cases and
blocks/options".


> We may also chose to focus on finishing the spec with "normal network
> capture related" blocks, leaving out complex blocks used by Hone,
> Crypto and SysDig for now, but adding helpers like IEBs, which should
> be achievable in a reasonable time frame. I doubt that we can evaluate
> and agree on all the others as they're much more complicated and may
> lead to a spec that we have to heavily modify in later versions.

Yeah - I think adding the IEB or not is really the question. The
down-side with not adding it now is that we'd have to go write yet
another document to define the IEB. On the other hand, making us do
that might be a good exercise to see how we want to handle future
registrations of blocks/option codes.


> P.S: I'd like to add an option to all block types to add flags
> indicating that they've been edited, sanitized, sliced, reduced in
> size by content replacement, expanded in size by content replacement
> and maybe a few others. But it can also wait for a later version if we
> agree to streamline and not add stuff :-)

Is this just for TraceWrangler and Wireshark's tools to use?  If so,
you could just use a Custom Option with your own PEN.

-hadriel