[ntar-workers] Re: [Ethereal-dev] Re: NTAR - PCAP next generationdump file formatimplementation

[Gianluca Varenni]

Comments marked with --GV--

Have a nice day
GV

----- Original Message ----- 
From: "LEGO" <luis.ontanon at gmail.com>
To: "Gianluca Varenni" <gianluca.varenni at gmail.com>
Cc: "ronnie sahlberg" <ronniesahlberg at gmail.com>; "Ethereal development" 
<ethereal-dev at ethereal.com>; "ntar-workers" <ntar-workers at winpcap.org>
Sent: Monday, June 27, 2005 4:21 AM
Subject: [ntar-workers] Re: [Ethereal-dev] Re: NTAR - PCAP next 
generationdump file formatimplementation


I was not clear at all in my last mail so I'll re-expose myself.

Please note that I'll refer to "chunks" for segments of data
containing more records and to "records" to refer to atoms. I find the
term block confusing (in my forma-mentis a block seems something that
contains more records).

The essential point I tried to make is to have the marker-records at
predictable locations in the file.
That would be N*chunk_size bytes.

--GV--
That's clear.
--GV--


To archieve that we need:
A padding record to fill in the space between the last packet and the
location of the next marker record.
And, a marker record that tell us what we have so far.

--GV--
Well, I don't see why you cannot use the sections for the same purpose. You 
can jump between jumps easily, and we can easily add some information 
(either in the interface description block or by means of a sort of marker 
block) that tells number of packets/bytes, timestamps... In my opinion, this 
is one of the main uses of sections. Any other opinion about that?

Moreover, the idea of having a padding block is troublesome: by definition, 
the smallest block in pcap-ng is 12 bytes. If we create a padding block 
because we want to support these markers at fixed locations, we need to be 
*very* careful of where to put these padding blocks (i.e. we need to take 
care of having either 0 bytes left before a new marker, or >= 12 bytes, 
otherwise we are screwed up).

Finally, as I said before having these markers at fixed locations is a real 
pain in the neck when you have a file with multiple sections *and* the SHB 
does not contain the length of the section.

--GV--

Another point is not to have to seek backwards to fill in fields while
dumping. Neither I like the Idea of having to keep a whole chunk in
memory nor to have to keep more than few very essential state
variables.

--GV--
I totally agree with you.
--GV--

I think either the file header or the HSB should tell which chunk-size
we are using.
The whole dump-file should use one an only one chunk-size.

--GV--
No. A file contains multiple sections (possibly coming from multiple cat'ted 
files). So *if* we really want to use this chunks approach, the chunk size 
should be saved in the SHB.
--GV--


On 6/27/05, Gianluca Varenni <gianluca.varenni at gmail.com> wrote:
> In general, I prefer your solution over Ronnie's one: the basic idea of
> pcap-ng was to create not too much nesting in the file, so at the moment 
> we
> basically have two levels, sections and blocks. In the future it's
> possible/probable that we will have some nesting, especially to support
> compressed and cyphered blocks.

I like it that way. Nesting stuff would mean either to seek back to
write the length or to keep in memory a whole meta-record before
writing. I do not like either solution.

[snip]
> In my opinion the marker should contain the following informations
> - number of packets up to the next "marker block"
This is a forward reference
> - number of bytes up to the next "marker block"
my point is we should know this before even reading the first marker-record.
> - timestamp of the first and last packet up to the next "marker block"
I would put those before the marker record.
> - offset to the next "marker block"
In my scenario we know it beforehand
> - other??

_______________________________________________
ntar-workers mailing list
ntar-workers at winpcap.org
https://www.winpcap.org/mailman/listinfo/ntar-workers