[ntar-workers] Re: [Ethereal-dev] Re: NTAR - PCAP next generationdump file formatimplementation

[Gianluca Varenni]

----- Original Message ----- 
From: "ronnie sahlberg" <ronniesahlberg at gmail.com>
To: "Michael Richardson" <mcr at sandelman.ottawa.on.ca>
Cc: "LEGO" <luis.ontanon at gmail.com>; "Ethereal development" 
<ethereal-dev at ethereal.com>; <ntar-workers at winpcap.org>
Sent: Sunday, June 26, 2005 8:27 PM
Subject: Re: [ntar-workers] Re: [Ethereal-dev] Re: NTAR - PCAP next 
generationdump file formatimplementation


>>   Can any of these things be written without seeking backwards in the
>> file? Can we make sure that the value 0, or some other nonsense value
>> can be inserted to mean, "I didn't calculate that", and let a
>> post-processor worry about fixing the actual values in.
>>
>
> I agree with the implied meaning above that a no seeking backward
> solution is absolutely required so that one can pass a capture through
> a pipe.

Yes, of course.

At the moment NTAR fully supports reading and writing from stdin/stdout. As 
a matter of facts, the only difference (apart from some settings to putt 
stdin/stdout in binary mode on windows) is the support (or lack of it) for 
seek operation. And at the moment backward seeks are actually  disabled even 
on normal files, due to the hell of having a portable fseek working on large 
files. I'm working on it...

>
>
> Number of bytes until the next marker should be possible to implement
> without seeking in the file. Just make a new marker once every 1Mbyte
> into the file   and if there is not enough space left to fit the next
> packet to write, just add some padding data until the next marker.

Uh, we should think of where to put such padding. pcap-ng does not specify 
any additional "random" padding (apart from the one used to align everything 
to 32bits).

In general, I would prefer not to have such "hacks" in the file format, as 
much as I'd like not to have these marker blocks required. The application 
should be  We already have the interface description block that is needed in 
every section...

>
> Then one marker would always know that the next marker starts 1 MB
> from now and no seeking of the file required.
>
> The number of packets until next marker and the timestamps can be left
> as zero   indicating "i didnt calculate that" and be left for a
> postprocessing tool to calculate if required.
>
> If the bytes until next marker points to beyong the end of the file
> this would just mean : no more markers.

Uhm... I'm not sure this will work: a file can contain multiple sections, 
either because the app writing the file created multiple sections, or simply 
because the user cat'ted two trace files together. A "blind" jump of 1MB 
ahead will result in an error (we will jump somewhere in the next section). 
A solution is to compute the effective length of "marked" packet blocks 
making use of the section length (this length is saved in the section header 
block), but this approach does not work all the times, since this length is 
written on file only if the underlying file descriptor support backward 
seeks (i.e. it will not work with pipes).

Have a nice day
GV



>
>
>
> On 6/27/05, Michael Richardson <mcr at sandelman.ottawa.on.ca> wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>>
>>
>> >>>>> "Gianluca" == Gianluca Varenni <gianluca.varenni at gmail.com> writes:
>>     Gianluca> In my opinion the marker should contain the following 
>> informations
>>     Gianluca> - number of packets up to the next "marker block"
>>     Gianluca> - number of bytes up to the next "marker block"
>>     Gianluca> - timestamp of the first and last packet up to the next 
>> "marker block"
>>     Gianluca> - offset to the next "marker block"
>>     Gianluca> - other??
>>
>>   Can any of these things be written without seeking backwards in the
>> file? Can we make sure that the value 0, or some other nonsense value
>> can be inserted to mean, "I didn't calculate that", and let a
>> post-processor worry about fixing the actual values in.
>>
>> - --
>> ] Michael Richardson          Xelerance Corporation, Ottawa, ON | 
>> firewalls  [
>> ] mcr @ xelerance.com           Now doing IPsec training, see   |net 
>> architect[
>> ] http://www.sandelman.ca/mcr/    www.xelerance.com/training/   |device 
>> driver[
>> ]                    I'm a dad: http://www.sandelman.ca/lrmr/ 
>> [
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG v1.2.2 (GNU/Linux)
>> Comment: Finger me for keys
>>
>> iQCVAwUBQr9mF4qHRg3pndX9AQFV4QQA1bLwRWvpq5MfRt0Tucppcc4w5iDZDL1f
>> amC/yf5PDyDqVV/0jHld4Xl9j2yeUyTSN+tfZzj2GeAdLMHh7LIpHI+3R9pZfwCv
>> nXm3muzNMdiDfCwVZZjyJH+/jCGsAohDZ2zMrsI2vcNUz7JU8jS4jhDSQRKQeVhl
>> M0YBD5KJK+M=
>> =2NZR
>> -----END PGP SIGNATURE-----
>>
>
> _______________________________________________
> ntar-workers mailing list
> ntar-workers at winpcap.org
> https://www.winpcap.org/mailman/listinfo/ntar-workers