[pcap-ng-format] TODO in pcap-ng specifications

[Jasper Bongertz]

On 25.07.2012 04:45, Guy Harris wrote:

>> opt_endofopt			only once, if at all
> 
> That's somewhat implicit - if opt_endofopt is the end of the
> options, there are no options after it, which means there are no
> opt_endofopt options after it.

Maybe my statement wasn't clear. I wrote it like this to include the
idea of not having to have the "opt_endofopt" option at all, because
the end of options can be deducted from the block content and header
lengths. So "if at all" means "if we chose to require it as an end of
option marker at all".

>> shb_hardware			multiple shb_os				multiple shb_userappl
>> multiple
> 
> That would require some way of determining, for instance of one of
> those options, which instances of the other options go along with
> it, if any.  If the capture program only supplied shb_hardware and
> shb_os, and the first program that processed the file after that
> only supplied shb_userappl, a naive program might think that was
> the application that captured the trace.  (Either that, or we
> should mandate that if any of those are present all should be
> present, but it might be tricky to get some of them on some
> platforms; I guess we could say "a zero-length string is OK, and it
> means "I have no clue"".)

Right, this is a little difficult. These are optional values, so
people will expect that they can be left out when writing files. I
could live with "shb_hardware" and "shb_os" being "once" only, but
"shb_userappl" is nice to have as "multiple". That way I can keep the
original "dumpcap" string and add a new string with the name of my
tool that I just used to write the modified file back to disk without
replacing the original string.

>> if_MACaddr			once if_EUIaddr			once
> 
> I think so - an interface might be set up to handle multiple
> *multicast* addresses, but I don't know whether any adapters can be
> given multiple individual addresses.

well, with IPv6 the interfaces will all have a solicited-node
multicast address assigned. I'm not sure if these (or any multicast
addresses) are of any interest in a capture file.

> (Note, BTW, that the table above assumes 8-space tabs; I suspect
> your table assumed 4-space tabs.  For better or worse, neither
> 4-space tabs nor 8-space tabs are a universal standard.)

It was just UltraEdit doing something when I pressed "tab" :-)

>> Text: "(TODO - It would be nice, to have a "invalid Interface ID"
>> defined, e.g. 0xFFFFFFFF"
>> 
>> I guess we can just say "Use 0xFFFFFFFF to signal an invalid
>> Interface ID" here.
> 
> If a Packet Block or Enhanced Packet Block has 0xFFFFFFFF as an
> interface ID, how is a program reading the file to determine what
> link-layer header type the packet has?  I'd say we should demand
> that all PBs and EPBs have a valid interface ID.

Agreed. This is probably one of the topics where we have to be strict.

>> Text: "Time zone for GMT support (TODO: specify better). 	TODO:
>> give a good example"
>> 
>> I'm not sure what the time zone is for, but I guess it can be
>> used to track in which time zone a trace was taken. That would
>> allow to adjust the UTC saved in the time stamps to the local
>> time of when the capture was made. I think the best way to
>> specify this would be to just note the offset from UTC in
>> minutes, like "UTC+60" (for a trace taking in Germany) or just
>> "60". Storing it in minutes instead of hours takes care of
>> offsets that are fractional, (e.g. Iran has a time zone of
>> UTC+4.5)
>> 
>> For daylight saving time I'd just write the UTC offset including
>> the additional minutes.
> 
> What if a change to or from DST/summer time takes place during the
> capture?  Handling that would require something such as an
> Olson/IANA time zone database:
> 
> http://www.iana.org/time-zones/
> 
> zone name (or a POSIX TZ specification), which would also specify
> the offset(s) from UTC.

Okay, timezone stuff is complicated... maybe someone can find a good
way to handle this? :-)

>> By the way, I just scanned the whole document to find the
>> specification that all timestamps have to be written in UTC and
>> didn't find it. Did I miss it, or is it really not specified at
>> the moment? If so, this is probably a critical topic to add,
>> isn't it?
> 
> Well, the EPB specification says
> 
>> Timestamp (High) and Timestamp (Low): high and low 32-bits of a
>> 64-bit quantity representing the timestamp. The timestamp is a
>> single 64-bit unsigned integer representing the number of units
>> since 1/1/1970. The way to interpret this field is specified by
>> the 'if_tsresol' option (see Figure 9) of the Interface
>> Description block referenced by this packet. Please note that
>> differently from the libpcap file format, timestamps are not
>> saved as two 32-bit values accounting for the seconds and
>> microseconds since 1/1/1970. They are saved as a single 64-bit
>> quantity saved as two 32-bit words.
> 
> but what they really presumably meant is "since 1/1/1970 00:00:00
> UTC".

Yes, I guess so. I think we should just clarify this in section 2.6.

> Of course, that then raises the question of whether that count is a
> strict count of seconds and fractions of a second, counting up by 1
> every second, or if the clock stops during a positive leap second
> and jumps by two over a negative leap second.  My strong
> inclination is not to address that question right now, given that
> people wanting to do calculations of elapsed time between packets
> would want "strict count of seconds" and people forced to implement
> packet capture on OSes either forced to do weird things during leap
> seconds by POSIX or by compatibility would want the former.  I
> might be sorely tempted to, in a future version of the spec:
> 
> add a new packet block type, wherein the time stamp is a strict
> count of seconds and fractions of a second since the capture
> started;
> 
> add an option to the SHB for "time the capture started", and
> perhaps specify that as year/month/day/hour/minute/second/fraction
> UTC (and duck as people ask whether, if the capture starts during a
> positive leap second, "second" can be 60 :-)), thus perhaps
> allowing us to punt on leap second hell (other than the
> aforementioned issue) *AND* not have to worry about what happens if
> the clock gets changed during the capture (for POSIX weenies, think
> of the packet block containing a CLOCK_MONOTONIC rather than
> CLOCK_REALTIME value), or, alternatively, use if_tsoffset for
> that.
> 
> (But don't get me started about POSIX and leap seconds and what
> "UTC" means....)

Okay, lets leave that to a future version of the spec then. This topic
needs more discussion I guess.

>> Text: "The filter (e.g. "capture only TCP traffic") used to
>> capture traffic. The first byte of the Option Data keeps a code
>> of the filter used (e.g. if this is a libpcap string, or BPF
>> bytecode, and more). More details about this format will be
>> presented in Appendix XXX (TODO). (TODO: better use different
>> options for different fields? e.g. if_filter_pcap, if_filter_bpf,
>> ...)"
>> 
>> Maybe this is something for someone who is more specialized in
>> the capture filter business. I'm not sure if we need different
>> fields for this.
> 
> I don't think so.  For one thing, what if you have more than one
> such option?  Is a program that cares about it required to assume
> that, say, an if_filter_bpf value is the result of compiling, on
> the machine on which the compilation was done, the if_filter_pcap
> value, or does it need to decide which of those filters was the one
> actually used?
> 
> We *should*, however, nail down the code for the first byte.

Okay, how about

Code 1 for libpcap string
Code 2 for BPF bytecode

do we need more at the moment? I am not that familiar with other
filter languages/strings.

>> 3.3 Enhanced Packet Block
>> 
>> Text: "(TODO: the text above uses "first bit", but shouldn't this
>> be "first byte"?!?)"
>> 
>> I think it is already fixed, so this TODO is deprecated. Or am I
>> missing something here?
> 
> It's not fixed in
> 
> http://www.winpcap.org/ntar/draft/PCAP-DumpFileFormat.html
> 
> Are you referring to a newer version in SVN?

No, you're right, I overlooked it apparently. It is still there, at
the end of the sentence. I will replace "bit" with "byte" then.



-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4484 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://www.winpcap.org/pipermail/pcap-ng-format/attachments/20120726/525c649a/attachment-0001.bin>