[pcap-ng-format] Questions about the ISB statistic counter options

[Hadriel Kaplan]

On Sun, Sep 6, 2015 at 4:16 AM, Guy Harris <guy at alum.mit.edu> wrote:
> So, while checking to see which ISB statistics options correspond to which statistics various capture mechanisms can provide, some questions came up:
>
>         isb_filteraccept: "The isb_filteraccept option specifies the 64-bit unsigned integer number of packets accepted by filter starting from the beginning of the capture."
>
>         Does this include packets accepted but then dropped due to lack of capture buffer space?  I infer from the description of isb_usrdeliver that it does, but we should probably explicitly indicate that.

Yes, I assumed it was purely the number of packets that passed the
filter. If they got dropped later due to other reasons, they still
passed the filter and should be counted in this stat.

One would think there should also be a isb_iffilterdrop, to count how
many did *not* pass the filter, because I'm not sure subtraction of
other counters safely gets the value - is it 'isb_ifrecv - (isb_ifdrop
+ isb_filteraccept)'? Or is it 'isb_ifrecv - (isb_ifdrop +
isb_filteraccept + isb_osdrop)'? Wouldn't you have to know how/where
the filter was implemented to know which one?


>         isb_osdrop: "The isb_osdrop option specifies the 64-bit unsigned integer number of packets dropped by the operating system starting from the beginning of the capture."
>
>         What if packets were dropped by OS code *before* being handed to the capture mechanism?  If those are counted, most if not all packet capture mechanisms would be incapable of providing that statistic - those capture mechanisms *do* count packets *they* dropped because the capture buffer was too small, but don't count packets dropped by the OS before they even got to see them.

I have no idea what this counter means. The wording later in the doc
implies if_osdrop is somewhere above/after the filters, like maybe the
pipe between dumpcap and an application using it - like for example if
packets got dropped between dumpcap and tshark/wireshark?


>         isb_ifdrop: "The isb_ifdrop option specifies the 64-bit unsigned integer number of packets dropped by the interface due to lack of resources starting from the beginning of the capture."
>
>         Does this include "dropped because the interface overran the receive ring for the adapter" but not include, for example, "dropped because the CRC was bad"?  And does "resources" include packet capture mechanism buffers?

I assumed it was a low-layer adapter counter - i.e., rx buffer full
issues, but does not include drops by the capture mechanism. But I
guess there are NICs with hardware specifically made for the sole
purpose of capturing, for which this counter might be very confusing.

I would not expect CRC errors to count for this stat, since they are
not due to a lack of resources.


> The packet capture mechanisms out there that provide statistics provide some or all of:
>
>         the number of packets handed to the packet capture mechanism - there's no ISB statistic for that;
>
>         the number of those packets that passed the filter in that mechanism, *even if they were then discarded because there wasn't any space for them in the capture mechanism's buffers* - that sounds like isb_filteraccept;
>
>         the number of *those* packets that were discarded because there wasn't space for them in the capture mechanism's buffers - is that what isb_osdrop was intended to be?

I guess so?


> There's also
>
>         isb_usrdeliv: "The isb_usrdeliv option specifies the 64-bit unsigned integer number of packets delivered to the user            starting from the beginning of the capture.  The value contained in this field can be different from the value 'isb_filteraccept - isb_osdrop' because some packets could still be in the OS buffers when the capture ended."
>
> That would presumably maintained by libpcap/WinPcap itself, if by "the user" we mean "the application doing the capture" (rather than "delivered to the user" meaning "provided to userland code by the kernel").

More confusion - I had assumed it meant the number of packets that
made it all the way into the pcapng file.

There is also no stat for packets filtered due to something other than
a BPF filter, like due to a tshark/wireshark read filter for example.

-hadriel