[pcap-ng-format] Questions about the ISB statistic counter options
Guy Harris
guy at alum.mit.edu
Mon Sep 7 03:09:22 UTC 2015
On Sep 6, 2015, at 3:36 PM, Hadriel Kaplan <the.real.hadriel at gmail.com> wrote:
> On Sun, Sep 6, 2015 at 4:16 AM, Guy Harris <guy at alum.mit.edu> wrote:
>
>> isb_filteraccept: "The isb_filteraccept option specifies the 64-bit unsigned integer number of packets accepted by filter starting from the beginning of the capture."
>>
>> Does this include packets accepted but then dropped due to lack of capture buffer space? I infer from the description of isb_usrdeliver that it does, but we should probably explicitly indicate that.
>
> Yes, I assumed it was purely the number of packets that passed the
> filter. If they got dropped later due to other reasons, they still
> passed the filter and should be counted in this stat.
>
> One would think there should also be a isb_iffilterdrop, to count how
> many did *not* pass the filter, because I'm not sure subtraction of
> other counters safely gets the value - is it 'isb_ifrecv - (isb_ifdrop
> + isb_filteraccept)'? Or is it 'isb_ifrecv - (isb_ifdrop +
> isb_filteraccept + isb_osdrop)'? Wouldn't you have to know how/where
> the filter was implemented to know which one?
The standard BPF pcap_stat structure has (from the OS X BPF man page):
BIOCGSTATS (struct bpf_stat) Returns the following structure of
packet statistics:
struct bpf_stat {
u_int bs_recv; /* number of packets received */
u_int bs_drop; /* number of packets dropped */
};
The fields are:
bs_recv the number of packets received by the
descriptor since opened or reset (including
any buffered since the last read call); and
bs_drop the number of packets which were accepted by
the filter but dropped by the kernel because
of buffer overflows (i.e., the application's
reads aren't keeping up with the packet
traffic).
NetBSD and WinPcap both add to their pcap_stat structure a bs_capt field (from the NetBSD man page):
BIOCGSTATS (struct bpf_stat) Returns the following structure of
packet statistics:
struct bpf_stat {
uint64_t bs_recv;
uint64_t bs_drop;
uint64_t bs_capt;
uint64_t bs_padding[13];
};
The fields are:
bs_recv the number of packets received by the
descriptor since opened or reset (including
any buffered since the last read call);
bs_drop the number of packets which were accepted by
the filter but dropped by the kernel because
of buffer overflows (i.e., the application's
reads aren't keeping up with the packet
traffic); and
bs_capt the number of packets accepted by the filter.
(WinPcap also adds bs_capt, but the structure is different. bs_capt and bs_drop are also handled a bit inconsistently, in that there are ways in which the count of dropped packets can be incremented for an accepted-by-the-filter *without* the count of packets accepted by the filter being incremented; that should be cleaned up.)
With bs_capt, the count of those dropped by the filter is bs_recv - bs_capt, i.e. the total number of packets delivered to the BPF instance (that's what "received by the descriptor" means here) minus the number of packets that passed the filter.
We could add either isb_filterdrop, with the number of packets delivered to the BPF instance being the sum of that and isb_filteraccept, or isb_filterreceive, with the number of packets dropped by the filter being the difference between isb_filterreceive and isb_filteraccept.
If we provide isb_filterreceive, we at least have a way of recording bs_recv on systems that lack bs_capt (i.e., all BSD-flavored OSes, including OS X and iOS, except for NetBSD), so my inclination is to provide isb_filterreceive and isb_filteraccept.
(I should probably push for the other *BSDs and OS X/iOS to somehow provide bs_capt in a way that satisfies their source and binary compatibility requirements; that might be more work for OS X than for other platforms.)
Or is isb_ifrecv: "The isb_ifrecv option specifies the 64-bit unsigned integer number of packets received from the physical interface starting from the beginning of the capture." supposed to count the packets received from the physical interface *and making it all the way to the capture mechanism's filter*, so
Linux provides tp_packets, which is isb_filteraccept and thus equivalent to bs_capt, and tp_drops, is the number of packets accepted by the filter but dropped due to insufficient buffer space being available, i.e. it's equivalent to bs_drop. It doesn't provide an equivalent to bs_recv, so, on Linux, we'd supply isb_filteraccept but neither isb_filterdrop nor isb_filterreceive, whichever of those we add.
>> isb_osdrop: "The isb_osdrop option specifies the 64-bit unsigned integer number of packets dropped by the operating system starting from the beginning of the capture."
>>
>> What if packets were dropped by OS code *before* being handed to the capture mechanism? If those are counted, most if not all packet capture mechanisms would be incapable of providing that statistic - those capture mechanisms *do* count packets *they* dropped because the capture buffer was too small, but don't count packets dropped by the OS before they even got to see them.
>
> I have no idea what this counter means. The wording later in the doc
> implies if_osdrop is somewhere above/after the filters, like maybe the
> pipe between dumpcap and an application using it - like for example if
> packets got dropped between dumpcap and tshark/wireshark?
If you're referring to
isb_usrdeliv: "The isb_usrdeliv option specifies the 64-bit unsigned integer number of packets delivered to the user starting from the beginning of the capture. The value contained in this field can be different from the value 'isb_filteraccept - isb_osdrop' because some packets could still be in the OS buffers when the capture ended."
I *suspect* that what "OS buffers" is intended to refer to is the buffer for the in-kernel capture mechanism. A packet that's handed to the capture mechanism, passes the filter, and isn't dropped due to lack of buffer space is put into the capture mechanism's buffers. The user-mode code using the mechanism, e.g. libpcap, extracts packets from that buffer either by doing read()/recv()/etc. system calls or by sharing a memory-mapped copy of the buffer. I'm guessing that the packets that "could still be in the OS buffers" refers to packets that are in the capture mechanism's buffers but that haven't yet been extracted and processed by the user-mode code.
If so, then the notion that you subtract isb_osdrop from isb_filteraccept suggests that isb_osdrop refers to packets that have been accepted by the filter but dropped after that for some reason, so it sounds like bs_drop. Perhaps its description should explicitly note that it refers only to packets dropped after being accepted by the filter, not dropped before being handed to the filter.
>> isb_ifdrop: "The isb_ifdrop option specifies the 64-bit unsigned integer number of packets dropped by the interface due to lack of resources starting from the beginning of the capture."
>>
>> Does this include "dropped because the interface overran the receive ring for the adapter" but not include, for example, "dropped because the CRC was bad"? And does "resources" include packet capture mechanism buffers?
>
> I assumed it was a low-layer adapter counter - i.e., rx buffer full
> issues, but does not include drops by the capture mechanism. But I
> guess there are NICs with hardware specifically made for the sole
> purpose of capturing, for which this counter might be very confusing.
>
> I would not expect CRC errors to count for this stat, since they are
> not due to a lack of resources.
For what it's worth, in libpcap 0.4 (the last release from LBL), the only platform on which ps_ifdrop was set was DEC OSF/1, sorry, Digital UNIX, sorry, Tru64 UNIX, which delivers, for each packet:
ens_dropped
The number of packets accepted by this filter but dropped because the
input queue was full; this is a cumulative count since the previous
enstamp was read from this packet filter file. This count may be
completely wrong if the ENNONEXCL mode bit is set for this filter.
ens_ifoverflows
The total number of input overflows reported by the network interface
since the system was booted.
and ps_ifdrop reflects the input overflows, so that presumably matches "packets dropped by the interface due to lack of resources", with "resources" being something *other* than capture mechanism buffer space, as that's counted by end_dropped.
Linux also now gets ps_ifdrop from /proc/dev/net's "drops" column, which, from looking at a couple of drivers, appears to be for problems such as "couldn't allocate an skb", so that's also "lack of resources".
We should probably make it clear that isb_ifdrop refers to packets dropped, by the adapter or its driver, *before* being handed to the capture mechanism and its filter for reasons *other* than PHY-layer errors (for example, not CRC errors). ("By the adapter or its driver" means that if layers of the networking stack *between* the driver and the capture mechanism drop it, that wouldn't count.)
>> The packet capture mechanisms out there that provide statistics provide some or all of:
>>
>> the number of packets handed to the packet capture mechanism - there's no ISB statistic for that;
>>
>> the number of those packets that passed the filter in that mechanism, *even if they were then discarded because there wasn't any space for them in the capture mechanism's buffers* - that sounds like isb_filteraccept;
>>
>> the number of *those* packets that were discarded because there wasn't space for them in the capture mechanism's buffers - is that what isb_osdrop was intended to be?
>
> I guess so?
Yeah, it looks that way.
>> There's also
>>
>> isb_usrdeliv: "The isb_usrdeliv option specifies the 64-bit unsigned integer number of packets delivered to the user starting from the beginning of the capture. The value contained in this field can be different from the value 'isb_filteraccept - isb_osdrop' because some packets could still be in the OS buffers when the capture ended."
>>
>> That would presumably maintained by libpcap/WinPcap itself, if by "the user" we mean "the application doing the capture" (rather than "delivered to the user" meaning "provided to userland code by the kernel").
>
> More confusion - I had assumed it meant the number of packets that
> made it all the way into the pcapng file.
It would count up by 1 for every packet delivered by libpcap/WinPcap to the application; if that doesn't make it to the pcapng file, that might be a result of:
> There is also no stat for packets filtered due to something other than
> a BPF filter, like due to a tshark/wireshark read filter for example.
filters such as that, for example.
More information about the pcap-ng-format
mailing list