[pcap-ng-format] "Hardware, OS, User application" - separate options for "what did the capture?" and "what's processed the file"?

[Anders Broman]

Skickat från min Sony Xperia™-smartphone

---- Guy Harris skrev ----

> On Jun 1, 2016, at 2:50 AM, Jasper Bongertz <jasper at packet-foo.com> wrote:
>
> >> Given the existence of tools that merge capture files, no guarantee
> >> can be made that any option in the Section Header Block will reflect
> >> "the machine on which the capture was done", so:
> >
> >> 1) programs shouldn't report the shb_hardware, shb_os, or
> >> shb_userappl options as pertaining to the machine on which the
> >> capture was done (yes, Wireshark does that, and, yes, it should stop doing so);
> >
> > Make sense to me; this is a change that requires increasing the minor
> > version number I guess. Minor, because it's just in the options, which
> > shouldn't break any reading process. But significant enough to
> > indicate a difference to 1.0.
>
> The spec, at least as far back as the version on the WinPcap site:
>
> https://www.winpcap.org/ntar/draft/PCAP-DumpFileFormat.html#sectionshb
>
> describes them as descriptions of the {hardware, OS, application} "used to create this section", which is somewhat ambiguous - does "create this section" refer to the creation of the very first pcapng file with the packets in the file, or the creation of the file that's currently being read?
>
> If it's the first of those, then:
>
> *if* the file was created by a capture program, they refer to the machine on which the capture was done;
>
> if the file was created by fuzzing a pcapng file, it's not clear whether it's appropriate to preserve that information, as that's not the data that was captured.
>
> if the file was created by otherwise processing a non-pcapng file without that information (simple conversion, filtering out packets, anonymizing) and writing a pcapng file (text2pcap, editcap), then it's impossible for them to refer to the machine on which the capture was done, so either we can't record that the file was created by the program in question or we need to interpret as "created by" as not necessarily meaning "captured by";
>
> if it was created *ab initio* (randpkt), it obviously can't refer to the machine on which the capture was done, as no capture was done;
>
> if the file was created by merging multiple pcapng files, there isn't necessary *a* machine on which the capture was done.
>
> If it's the second of those, then it obviously won't necessarily be information about the machine on which the packets were captured.
>
> Another idea I had:
>
> Have the options in question refer to the machine that "originated" the packets, with "originating" referring to capturing, or generating (e.g., randpkt), the packet data - this means that programs such as tcpdump (a future version that can write pcapng files), dumpcap, randpkt, etc. would be the only programs setting them, and text2pcap, editcap, etc. would *not* add or change those options.
>
> Have a "merged from" option, containing a set of "merge ID" values (which could be just integral indices), one for each of the files from which the merge is done.
>
> Have "merged-from identifier" options, corresponding to shb_hardware, shb_os, and shb_userappl, with a "merge ID" followed by a UTF-8 string.
>
> The merging program would add a "merged from" option and, for any shb_hardware, shb_os, and shb_userappl options found in the input files, write the string values out as shb_mergedfrom_hardware, shb_mergedfrom_os, and shb_mergedfrom_userappl options with the appropriate "merge ID" values.
>
> We could also have a "merge ID" option for the IDB, so that the merging program would, if it doesn't combine IDBs, tag the copied-over IDBs with the "merge ID" corresponding to the file from which the IDB was taken. If it *does* combine IDBs, it could give a combined IDB multiple "merge ID"s.
>
> With that, we wouldn't need the if_remote_ options - but we probably would want the if_hardware, if_interface_hw, and if_userappl options, all of which, along with if_os, refer to the machine to which the adapter is attached.
>
> >> if_interface_hw - a hardware-level description of the
> >> adapter on which the capture is being done (local adapter for a
> >> local machine, remote adapter for a remote machine).
> >
> > Sounds good to me - we should try to have a unique identifier per
> > interface (the MAC/hardware address?) so that merge routines can have
> > a chance to know that two interfaces in two different files/sections are
> > actually (or at least "most likely") one and the same. Otherwise we'll
> > end up with merged files containing sometimes hundreds of IDBs.
>
> Yes, programs should try to provide a MAC address, although:
>
> 1) there really need to be libpcap/WinPcap APIs to make this easier (it's on my to-do list);
>
> 2) not all interfaces have MAC addresses (such as PPP and loopback interfaces);
>
> 3) that raises the question of what to do if you have two interfaces with 00:fa:11:de:ad:00, one named "en0" and one named "en1" - presumably you don't merge them, even though they're probably the same interface, just renamed (this is unlikely on NT 5 and later, as they assign UUIDs to interfaces and the UUID is part of the interface name, but it could happen on UN*Xes if interfaces are added and removed over time, depending on how interface unit numbers are assigned).
>
> >> The current description of the SHB options in question speaks of
> >> them as being for the {hardware,OS,application} "used to create the
> >> section", which, arguably, means that, in a merged capture, it
> >> should refer to the hardware and OS on which the merging program was
> >> run and the merging application itself.
> >
> >> Whether editing a capture, by removing packets,
> >> adding/removing/changing comments, etc. should result in the SHB
> >> options being removed or changed is another question; my inclination is "no".
> >
> > I agree, they shouldn't change the SHB except it is intended to do
> > just that. Normal packet/comment editing shouldn't touch it.
>
> So what about changing packet *contents*, by fuzzing or anonymization?
>
> Should we have additional SHB options to handle processing of that sort?
>
> Should we also have options to handle creation of pcapng files from sources that don't include capture-machine information?
>
> Do we need, in those cases, to give anything other than the application doing the processing/creation? The hardware and OS would probably not be of interest to the user - they'd probably mainly be interesting only if somebody's trying to debug the writer of the file - so maybe we could just have shb_userappl equivalents for those.
>

At some point someone raised concerns about privacy issues with the pcap-ng options present and the ability to track a file back to the machine creating it. If not somthing for the specification at least somthing to think about when creating a program using pcap-ng. E.g add ability to turn off writing some options.

_______________________________________________
> pcap-ng-format mailing list
> pcap-ng-format at winpcap.org
> https://www.winpcap.org/mailman/listinfo/pcap-ng-format
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.winpcap.org/pipermail/pcap-ng-format/attachments/20160602/38c8ae10/attachment-0001.html>