[pcap-ng-format] Proposing new block type for PCAP-NG (UNCLASSIFIED)
Jasper Bongertz
jasper at packet-foo.com
Wed Apr 1 20:33:36 UTC 2015
> Classification: UNCLASSIFIED
> Caveats: NONE
>> Wireshark, at least, allows multiple capture files to be merged into a
>> single capture file. What happens if you take two captures, perhaps
>> from two different machines in different positions traveling over
>> different paths, taken over the same time period, and merge them?
> I assume this means merge them into the same section?
If this hasn't changed in the meantime: no, merging files with
multiple IDBs isn't possible at this time, at least not with
mergecap/Wireshark. You get an error message if you try, and I guess
it's because of the multiple IDBs. TraceWrangler can merge files with
multiple IDBs and offers three modes of how to merge them. See
https://www.tracewrangler.com/documentation/TraceWrangler.html?MergeSettings.html
The problem is that the merge process needs to determine if one or
more of the IDBs are the same interface, which is hard if all of them
are labeled "eth0" (unless their link type is different). If they are
the same they should be reduced to a single IDB in the new file, if
not, two IDBs need to be written.
When you're going to add a new block type that has the same potential
problem when merging files (like the new location provider block type
you proposed) I can heavily recommend adding a unique identifier
option to each block, like a GUID or something else. That way a merge
process can tell that two captures are taken on the same device and
the blocks can be merged into a single output block. This GUID should
at least be kept consistent for a capture task for when it spans
multiple files.
>> Perhaps, instead, we could treat position providers similarly to
>> interfaces, and have a "location provider information block" that gives
>> information such as time resolution - and perhaps other information,
>> such as source (GPS, GLONASS, Galileo, etc.), resolution, etc.? A
>> Location Information Block would have, instead of an interface ID, a
>> "location provider" ID, which would be the ordinal number of the
>> "location provider information block" within the section.
> I like this idea a lot! Put an optional origin reference in there, too.
It might be a good idea to keep the way the ID is assigned/used
consistent with the Interface IDs, meaning that the first location
provider block has the index of 0, the second is index 1, and so on.
That way no ID needs to be written to the location provider block, as
their order in the file IS the index.
Also, we should allow for such location provider blocks to appear
anywhere in the file as long as they're written before the first
location block referencing them (same as IDBs). This may sound like
making it more complex than necessary, but we had that problem before
with IDBs, where a PCAPng file was created from a text stream and the
interfaces would only be defined when first used (so you simply do not
know at the beginning of the file which IDBs you'd end up with).
>> A machine could have multiple location providers, including one per
>> interface, so that'd support both "my mobile computer with its GPS, Wi-
>> Fi, and mobile phone interface" and "my machine, gathering packets from
>> multiple probes moving independently".
> Should there be a way to tie a location provider to an interface identifier?
> Maybe as an option?
That should be simple. Add an option field for this to the location
provider block - maybe it comes in handy in some situations (e.g.
after merging when there's multiple IDB/location blocks and you need
to know which was originally used with which device and where)
>> Merging captures would mean assigning a new set of location provider
>> IDs and updating LIBs to use the new IDs.
> Assuming they are in the same section, yes, you would have to merge and
> resolve IDs for both interfaces and location providers.
No, as I proposed above, the location provider ID is determined by the
block order in the file, but you need to update the location blocks
(that's the same challenge as with IDBs when merging)
Merging requires:
1) determine list of unique blocks (e.g. merge identical blocks into a
single block)
2) write them to file and create a mapping of Old Index -> New Index
3) write location blocks with the old index remapped to the new one
>> If we went for per-packet location information, the option to give a
>> location report with a packet would include both a location provider ID
>> and location information.
> I would hope that per-packet location information would only be used on
> low-rate captures. If that's a safe assumption, then adding a full Location
> Block of 40-60 bytes isn't too much overhead.
>> Or, instead, an Interface Description Block could include a location
>> provider ID giving the location provider for that interface.
> See below.
> Here is a draft of how these blocks might look. This is intended as a
> reference for discussion and still needs work. Apologize for any line-wrap
> issues.
> -Ken
> =====================================================
> Two New Block Types
> 1. Location Provider Block [Optional]
> Establish information about the provider of location information. This
> Could be something such as a specific technology that is used for location
> information (e.g. GPS, 3GPP, INS), some reference to an origin. Zero or
> more LPBs are allowed per Section of a file. Location Providers should
> be stored in a table and referenced by Location Blocks (similar to how
> packets
> reference interfaces). Location Providers MUST be defined before they
> are referenced in Location Blocks.
> Provider ID: Tools that write / read the capture file associate a
> progressive 32-bit number (starting from '0') to each Location
> Provider Block. This number is unique within each Section and
> uniquely identifies the provider (inside the current section);
> therefore, two Sections can have providers identified by the same
> identifiers. This unique identifier is referenced by other blocks
> (e.g. Location Block) to point out the provider the block refers to
> (e.g. the source of the location information).
> 0 1 2 3
> 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
>
> +---------------------------------------------------------------+
> 0 | Block Type = 0x00000009
>
> +---------------------------------------------------------------+
> 4 | Block Total Length
>
> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> 8 | Source Type
>
> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> /
> / Options (variable)
> /
>
> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> | Block Total Length
>
> +---------------------------------------------------------------+
> o Source Types are an enumerated type with a defined list such as:
> 0 Undefined source
> 1 GPS
> 2 3GPP
> 3 INS
> [...]
> Option Name Code Length Description
> Example
> lp_device 2 variable A UTF-8 string containing
> "Garmin 430"
> a description of the
> measuring device.
> lp_origin 3 variable A UTF-8 string representing
> "Earth Center",
> a reference to an origin or
> some specific
> for XYX location information
> lat/lon/alt
> lp_name 4 variable A UTF-8 string containing
> "Vehicle X",
> a description of the
> location provider.
> lp_error_type 5 32-bit An enumeration of the type
> NUMBER_OF_SATELLITES,
> unsigned int of error described in location
> CONFIDENCE_PERCENTAGE,
> information blocks that
> ACCURACY_IN_METERS
> reference this location
> provider.
> lp_tsresol 6 1 byte Resolution of time stamps.
> If the Most Significant
> Bit is equal to zero, the
> remaining bits indicates
> the resolution of the
> timestamp as as a negative
> power of 10 (e.g. 6 means
> microsecond resolution,
> time stamps are the number
> of microseconds since
> 1/1/1970). If the Most
> Significant Bit is equal
> to one, the remaining
> bits indicates the
> resolution as a negative
> power of 2 (e.g. 10 means
> 1/1024 of second).
> 2. Location Block
> The Location Block provides some idea of where the capture is taking place.
> This could be a descriptive location such as "DMZ", "Server Room", or
> "Starbucks". It could also be a geographic location specified in some
> format.
> Examples of specific formats for geographic location are: "Lattitude-
> Longitude-Altitude", "Orientation" (pitch, yaw, roll), or "XYZ" (meters).
> Location Blocks reference a location provider that help define how to
> interpret the location data.
> 0 1 2 3
> 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
>
> +---------------------------------------------------------------+
> 0 | Block Type = 0x0000000A
>
> +---------------------------------------------------------------+
> 4 | Block Total Length
>
> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> 8 | Location Provider Identifier
>
> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> 12 | Timestamp High
>
> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> 16 | Timestamp Low
>
> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> /
> / Options (variable)
> /
>
> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> | Block Total Length
>
> +---------------------------------------------------------------+
> o Location Provider ID: it specifies the location provider this location
> information comes from; the correct location provider will be the
> one whose Interface Description Block (within the current Section of
> the file) is identified by the same number of this field. The
> Location Provider ID MUST be valid, which means that an matching
> Location Provider Block MUST exist.
> o Timestamp (High) and Timestamp (Low): high and low 32-bits of a
> 64-bit quantity representing the timestamp. The timestamp is a
> single 64-bit unsigned integer representing the number of units
> since 1/1/1970 00:00:00 UTC. The way to interpret this field is
> specified by the 'lp_tsresol' option of the Location Provider
> Block referenced by this packet. Please note that differently
> from the libpcap file format, timestamps are not saved as two
> 32-bit values accounting for the seconds and microseconds since
> 1/1/1970. They are saved as a single 64-bit quantity saved as
> two 32-bit words.
> Option Name Code Length Description
> Example
> lb_lla 2 12 bytes Three 32-bit signed integers
> 32786026, -96801908, 42
> representing the Latitude,
> represents 32.786026 degrees
> Longitude, and altitude of
> latitude, -96.801908 degrees
> the location. Latitude and
> longitude, and 42 meters
> Longitude are expressed as
> above MSL
> millionths of degrees
> (10^-6 degrees) and altitude
> is expressed as meters above
> mean sea level (MSL)
> lb_orientation 3 12 bytes Three 32-bit signed integers
> 45000000,115000000,-500000
> representing the pitch, yaw,
> represents 45.0 degrees pitch,
> and roll angles expressed in
> 114.0 degrees roll, and -0.5
> millionths of degrees
> degrees yaw.
> (10^-6 degrees). The
> reference plane for these
> measurements should be a
> tangent to the base surface,
> or the Z=0 (altitude=0)
> surface
> lb_xyz 4 12 bytes Three 32-bit signed integers
> 42, -166, 0 represents 42 meters
> representing the distance from
> from x=0 axis, -166 meters from
> the X, Y, and Z axis in meters.
> y=0 axis, and 0 meters from the
> A reference to the origin point
> z=0 axis.
> may be found in the location
> provider block.
> lb_description 5 variable A UTF-8 string containing
> "Server Room", "DMZ"
> a description of the location.
> Interpretation of this field
> is up to the applications that
> generates and use this field
> lb_hiprexyz 6 28 bytes A high-precision offset along 3,
> 42, -166, 0 represents
> the X, Y, and Z axis. It is a
> distance of 42km from the
> made up of a 32-bit signed
> x=0 axis, -166km from the
> value that represents a
> y=0 axis, and 0km from the
> resolution, followed by three
> z=0 axis
> 64-bit signed values
> representing the distance in
> the X, Y, and Z directions
> respectively. The resolution
> is a signed integer that
> specifies the power of 10 for
> the units of the X, Y, and Z
> values in meters. If the
> resolution value is -6, then
> the X, Y, and Z offsets are in
> micrometers from the origin.
> It the resolution value is 3,
> the X, Y, and Z offsets are in
> kilometers from the origin. A
> eeference to the origin point
> may be found in the location
> provider block.
> lb_error 7 variable A UTF-8 string containing
> some indication of the error
> tolerance expected from this
> location measurement.
> Interpretation of this field
> is dependent on the lp_error_type
> defined in the location provider
> block.
> Classification: UNCLASSIFIED
> Caveats: NONE
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3681 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://www.winpcap.org/pipermail/pcap-ng-format/attachments/20150401/12bcbb5d/attachment.bin>
More information about the pcap-ng-format
mailing list