[pcap-ng-format] Proposal for storing decryption secrets in a pcapng block

[Anders Broman]

-----Original Message-----
From: pcap-ng-format <pcap-ng-format-bounces at winpcap.org> On Behalf Of Jasper Bongertz
Sent: den 5 oktober 2018 12:25
To: Pcap-ng file format <pcap-ng-format at winpcap.org>
Subject: Re: [pcap-ng-format] Proposal for storing decryption secrets in a pcapng block

Friday, October 5, 2018, 11:01:58 AM, Anders Broman wrote:
> -----Original Message-----
> From: pcap-ng-format <pcap-ng-format-bounces at winpcap.org> On Behalf Of 
> Guy Harris
> Sent: den 5 oktober 2018 08:11
> To: Pcap-ng file format <pcap-ng-format at winpcap.org>
> Cc: Developer support list for Wireshark <wireshark-dev at wireshark.org>
> Subject: Re: [pcap-ng-format] Proposal for storing decryption secrets 
> in a pcapng block

> On Sep 30, 2018, at 10:47 AM, Peter Wu <peter at lekensteyn.nl> wrote:

>> Requirements for block placement:
>> - No requirement. Producers are allowed to write the block anywhere.
>>  Disadvantages for consumers: requires a two-pass scan to collect 
>> secrets before they are used.
>> - Place secrets before the packet blocks that require them. Consumers 
>> can read and decrypt in one pass. Disadvantage: producers cannot 
>> always guarantee availability of secrets while writing the capture.
>> - Place a single secret block before the first packet block. 
>> Consumers can read and decrypt in one pass. Disadvantage: requires 
>> producers to post-process (rewrite) the capture file to insert secrets.

>>The third of those appears to be a special case of the second of those.  I don't see any need to require the secrets to be before the *first* packet block if the first packet block doesn't require the secret; presumably >"before the packet blocks that require them" just means "*somewhere* before the packet blocks that require them", which is *allowed* to be "before all packet blocks in the file" but not *required* to be "before all >packet blocks in the file".
>>
>>If the secret isn't available by the time the first packet requiring the secret for decryption is ready to be written to the capture, *somebody* will have to do some form of two-pass processing.
>>
>>The first option says the consumer must do so; that's inconvenient for a consumer doing one-pass processing (tcpdump, TShark without the -2 option), and isn't even really good for at least some consumers doing two->pass processing (Wireshark, TShark with the -2 option), because dissection is done on the first pass.

>>The second and third option require either the producer, or some post-processor, to write a new version of the file putting the secrets before the packets that require them.  The producer isn't necessarily responsible >for doing so; one might have tcpdump, or dumpcap (or some program using dumpcap, such as TShark or Wireshark) write out a capture with no secrets, and then have another program (a utility, or Wireshark after >having read in the file and then given the secret in question) write out a new file with the secrets early enough in the file ("before all the packet blocks" is probably the simplest implementation).
>>
>>A producer that *does* happen to have the secret available before seeing any packets that require the secret *could* write it directly.

> For the sake of the file format as such would it make sense to say 
> that the decryption block can occur at any place in the file but if 
> that's the case all file readers may not be able to do decryption 
> without first Rearranging the file.

>I concur. My strategy would be like this:
>- try to write the decryption block right before the first packet block so that it's easily found when scanning the header blocks (SDB, IDB, etc.)
>- if that's not possible, try to write it before any packet block that requires it
>- if that's still not possible, write it whenever you have the information, even if it's the last block of the file

>Thinking of that - since pcap-ng offers the wonderful option of reading blocks backwards we could put them all at the end. That way any tool could read them easily and has the information ready when starting to read >packet blocks from the beginning. The downside is that if the capture crashes the blocks are never written, so maybe it's not that cool...

"... since pcap-ng offers the wonderful option of reading blocks backwards we could put them all at the end."
Perhaps not such a good idea in this case as stated above, but interesting. I noted that a capture I have, has the address resolution block at the end which makes it unusable as the resolved
Addresses are written on the first pass(?), perhaps we should peek at the end of the file to see if there's something there...

In addition to be able to write the decryption block at any place in the file I think it should contain a TLV type structure

Tag = Type of "secret" 32 bits. A number allocated by the pcap-ng format responsible on a first come first served basis
Value = "The secret formatted as specified by tag"
Length = The length of the secret data
Pad to 32 bit boundary.

In this way we can calculate if options are present or not.

Regards
Anders

Regards
Anders
_______________________________________________
pcap-ng-format mailing list
pcap-ng-format at winpcap.org
https://www.winpcap.org/mailman/listinfo/pcap-ng-format