[pcap-ng-format] Need to figure out what to do with Experimental Blocks

[Michael Haney]

Wait, wait, hold everything.  I graduated in May with a Ph.D. in computer science, and have been moving and starting a new job and otherwise stressing out on life.  This summer I've been improving the prototype code and getting ready for a public release.  ...  So it's time to break radio silence. My dissertation was on privacy in network surveillance and 'lawful intercept' and is based entirely on pcapng.

I have developed a detailed spec for the Crypto Block, including multiple supported algorithms.  You're absolutely right about hashing for integrity. So I developed details for the Hash Option of the EPB, supporting multiple message digest algorithms.  Basically the body of the encrypted block decrypts to the full Enhanced Packet Block. And then checking the hash after decrypt is done to verify.  The hash is taken of all the EPB required fields and none of the options.  So the hash is an option itself, comments are okay, and option order doesn't matter. 

The real kicker is the way you specify the keys for the encrypted blocks.  Asymmetric crypto could be used, but I've implemented symmetric key because it's so much faster and lighter.  A serial number is used to reference the key for later decryption.  And I use a unique key for each network flow.

Thus I've also designed a Flow Block which gives the details (5-tuple for most packet types)  of each network session and uses a sort of serial number to identify the crypto blocks that go together in a network flow.  I stopped short of implementing all of IPFIX/Netflow but basically use the same techniques of wireshark's 'follow stream'.  

In short, packet capture process encrypts everything recorded, separate key for each flow, writes a flow block for each flow with a flow ID, hashes each packet and encrypts to a crypto block, and tags the block with its flow ID.  The  keys necessary to decrypt are protected in hardware and can be 'checked out' if some authorization is granted.  

This can all be done in software or done differently with a few compatible operational changes.  But the full design I intended to support law enforcement or corporate large-scale full packet capture and preserve greater levels of privacy requires a hardware approach.  I (and the University of Tulsa) have filed a patent for the hardware and am building prototypes.  But I always intended to release the pcapng specs and the software open source.  

Please excuse the typos and lack of detail.  You guys caught me on my tiny phone keyboard and my thumbs are cramping up.  This week I'll send much better write-ups with more detailed explanations of the components.  But don't drop the Ceypto Block just yet.  I'm more than happy to support pcapng as a 1.0 version RFC and intend to make that one of my research focus areas for my faculty appointment.  So I'm now getting paid to help out. :-) 

Nice to meet you all.

Regards,
Michael


On August 22, 2015 5:43:12 PM MDT, Jasper Bongertz <jasper at packet-foo.com> wrote:
>Sunday, August 23, 2015, 12:39:12 AM, Hadriel Kaplan wrote:
>
>> If we're going to publish this doc - either through the IETF, or even
>> independently create a "version 1.0" - the Experimental Blocks have
>to
>> go. Their formats and semantics are incomplete, they don't have
>> reserved codes, etc. They just take up space, and add confusion.
>
>I fully agree; I had proposed to do that last year during Sharkfest if
>I
>remember correctly; in the current state they are more like idea
>drafts, not really specs. AFAIK nobody has even tried to implement any
>of this as they're to unspecific.
>
>> For each Experimental Block, we need to decide to either:
>
>> 1. Complete it and make it no longer experimental, or...
>> 2. Remove it from the file and add it to this github repo's wiki
>pages
>> in a "Future Work" section or some such.
>
>We can maybe think about completing
>
>- Compression Block
>  - need to define compression algorithms and compression type byte
>  - Maybe there is an interest for this as it can reduce file size
>without having to gz the whole pcapng file
>- Wireshark/libpcap would have to support it to have any chance of
>being
>used
>
>- Crypto Block
>  - need to define encryption algorithms and encryption type byte id
>- Maybe there is an interest for this, e.g. for secure lawful
>interception
>  file storage
>- would also need some sort of hashing to prevent modification of the
>crypted data
>- Wireshark/libpcap would have to support it to have any chance of
>being
>used
>
>otherwise, push those two to "Future Work" and get 1.0 done with the
>block types we already have (and know to be working).
>
>I'd vote to remove/"Future work":
>
>- Fixed Length Block
>  - I'm not sure who/what may benefit from this; maybe some
>specialized hardware capture device that has a file system that needs
>this?
>
>- Directory Block
>  - needs to be defined; maybe in a later RFC if use case comes up
>
>- Traffic Statistics and Monitoring Block
>  - needs to be defined; maybe in a later RFC if use case comes up
>
>- Event/Security Block
>  - needs to be defined; maybe in a later RFC if use case comes up
>
>That way we'd get a clean specification with block types that we know
>are already implemented and working (if maybe not to the last little
>detail). We can always add more block types later in another RFC (or
>newer document version).
>
>Just may 2 cents :-)
>
>Cheers,
>Jasper
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>pcap-ng-format mailing list
>pcap-ng-format at winpcap.org
>https://www.winpcap.org/mailman/listinfo/pcap-ng-format
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.winpcap.org/pipermail/pcap-ng-format/attachments/20150822/8c90dd78/attachment.html>