[pcap-ng-format] Comments for unknown blocks [was Re: opsarea presentation?]

Sun Jul 27 18:32:20 UTC 2014

On 27 Jul 2014, at 19:39, Marc Petit-Huguenin <marc at petit-huguenin.org> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
> 
> On 07/27/2014 10:38 AM, Jasper Bongertz wrote:
>> Sunday, July 27, 2014, 2:35:14 PM, Marc Petit-Huguenin wrote:
>> 
>>> On 07/23/2014 11:21 AM, Michael Tuexen wrote:
>>>> On 23 Jul 2014, at 13:16, Michael Richardson <mcr at sandelman.ca> wrote:
>>>> 
>>>>> 
>>>>> So, was pcap-ng well receives by opsarea WG this morning?
>>>> There were a couple of people willing to review and contribute. I also
>>>> got or will get contacts within Apple, Google, Microsoft and hopefully
>>>> others. The WG chairs will talk to the AD if it is OK to work on that
>>>> within the OPSAWG. I guess we mostly need concrete names of people who
>>>> are willing to (and actually will) contribute and review. And then 
>>>> move the discussion to the WG mailing list.
>>>> 
>> 
>>> I was one of the persons who volunteered on reviewing this draft.
>> 
>>> One issue I found in the pcapng format is that it is not possible to add
>>> a comment (which is generic to all block formats) in an unknown block 
>>> because there is no way to know where the options starts without knowing
>>> the size of the content of a block.  A related problem is that it is not
>>> possible to allocate new generic options, as all specific options starts
>>> at codepoint 2.
>> 
>> Good  points, thanks for reviewing!
>> 
>> My point of view would be that there should not be any "unknown block" 
>> types, meaning that if you want to add a new block type to PCAPng you need
>> to officially specify it.
> 
> I am already using new blocks (with bit 31 set to 1) in my application, and
> have no plan to specify them. It would still be great to be able to annotate
> them with comments in an application that does not support my extensions.
How would an application do that if it skipped it while reading the file?
Hmm. Would it make sense to allow registering them with specifying them?
Without this, several people can use the same block types for different
purposes...
> 
> If you want to go that way, you will need to either remove the possibility to
> use private blocks types, or remove the possibility to write generic options
> in private blocks.
Why can't you use a generic option type on the unknown block. The option is
just skipped, being part of something which is not understood by the reader...
> 
>> Also, any program reading the file would probably not care if an unknown 
>> block type contains a comment, as it cannot read the block anyway. So just
>> being able to pull a comment from an otherwise unreadable block is not 
>> that important I think (but  I  can  see  the  point of wanting to be able 
>> to). To be able to allow  reading generic comments for otherwise unknown 
>> blocks we'd need to add a start-of-options offset indicator I guess.
> 
> Here's the type of comments that I would like to be able to write:
> 
> "I do not know what this block contains, but it seems to disclose private
> information"
> 
> "TODO: try to track what application is writing this block"
This kind of option could be at the file level, as a comment in the SHB.
This way it could be understood by all programs reading that file. And if
the application writing this file is really good, it puts its name as
an shb_userappl option in the SHB block. At least dumpcap does including
its version.
> 
>> 
>> Regarding the ability to add generic options - there's two ways to deal 
>> with this I can think of in a heartbeat:
>> 
>> 1. renumber specific option IDs to leave some room between generic and 
>> specific options (which would require code changes for all existing 
>> implementations)
> 
> That would require a new pcapng version.
> 
>> 
>> 2. make all further options non generic, even if they are useful for all 
>> block types - which would mean that all block types have them specified as
>> if unique for that block. I can't think of any right now, but that doesn't
>> mean that there could be some.
> 
> I have a need for an options that is generic in my application (generic here
> means that the content of the block does not matter to the content of the
> option, like for the comment option). It probably does not matter yet because
> I allocated it in the private range (MSB set) but that means that I cannot
> request a generic one.
I really don't like the idea of having only a limited fixed set of generic
ones... We could use the range with the highest bits 01. This would require
us to register new values for opt_endofopt and opt_comment, but we could move
the opt values 0 and 1 in all individual registries. That would not break existing
code and gives ranges for generic and specific options. We could do the same for
the MSB 1.

Best regards
Michael
> 
> - -- 
> Marc Petit-Huguenin
> Email: marc at petit-huguenin.org
> Blog: http://blog.marc.petit-huguenin.org
> Profile: http://www.linkedin.com/in/petithug
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1
> 
> iQIcBAEBCAAGBQJT1Tk/AAoJECnERZXWan7Emj0P/0iZVPKkgRblI2V6I/x30mT3
> 7+eXL+wm4C42sgnQON9Vnu9wQP4N15I4BlUwH50vxKmZPpcrgNkmHTN99GSyvTXJ
> WZ9KsS3Ds0YMzqZPVEFMgRblJRz6+lAvPPXe3eNaepU1BE6NhSlPdYSOC1zJeKmf
> T8x5Sv0B4SJnTKAo3bomqF3Rftse7VGIBB7aXsgxG/GCqaYCaNbzrzCcIzpUNG0b
> l2I/3v+ZNlpQ/icdFhE/fu6eiKu96Am3iEf8ZYUYwE1g+CMet2IsmQjiZYPP53lr
> sj7NEI0QolMXtCKiW8FMWOk4y3U1W8W0R+ykfveC+oopYX8A+F6lLn7WlZW5xtym
> BSzeJe9N4hLtcISs9LWlU9DgU5QTKKS60wMSYc+H5t5UWAGYXgApQ8pwDwXyDxXF
> FIlFT/iPrBVkJwFJnxCEDBxMj/+yNWTlzWIpRqaHpZRlYO/QdQbtsOWRUvHbzYpH
> F5RwlOTyxL+xainl4gVr29W2gaE6n0T0fLG4KeiEl3sgkPk4A8A7VOXtuSMUoGBJ
> E4AD0xfv2WZNvFsXsFfY78pjIqO52DeSm+2qiPxqXYGDktd9BL86pjVTDFhxD2FL
> 7nbvPYk1GRU2ujVbeYYDdY6PfmW+zQs3YGuWZuRaAV46WvgU1qK09FoPrutXa0Ic
> SZW39Ys6487MeVWRGWz9
> =wBq6
> -----END PGP SIGNATURE-----
> _______________________________________________
> pcap-ng-format mailing list
> pcap-ng-format at winpcap.org
> https://www.winpcap.org/mailman/listinfo/pcap-ng-format
> 