[pcap-ng-format] sane maximum length for block_length

[Guy Harris]

On Sep 18, 2019, at 3:59 PM, Michael Richardson <mcr at sandelman.ca> wrote:

> It seems to be that numbers bigger than 1MB here for a *Section Header Block*
> should be more than enough.   If we knew the size of the file (it's not a
> pipe, and it's not coming from a stream de-compressor), then we could set
> another reasonable maximum.  In that case, we could mmap() the file and win
> much more.
> 
> I'm going to insert some sane value checks into the block_total_length.

Currently, Wireshark's pcapng reading code imposes a block size limit for all blocks:

/*          
 * In order to keep from trying to allocate large chunks of memory,
 * which could either fail or, even if it succeeds, chew up so much
 * address space or memory+backing store as not to leave room for
 * anything else, we impose upper limits on the size of blocks we're
 * willing to handle.
 *         
 * We pick a limit of an EPB with a maximum-sized D-Bus packet and 128 KiB
 * worth of options; we use the maximum D-Bus packet size as that's larger
 * than the maximum packet size for other link-layer types, and the maximum
 * packet size for other link-layer types is currently small enough that
 * the resulting block size would be less than the previous 16 MiB limit.
 */
#define MAX_BLOCK_SIZE (MIN_EPB_SIZE + WTAP_MAX_PACKET_SIZE_DBUS + 131072)

WTAP_MAX_PACKET_SIZE_DBUS is 16 MiB.

Wireshark also imposes file-type-independent maximum packet size limits (which are link-layer-type dependent, just as the limits in libpcap are).  Those apply only to packet blocks in pcapng.

> I'll put them clearly in a header, and they ought to be per block-type.
> Is it reasonable for the internet-draft to say something about each type?

Given that we may get complaints in the future that GigantoBus messages can be up to 1 GiB, so we need to increase the maximum packet size for LINKTYPE_GIGANTOBUS, we can't provide a maximum EPB/PB/SPB size that won't get in somebody's way in the future (other than the obvious 2^31-1 - if *that's* a problem, we'd have to introduce pcapng 2.0, with 64-bit block sizes and packet lengths, add new APIs, and, on 32-bit platforms, just say "we don't support reading RidiculoBus captures")

Perhaps we could, as per the above arbitrarily say "don't have more than 128 KiB of options" and "for now, don't have packets bigger than 16 GiB", with a warning that the maximum packet size is subject to change, and have recommended maxima based on the minimum block sizes and that.