[Winpcap-users] Circular buffer
gianluca.varenni at cacetech.com
Fri Jun 23 18:13:17 GMT 2006
----- Original Message -----
From: "Ioan Popescu" <ipopescu at dataq.com>
To: <winpcap-users at winpcap.org>
Sent: Friday, June 23, 2006 10:09 AM
Subject: Re: [Winpcap-users] Circular buffer
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA512
> Loris Degioanni wrote:
>> 2. According to the documentation, the circular buffer should
>> overwrite the
>> older packets with the latest "wire" packets while at the same time
>> the oldest available packets to the application. Is my understanding
>>> The driver overwrites a packet only after it's been "consumed" by the
>>> application. If the application is stuck, the driver starts dropping
>>> packets when it reaches the tail of the buffer.
> So these "purposely" dropped packets are reported in the pcap_stat
> structure? Is there a way to reset these stats? Besides closing and
> reopening the adapter. Although, I could keep track of when I check it and
> simply subtract off what I consider "handled".
>> 3. I have noticed something peculiar related to kernel memory size and
>> usage. As I increase the kernel (and user) buffer, the % CPU usage of my
>> test application goes up. I'm testing this using Task Manager. The test
>> application stays the same, it uses a command line argument to set the
>> memory size. It also works on the same set of data. What reason's might
>> there be for such behavior?
>>> How big is the buffer that you set?
>>> The driver allocates this buffer from the nonpaged memory pool, and
>>> setting it to a very big size could impact on the OS performance. Other
>>> than that, the only reason why changing the size of the buffers could
>>> slow your application down is caching issues.
> The effect is shown over a range of sizes. I've tried values from 100,000
> 50,000,000. The effect is "skewed" depending on the machine, but it's
> there. My first thought was that more processing needed to be done to
> "handle" the larger amount of memory, but I've not yet come up with any
> ideas as to what might cause such a thing.
The driver does not do any more processing for bigger buffers: apart from
some other details of no importance here, we basically just store a pointer
to the buffer, and two counters of head and tail in the buffer. It doesn't
matter much if the buffer is big or small.
As Loris said changing buffer size could cause performance issues because of
caching. There's also another thing that should be taken into consideration:
the amount of data transferred from the kernel buffer to the user buffer at
every ReadFile system call. If the buffer is too small, you will probably
have a higher packet loss. After a certain threshold, you will probably stop
losing packets, and if your application processes packets at a quite
constant rate, the amount of bytes transferred in the read operation is more
or less constant, so at user level you will always use the same bytes in the
user buffer. In the kernel, since the buffer is circular, you will continue
using all the kernel buffer bytes in a sequential way, so in this case
caching can be the bottleneck. Last but not least, the value of mintocopy
and timeout you are currently using can affect the overall performance.
This is just to give you an idea of all the possible reasons why changing
the buffer size can impact on performance. And as you've seen, a bigger
buffer doesn't mean better performance at all...
Have a nice day
> I understand your reasoning for large buffers, but why is the effect
> noticeable even at smaller ranges? I'll try to see if I can come up with a
> good size at which an "outlier" occurs.
> Is there any reason for me to test the kernel and user buffers
> independently? Up until now, I've set both of them to the same values.
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v188.8.131.52 (MingW32)
> -----END PGP SIGNATURE-----
> Winpcap-users mailing list
> Winpcap-users at winpcap.org
More information about the Winpcap-users