<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML xmlns:v = "urn:schemas-microsoft-com:vml" xmlns:o =
"urn:schemas-microsoft-com:office:office" xmlns:w =
"urn:schemas-microsoft-com:office:word" xmlns:m =
"http://schemas.microsoft.com/office/2004/12/omml"><HEAD>
<META content=text/html;charset=utf-8 http-equiv=Content-Type>
<META name=GENERATOR content="MSHTML 8.00.7600.16625">
<STYLE>
<!--
/* Font Definitions */
@font-face
{font-family:SimSun;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:"Comic Sans MS";
panose-1:3 15 7 2 3 3 2 2 2 4;}
@font-face
{font-family:"\@SimSun";
panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
span.apple-style-span
{mso-style-name:apple-style-span;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:1646936997;
mso-list-type:hybrid;
mso-list-template-ids:1931236606 67698703 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l0:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
-->
</STYLE>
</HEAD>
<BODY style="PADDING-LEFT: 10px; PADDING-RIGHT: 10px; PADDING-TOP: 15px"
id=MailContainerBody lang=EN-US leftMargin=0 link=blue topMargin=0 bgColor=white
vLink=purple CanvasTabStop="true" name="Compose message area">
<DIV><FONT face=Calibri>Two notes: </FONT></DIV>
<DIV><FONT face=Calibri></FONT> </DIV>
<DIV><FONT face=Calibri>1. if the objective is dumping the packets to disk, then
the bottleneck is mostly the disk. </FONT></DIV>
<DIV><FONT face=Calibri>2. checksum offloading does not play any role in packet
capture. It plays a role when packets have to be processed by the TCP/IP stack
(because the validation is done in hardware rather software). But when you do
packet capture, you don't care about that. In fact I suggested to totally
disable TCP/IP from the NIC where you capture the packets from.</FONT></DIV>
<DIV><FONT face=Calibri></FONT> </DIV>
<DIV><FONT face=Calibri>GV</FONT></DIV>
<DIV style="FONT: 10pt Tahoma">
<DIV><FONT size=3 face=Calibri></FONT><BR></DIV>
<DIV style="BACKGROUND: #f5f5f5">
<DIV style="font-color: black"><B>From:</B> <A title=fish@infidels.org
href="mailto:fish@infidels.org">"Fish" (David B. Trout)</A> </DIV>
<DIV><B>Sent:</B> Sunday, September 19, 2010 4:44 AM</DIV>
<DIV><B>To:</B> <A title=winpcap-users@winpcap.org
href="mailto:winpcap-users@winpcap.org">winpcap-users@winpcap.org</A> </DIV>
<DIV><B>Subject:</B> Re: [Winpcap-users] About the packets loss ,what is the
bottleneck ?</DIV></DIV></DIV>
<DIV><BR></DIV>
<DIV class=WordSection1>
<P class=MsoNormal><SPAN
style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 10.5pt">There
are other factors as well:<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN
style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 10.5pt"><o:p> </o:p></SPAN></P>
<P style="TEXT-INDENT: -0.25in; mso-list: l0 level1 lfo1"
class=MsoListParagraph><SPAN
style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 10.5pt"><SPAN
style="mso-list: Ignore">1.<SPAN
style="FONT: 7pt 'Times New Roman'">
</SPAN></SPAN></SPAN><SPAN
style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 10.5pt">Number
of processors and processor speed.<o:p></o:p></SPAN></P>
<P style="TEXT-INDENT: -0.25in; mso-list: l0 level1 lfo1"
class=MsoListParagraph><SPAN
style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 10.5pt"><SPAN
style="mso-list: Ignore">2.<SPAN
style="FONT: 7pt 'Times New Roman'">
</SPAN></SPAN></SPAN><SPAN
style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 10.5pt">Amount
of RAM<o:p></o:p></SPAN></P>
<P style="TEXT-INDENT: -0.25in; mso-list: l0 level1 lfo1"
class=MsoListParagraph><SPAN
style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 10.5pt"><SPAN
style="mso-list: Ignore">3.<SPAN
style="FONT: 7pt 'Times New Roman'">
</SPAN></SPAN></SPAN><SPAN
style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 10.5pt">Operating
system workload<o:p></o:p></SPAN></P>
<P style="TEXT-INDENT: -0.25in; mso-list: l0 level1 lfo1"
class=MsoListParagraph><SPAN
style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 10.5pt"><SPAN
style="mso-list: Ignore">4.<SPAN
style="FONT: 7pt 'Times New Roman'">
</SPAN></SPAN></SPAN><SPAN
style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 10.5pt">Network
Adapter Task Offloading capability (esp. checksum
offloading)<o:p></o:p></SPAN></P>
<P style="TEXT-INDENT: -0.25in; mso-list: l0 level1 lfo1"
class=MsoListParagraph><SPAN
style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 10.5pt"><SPAN
style="mso-list: Ignore">5.<SPAN
style="FONT: 7pt 'Times New Roman'">
</SPAN></SPAN></SPAN><SPAN
style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 10.5pt">Speed
of disk subsystem<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN
style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 10.5pt"><o:p> </o:p></SPAN></P>
<P class=MsoNormal><SPAN
style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 10.5pt">On
an modern fast quad-core system that is relatively idle (has no other work to
do) with lots of RAM and a high performance multi-disk RAID-0 array and with
checksum offloading enabled in the adapter, it should be entirely possible to
capture a gigabit speed without packet loss.<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN
style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 10.5pt"><o:p> </o:p></SPAN></P>
<P class=MsoNormal><SPAN
style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 10.5pt">If
your system is an overtaxed (busy) single processor system with only an average
amount of RAM and an average speed non-RAID disk subsystem however, then you’re
very likely going to experience significant packet loss beyond about 480
Mbps.<o:p></o:p></SPAN></P>
<DIV>
<P style="mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"
class=MsoNormal><SPAN
style="FONT-FAMILY: 'Arial','sans-serif'; COLOR: black; FONT-SIZE: 7.5pt">--
</SPAN><SPAN style="COLOR: #1f497d"><BR></SPAN><SPAN
style="FONT-FAMILY: 'Arial','sans-serif'; COLOR: black; FONT-SIZE: 13.5pt">
"</SPAN><B><SPAN
style="FONT-FAMILY: 'Arial','sans-serif'; COLOR: green">Fish</SPAN></B><SPAN
style="FONT-FAMILY: 'Arial','sans-serif'; COLOR: black; FONT-SIZE: 13.5pt">"</SPAN><SPAN
style="FONT-FAMILY: 'Arial','sans-serif'; COLOR: black; FONT-SIZE: 10pt">
(</SPAN><SPAN
style="FONT-FAMILY: 'Comic Sans MS'; COLOR: purple; FONT-SIZE: 10pt">David B.
Trout</SPAN><SPAN
style="FONT-FAMILY: 'Arial','sans-serif'; COLOR: #1f497d; FONT-SIZE: 10pt">)</SPAN><SPAN
style="COLOR: #1f497d"> <BR></SPAN><SPAN
style="FONT-FAMILY: 'Verdana','sans-serif'; COLOR: #1f497d; FONT-SIZE: 10pt"> </SPAN><SPAN
style="COLOR: #1f497d"> </SPAN><U><SPAN
style="FONT-FAMILY: 'Arial','sans-serif'; COLOR: blue; FONT-SIZE: 10pt">fish@softdevlabs.com</SPAN></U><SPAN
style="COLOR: #1f497d"><o:p></o:p></SPAN></P></DIV>
<P class=MsoNormal><SPAN
style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 10.5pt"><o:p> </o:p></SPAN></P>
<DIV
style="BORDER-BOTTOM: medium none; BORDER-LEFT: blue 1.5pt solid; PADDING-BOTTOM: 0in; PADDING-LEFT: 4pt; PADDING-RIGHT: 0in; BORDER-TOP: medium none; BORDER-RIGHT: medium none; PADDING-TOP: 0in">
<DIV>
<DIV
style="BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<P class=MsoNormal><B><SPAN
style="FONT-FAMILY: 'Tahoma','sans-serif'; FONT-SIZE: 10pt">From:</SPAN></B><SPAN
style="FONT-FAMILY: 'Tahoma','sans-serif'; FONT-SIZE: 10pt">
winpcap-users-bounces@winpcap.org [mailto:winpcap-users-bounces@winpcap.org]
<B>On Behalf Of </B>yulou liu<BR><B>Sent:</B> Sunday, September 19, 2010 12:50
AM<BR><B>To:</B> winpcap-users@winpcap.org<BR><B>Subject:</B> [Winpcap-users]
About the packets loss , what is the bottleneck ?<BR><B>Importance:</B>
High<o:p></o:p></SPAN></P></DIV></DIV>
<P class=MsoNormal><o:p> </o:p></P>
<P style="MARGIN-BOTTOM: 12pt" class=MsoNormal><SPAN
style="FONT-SIZE: 11.5pt"><BR></SPAN><SPAN class=apple-style-span><SPAN
style="FONT-FAMILY: 'Calibri','sans-serif'; FONT-SIZE: 10pt">There is still the
question about packets loss.</SPAN></SPAN><SPAN
style="FONT-SIZE: 11.5pt"><BR></SPAN><SPAN class=apple-style-span><SPAN
style="FONT-FAMILY: 'Calibri','sans-serif'; FONT-SIZE: 10pt"> </SPAN></SPAN><SPAN
class=apple-style-span><SPAN style="FONT-SIZE: 11.5pt"> </SPAN></SPAN><SPAN
style="FONT-SIZE: 11.5pt"><BR></SPAN><SPAN class=apple-style-span><SPAN
style="FONT-FAMILY: 'Calibri','sans-serif'; FONT-SIZE: 10pt">According to the
essay ' </SPAN></SPAN><SPAN class=apple-style-span><SPAN
style="FONT-FAMILY: 'SimSun','serif'; FONT-SIZE: 10pt">Profiling and
Optimization of Software-Based Network-Analysis Applications</SPAN></SPAN><SPAN
class=apple-style-span><SPAN
style="FONT-FAMILY: 'Calibri','sans-serif'; FONT-SIZE: 10pt">'
, every packet is copied twice in the main memory before
reaching the user. In order to reduce the cost of CPU and the bus
occupying of the SDRAM of pc, is it possible to copy data directly
from the kernel buffer to the final buffer , which I want the date
kept in ? </SPAN></SPAN><SPAN
style="FONT-SIZE: 11.5pt"><BR></SPAN><SPAN class=apple-style-span><SPAN
style="FONT-FAMILY: 'Calibri','sans-serif'; FONT-SIZE: 10pt"> </SPAN></SPAN><SPAN
class=apple-style-span><SPAN style="FONT-SIZE: 11.5pt"> </SPAN></SPAN><SPAN
style="FONT-SIZE: 11.5pt"><BR></SPAN><SPAN class=apple-style-span><SPAN
style="FONT-FAMILY: 'Calibri','sans-serif'; FONT-SIZE: 10pt">Here is another
idea --- </SPAN></SPAN><SPAN class=apple-style-span><SPAN
style="FONT-FAMILY: 'SimSun','serif'; FONT-SIZE: 10pt"> allocate several
different user buffers , once </SPAN></SPAN><SPAN
class=apple-style-span><SPAN
style="FONT-FAMILY: 'Calibri','sans-serif'; FONT-SIZE: 10pt">a </SPAN></SPAN><SPAN
class=apple-style-span><SPAN
style="FONT-FAMILY: 'SimSun','serif'; FONT-SIZE: 10pt">user buffer is fulled ,
then let the next user buffer to save the </SPAN></SPAN><SPAN
class=apple-style-span><SPAN
style="FONT-FAMILY: 'Calibri','sans-serif'; FONT-SIZE: 10pt">new</SPAN></SPAN><SPAN
class=apple-style-span><SPAN
style="FONT-FAMILY: 'SimSun','serif'; FONT-SIZE: 10pt"> datas from kernel
buffer. Meanwhile copy datas from the first user buffer to disk (assume
that the hard disk write rate is fast enough).</SPAN></SPAN><SPAN
class=apple-style-span><SPAN
style="FONT-FAMILY: 'Calibri','sans-serif'; FONT-SIZE: 10pt"> Is this idea
work with the winpcap ?</SPAN></SPAN><SPAN class=apple-style-span><SPAN
style="FONT-SIZE: 11.5pt"> </SPAN></SPAN><SPAN
style="FONT-SIZE: 11.5pt"><BR></SPAN><SPAN class=apple-style-span><SPAN
style="FONT-FAMILY: 'Calibri','sans-serif'; FONT-SIZE: 10pt"> </SPAN></SPAN><SPAN
class=apple-style-span><SPAN style="FONT-SIZE: 11.5pt"> </SPAN></SPAN><SPAN
style="FONT-SIZE: 11.5pt"><BR></SPAN><SPAN class=apple-style-span><SPAN
style="FONT-FAMILY: 'Calibri','sans-serif'; FONT-SIZE: 10pt"> </SPAN></SPAN><SPAN
class=apple-style-span><SPAN style="FONT-SIZE: 11.5pt"> </SPAN></SPAN><SPAN
style="FONT-SIZE: 11.5pt"><BR></SPAN><SPAN class=apple-style-span><SPAN
style="FONT-FAMILY: 'Calibri','sans-serif'; FONT-SIZE: 10pt">Thank
you!</SPAN></SPAN><SPAN class=apple-style-span><SPAN
style="FONT-SIZE: 11.5pt"> </SPAN></SPAN><SPAN
style="FONT-SIZE: 11.5pt"><BR></SPAN><SPAN class=apple-style-span><SPAN
style="FONT-FAMILY: 'Calibri','sans-serif'; FONT-SIZE: 10pt"> </SPAN></SPAN><SPAN
class=apple-style-span><SPAN style="FONT-SIZE: 11.5pt"> </SPAN></SPAN><SPAN
style="FONT-SIZE: 11.5pt"><BR></SPAN><SPAN class=apple-style-span><SPAN
style="FONT-FAMILY: 'Calibri','sans-serif'; FONT-SIZE: 10pt">==============================================================</SPAN></SPAN><SPAN
class=apple-style-span><SPAN style="FONT-SIZE: 11.5pt"> </SPAN></SPAN><SPAN
style="FONT-SIZE: 11.5pt"><BR></SPAN><SPAN class=apple-style-span><SPAN
style="FONT-FAMILY: 'Calibri','sans-serif'; FONT-SIZE: 10pt"> </SPAN></SPAN><SPAN
class=apple-style-span><SPAN style="FONT-SIZE: 11.5pt"> </SPAN></SPAN><SPAN
style="FONT-SIZE: 11.5pt"><BR></SPAN><SPAN class=apple-style-span><SPAN
style="FONT-FAMILY: 'SimSun','serif'; FONT-SIZE: 10pt">Q1: tough question to
answer, as it depends on a number of factors. What is the average packet size?
Are you just counting the packets, or dumping them to disk/DB/...? Do you see
packet drops in the pcap_stats?</SPAN></SPAN><SPAN
style="FONT-FAMILY: 'SimSun','serif'; FONT-SIZE: 10pt"><BR><SPAN
class=apple-style-span>Q2: I will need to run some tests on this. Do you have
some minimal sample code that shows the issue?</SPAN><BR><SPAN
class=apple-style-span>Q3: Yes, *if* the NIC (and NIC driver) are not dropping
packets themselves. </SPAN><BR><BR><SPAN class=apple-style-span>Did you
disable any protocol bound to the NIC where you receive all these
packets?</SPAN><BR><BR><SPAN class=apple-style-span>Have a nice
day</SPAN><BR><SPAN class=apple-style-span>GV</SPAN><BR><BR><BR><BR><BR><SPAN
class=apple-style-span>From: yulou liu </SPAN><BR><SPAN
class=apple-style-span>Sent: Wednesday, August 25, 2010 11:08 AM</SPAN><BR><SPAN
class=apple-style-span>To: winpcap </SPAN><BR><SPAN
class=apple-style-span>Subject: [Winpcap-users] About the packets loss , what is
the bottleneck ?</SPAN><BR><BR><BR><SPAN class=apple-style-span>I'm using
winpcap to capture datas from FPGA board via 1G Ethernet connection(directly
connected).</SPAN><BR><SPAN class=apple-style-span>the FPGA is configed to send
data at a special rate. With the fpga sending rate increases,especially
above 500Mbps, it sometimes loss packets. </SPAN><BR><BR><SPAN
class=apple-style-span>Q1: Is it possible to totally avoid packets loss by
optimism the code? I want to collect datas at speed 614Mbps without packet
loss (collecting all datas </SPAN><BR><BR><SPAN class=apple-style-span>last
about 1 minutes). My workstations features with 2 Xeon CPU (each has 4core),
DDR3 SDRAM, 1 G onboard netcard. Which part is the most probably
bottleneck ?</SPAN><BR><BR><SPAN class=apple-style-span>Q2: As a test , I
found the pcap_next_ex() can't get any packets when the user buffer is set
over 64 MB, what does it happen ?</SPAN><BR><BR><SPAN
class=apple-style-span>Q3: if the Kernel buffer's size is 16MB , then the first
16Mb packets from fpga won't be lost, so , if I set the kernel
buffer as 128MB , then at least </SPAN><BR><BR><SPAN
class=apple-style-span>the first coming 128MB data from FPGA won't be lost
either ? but I found When I set the kernel buffer bigger than 100 MB, the
packets drops is getting </SPAN><BR><BR><SPAN
class=apple-style-span>worse. </SPAN><BR><BR><BR><SPAN
class=apple-style-span>Thanks a lot!</SPAN><BR><BR></SPAN><SPAN
class=apple-style-span><SPAN
style="FONT-SIZE: 11.5pt"><o:p></o:p></SPAN></SPAN></P><PRE><o:p> </o:p></PRE></DIV></DIV>
<P>
<HR>
<P></P>_______________________________________________<BR>Winpcap-users mailing
list<BR>Winpcap-users@winpcap.org<BR>https://www.winpcap.org/mailman/listinfo/winpcap-users<BR></BODY></HTML>