<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 14 (filtered medium)"><!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>There is a bit of confusion here.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>He is using WinPcap to capture packets, so there is no concept of IP queue, or even UDP packets. WinPcap treats all the packets in the same way, it totally ignores the fact that they are IP, UDP or anything else.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Also, out-of order packets are normal and can happen on an Ethernet network, period. <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Finally (and even if you have two machines with a direct Ethernet cable, no switches/routers…) you can still “receive” packets out-of-order when running on a multicore system. The reason is that the NIC itself receives the packets probably in order, but then the NIC driver and OS will try to distribute the received packets among all the CPUs on the system. WinPcap (in its kernel driver) will receive packets concurrently on multiple cores, and will put all the packets in the same buffer. It has no way of knowing which packet was actually received first by the NIC (and has no way to control such behavior). <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Have a nice day<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>GV<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><div><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'><p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> winpcap-users-bounces@winpcap.org [mailto:winpcap-users-bounces@winpcap.org] <b>On Behalf Of </b>Black, Michael (IS)<br><b>Sent:</b> Friday, February 25, 2011 5:05 AM<br><b>To:</b> winpcap-users@winpcap.org<br><b>Cc:</b> neil.powell neil.powell<br><b>Subject:</b> Re: [Winpcap-users] EXT : WinPCAP Packet ordering<o:p></o:p></span></p></div></div><p class=MsoNormal><o:p> </o:p></p><div><p><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black'>I've been trying to tell you that UDP does NOT guarantee ordering. And guess what you found...it's out of order.<o:p></o:p></span></p><p><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black'>It also doesn't guarantee delivery. Your lucky you're not dropping packets...if you overrun the IP queue you would be dropping them. You shouldb be using TCP. Or...you need a sequence number in the packet so you can reassemble the order yourself.<o:p></o:p></span></p><p><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black'> <o:p></o:p></span></p><p><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black'>That's why its nickname is "Unreliable Datagram Protocol".<o:p></o:p></span></p><p><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black'><a href="http://en.wikipedia.org/wiki/User_Datagram_Protocol">http://en.wikipedia.org/wiki/User_Datagram_Protocol</a><o:p></o:p></span></p><p><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black'> <o:p></o:p></span></p><p><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black'>You're probably seeing multi-core processing of packets...so it can pick them out of order since it doesn't violate the protocol.<o:p></o:p></span></p><p><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black'> <o:p></o:p></span></p><p><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black'><a href="http://stackoverflow.com/questions/2533873/why-do-i-get-udp-datagrams-out-of-order-even-with-processes-runnning-locally">http://stackoverflow.com/questions/2533873/why-do-i-get-udp-datagrams-out-of-order-even-with-processes-runnning-locally</a><o:p></o:p></span></p><div><p><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black'> <o:p></o:p></span></p><div><p><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black'>Michael D. Black<o:p></o:p></span></p><p><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black'>Senior Scientist<o:p></o:p></span></p><p><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black'>NG Information Systems<o:p></o:p></span></p><p><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black'>Advanced Analytics Directorate<o:p></o:p></span></p><p><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black'> <o:p></o:p></span></p></div></div><div><div class=MsoNormal align=center style='text-align:center'><span style='color:black'><hr size=2 width="100%" align=center></span></div><div id=divRpF755259><p class=MsoNormal style='margin-bottom:12.0pt'><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black'> winpcap-users-bounces@winpcap.org [winpcap-users-bounces@winpcap.org] on behalf of neil.powell neil.powell [neil.powell@HERRHAIR.COM]<br><b>Sent:</b> Friday, February 25, 2011 2:44 AM<br><b>To:</b> winpcap-users@winpcap.org<br><b>Cc:</b> neil.powell neil.powell<br><b>Subject:</b> EXT :[Winpcap-users] WinPCAP Packet ordering</span><span style='color:black'><o:p></o:p></span></p></div><div><p><span style='color:black'>Hello,<br> <br>I would like to ask a question to the mailing list; I apologise if this has been<br>answered before I did search the archive but didn't see anything relating to my<br>question.<br> <br>Firstly let me explain what I am trying to achieve...<br> <br> <br>I am writing a Windows application that is targeting Windows 7 64bit<br>multiprocessor machines. The basic idea is as follows.... I have a piece of<br>hardware (FPGA based) that is delivering layer II packets over a Gbit network;<br>each packet is NO more than 1500 bytes. The data rate pushed out of the FPGA is<br>typically 50Mbytes per sec or lower.<br> <br>The purpose of the Windows application is the capture each packet off the<br>network, do some quick data integrity checks on the payload and dump it to disk<br>ASAP. The FPGA has a 12bit packet counter (0-4095); this count is placed inside<br>our bespoke payload so that the data can be reconstituted on the PC. During the<br>data integrity checks I check the packet count to ensure the packets are in<br>order before I begin reconstituting the data; I also sanity check the timestamp<br>of each packet to make sure it too is increasing.<br> <br> <br> Here is my question/problem.....<br> On multicore 64 bit machines (with a 64 bit OS) can the packets come out of<br>order?<br> My justification for the question is as follows. During debug of the windows<br>application I report the timestamp and pkt number of each packet. Whilst it<br>appears that I do not drop packets I do see packets numbers come out of order. <br>Here is a debug trace.<br> <br> Timestamp Pkt #<br> 15:38:40:415412 2222<br> 15:38:40:415413 2223<br> 15:38:40:415424 2224<br> 15:38:40:415425 2225<br> 15:38:40:415426 2226<br> 15:38:40:415428 2227<br> 15:38:40:415428 Found Packet: 2236 Expecting Packet: 2228<br> 15:38:40:415430 2237<br> 15:38:40:415431 2238<br> 15:38:40:415433 2239<br> 15:38:40:415433 Found Packet: 2228 Expecting Packet: 2240<br> 15:38:40:415434 2229<br> 15:38:40:415445 2230<br> 15:38:40:415447 2231<br> 15:38:40:415447 2232<br> 15:38:40:415448 2233<br> 15:38:40:415451 2234<br> 15:38:40:415452 2235<br> 15:38:40:415453 Found Packet: 2240 Expecting Packet: 2236<br> 15:38:40:415454 2241<br> 15:38:40:415456 2242<br> 15:38:40:415457 2243<br> 15:38:40:415467 2244<br> 15:38:40:415469 2245<br> 15:38:40:415471 2246<br> <br>If you follow the trace you can see that I don't drop packets but they are<br>indeed out of order. <br>I must stress that this situation is NOT frequent but is exacerbated by high<br>data rates or heavy disk usage.<br>I have spent some time eliminating all the obvious… debug output buffering, <br>FPGA output and disk writes all of which have drawn a blank.<br> <br>I am using WinPCAP in the following manner:<br> <br>pcap_open:<br>snaplen = 1500;<br>flags =<br>PCAP_OPENFLAG_PROMISCUOUS|PCAP_OPENFLAG_NOCAPTURE_LOCAL|PCAP_OPENFLAG_MAX_RESPONSIVENESS;<br>read_timeout = 0;<br> <br>pcap_setbuff:<br>100Mb<br> <br>pcap_setmintocopy:<br>0<br> <br> <br>Thank you in advance<br> <br> <br> <br>Neil Powell<o:p></o:p></span></p></div></div></div></div></body></html>