[Winpcap-users] is winpcap appropriate for my gigabit ethernet device?

max reason maxreason at maxreason.com
Thu Sep 4 09:52:52 GMT 2008

 This is my first post, since I just learned about winpcap.
 I suspect my application for winpcap is unusual, so I'd like
 to ask others with winpcap experience whether winpcap will
 do what I need.  So, here goes.

 I designed an inexpensive but high-speed, high-resolution
 digital video camera with 5 megapixel [and 8 megapixel] CCDs.
 To keep the per-unit cost low, I had to find some way to get
 151,165,440 bytes per second from camera to PC - which equals
 roughly 1.2 gigabits per second.  My solution was to base the
 camera circuit on a cyclone3 FPGA and gigabit ethernet PHY.
 Plus I had to concoct a lossless compression system that is
 both brain-dead simple, and simple and fast enough to compress
 100 million 16-bit pixels per second enough to reduce the
 bandwidth far enough below 1 gigabit per second to squeeze
 the data through the PHY along with ethernet packet overhead.

 The bottom line is the following.  For this to work reliably,
 the packets need to be captured, then stored and/or displayed
 with as little overhead as possible.  Thet's probably obvious!

 To make this work, the ethernet cable from the camera must plug
 directly into the RJ45 connector on the PC - no routers/hubs/etc.
 The connection must be direct connect, or no way.

 My plan has been to send the data in a simple ethernet packet:
   6-byte target-MAC-address
   6-byte source-MAC-address
   2-byte length-field
   data (2592 pixels)

 I didn't learn until recently that PC operating systems do not
 typically receive or send such fundamental ethernet packets,
 and support might be non-trivial.  I wrote some prototype code
 on Linux with PF_PACKET that appears to provide me a way to
 process these fundamental packets efficiently, and avoid ALL
 subsequent [higher-level protocol] processing by the OS.

 But so far, I have not found a similar mechanism in win32.
 However, if I understand the nature of winpcap correctly,
 winpcap appears to provide similar capabilities.

 So my first question is, "does winpcap do what I need?".

 I think "what I need" is fairly obvious, but I'll rephrase
 it this way.  On the "PC receives data" side, I need a way to
 receive the packet - at least the data payload, but also the
 header if feasible.  I prefer to receive the CRC32 at the end
 of the packet (after the data), but can live without it.

 On the "PC sends data" side, I need to send short, simple
 command packets fairly rarely - no more often than one per
 image == 1944 ~ 2448 received packets.  And that packet
 can and should be sent when no data is being received
 (during a short pause between images).  These packets
 should also be plain, ordinary ethernet packets with
 the same fields shown above.

 I very much prefer to ***avoid*** IP, UDP, TCP/IP packets!
 It is difficult enough to make the FPGA control and receive
 data from four CCDs simultaneously (?did I mention that?),
 plus perform realtime lossless data compression, plus send
 the compressed data through the gigabit ethernet PHY also
 in realtime (albeit one scanline later == pipelined).

 To waste space, bandwidth and complexity in the packet header
 on totally useless NOTHINGNESS (to my application anyway),
 is just stupid.  To create those larger packets just tempts
 the OS to perform some packet processing, which just consumes
 CPU bandwidth that my application need for other work (say,
 saving the data to disk, or blasting it into a texture-map
 in a GPU card, or etc).  It also throws a complexity-wrench
 into my plans for computing the CRC32 in the FPGA.

 I hope winpcap can simply "give my application the packet"
 and do no [additional] processing whatsoever.  It is also
 very important that the OS not waste its time trying to
 process the packet too.  I mean, what on earth for?

 And no, I can't just slow down the frame rate a little, to
 whatever bandwidth is "feasible".  Some of the applications
 of this device require "super-cinema HDTV" media output,
 which is 2592 x 1080 x 24 FPS --- and the frame rate is
 fixed by convention (and the equipment that processes it).
 And that's not the only application that requires this
 little device crank out data out at nearly 1 gigabit.
 PS:  I suspect the actual data-payload rate can be in
 the neighborhood of 900Mbps (maybe even 850Mbps), but
 it must not "studder or skip" on frames that contain
 "complexity" the lossless compression cannot compress
 as much as "easy" frames.  That's an absolute no-can-do.

 I thank whoever decides to answer my question.  I am happy
 to dig-in, read and learn how to hook winpcap into my code.
 I just want to make sure I don't do all that work just to
 discover winpcap can't do what I need.  Thanks, Max.

More information about the Winpcap-users mailing list