[ntar-workers] On Markers and Bookmarks

[Jose M. Gonzalez]

Loris Degioanni wrote:
> I like the FF/REW blocks idea, and it should be quite simple to 
> implement. However, do you really think that repeating interface blocks 
> will be a big overhead or performance hit? It's really not cler to me.
It depends on the distance you want between consecutive FF/REW blocks. 
If you plan to have just a few packets in the middle, repeating the IDBs 
is probably not justifiable. 

Regards. 
-Chema

PS: I'm adding a more clear argument on the FF/REW issue, which I missed 
yesterday. 

One of the things I like about the new format is that it permits 
navigating traces in both directions.

With the old format, the only possible navigation method (again w/o 
considering tcpslice's hack) is "go to next packet." It was implemented 
using the caplen field in the pcap header (4 bytes per packet). You have 
to provide at least one navigation method, and sequential-forward seems 
clearly the most useful one. 

	   /------------\  /----->
	  |              ||      
	+-|------+------+v|------+------+----
	| header | data | header | data | ...
	+--------+------+--------+------+----

In the new format, we recognize other navigation methods are desirable. 
One is "go to the previous packet." You need at least another pointer, 
which in the current Draft is the second "Block Total Length" field. The 
cost is another 4 bytes per packet. 

	       /-----------------\      /----------------\
	      |                   |    |                  |
	+----+|----+------+-----++v---+|----+------+----++v---+----
	| BT | BTL | data | BTL || BT | BTL | data | BT || BT | ...
	+^---+-----+------+|----++^---+-----+------+|---++----+----
	 |                 |      |                 |
	  \---------------/        \---------------/

Yet another navigation method requires adding fast paths to navigate the 
trace. This can be done with the FF/REW marker blocks, or 16 bytes per 
marker. What constitutes a "fast path," though, depends on the use. You 
may want to tradeoff space for availability, and therefore set an FF/REW 
pair every 10 packets. In this case, you don't want to repeat the IDBs 
after each FF/REW. 

Actually, I'm thinking that you may want to have several parallel FF/REW. 
For example, one that advances 100 packets, another that advances 10K 
packets, and so on (note that this numbers will be approximate, as the 
FF must chose in advance the offset that will correspond to the 100/10K 
packets).  [Figure only represents forward path]

	  /-----------\  /-\  /--\  /-\  /-\  /-\  /--\  /-\  /->
	 |             ||   ||    ||   ||   ||   ||    ||   ||
	+|------------+v|--+v|---+v|--+v|--+|---+|----+|---+|---+----
	| PacketBlock | PB | REW | FF | PB | PB | REW | FF | PB | ...
	+-------------+----+-----+^|--+----+----+-----+^|--+----+----
	                          ||                   ||
	          ---------------/  \-----------------/  \---->

Note: this is not indexing at all. I agree with Christian that indexing 
is better done offline, in a separate file. The FF/REW approach is only a 
method for quick sequential navigation.