Aaron Turner <aturner_AT_pobox.com>
Matt Bing <mbing_AT_nfr.net>
http://www.sourceforge.net/projects/tcpreplay/
FAQ Version 0.8
Tcpreplay is a powerful tool, but with that power comes complexity. While we have done our best to write good man pages for tcpreplay and it's associated utilities, we understand that many people may want more information then we can provide in the man pages. Additionally, this FAQ attempts to cover material which we feel will be of use to people using tcpreplay, as well as common questions that occur on the Tcpreplay-Users mailing list.
In the simplest terms, tcpreplay is a tool to send network traffic stored in pcap format back onto the network; basically the exact opposite of tcpdump.
Tcpreplay is not a tool to replay captured traffic to a server or client. Specifically, tcpreplay does not have the ability to rewrite IP addresses to a user-specified value or synchronize TCP sequence and acknowledgment numbers. In other words, tcpreplay can't ``connect'' to a server or be used to emulate a server and have clients connect to it.
The source code to tcpreplay is available on SourceForge: http://www.sourceforge.net/projects/tcpreplay
Two easy steps:
A few comments about 'make test':
Occasionally. And even when we do, generally only for one or two operating systems. Generally speaking, we assume people who want to use a tool like this can figure out how to compile it.
Sorry, none of the developers have yet to sell their soul.
Tcpreplay is licensed under the BSD License. For details, see Appendix A.
Originally, tcpreplay was written to test network intrusion detection devices, however tcpreplay has been used to test firewalls, routers, and other in-line devices. In it's current form, tcpreplay is not very well suited for testing IP stacks or applications running on servers or clients since it is unable to synchronize TCP sequence numbers nor is able to react or use packets sent by other devices as input.
The best place to get help or report a bug is the Tcpreplay-Users mailing list:
http://lists.sourceforge.net/lists/listinfo/tcpreplay-users
One of the most frustrating things for any developer trying to help a user with a problem is not enough information. Please be sure to include at minimum the following information, however any additional information you feel may be helpful will be appreciated.
To replay a given pcap as it was captured all you need to do is specify the pcap file and the interface to send the traffic out of:
tcpreplay -i eth0 sample.pcap
You can also replay the traffic at different speeds then it was originally captured1. To support this, tcpreplay supports four different flags: -R, -r, -m, and -p
Some examples:
Using the loop flag (-l) you can specify that a pcap file will be sent two or more times2:
Tcpreplay can also split traffic so that each side of a connection is sent out a different interface. In order to do this, tcpreplay needs the name of the second interface (-j) and a way to split the traffic. Currently, there are two ways to split traffic:
Sometimes, you want to do some post-capture filtering of packets. Tcpreplay let's you have some control over which packets get sent.
If you ever want to send traffic to another device on a switched LAN, you may need to change the destination MAC address of the packets. Tcpreplay allows you to set the destination MAC for each interface independently using the -I and -J switches. Example:
Occasionally, it is necessary to have tcpreplay rewrite the source and destination IP addresses, yet maintain the client/server relationship. Such a case might be having multiple copies of tcpreplay running at the same time using the same pcap file while trying to stress test firewall, IDS or other stateful device. If you didn't change the source and destination IP addresses, the device under test would get confused since it would see multiple copies of the same connection occurring at the same time. In order to accomplish this, tcpreplay accepts a user specified seed which is used to generate pseudo-random IP addresses. Also, when this feature is enabled, tcpreplay will automatically recalculate the layer 3 and 4 checksums as needed. Example:
tcpreplay -i eth0 -s 1239 sample.pcap &
tcpreplay -i eth0 -s 76 sample.pcap &
tcpreplay -i eth0 -s 239 sample.pcap &
tcpreplay -i eth0 sample.pcap
Occasionally, it is necessary to replay traffic which has been truncated by tcpdump. This occurs when the tcpdump snaplen is smaller then the actual packet size. Since this will create problems for devices which are expecting a full-sized packet or attempting checksum calculations, tcpreplay allows you to either pad the packet with zeros or reset the packet length in the headers to the actual packet size. In either case, the layer 3 and 4 checksums are recalculated. Examples:
There are about as many different capture file formats as there are sniffers. In the interest of simplicity, tcpreplay only supports libpcap4. If you would like to replay a file in one of these multitude of formats, the excellent open source tool Ethereal easily allows you to convert it to libpcap. For instance, to convert a file in Sun's snoop format to libpcap, issue the command:
tethereal -r blah.snoop -w blah.pcap
and replay the resulting file.
A common question on the tcpreplay-users list is how does one the client side of a connection back to a server. Unfortunately, tcpreplay doesn't support this right now. The major problem concerns syncing up TCP Seq/Ack numbers which will be different. ICMP also often contains IP header information which would need to be adjusted as well in many cases. About the only thing that could be easy to do is UDP which doesn't isn't usually requested.
This is however a feature that we're looking into implementing. If you're interested in helping add this feature, please contact us and we'd be more then happy to work with you. At this time however, we don't have an ETA when this will be implemented, so don't bother asking.
Tcpreplay can send traffic out two network cards, however it requires the calculations be done in real-time. These calculations can be expensive and can significantly reduce the throughput of tcpreplay.
Tcpprep is a libpcap pre-processor for tcpreplay which enables using two network cards to send traffic without the performance hit of doing the calculations in real-time.
Tcpprep reads in a libpcap (tcpdump) formatted capture file and does some processing to generate a tcpprep cache file. This cache file tells tcpreplay which interface a given packet should be sent out of.
No.
There are three major reasons to use tcpprep:
Cache files have nothing linking them to a given libpcap file, so there is nothing to stop you from doing this. However running tcpreplay with a cache file from a different libpcap source file is likely to cause a lot of problems and is not supported.
Tcpreplay traditionally is good for putting traffic on a given network, often used to test a network intrusion detection system (NIDS). However, there are cases where putting traffic onto a subnet in this manner is not good enough- you have to be able to send traffic *through* a device such as a router, firewall, or bridge.
In these cases, being able to use a single source file (libpcap) for both ends of the connection solves this problem.
Very small. Actual size depends on the number of packets in the dump file. Four bits of data is stored for each packet. On a test using a 900MB dump file containing over 500,000 packets, the cache file was only 300K.
Tcpprep has two basic modes which require the user to specify how to split traffic.
Tcpprep supports the same CIDR mode that tcpreplay supports using the -c flag (tcpreplay uses -C). Additionally, tcpprep also supports regex(7) regular expressions to match source IP addresses using the -r flag.
Tcpprep does an initial pass over the libpcap file to build a binary tree (one node per IP). For each IP, it keeps track of how many times it was a client or server. It then does a second pass of the file using the data in the tree and the ratio to determine if an IP is a client or server. If tcpprep is unable to determine the type (client or server) for each and every packet, then auto/bridge mode will fail. In these cases, it is best to use auto/router mode.
Tcpprep does the same first pass as Auto/Bridge mode. It then trys to convert the binary tree into a list of networks containing the servers. Finally it uses the CIDR mode with the list of server networks in a second pass of the libpcap file. Unlike auto/bridge mode, auto/router mode can always successfully split IP addresses into clients and servers.
Tcpprep uses the following methods in auto/router and auto/bridge mode to determine if an IP address is a client or server:
Since a system may send traffic which would classify it as both a client and server, it's necessary to be able to weigh the traffic. This is done by specifying the client/server ratio (-R) which is by default set to 2.0. The ratio is the modifier to the number of client connections. Hence, by default, client connections are valued twice as high as server connections.
Tcpprep supports the same -x and -X options to selectively send or drop packets.
Just run:
tcpreplay -c sample.cache -i eth0 -j eth1 sample.pcap
Regardless of the size of physical memory, UNIX kernels will only allocate a static amount for network buffers. This includes packets sent via the "raw" interface, like with tcpreplay. Most kernels will allow you to tweak the size of these buffers, drastically increasing performance and accuracy.
NOTE: Please note that the following information is provided based upon our own experiences or the reported experiences of others. Depending on your hardware and specific hardware, it may or may not work for you. It may even make your system horribly unstable, corrupt your harddrive, or worse.
NOTE: Different operating systems, network card drivers, and even hardware can have an effect on the accuracy of packet timestamps that tcpdump or other capture utilities generate. And as you know: garbage in, garbage out.
The following is known to apply to the 2.4.x series of kernels. If anyone has any information regarding other kernel versions, please let us know. By default Linux's tcpreplay performance isn't all that stellar. However, with a simple tweak, relatively decent performance can be had on the right hardware. By default, Linux specifies a 64K buffer for sending packets. Increasing this buffer to about half a megabyte does a good job:
echo 524287 >/proc/sys/net/core/wmem_default
echo 524287 >/proc/sys/net/core/wmem_max
echo 524287 >/proc/sys/net/core/rmem_max
echo 524287 >/proc/sys/net/core/rmem_default
On one system, we've seen a jump from 23.02 megabits/sec (5560 packets/sec) to 220.30 megabits/sec (53212 packets/sec) which is nearly a 10x increase in performance. Depending on your system, different numbers may provide different results.
*BSD systems typically allow you to specify the size of network buffers with the NMBCLUSTERS option in the kernel config file. Experiment with different sizes to see which yields the best performance. See the options(4) man page for more details.
As of tcpreplay v1.4, you'll need to have Libpcap installed on your system. Any recent Libpcap version should do, but we only test our code with the latest version: 0.7.1. Libpcap can be obtained here: http://www.tcpdump.org/
Tcpreplay v1.3 is the last version to support the old Libnet API (everything before 1.1.x) however it is not longer supported. As of v1.4 you will need to use Libnet 1.1.0 or better which can be obtained from the Libnet homepage: http://www.packetfactory.net/Projects/Libnet/
Generally this occurs when the interface (eth0 in this example) is not up or
doesn't have an IP address assigned to it. For example:
ifconfig eth0 1.1.1.1 netmask 255.255.255.0 up
Will do the trick on most UNIX and UNIX-like operating systems.
Version 1.1.0 of Libnet is unable to send traffic on the loopback device. Try upgrading to the 1.1.1 (pre)release of the Libnet library to solve this problem.
Tcpreplay requires that you run it as root.
When tcpreplay displays a message like "100000 write attempts failed from full buffers and were repeated", this usually means the kernel buffers were full and it had to wait until memory was available. This is quite common when replaying files as fast as possible with the "-R" option. See the tuning OS section in this document for suggestions on solving this problem.
Currently tcpreplay reserves the MAC address of 00:00:00:00:00:00 as reserved for internal use. Hence you can't rewrite the destination MAC address of packets to be all zeros. While we intend to fix this someday it's not currently high on our priority list, so let us know if we should re-prioritize things.
Cache files generated by tcpprep and read by tcpreplay are versioned to allow enhancements to the cache file format. Anytime the cache file format changes, the version is incremented. Since this occurs on a very rare basis, this is generally not an issue; however anytime there is a change, it breaks compatibility with previously created cache files. The solution for this problem is to use the same version of tcpreplay and tcpprep to read/write the cache files. Cache file versions match the following versions of tcpprep/tcpreplay:
Copyright (c) 1999-2003 Matt Bing, Aaron Turner. All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. All advertising materials mentioning features or use of this software must display the following acknowledgement: This product includes software developed by Anzen Computing, Inc. 4. Neither the name of Anzen Computing, Inc. nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
This document was generated using the LaTeX2HTML translator Version 2002 (1.62)
Copyright © 1993, 1994, 1995, 1996,
Nikos Drakos,
Computer Based Learning Unit, University of Leeds.
Copyright © 1997, 1998, 1999,
Ross Moore,
Mathematics Department, Macquarie University, Sydney.
The command line arguments were:
latex2html -nonavigation -no_subdir -split 0 -show_section_numbers FAQ.tex
The translation was initiated by Aaron Turner on 2003-12-05