How can I improve my throughput performance on Linux?

One potential cause of poor performance on a Linux system is the amount of buffer space the RHEL 4u4 kernel uses to reassemble IP fragments.

The parameters in /proc/sys/net/ipv4 control various aspects of the network, including a parameter that controlls the reassembly buffer size.

ipfrag_high_threshold specifies that maximum amount of memory used to reassemble IP fragments. When the memory used by fragments reaches ipfrag_high_threshold, old entries are removed until the memory used declines to ipfrag_low_threshold.

If the output of netstat shows increasing amounts of IP fragment reassemblies failing, we recommend to increase ipfrag_high_threshold. The impact can be significant. In some use cases, increasing this buffer space improved throughput from 32MB/sec to 80MB/sec.

To temporarily change the value of ipfrag_high_threshold, use this command as root:

echo "8388608" > /proc/sys/net/ipv4/ipfrag_high_threshold

To make this change permanent across reboots, edit the /etc/sysctl.conf file and add the line:

net.ipv4.ipfrag_high_threshold = 8388608;