Fast Ethernet controllers
0verflows and underflows are two important performance metrics that affect network and system throughput, especially in the window-based protocol schemes like TCP and IPX. Each missed frame in these protocols involves retransmission of that particular frame (or more frames within that window). Retransmission is expensive as it uses both hardware and software resources of the system, as well as the network bandwidth. It is desirable to have the missed frame rate to be no higher than the frame loss rate due to the acceptable bit error rate. To avoid overflows and to take maximum advantage of large TCP and IPX window sizes, a system should be designed with optimum buffer sizes for that configuration.
An underflow occurs when the transmit buffer receives data from the system at a rate slower than the network rate. Overflow occurs when a system cannot drain out the data from the receive buffer at a sufficient rate, causing the buffer to fill up and the rest of the data on the wire to be thrown away. Both TCP and the latest versions of IPX use a windowing protocol scheme for data transmission. Instead of sending one frame and waiting for the acknowledgment from the receiver before transmitting the next frame, the protocol allows for sending a burst of frames by the transmitter before receiving an acknowledgment from the receiver. The amount of burst is determined by the size of the window. It has been reported that increasing the window size greatly improves the overall network. If the sender does not receive an ACK within a predefined window, some form of retransmission has to occur. Besides network infrastructure problems, the most probable cause for an absence of ACK is the dropping of a data frame by the receiver! Frames will be dropped by the receiver when it does not have enough buffer space.
TCP uses the sliding window protocol. As the ACK for the leftmost frame in the window is received, the window is moved to the right. If the sender reaches the end of the window before receiving an ACK for the leftmost frame, retransmission of the frame missing an ACK is required. Some implementations handle this by retransmitting the whole window, while others may only retransmit the missed frame. In any case, the overhead is not trivial. With the understanding that overflows and underflows can affect the system and network performance, a study was done to determine the buffer size required to ensure zero underflows and overflows in various environments for a typical network controller card.
AMDs PCnet-FAST device provides a flexible FIFO-SRAM architecture and was used as a model for system performance. A system-level model of the PCnetFAST's buffer architecture was developed using a modeling tool called Workbench from SES. For modeling purposes, the PCnetFAST device is divided into three distinct interfaces: PCI interface, buffer interface and network interface. The network interface models the reception and transmission of frames over the wire. Almost all the operational parameters in this interface are defined in the IEEE 802.3 standard. Two important network traffic related parameters, frame size distributions and frame arrival rates, were derived based on measurements for a typical client-server configuration. The buffer interface includes the FIFO-SRAM subsystem.
Bus latency and burst size are the two most important parameters for the PCI interface model. Bus latency is defined as the interval from the time a PCI device requests the bus to the time it can begin the first data transfer. Burst size is the number of bytes that can be transferred by a master each time it gains control of the PCI bus. The bus latency and burst size together determine the ability of the system to drain the data out of the network controller buffer. The burst sizes and bus latency vary widely from system to system, depending on configurations and applications.
For the transmit section, to prevent any underflows, the buffer need not be much larger than the maximum frame size (maximum frame size in IEEE 802.3 based networks is 1518 bytes). A transmit buffer of 2-4Kbytes is sufficient for most applications. For the receive section, the situation is quite different. The network controller buffer requirements vary widely, depending on the type of network and system environment. For a heavily loaded system where the latency can be high (for example, a PC with multiple PCI and ISA bus cards running disk and network intensive applications), network controller buffer requirements can be large in order to prevent overflows. Following are some of the results of simulation runs for different environments. Various cases covering half- and full-duplex 100-Mbps Ethernet networks are considered, and buffer requirements for the receive section are illustrated in the graphs under each case. Simulation parameters are as shown in Table 1.
|Inter Frame Space (IFS)
||256 &1500 Bytes
|PCI Clock Speed
|PCI Bus Width
|PCI Bus Cycle
|PCI Burst Size
|PCI Bus Latency
100Mbps half-duplex for a 24Kbyte window - The latest versions of IPX (VLM) use a sliding window protocol with window size of 16 frames for read and 10 frames for write. Assuming a frame size of 1500 bytes, these window sizes in terms of bytes are 24Kbytes for read and 15Kbytes for write. To find the network controller's buffer requirements for the receive section, the 24Kbyte window is considered. The resulting buffer requirement is as shown in Figure 1.
Figure 1 - Receive Buffer requirement for half-duplex IPX traffic at various latencies and burst sizes
100Mbps full-duplex for a 24Kbyte window - The IPX/VLM type traffic is simulated for a 100Mbps full-duplex operation. Results are shown in Figure 2. The results presented in Case 1 and Case 2 give an estimate of the buffer requirements for a network controller operating in a 100-Mbps Ethernet network under IPX environment. As can be seen, for low bus acquisition latencies, buffer requirement is minimal. However, as bus latency increases beyond 2ms, buffer requirements are non-trivial. Providing a buffer of the size indicated in the graphs guarantees zero overflows for 16 back-to-back frames of 1500 bytes, typical IPX/VLM traffic. To avoid boundary cases, it is advisable to design for the worst case, especially while designing network adapter cards that may be installed into the existing older machines.
Figure 2 - Receive Buffer requirement for full-duplex IPX traffic at various latencies and burst sizes
100Mbps full-duplex for a 32Kbyte window (for low performance chipsets) - As mentioned earlier, most of the study is targeted for high performance PCI chipsets. However, some of the chipsets available in the market can only do a burst of at the most 16 to 32 bytes. This is a special case where we look at the effect of systems having one of these low performance chipset. As seen in Figure 3, at lower burst sizes of 16 or 32 bytes, the buffer requirement is significant for typical latencies. This means that if one is designing a network adapter card that may be installed into low performance machines, it is necessary to provide a sufficiently large receive buffer. A network controller should, therefore, support flexible buffer architecture to permit acceptable network performance with a variety of PCI chipsets.
Figure 3 - Receive Buffer requirement in a system with low performance chipset
Since the system and network environments vary widely from one configuration to another, the data is meant to provide an insight into the effects of various parameters on network controller buffer requirements. By identifying the typical parameters in your environment and then using the PCnet-FAST's flexible FIFO architecture, overflows and underflows in a system can be avoided, thereby, enhancing the overall system and network throughput. Several cases not shown were also examined. Based on the data, the following conclusions can be drawn for various system and network configurations:
- PCI chipsets - The PCI burst size is largely dependent on the chipset being used in the system. The burst size is an important parameter in determining the receive buffer requirement.
- PCI PC system - The PCI bus acquisition latency is another very important parameter in determining the buffer requirement for a network controller. In a system with ISA bus controllers, multiple PCI bus controllers, and/or PCI-toPCI bridges, the bus acquisition latencies can be high. This will increase the receive buffer requirement to prevent the overflows.
- PCI clock speed - The clock speed has a mitigating effect on the network controller's buffer requirements as the PCI bus acquisition latency increases. At higher latencies, PCI clock speeds of 25 and 33MHz have almost the same buffer requirement.
- Half-duplex versus full-duplex - As expected, the buffer requirement in fullduplex operation is higher than that for the half-duplex operation.
- TCP and IP protocols - For currently available systems, the buffer requirements of a network controller may not be trivial unless low latency can be guaranteed for all times. To avoid costly overflows and to take maximum advantage of large TCP and IPX window sizes, a system should be designed with optimum buffers.
The transmit buffer need not be much larger than the maximum frame size. A transmit buffer of 24Kbytes is sufficient for most applications. However, in full-duplex 100Mbps Ethernet environment with high bus latency, underflows can be prevented by buffering an entire frame before the start of transmission. Under the present PCI bus based PC system scenario, it can be concluded that the network controller buffer requirements, to ensure maximum network and system throughput, will vary widely. Clearly a flexible buffer architecture is necessary to support the current needs, especially while designing network adapter cards.
Click here to go back to my Home Page
This page has been accessed
Last revised: Saturday, 15 May 1999