Internet Engineering Task Force Gorry Fairhurst INTERNET DRAFT University of Aberdeen Lloyd Wood Cisco Systems Ltd draft-ietf-pilc-link-arq-issues-01.txt March 2001 Expires: September 2001 Link ARQ issues for IP traffic STATUS OF THIS MEMO This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress". The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html ABSTRACT This document discusses the use of automatic repeat request (ARQ) procedures as an option in the design of a link protocol, and the impact of the use of ARQ on IP traffic travelling over the link. ARQ is described in a general way that includes its use over a wide range of underlying physical media, including cellular wireless, wireless LANs, RF links, and other types of bearer channel. This document is intended to outline matters for consideration by, and to provide guidance to, designers and implementers of future link ARQ mechanisms. The material presented here is intended to be combined with the PILC 'Advice for Internet Subnetwork Designers' draft [DRAFTKARN00] to provide a single document giving guidance to subnetwork designers. PILC WORKING GROUP INTERNET DRAFT expires September 2001 1 Link ARQ issues for IP traffic March 2001 CONTENTS 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 1.1 Link ARQ 3 1.2 Causes of packet loss on a link . . . . . . . . . . . . .3 1.3 Why use ARQ? 4 1.4 Commonly-used ARQ techniques . . . . . . . . . . . . . . 5 1.4.1 Stop-and-wait ARQ 5 1.4.2 Sliding-window ARQ. . . . . . . . . . . . . . . . .5 1.5 Causes of delay across a link 6 2. ARQ persistence. . . . . . . . . . . . . . . . . . . . . . . 6 2.1 Perfectly-persistent (reliable) ARQ protocols 7 2.2 High-persistence (highly-reliable) ARQ protocols. . . . .8 2.3 Low-persistence (partially-reliable) ARQ protocols 9 2.4 Choosing your persistency. . . . . . . . . . . . . . . .10 3. Packet ordering 11 3.1 Using link ARQ to support multiple flows. . . . . . . . 12 3.2 Differentiation of link service classes 13 4. Conclusions. . . . . . . . . . . . . . . . . . . . . . . . .14 5. Security implications 16 6. Acknowledgements. . . . . . . . . . . . . . . . . . . . . . 16 7. References 16 Authors' addresses. . . . . . . . . . . . . . . . . . . . . . .19 Full copyright statement 19 1. INTRODUCTION Over the years, traffic using the Internet Protocol (IP [RFC791]) has been carried over a wide variety of links, of vastly different capacities, delays and loss characteristics. IP traffic can be expected to be carried over a very wide variety of new and existing link designs in the future. This document is intended for three distinct groups of readers: a. Link designers wishing to configure (or tune) a link for the IP traffic that it will carry, using standard link-layer mechanisms such as the ISO High-level Data Link Control (HDLC) [ISO4335] or its derivatives. b. Link implementers who may wish to design new link mechanisms that perform well for IP traffic when carrying IP. c. The community of people using or developing TCP, UDP and related protocols, who may wish to be aware of the ways in which links can operate. The primary audiences are intended to be groups a. and b. Group c. should not need to be aware of the exact details of an ARQ scheme across a single link, and should not have to consider it in their PILC WORKING GROUP INTERNET DRAFT expires September 2001 2 Link ARQ issues for IP traffic March 2001 implementations; much of the Internet runs across links that do not use any form of ARQ. However, the TCP/IP community does need to be aware that the IP protocol operates over a diverse range of underlying subnetworks. This document may help to raise that awareness. Some familiarity with TCP [RFC2581, RFC2488, STE94] is assumed here. There are a number of variants of TCP, with differing levels of sophistication in their procedures for handling loss recovery and congestion avoidance. Far from being static, the TCP protocol is itself subject to ongoing gradual refinement and evolution, e.g. [RFC2760]. This document should be read in conjunction with other documents produced by the Performance Implications of Link Characteristics (PILC) IETF workgroup [DRAFTDAW00, DRAFTKARN00, DRAFTBOR00]. 1.1 LINK ARQ At the link layer, ARQ operates on blocks of data, known as frames, and attempts to deliver frames from the sender to the receiver over a physical link. Frames often have a small maximum size for convenience of processing by media-access control (MAC) and link- layer protocols. This contrasts with the variable lengths of IP datagrams, or 'packets'. The ARQ link mechanism provides an integrity check and a retransmission process to retransmit lost or errored frames. Link ARQ protocols operate within the link layer of the OSI reference model [DRAFTKARN00]. A frame may contain one or more IP datagrams, although transparent implicit fragmentation is often used so that each frame will contain only a fraction of an IP datagram [KEN88]. Explicit network-layer IP fragmentation is undesirable for a variety of reasons, and should be avoided [KEN88]. Its use can be minimised with use of Path MTU discovery for TCP connections [RFC1191, RFC1435]. 1.2 CAUSES OF PACKET LOSS ON A LINK Not all packets sent to a link are necessarily received successfully by the receiver at the other end of the link. There are a number of possible causes of packet loss that may occur as frames travel across a link. These include: a. Loss due to channel noise, often characterised by random frame loss. PILC WORKING GROUP INTERNET DRAFT expires September 2001 3 Link ARQ issues for IP traffic March 2001 b. Frame loss due to channel interference. This interference can be random, structured, and in some cases even periodic. c. A link outage, causing loss due to unavailability of the physical link for a period of time. This is a common characteristic of some types of link, e.g. mobile cellular radio. In the outage, the link loses all or virtually all frames for a period of time, until the link is restored. Other forms of packet loss are not related to channel conditions, but include: i. Packet discards due to congestion. Queues will eventually overflow as the arrival rate of new packets to send continues to exceed the outgoing packet transmission rate over the link. ii. Loss due to implementation errors, including hardware faults and software errors. The levels of loss and patterns of loss experienced are functions of the design of the link's physical and link layers. These vary significantly across different link configurations. The performance of a specific implementation may also vary considerably across the same link configuration when operated over different types of physical channel. 1.3 WHY USE ARQ? Reasons that encourage considering the use of ARQ include: a. ARQ across a single link has a shorter control loop than the end- to-end path over which TCP must operate. ARQ can therefore provide more rapid retransmission for TCP segments lost on the link, at least for a reasonable number of retries [DRAFTDAW00, SAL81]. b. Link ARQ can operate on individual frames, which are much smaller than IP datagrams and may therefore permit more efficient recovery in terms of repetition of data after occasional losses. For any specific target error rate there is an optimal trade-off between header overhead and frame size. c. A link ARQ procedure may be able to use local knowledge that is not available to endhosts, including information about the state of the link and physical layer techniques, in order to optimise delivery performance for the current link conditions. This information can include knowledge of the current available transmission rate, the prevailing error environment, or available transmit power in wireless links. PILC WORKING GROUP INTERNET DRAFT expires September 2001 4 Link ARQ issues for IP traffic March 2001 1.4 COMMONLY-USED ARQ TECHNIQUES Two basic categories of ARQ retransmission process are widely used: 1.4.1 STOP-AND-WAIT ARQ A stop-and-wait ARQ sender transmits a single frame and then waits for an acknowledgement from the receiver for that frame, before either continuing transmission with the next frame, or repeating transmission of the same frame if the original frame is errored or lost. Stop-and-wait ARQ is simple for protocol designers to implement, e.g. tftp [RFC1350]. However, when stop-and-wait ARQ is used in the link layer, it is well-suited only to links with low bandwidth-delay products. This technique is not discussed further in this document. 1.4.2 SLIDING-WINDOW ARQ A sliding-window link ARQ protocol numbers every frame with a unique sequence number, according to a modulus. TCP is itself a sliding- window protocol, so similarities between this link-interface-to- link-interface protocol and end-to-end TCP may be recognisable. A sliding-window link protocol is much more complex in implementation than a simpler stop-and-wait protocol, particularly if per-flow ordering is preserved. At any time the link sender may have a number of frames outstanding and awaiting acknowledgement, up to the space available in its transmission window. A sufficiently large link sender window (equivalent to or greater than the number of frames sent, or larger than the bandwidth*delay capacity of the link) enables continuous transmission of new frames. A smaller link sender window causes the sender to pause transmission of new frames until a timeout or a control frame, such as an acknowledgement, is received. When frames are lost, a larger window, i.e. more than the link's bandwidth*delay product, is needed to allow continuous operation while frame retransmission takes place. The modulus numbering space determines the size of the frame header sequence number field. This sequence space needs to be larger than the link window size, and if using selective frame repeat, larger than twice the link window size. For continuous operation, the sequence space should be larger than the bandwidth-persistence product, so that in-flight frames can be identified uniquely. As with TCP, which provides sliding-window delivery across an entire path, rather than a single link, there are a number of variations on the basic sliding-window implementation, with increased complexity and sophistication to make them suitable for various conditions. PILC WORKING GROUP INTERNET DRAFT expires September 2001 5 Link ARQ issues for IP traffic March 2001 1.5 CAUSES OF DELAY ACROSS A LINK Links and link protocols contribute to the total path delay experienced between communicating applications on hosts. Delay has a number of causes, including: a. Input packet queuing and frame buffering at the link head before transmission over the link. b. Waiting for access to the allocated channel when the physical medium is shared. There may be processing or protocol-induced delay before transmission takes place [FER99, PAR00]. c. Frame serialisation and transmission processing. These are functions of frame size and the transmission speed of the link. d. Physical link propagation time, caused by the speed of light in the channel medium. e. Processing, including buffering frame contents for packet reassembly, at the receiver before onward transmission of the packet. When link ARQ is used, steps b., c. and d. may be repeated together at intervals a number of times after a frame is lost, increasing overall delay for the packet that the frame partially transmits. It is important to understand that applications at the endhosts are unaware of the individual delays contributed by each link in the path, and only see the overall path delay. Application performance is determined by the cumulative delay of the entire end-to-end Internet path. This path may include an arbitrary or even widely- varying number of links, where each link may or may not use ARQ. 2. ARQ PERSISTENCE ARQ protocols may be characterised by their persistence. Persistence is the willingness of the protocol to retransmit lost frames to ensure reliable delivery of the stream of traffic across the link. A link's retransmission persistency defines how long the link is allowed to delay an IP packet, in an attempt to transmit all the frames carrying the packet and its content over the link, before giving up and discarding the packet. This persistency is normally measured in milliseconds, but may, if the link propagation delay is specified, also be expressed in terms of the maximum number of link retransmission attempts permitted. PILC WORKING GROUP INTERNET DRAFT expires September 2001 6 Link ARQ issues for IP traffic March 2001 An example of a reliable link protocol that is perfectly persistent is the ISO HDLC protocol in the Asynchronous Balanced Mode (ABM) [ISO4335]. A protocol that only retransmits a fixed number of times before giving up is less persistent, e.g. Ethernet [FER99] or GSM RLP [RFC2757]. Here, lower persistence ensures stability and fair sharing of the shared subnetwork, even when many senders are attempting retransmissions. In a broadcast wireless environment, the interval between repeat attempts will be set to be greater than the link round trip delay, and frequently increases with the number of retransmission attempts. This is similar to having increasing backoffs in Ethernet / CSMA to decrease offered channel load in the presence of impairments. A wireless receiver may experience lost or errored frames due to fading, shadowing or interference on its channel from the wireless sender. In this situation, it is wasteful for the sender to consume shared capacity when retransmissions are likely to experience the same conditions; other receivers can benefit from that capacity. One simple solution is to delay retransmissions to that receiver in the hope that the channel conditions will improve. A complementary approach is to adapt the way in which retransmitted information is sent over the channel, e.g. by adapting modulation or coding. The interval or delay between retransmission attempts may be varied as a function of the perceived channel state and the number of previous retransmission attempts. Perfect reliability is not a requirement for IP subnetworks [DRAFTKARN00]. IP networks may discard packets due to a variety of reasons entirely unrelated to link errors, including lack of queuing space, congestion management, faults, and route changes. It has long been widely understood that perfect reliability can be obtained only at the transport layer [SAL81]. TCP [RFC2488, RFC2581, STE94] and a number of applications using UDP implement their own end-to-end reliable delivery mechanisms. Many TCP and UDP applications, e.g. streaming multimedia, benefit from timely delivery from lower layers with sufficient reliability, rather than perfect reliability with increased link delays. 2.1 PERFECTLY-PERSISTENT (RELIABLE) ARQ PROTOCOLS Most well-known link protocols are reliable protocols, which use perfectly-persistent ARQ to ensure that all frames (and therefore all packets) are received by the other side of the link. The perfectly-persistent ARQ protocol will repeat a lost or errored frame an indefinite number of times until the frame is successfully received. If traffic is going no further than across one link, this ensures reliability between the two link ends without requiring higher- PILC WORKING GROUP INTERNET DRAFT expires September 2001 7 Link ARQ issues for IP traffic March 2001 layers protocols. This reliability can be counterproductive for traffic traversing multiple links, as it interacts with protocol mechanisms at higher layers. Arguments against perfect persistence for IP traffic include: a. Variable link delay; the impact of ARQ introduces a degree of jitter, a function of the link's physical delay and frame serialisation and transmission times, to all flows sharing the link performing frame retransmission. b. High persistence does not provide a clear upper bound on the maximum retransmission delay for the link. Significant changes in path delay caused by excessive link retransmissions may lead to timeout of TCP retransmission timers, although a high variance in link delay and the resulting overall path delay may also cause a large TCP Retransmission Timeout (RTO) value to be used [LUD99a, PAR00]. Future TCP implementations may use techniques that reduce premature expiry of this timer, e.g. Eifel [LUD00] and Rapid Packet Loss Detection [SAM98], but further research is required in this area. c. Applications needing perfectly-reliable delivery can implement a form of perfectly-persistent ARQ themselves, or use a reliable transport protocol within the endhosts. Implementing perfect persistence at each link along the path between the endhosts is redundant, and cannot guarantee the same reliability [SAL81]. It may also affect other end-to-end protocol mechanisms, e.g. using Explicit Congestion Notification (ECN). Examples of ARQ protocols that are perfectly persistent include ISO/ITU-T LAP-B [ISO7776] and ISO HDLC in the Asynchronously Balanced Mode (ABM) [ISO4335], e.g. using Multiple Selective Reject (MSREJ [ISO4335a]). These protocols will retransmit a frame an unlimited number of times until receipt of the frame is acknowledged. 2.2 HIGH-PERSISTENCE (HIGHLY-RELIABLE) ARQ PROTOCOLS High-persistence ARQ protocols attempt only a finite, large, number of transmissions of each frame before giving up. Arguments against high persistence are similar to those against perfect persistence, except that infinite delay and perfectly reliable delivery are traded off against imperfect reliability and an upper bound on link delay. It has been recommended that a single IP datagram should never be delayed by more than the Maximum Segment Lifetime (MSL) of 120 seconds defined for TCP [RFC1122]. Failure to limit the maximum datagram lifetime can result in sequence number wrapping at high TCP transmission rates [RFC1323], where old data segments may be PILC WORKING GROUP INTERNET DRAFT expires September 2001 8 Link ARQ issues for IP traffic March 2001 confused with newer segments where the sequence number space has been exhausted and reused in the interim. Some TCP implementations use timestamps to remove this ambiguity [RFC1323]. (One case where segment lifetime may be significant is where alternate paths of different delays exist between hosts. Some TCP segments may follow a short path, while others follow a much longer path, e.g. using persistent ARQ over a link outage.) In practice, the MSL is usually very large compared to the typical TCP RTO. The calculation of RTO is based on estimated round-trip path delay. If the number of link retransmissions causes a path delay larger than the value of RTO, the TCP retransmission timer will expire, leading to a timeout and retransmission by the TCP sender. Although high persistency may benefit bulk flows, the additional delay (and variations in delay) may be highly undesirable for other types of flows. Being able to treat flows separately with different classes of link service is useful, and is discussed in section 3.2. Examples of highly-persistent ARQ include [BHA97, ECK98, LUD99, MEY99]. 2.3 LOW-PERSISTENCE (PARTIALLY-RELIABLE) ARQ PROTOCOLS The characteristics of a link using a low-persistence ARQ protocol may be summarised as: a. The link is not perfectly reliable and does not provide an absolute guarantee of delivery; i.e. the transmitter will discard some frames as it 'gives up' before receiving an acknowledgement of successful transmission across the link. b. There is a lowered limit on the maximum added delay that IP datagrams will experience when using the link. This reduces interaction with TCP timers or with UDP-based error-control schemes. c. The link bounds the time that retransmission for any one frame can block completed transmission and assembly of other correctly- received datagrams originally sent before the missing frame. Limiting delay avoids aggravating contention and interaction between different packet flows (see also 3.2). Selection of even a quite high persistence, e.g. allowing tens of attempts to deliver a single frame, would usually satisfy a bound for b. for low-delay links. However, a packet may traverse a number PILC WORKING GROUP INTERNET DRAFT expires September 2001 9 Link ARQ issues for IP traffic March 2001 of successive links on its total end-to-end path. This is an argument for much lower persistency on individual links. 2.4 CHOOSING YOUR PERSISTENCY The TCP Maximum RTO is an upper limit on the maximum time that TCP will wait until it performs a retransmission. Most TCP implementations will generally have a TCP RTO of typically a few times the path delay. Setting a lower link persistency (of the order 2-5 retransmission attempts) reduces interaction with the RTO timer, and may therefore reduce the probability of duplicate copies of the same packet being present in the link transmit buffer under some patterns of loss. The following table is calculated from the probability of loss, assuming the 'ideal' case of random loss. It shows the probability of successful frame transmission using an idealised ARQ procedure, i.e. one in which the control frames are not corrupted. Frame loss (random) Persistence (Link RTT) Average frame loss after link ARQ 1% x3 0.0001% 1% x5 0.00000001% 3% x3 0.003% 3% x5 0.000002% 10% x3 0.1% 10% x5 0.001% 10% x7 0.00001% 30% x3 3% 30% x5 0.2% 30% x7 0.02% 30% x9 0.002% 50% x3 12% 50% x5 3% 50% x7 0.8% 50% x9 0.2% When interpreting these results, remember that: a. Link protocols normally employ transparent fragmentation and all frames containing fragments must be delivered to reassemble an IP packet. b. Channel error conditions are seldom characterised by random loss. PILC WORKING GROUP INTERNET DRAFT expires September 2001 10 Link ARQ issues for IP traffic March 2001 c. In these calculations, link persistence is expressed in terms of multiples of the Link Round Trip Time (Link RTT). Some implementers have chosen a lower persistence, falling back on the judgement of TCP or a UDP application to retransmit any packets that are not recovered by the link ARQ protocol. Examples of low- persistence ARQ protocols include [SAM96, WAR95, CHE00]. 3. PACKET ORDERING A common debate is whether a link should be allowed to forward packets in an order different to that in which they were originally received at the link's transmit interface. IP networks are not required to deliver all datagrams in order, although generally networks do deliver most datagrams in their original transmission order. Routers supporting class-based queuing do reorder packets received, by reordering packets in different flows -- but these usually retain per-flow ordering. Policy-based queuing, allowing fairer access to the link, may reorder packets. There is still much debate on optimal algorithms, and on optimal queue sizes for particular link speeds. This, however, is not related to use of link ARQ and applies to any (potential) bottleneck router. Although small amounts of reordering are common in IP networks [BEN00], significant re-ordering within a flow is undesirable, as it can have several effects: a. reordering may cause unwanted side effects in poorly-implemented IP fragment reassembly code. b. reordering may cause unwanted side effects in poorly-implemented IP demultiplexing (filter) code. c. reordering may cause unwanted side effects in poorly-implemented UDP applications. d. reordering will increase packet jitter for real-time applications. This may lead to application data loss if a small play-out buffer is used by the receiving application. e. reordering may impact congestion avoidance and may negatively impact overall throughput on paths. f. reordering will interleave arrival of TCP segments, leading to generation of duplicate ACKs (dupacks [STE94, BAL97]). A sequence of three identical dupacks causes the TCP sender to trigger fast retransmission and recovery, a form of congestion avoidance, PILC WORKING GROUP INTERNET DRAFT expires September 2001 11 Link ARQ issues for IP traffic March 2001 since TCP always presumes loss due to congestion [RFC2581, STE94]. This reduces TCP throughput efficiency as far as the application is concerned, but it should not impact data integrity. Ordering effects must be considered when breaking the end-to-end paradigm and evaluating transport-level relays such as split-TCP implementations or Protocol Enhancing Proxies [DRAFTBOR00]. As with total path delay, reordering effects accumulate over multiple links. Do not assume that your link is the only link undertaking packet reordering, as some level of reordering may be introduced by other links along the same path, or by router processing within the network [BEN00]. TCP and UDP flows are impacted by the cumulative effect of reordering along the entire path. The link protocol should ideally eliminate reordering within a flow, or at least ensure that it does not significantly increase the level of reordering within the flow. A number of ARQ protocols have allowed out-of-order delivery [BAL95, SAM96, WAR95]. 3.1 USING LINK ARQ TO SUPPORT MULTIPLE FLOWS Most links can be expected to carry more than one IP traffic flow at a time. Some high-capacity links are expected to carry a very large number of simultaneous flows, often from and to a large number of different hosts. With use of multiple applications at a host, multiple flows can be considered the norm rather than the exception, even for last-hop links. When packets from several flows are simultaneously in transit within a link ARQ protocol, ARQ may cause a number of additional effects: a. ARQ introduces jitter to a TCP flow sharing a link with another flow experiencing loss. In-sequence link-level delivery of packets may adversely impact other applications sharing the link, and can increase the duration of the initial slow-start period for TCP flows for these applications. This is significant for short-lived TCP flows (e.g. those used by HTTP/1.0 and earlier), which spend most of their lives in slow-start. b. High-persistence ARQ may cause premature timeout of another TCP flow's RTO timer, although modern TCP implementations should not experience this since their computed RTO values should leave sufficient margin over path RTTs to cope with such jitter. c. ARQ introduces jitter to UDP flows. An end-to-end protocol may not require reliable delivery, particularly if it is delay- sensitive. PILC WORKING GROUP INTERNET DRAFT expires September 2001 12 Link ARQ issues for IP traffic March 2001 Reordering of packets belonging to different flows may be desirable to achieve fair sharing of the link between established bulk data transfer sessions and sessions that transmit less data but would benefit from lower link transit delay. Preserving ordering within each flow, to avoid the effects of reordering described earlier, is worthwhile. 3.2 DIFFERENTIATION OF LINK SERVICE CLASSES High ARQ persistency is generally considered unsuitable for many applications using UDP, where reliable delivery is not always required and where it may introduce unacceptable jitter, but may benefit bulk data transfers under certain link conditions. A scheme that differentiates packet flows into two or more classes, to provide different service to each class, is therefore desirable. Observation of flow behaviour can tell you which flows are controlled and congestion-sensitive, or uncontrolled and not, so that you can treat them differently and ensure fairness. However, this cannot tell you whether a flow is intended as reliable or unreliable by its application, or what the application requires for best operation. Supporting different link services for different classes of flows therefore needs an explicit indication of the class of each IP packet. Some potential schemes for indicating the class of a packet include: a. Using the Type of Service (ToS) bits in the IP header [RFC791]. The IETF has replaced these globally-defined bits, which were rarely used, with a much more complex identification scheme for use with differentiated services (diffserv [RFC2475]). Diffserv defines classes of service that are mapped to various Per Hop Behaviours (PHBs) as the packet is processed within the network. Its uses include policy-based routing, class-based queuing, and support for other QoS metrics, including packet priority, delay, reliability, cost, etc. b. Inspecting the network packet header and viewing the IP protocol type [RFC791], to gain an idea of the transport protocol used and thus guess its needs. This is not possible when carrying an encrypted payload, e.g. using the IP security extensions (IPSec) with the Encapsulation Security Payload (ESP) [RFC1827]. c. By inspecting the transport packet header information to view the TCP or UDP headers and port numbers (e.g. [PAR00, BAL95]). This is not possible when using payload encryption, e.g. IPSec with ESP [RFC1827], and incurs processing overhead in routers. PILC WORKING GROUP INTERNET DRAFT expires September 2001 13 Link ARQ issues for IP traffic March 2001 There are, however, some drawbacks to these schemes: i. The ToS/Differentiated Services Code Point (DSCP) values [RFC2475] may not be set reliably, and may be overwritten by intermediate routers along the packet's path. These values may be set by an ISP, and do not necessarily indicate the level of reliability required by the end application. ii. Tunnelling of traffic (e.g. GRE, MPLS, L2TP, IP-in-IP encapsulation) can aggregate flows of different transport classes, complicating individual flow classification with schemes b. and c. and incurring further header processing if tunnel contents are inspected. iii. Use of the port number makes assumptions about application behaviour and requirements. New applications or protocols can invalidate these assumptions. iv. In IPv6, locating the transport layer protocol type requires parsing the entire IPv6 header, adding complexity to header inspection. Again, this assumes that IPSec encryption is not used. Flow management is desirable; clear flow identification increases the number of tools available for the link designer to use, and permits more complex ARQ strategies that may otherwise make misassumptions about traffic requirements and behaviour without flow identification. Despite the difficulties in providing a framework for accurate flow identification, [RFC3002] advocates approach (a), and urges against adding optimisations to future links that are triggered by inspecting the contents of specific IP packets. Links that are unable to distinguish clearly and safely between delay-sensitive flows, e.g. real-time multimedia, DNS queries or telnet, and delay-insensitive flows, e.g. bulk ftp transfers or reliable multicast file transfer, cannot separate link service classes safely. All flows should therefore experience the same link behaviour. In general, if separation of flows according to class is not practicable, a low persistency is best for link ARQ. 4. CONCLUSIONS It is currently difficult to make simple statements about arbitrary patterns of loss that cover a wide range of link speeds and propagation delays. Some general statements follow, summarising the preceding discussion. In each case, it is recommended that specific PILC WORKING GROUP INTERNET DRAFT expires September 2001 14 Link ARQ issues for IP traffic March 2001 details of the link characteristics and mechanisms are also considered; solutions vary with conditions. ARQ has been and continues to be used on links that carry IP traffic. At low error rates, many of the details of ARQ, such as degree of persistence or resulting out-of-order delivery, become unimportant. Most frame losses will be resolved in one or two retransmission attempts, and this is generally unlikely to cause significant impact to e.g. TCP. However, on shared high-delay links, the impact of ARQ on other UDP or TCP flows may lead to unwanted jitter. In a link outage, interactions between ARQ and multiple flows are less significant; the ARQ protocol is likely to be equally unsuccessful in retransmitting frames for all flows. High persistence may also be desirable to enable prompt recovery once the channel is restored. For links with highly-variable error rates, high ARQ persistence may provide good performance for a single TCP flow. However, lower persistence may in some cases have merit, particularly when many flows share and compete for capacity on the same link. The reasoning here is that the link can perform useful work forwarding some complete packets, and that blocking all flows by retransmitting the frames of a single packet with high persistence is undesirable. Low ARQ persistence is particularly useful where it is difficult or impossible to classify traffic flows, and hence treat each flow independently, and where the link capacity can accommodate a large number of simultaneous flows. In summary, low persistence is generally desirable; preserving packet ordering is generally desirable. Extremely high persistence is generally undesirable. There is currently insufficient experience to recommend a specific ARQ scheme for any class of link. It is also important to realise that link ARQ is just one method of error recovery, and that other complementary physical-layer techniques may be used instead of, or together with, ARQ to improve overall link throughput for IP traffic. The choice of potential schemes includes adapting the data rate, adapting the signal bandwidth, adapting the transmission power, adaptive modulation, and adaptive information redundancy / forward error control (FEC). All of these schemes can be used to improve the received signal energy per bit, and hence reduce error, frame loss and resulting packet loss rates given specific physical channel conditions. There is a need for more research to more clearly identify the importance of and trade-offs between the above issues over various PILC WORKING GROUP INTERNET DRAFT expires September 2001 15 Link ARQ issues for IP traffic March 2001 types of link. It would be useful if researchers and implementers clearly indicated the loss model, link capacity and characteristics, link and end-to-end path delays, details of TCP, and the number (and details) of flows sharing a link when describing their experiences. 5. SECURITY IMPLICATIONS No security implications have been identified as directly impacting IP traffic. However, an unreliable link service may adversely impact some link-layer key management distribution protocols if encryption is used over such a link. Denial-of-service attacks exploiting the behaviour of the link protocol, e.g. using knowledge of its retransmission behaviour and propagation delay to cause a particular form of jamming, may be specific to an individual link scenario. 6. ACKNOWLEDGMENTS Much of what is described here has been developed from a summary of a subset of the discussions on the archived IETF PILC mailing list. We thank the contributors to PILC for vigorous debate. 7. REFERENCES References of the form RFCnnnn are Internet Request for Comments (RFC) documents available online at http://www.rfc-editor.org/. [BAL95] H. Balakrishnan, S. Seshan and R. H. Katz, Improving Reliable Transport and Handoff Performance in Cellular Wireless Networks, ACM MOBICOM, Berkeley, 1995. [BAL97] H. Balakrishnan, V. N. Padmanabhan, S. Seshan and R. H. Katz, A Comparison of Mechanisms for Improving TCP Performance over Wireless Links, IEEE/ACM Transactions on Networking, 5(6), pp. 756-759, 1997. [BEN00] J.C. Bennett, C. Partridge, and N. Schectman, Packet Reordering is Not Pathological Network Behaviour, IEEE/ACM Transactions on Networking, 7(6), pp. 789-798, 2000. [BHA97] P. Bhagwat, P. Bhattacharya, A. Krishna and S. K. Tripathi, Using channel state dependent packet scheduling to improve TCP throughput over wireless LANs, ACM/Baltzer Wireless Networks Journal, (3)1, 1997. PILC WORKING GROUP INTERNET DRAFT expires September 2001 16 Link ARQ issues for IP traffic March 2001 [CHE00] H. S. Cheng, G. Fairhurst et al., An Efficient Partial Retransmission ARQ Strategy with Error Codes by Feedback Channel, IEE Proceedings - Communications, (147)5, pp. 263-268, 2000. [DRAFTBOR00] J. Border, M. Kojo, J. Griner et al., Performance Enhancing Proxies, draft-ietf-pilc-pep-nn.txt, work in progress as internet-draft. [DRAFTDAW00] S. Dawkins, G. Montenegro, M. Kojo, V. Magret et al., End-to-end Performance Implications of Links with Errors, draft-ietf-pilc-error-nn.txt, work in progress as internet-draft. [DRAFTKARN00] P. Karn, A. Falk, J. Touch et al., Advice for Internet Subnetwork Designers, draft-ietf-pilc-link-design-nn.txt, work in progress as internet-draft. [ECK98] D. A. Eckhardt and P. Steenkiste, Improving Wireless LAN Performance via Adaptive Local Error Control, IEEE ICNP, 1998. [FER99] A. Ferrero, The Eternal Ethernet, Addison-Wesley, 1999. [ISO4335] HDLC Procedures: Specification for Consolidation of Elements of Procedures, ISO 4335 and AD/1, International Standardization Organization, 1985. [ISO4335a] HDLC Procedures: Elements of Procedures, Amendment 4: Multi-Selective Reject Option, ISO 4335/4, International Standards Organization, 1991. [ISO7776] Specification for X.25 LAPB-Compatible DTE Data Link Procedures, ISO 4335/4, International Standards Organization, 1985. [KEN88] C. A. Kent and J. C. Mogul, Fragmentation Considered Harmful, Proceedings of ACM SIGCOMM, pp. 390-401, 1988. [LUD99] R. Ludwig, B. Rathonyi, A. Konrad, K. Oden and A. Joseph, Multi-Layer Tracing of TCP over a Reliable Wireless Link, ACM SIGMETRICS, pp. 144-154, 1999. [LUD99a] R. Ludwig, A. Konrad, A. Joseph, and R. Katz, Optimizing the End-to-End Performance of Reliable Flows over Wireless Links, ACM MobiCOM, 1999. [LUD00] R. Ludwig and R. Katz, The Eifel Algorithm: Making TCP Robust Against Spurious Retransmissions, ACM Computer Communication Review, (30)1, January 2000. [MEY99] M. Meyer, TCP Performance over GPRS, IEEE WCNC, 1999. [PAR00] C. Parsa and J. J. Garcia-Luna-Aceves, Improving TCP Performance over Wireless Networks at the Link Layer, Mobile Networks and Applications, (1)5, pp. 57-71, 2000. PILC WORKING GROUP INTERNET DRAFT expires September 2001 17 Link ARQ issues for IP traffic March 2001 [RFC791] J. Postel, Internet Protocol, RFC 791, 1981. [RFC1122] R. Braden et al., Requirements for Internet Hosts -- Communication Layers, RFC 1122, 1989. [RFC1191] J. Mogul and S. Deering, Path MTU Discovery, RFC 1191, 1990. [RFC1323] V. Jacobson and R. Braden, TCP Extensions for High Performance, RFC 1323, 1992. [RFC1350] K. Solins, The TFTP Protocol (Revision 2), IETF RFC 1350, 1992. [RFC1435] S. Knowles, IESG Advice from Experience with Path MTU Discovery, RFC 1435, 1993. [RFC1827] R. Atkinson, IP Encapsulating Security Payload (ESP), RFC 1827, 1995. [RFC2475] S. Blake, D. Black, M. Carlson, E. Davies, Z. Wang and W. Weiss, An Architecture for Differentiated Services, RFC 2475, 1998. [RFC2488] M. Allman, D. Glover and L. Sanchez, Enhancing TCP Over Satellite Channels using Standard Mechanisms, RFC 2488, 1999. [RFC2581] M. Allman, V. Paxson and W. Stevens, TCP Congestion Control, RFC 2581, 1999. [RFC2757] S. Dawkins, M. Kojo, V. Magret and N. Vaidya, Long Thin Networks, RFC 2757, 2000. [RFC2760] M. Allman, S. Dawkins, D. Glover, J. Griner et al., Ongoing TCP Research Related to Satellites, RFC 2760, 1999. [RFC3002] D. Mitzel, Overview of 2000 IAB Wireless Internetworking Workshop, RFC 3002, 2000. [SAL81] J. H. Saltzer, D. P. Reed and D. Clark, End-to- End Arguments in System Design, Second International Conference on Distributed Computing Systems, pp. 509-512, 1988. Published with minor changes in ACM Transactions in Computer Systems (2)4, pp. 277-288, 1984. [SAM96] N. Samaraweera and G. Fairhurst, Robust Data Link Protocols for Connection-less Service over Satellite Links, International Journal of Satellite Communications, 14(5), pp. 427-437, 1996. [SAM98] N. Samaraweera and G. Fairhurst, Reinforcement of TCP/IP Error Recovery for Wireless Communications, ACM CCR, 28(2), pp. 30-38, 1998. PILC WORKING GROUP INTERNET DRAFT expires September 2001 18 Link ARQ issues for IP traffic March 2001 [STE94] W. R. Stevens, TCP/IP Illustrated, Volume 1, Addison-Wesley, 1994. [WAR95] C. Ward et al., A Data Link Control Protocol for LEO Satellite Networks Providing a Reliable Datagram Service, IEEE/ACM Transactions on Networking, 3(1), 1995. AUTHORS' ADDRESSES Gorry Fairhurst (gorry@erg.abdn.ac.uk) Department of Engineering, University of Aberdeen, Aberdeen AB24 3UE, United Kingdom. Lloyd Wood (lwood@cisco.com) Cisco Systems Ltd, 4 The Square, Stockley Park, Uxbridge UB11 1BY, United Kingdom. FULL COPYRIGHT STATEMENT Copyright (C) The Internet Society (2000-2001). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. PILC WORKING GROUP INTERNET DRAFT expires September 2001 19