Network Working Group D. Walton Internet Draft A. Retana Expiration Date: May 13, 2010 E. Chen Cisco Systems J. Scudder Juniper Networks November 12, 2009 BGP Persistent Route Oscillation Solutions draft-walton-bgp-route-oscillation-stop-02.txt Status of this Memo This Internet-Draft is submitted to IETF in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/1id-abstracts.html The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html This Internet-Draft will expire on May 13, 2010. Copyright Notice Copyright (c) 2009 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents in effect on the date of publication of this document (http://trustee.ietf.org/license-info). Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Walton, et al [Page 1] draft-walton-bgp-route-oscillation-stop-02.txtNovember 2009 Abstract In this document we present two sets of paths for an address prefix that can be advertised by a BGP route reflector or confederation ASBR to eliminate the MED-induced route oscillations in a network. The first set involves all the available paths, and would achieve the same routing consistency as the full IBGP mesh. The second set, which is a subset of the first one, involves the neighbor-AS based Group Best Paths, and would be sufficient to eliminate the MED- induced route oscillations (subject to certain commonly adopted topological constrains). 1. Introduction As documented in [RFC3345], the routing information reduction by BGP Route Reflection [RFC4456] or BGP Confederation [RFC5065] can result in persistent IBGP route oscillations with certain routing setup and network topologies. Except for a couple artificially engineered network topologies, the MED attribute [RFC4271] has played a pivotal role in virtually all of the known persistent IBGP route oscillations. For the sake of brevity, we use the term "MED-induced route oscillation" hereafter to refer to a persistent IBGP route oscillation in which the MED plays a role. In order to eliminate the MED-induced route oscillations and to achieve consistent routing in a network, clearly a route reflector or a confederation ASBR needs to advertise more than just the best path for an address prefix. Our goal is to identify the "right" set of paths for an address prefix that needs to be advertised by a route reflector or a confederation ASBR. In this document we present two sets of paths for an address prefix that can be advertised by a BGP route reflector or confederation ASBR to eliminate the MED-induced route oscillations in a network. The first set involves all the available paths, and would achieve the same routing consistency as the full IBGP mesh. The second set, which is a subset of the first one, involves the neighbor-AS based Group Best Paths, and would be sufficient to eliminate the MED- induced route oscillations (subject to certain commonly adopted topological constrains). These paths can be advertised using the mechanism described in [ADD- PATH] for advertising multiple paths. Walton, et al [Page 2] draft-walton-bgp-route-oscillation-stop-02.txtNovember 2009 1.1. Specification of Requirements The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. 2. Advertise the Available Paths Observe that in a network that maintains a full IBGP mesh all the BGP speakers have consistent and equivalent routing information. Such a network is thus free of the MED-induced route oscillations and other routing inconsistencies such as forwarding loops. Therefore one approach is to allow a route reflector or a confederation ASBR to advertise all the available paths for an address prefix. Clearly this approach would yield the same amount of routing information and achieve the same routing consistency as the full IBGP mesh in a network. This approach can be implemented using the mechanism described in [ADD-PATH] for advertising multiple paths for certain prefixes. For the sake of scalability the advertisement of multiple paths should be limited to those prefixes which are affected by MED-induced route oscillation in a network carrying a large number of alternate paths. 3. Advertise the Group Best Paths The term neighbor-AS for a route refers to the neighboring AS from which the route was received. The calculation of the neighbor-AS is specified in Sect. 9.1.2.2 of [RFC4271], and Section 7.2 of [RFC5065]. By definition the MED is comparable only among routes with the same neighbor-AS. Thus the route selection procedures specified in [RFC4271] would conceptually involve two steps: first organize the paths for an address prefix into groups according to their respective neighbor-AS's, and calculate the most preferred one (termed "Group Best Path") for each of the groups; Then calculate the overall best path among all the Group Best Paths. As a generally recommended [RFC4456, RFC5065] and widely adopted practice, a route reflection cluster or a confederation sub-AS should be designed such that the IGP metrics for links within a cluster (or confederation sub-AS) are much smaller than the IGP metrics for the links between the clusters (or confederation sub-AS). This practice helps achieve consistent routing within a route reflection cluster or Walton, et al [Page 3] draft-walton-bgp-route-oscillation-stop-02.txtNovember 2009 a confederation sub-AS. When the aforementioned practice for devising a route reflection cluster or confederation sub-AS is followed in a network, we claim that the advertisement of all the Group Best Paths by a route reflector or a confederation ASBR is sufficient to eliminate the MED-induced route oscillations in the network. This claim is validated in Appendix A. Note that a Group Best Path for an address prefix can be identified by the combination of the address prefix and the neighbor-AS. Thus this approach can be implemented using the mechanism described in [ADD-PATH] for advertising multiple paths, and in this case the neighbor-AS of a path may be used as the path identifier of the path. It should be noted that the approach of advertising the Group Best Paths requires certain topological constrains to be satisfied in order to eliminate the MED-induced route oscillation. In addition, the BGP speakers still depend on the route selection by the route reflector or the confederation ASBR. As the route selection involves the comparison of the nexthop's IGP metrics which are specific to a particular BGP speaker, the routing information advertised by a route reflector or a confederation ASBR may still be inadequate to avoid other routing inconsistencies such as forwarding loops in certain networks. 4. Route Reflection and Confederation To allow a route reflector or a confederation ASBR to advertise either the Available Paths or Group Best Paths using the mechanism described in [ADD-PATH], the following revisions are proposed for BGP route reflection and BGP Confederation. 4.1. Route Reflection Depending on the configuration, for a particular a route reflector SHOULD include the with the "Send/Receive" field set to 2 or 3 in the ADD-PATH Capability [ADD-PATH] advertised to an IBGP peer. When the ADD-PATH Capability is also received from the IBGP peer with the "Send/Receive" field set to 1 or 3 for the same , then the following procedures shall be followed: If the peer is a route reflection client, the route reflector SHOULD advertise to the peer the Group Best Paths (or the Available Paths) received from its non-client IBGP peers. Depending on the configuration, the route reflector MAY also advertise to the peer the Walton, et al [Page 4] draft-walton-bgp-route-oscillation-stop-02.txtNovember 2009 Group Best Paths (or the Available Paths) received from its clients. If the peer is a non-client, the route reflector SHOULD advertise to the peer the Group Best Paths (or the Available Paths) received from its clients. 4.2. Confederation Depending on the configuration, for a particular a confederation ASBR SHOULD include the with the "Send/Receive" field set to 2 or 3 in the ADD-PATH Capability [ADD- PATH] advertised to an IBGP peer, and to a confederation external peer. When the ADD-PATH Capability is also received from the IBGP peer or the confederation external peer with the "Send/Receive" field set to 1 or 3 for the same , then the following procedures shall be followed: If the peer is internal, the confederation ASBR SHOULD advertise to the peer the Group Best Paths (or the Available Paths) received from its confederation external peers. If the peer is confederation external, the confederation ASBR SHOULD advertise to the peer the Group Best Paths (or the Available Paths) received from its IBGP peers. 5. Deployment Considerations Some route oscillations, once detected, can be eliminated by simple configuration workarounds. As carrying additional paths impacts the memory usage and routing convergence in a network, it is recommended that the impact be evaluated and the approach of using a configuration workaround be considered in deciding whether to deploy the proposed mechanism in a network. In addition, the advertisement of multiple paths should be limited to those prefixes which are affected by MED-induced route oscillation While the route reflectors or confederation ASBRs in a network need to advertise the Group Best Paths or Available Paths, the vast majority of the BGP speakers in the network only need to receive the Group Best Paths or Available Paths, which would involve only minor software changes. It should be emphasized that in order to eliminate the MED-induced route oscillations in a network using the approach of advertising the Group Best Paths, the recommended practice for devising a route reflection cluster or confederation sub-AS with respect to the IGP Walton, et al [Page 5] draft-walton-bgp-route-oscillation-stop-02.txtNovember 2009 metrics [RFC4456, RFC5065] should be followed. It is expected that the approach of advertising the Group Best Paths would be adequate to achieve consistent routing for the vast majority of the networks. For a network that has large number of alternate paths, the approach should be a good choice as the number of paths advertised by a reflector or a confederation ASBR is bounded by the number of the neighbor-AS's for a particular address prefix. The additional states for an address prefix would also be per neighbor-AS based rather than per path based. The number of the neighbor-AS's for a particular address prefix is typically small because of the limited number of upstream providers for a customer and the nature of advertising only customer routes at the inter-exchange points. The approach of advertising the Group Best Paths, however, may still be inadequate for certain networks to avoid other routing inconsistencies such as forwarding loops. The required topological constrains could also be operationally challenging. In these cases the approach of advertising the Available Paths may be used, but should be limited to those prefixes which are affected by MED-induced route oscillation in a network carrying a large number of alternate paths. Note that the number of paths that need to be maintained and advertised can be greatly reduced by accepting the IGP metric based MEDs from other peering networks. 6. Security Considerations This extension to BGP does not change the underlying security issues inherent in the existing BGP [RFC4271]. 7. Acknowledgments We would like to thank David Cook and Naiming Shen for their contributions to the design and development of the solutions. Walton, et al [Page 6] draft-walton-bgp-route-oscillation-stop-02.txtNovember 2009 8. References 8.1. Normative References [RFC4271] Rekhter, Y., T. Li, and S. Hares, "A Border Gateway Protocol 4 (BGP-4)," RFC 4271, January 2006. [RFC4456] Bates, T., R. Chandra, and E. Chen "BGP Route Reflection - An Alternative to Full Mesh Internal BGP (IBGP)", RFC 4456, April 2006. [RFC5065] Traina, P., D. McPherson, and J. Scudder, "Autonomous System Confederations for BGP", RFC 5065, August 2007. [ADD-PATH] Walton, D., A. Retana, E. Chen, and J. Scudder "Advertisement of Multiple Paths in BGP", draft-ietf-idr-add-path- 02.txt, August 2009. [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels," RFC 2119, BCP 14, March 1997. 8.2. Informative References [RFC3345] McPherson, D., V. Gill, D. Walton, and A. Retana, "Border Gateway Protocol (BGP) Persistent Route Oscillation Condition", RFC 3345, August 2002. Appendix A. Why the Group Best Paths Are Adequate? It is assumed that the following common practice is followed. A route reflection cluster or a confederation sub-AS should be designed such that the IGP metrics for links within a cluster (or confederation sub-AS) are much smaller than the IGP metrics for the links between the clusters (or confederation sub-AS). This practice helps achieve consistent routing within a route reflection cluster or a confederation sub-AS. Observe that in a network that maintains full IBGP mesh only the paths that survive the (Local_Pref, AS-PATH Length, Origin, MED) comparisons [RFC4271] would contribute to the route selection in the network. Consider a route reflection cluster that sources one or more paths that would survive the (Local_Pref, AS-PATH Length, Origin, MED) comparisons among all the paths in the network. One of these Walton, et al [Page 7] draft-walton-bgp-route-oscillation-stop-02.txtNovember 2009 surviving paths would be selected as the Group Best Path by the route reflector in the cluster. Due to the constrain on the IGP metrics as described previously, this path would remain as the Group Best Path and would be advertised to all other clusters even after a path is received from another cluster. On the other hand, when no path in a route reflection cluster would survive the (Local_Pref, AS-PATH Length, Origin, MED) comparisons among all the paths in the network, the Group Best Path (when exists) for a route reflector would be from another cluster. Clearly the advertise of the Group Best Path by the route reflector to the clients only depends on the paths received from other clusters. Therefore there is no MED-induced route oscillation in the network as the advertisement of a Group Best Path to a peer does not depend on the paths received from that peer. The claim for the confederation can be validated similarly. Authors' Addresses Daniel Walton Cisco Systems, Inc. 7025 Kit Creek Rd. Research Triangle Park, NC 27709 Email: dwalton@cisco.com Alvaro Retana Cisco Systems, Inc. 7025 Kit Creek Rd. Research Triangle Park, NC 27709 Email: aretana@cisco.com Enke Chen Cisco Systems, Inc. 170 W. Tasman Dr. San Jose, CA 95134 Email: enkechen@cisco.com John Scudder Juniper Networks Walton, et al [Page 8] draft-walton-bgp-route-oscillation-stop-02.txtNovember 2009 Email: jgs@juniper.net Walton, et al [Page 9]