BGP Protocol.pdf

About Global Documents

Global Documents provides you with documents from around the globe on a variety of topics for your enjoyment.
Global Documents utilizes edocr for all its document needs due to edocr's wonderful content features. Thousands of professionals and businesses around the globe publish marketing, sales, operations, customer service and financial documents making it easier for prospects and customers to find content.
Cisco System reserved APNIC BGP route peer Community route Reflector Path
1
© 2005 Cisco Systems, Inc. All rights reserved.
Session Number
Presentation_ID
Cisco Confidential
APNIC 22
BGP Tutorial
Philip Smith
<pfs@cisco.com>
APNIC 22
4th-8th September 2006
Kaohsiung, Taiwan
2
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Tutorial Outline
• Part 1
Introduction to BGP
Coffee Break
• Part 2
Deploying BGP
Lunch
• Part 3
Multihoming
Coffee Break
• Part 4
Troubleshooting
3
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Presentation Slides
• Are available on
ftp://ftp-eng.cisco.com
/pfs/seminars/APNIC22-BGP-part1+2.pdf
And on the APNIC 22 website
• Feel free to ask questions any time
4
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
BGP Techniques for Internet Service
Providers
• BGP Basics
• Scaling BGP
• Using Communities
• Deploying BGP in an ISP network
5
© 2005 Cisco Systems, Inc. All rights reserved.
Session Number
Presentation_ID
Cisco Confidential
APNIC 22
BGP Basics
What is this BGP thing?
6
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Border Gateway Protocol
• Routing Protocol used to exchange routing information between
networks
exterior gateway protocol
• Described in RFC4271
RFC4276 gives an implementation report on BGP-4
RFC4277 describes operational experiences using BGP-4
• The Autonomous System is BGP’s fundamental operating unit
It is used to uniquely identify networks with common routing
policy
7
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Autonomous System (AS)
• Collection of networks with same routing policy
• Single routing protocol
• Usually under single ownership, trust and administrative
control
• Identified by a unique number
AS 100
8
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Autonomous System Number (ASN)
• An ASN is a 16 bit number
1-64511 are assigned by the RIRs
64512-65534 are for private use and should never appear on
the Internet
0 and 65535 are reserved
• 32 bit ASNs are coming soon
www.ietf.org/internet-drafts/draft-ietf-idr-as4bytes-12.txt
With AS 23456 reserved for the transition
9
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Autonomous System Number (ASN)
• ASNs are distributed by the Regional Internet Registries
• Also available from upstream ISPs who are members of
one of the RIRs
Current ASN allocations up to 41983 have been made to the
RIRs
Of these, around 23000 are visible on the Internet
See www.iana.org/assignments/as-numbers
10
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
BGP Basics
• Runs over TCP – port 179
• Path vector protocol
• Incremental updates
• “Internal” & “External” BGP
AS 100
AS 101
AS 102
E
B
D
A
C
Peering
11
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
AS 100
AS 101
AS 102
DMZ
Network
A
B
C
D
E
• Shared network between ASes
Demarcation Zone (DMZ)
12
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
BGP General Operation
• Learns multiple paths via internal and external BGP
speakers
• Picks the best path and installs in the forwarding
table
• Best path is sent to external BGP neighbours
• Policies applied by influencing the best path
selection
13
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
eBGP & iBGP
• BGP used internally (iBGP) and externally (eBGP)
• iBGP used to carry
some/all Internet prefixes across ISP backbone
ISP’s customer prefixes
• eBGP used to
exchange prefixes with other ASes
implement routing policy
14
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
BGP/IGP model used in ISP networks
• Model representation
IGP
iBGP
IGP
iBGP
IGP
iBGP
IGP
iBGP
eBGP
eBGP
eBGP
15
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
External BGP Peering (eBGP)
AS 100
AS 101
C
A
• Between BGP speakers in different AS
• Should be directly connected
• Never run an IGP between eBGP peers
B
16
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Internal BGP (iBGP)
• BGP peer within the same AS
• Not required to be directly connected
IGP takes care of inter-BGP speaker connectivity
• iBGP speakers need to be fully meshed
they originate connected networks
they do not pass on prefixes learned from other iBGP
speakers
17
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Internal BGP Peering (iBGP)
• Topology independent
• Each iBGP speaker must
peer with every other
iBGP speaker in the AS
AS 100
A
D
C
B
18
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Peering to loopback addresses
• Peer with loop-back address
Loop-back interface does not go down – ever!
• iBGP session is not dependent on
State of a single interface
Physical topology
AS 100
19
© 2005 Cisco Systems, Inc. All rights reserved.
Session Number
Presentation_ID
Cisco Confidential
APNIC 22
BGP Attributes
Information about BGP
20
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
• Sequence of ASes a route has
traversed
• Loop detection
• Apply policy
AS-Path
AS 100
AS 300
AS 200
AS 500
AS 400
170.10.0.0/16
180.10.0.0/16
150.10.0.0/16
180.10.0.0/16
300  200 100
170.10.0.0/16
300  200
150.10.0.0/16
300  400
180.10.0.0/16   300  200  100
170.10.0.0/16   300  200 
21
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
AS-Path loop detection
AS 100
AS 300
AS 200
AS 500
170.10.0.0/16
180.10.0.0/16
180.10.0.0/16
300  200 100
170.10.0.0/16
300  200
140.10.0.0/16
300
140.10.0.0/16
500  300
170.10.0.0/16
500  300  200
140.10.0.0/16
180.10.0.0/16 is not
accepted by AS100 as the
prefix has AS100 in its
AS-PATH attribute – this
is loop detection in action
22
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Next Hop
160.10.0.0/16
150.10.0.0/16
150.10.1.1
150.10.1.2
AS 100
AS 300
AS 200
A
B
C
150.10.0.0/16   150.10.1.1
160.10.0.0/16   150.10.1.1
eBGP
iBGP
eBGP – address of external neighbour
iBGP – NEXT_HOP from eBGP
23
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
iBGP Next Hop
AS 300
B
C
120.1.1.0/24   120.1.254.2
120.1.2.0/23   120.1.254.3
iBGP
120.1.1.0/24
120.1.2.0/23
Loopback
120.1.254.2/32
Loopback
120.1.254.3/32
Next hop is ibgp router loopback address
Recursive route look-up
D
A
24
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Third Party Next Hop
120.68.1.0/24
150.1.1.3
150.1.1.1
150.1.1.2
120.68.1.0/24      150.1.1.3
AS 201
AS 200
C
A
B
• eBGP between Router A
and Router C
• eBGP between Router A
and Router B
• 120.68.1/24 prefix has next
hop address of 150.1.1.3 –
this is passed on to Router
C instead of 150.1.1.2
• More efficient
• No extra config needed
25
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Next Hop (Summary)
• IGP should carry route to next hops
• Recursive route look-up
• Unlinks BGP from actual physical topology
• Allows IGP to make intelligent forwarding decision
26
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Origin
• Conveys the origin of the prefix
• Historical attribute
Used in transition from EGP to BGP
• Influences best path selection
• Three values: IGP, EGP, incomplete
IGP – generated by BGP network statement
EGP – generated by EGP
incomplete – redistributed from another routing protocol
27
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Aggregator
• Conveys the IP address of the router or BGP
speaker generating the aggregate route
• Useful for debugging purposes
• Does not influence best path selection
28
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Local Preference
AS 400
AS 200
160.10.0.0/16
AS 100
AS 300
    160.10.0.0/16    500
>  160.10.0.0/16    800
500
800
E
B
C
A
D
29
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Local Preference
• Local to an AS – non-transitive
Default local preference is 100 (IOS)
• Used to influence BGP path selection
determines best path for outbound traffic
• Path with highest local preference wins
30
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Multi-Exit Discriminator (MED)
AS 201
AS 200
120.68.1.0/24
C
A
B
120.68.1.0/24    1000
120.68.1.0/24     2000
31
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Multi-Exit Discriminator
• Inter-AS – non-transitive & optional attribute
• Used to convey the relative preference of entry points
determines best path for inbound traffic
• Comparable if paths are from same AS
bgp always-compared-med allows comparisons of MEDs
from different ASes
• Path with lowest MED wins
• Absence of MED attribute implies MED value of zero
(RFC4271)
32
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Multi-Exit Discriminator
“metric confusion”
• MED is non-transitive and optional attribute
Some implementations send learned MEDs to iBGP peers
by default, others do not
Some implementations send MEDs to eBGP peers by
default, others do not
• Default metric value varies according to vendor
implementation
Original BGP spec made no recommendation
Some implementations said no metric was equivalent to
232-1 (the highest possible) or 232-2
Other implementations said no metric was equivalent to 0
• Potential for “metric confusion”
33
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Community
• Communities are described in RFC1997
Transitive and Optional Attribute
• 32 bit integer
Represented as two 16 bit integers (RFC1998)
Common format is <local-ASN>:xx
0:0 to 0:65535 and 65535:0 to 65535:65535 are reserved
• Used to group destinations
Each destination could be member of multiple communities
• Very useful in applying policies within and between
ASes
34
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
160.10.0.0/16 300:1
Community
AS 200
160.10.0.0/16 300:1
170.10.0.0/16 300:1
170.10.0.0/16    300:1
AS 400
D
C
F
B
170.10.0.0/16
AS 100 A
160.10.0.0/16
ISP 1
200.10.0.0/16 300:9
X
ISP 2
200.10.0.0/16
AS 300
E
35
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Well-Known Communities
• Several well known communities
www.iana.org/assignments/bgp-well-known-communities
• no-export
65535:65281
do not advertise to any eBGP peers
• no-advertise
65535:65282
do not advertise to any BGP peer
• no-export-subconfed
65535:65283
do not advertise outside local AS (only used with confederations)
• no-peer
65535:65284
do not advertise to bi-lateral peers (RFC3765)
36
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
No-Export Community
170.10.0.0/16
170.10.X.X
No-Export
170.10.0.0/16
AS 100
AS 200
170.10.X.X
C
F
G
D
A
B
E
• AS100 announces aggregate and subprefixes
aim is to improve loadsharing by leaking subprefixes
• Subprefixes marked with no-export community
• Router G in AS200 does not announce prefixes with no-export
community set
37
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
No-Peer Community
• Sub-prefixes marked with no-peer community are not sent to bi-
lateral peers
They are only sent to upstream providers
170.10.0.0/16
170.10.X.X
No-Peer
170.10.0.0/16
A
B
E
D
C
C&D&E are
peers e.g.
Tier-1s
upstream
upstream
upstream
38
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Community
Implementation details
• Community is an optional attribute
Some implementations send communities to iBGP peers by
default, some do not
Some implementations send communities to eBGP peers by
default, some do not
• Being careless can lead to community “confusion”
ISPs need consistent community policy within their own
networks
And they need to inform peers, upstreams and customers
about their community expectations
39
© 2005 Cisco Systems, Inc. All rights reserved.
Session Number
Presentation_ID
Cisco Confidential
APNIC 22
BGP Path Selection Algorithm
Why Is This the Best Path?
40
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
BGP Path Selection Algorithm for IOS
Part One
• Do not consider path if no route to next hop
• Do not consider iBGP path if not synchronised (Cisco
IOS)
• Highest weight (local to router)
• Highest local preference (global within AS)
• Prefer locally originated route
• Shortest AS path
41
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
BGP Path Selection Algorithm for IOS
Part Two
• Lowest origin code
IGP < EGP < incomplete
• Lowest Multi-Exit Discriminator (MED)
If bgp deterministic-med, order the paths before comparing
If bgp always-compare-med, then compare for all paths
otherwise MED only considered if paths are from the same
AS (default)
42
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
BGP Path Selection Algorithm for IOS
Part Three
• Prefer eBGP path over iBGP path
• Path with lowest IGP metric to next-hop
• Lowest router-id (originator-id for reflected routes)
• Shortest Cluster-List
Client must be aware of Route Reflector attributes!
• Lowest neighbour IP address
43
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
BGP Path Selection Algorithm
• In multi-vendor environments:
Make sure the path selection processes are understood for
each brand of equipment
Each vendor has slightly different implementations, extra
steps, extra features, etc
Watch out for possible MED confusion
44
© 2005 Cisco Systems, Inc. All rights reserved.
Session Number
Presentation_ID
Cisco Confidential
APNIC 22
Applying Policy with BGP
Control!
45
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Applying Policy in BGP:
Why?
• Policies are applied to:
Influence BGP Path Selection by setting BGP attributes
Determine which prefixes are announced or blocked
Determine which AS-paths are preferred, permitted, or
denied
Determine route groupings and their effects
• Decisions are generally based on prefix, AS-path and
community
46
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Applying Policy with BGP:
Tools
• Most implementations have tools to apply policies
to BGP:
Prefix manipulation/filtering
AS-PATH manipulation/filtering
Community Attribute setting and matching
• Implementations also have policy language which
can do various match/set constructs on the
attributes of chosen BGP routes
47
© 2005 Cisco Systems, Inc. All rights reserved.
Session Number
Presentation_ID
Cisco Confidential
APNIC 22
BGP Capabilities
Extending BGP
48
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
BGP Capabilities
• Documented in RFC2842
• Capabilities parameters passed in BGP open
message
• Unknown or unsupported capabilities will result in
NOTIFICATION message
• Codes:
0 to 63 are assigned by IANA by IETF consensus
64 to 127 are assigned by IANA “first come first served”
128 to 255 are vendor specific
49
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
BGP Capabilities
Current capabilities are:
 0
Reserved
[RFC3392]
 1
Multiprotocol Extensions for BGP-4
[RFC2858]
 2
Route Refresh Capability for BGP-4
[RFC2918]
 3
Cooperative Route Filtering Capability
[ID]
 4
Multiple routes to a destination capability
[RFC3107]
64
Graceful Restart Capability
[ID]
65
Support for 4 octet ASNs
[ID]
66
Deprecated 2003-03-06
67
Support for Dynamic Capability
[ID]
See www.iana.org/assignments/capability-codes
50
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
BGP Capabilities
• Multiprotocol extensions
This is a whole different world, allowing BGP to support more
than IPv4 unicast routes
Examples include: v4 multicast, IPv6, v6 multicast, VPNs
Another tutorial (or many!)
• Route refresh is a well known scaling technique –
covered shortly
• The other capabilities are still in development or not
widely implemented or deployed yet
51
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
BGP for Internet Service Providers
• BGP Basics
• Scaling BGP
• Using Communities
• Deploying BGP in an ISP network
52
© 2005 Cisco Systems, Inc. All rights reserved.
Session Number
Presentation_ID
Cisco Confidential
APNIC 22
BGP Scaling Techniques
53
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
BGP Scaling Techniques
• How does a service provider:
Scale the iBGP mesh beyond a few peers?
Implement new policy without causing flaps and route
churning?
Keep the network stable, scalable, as well as simple?
54
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
BGP Scaling Techniques
• Route Refresh
• Route Reflectors
• Confederations
55
© 2005 Cisco Systems, Inc. All rights reserved.
Session Number
Presentation_ID
Cisco Confidential
APNIC 22
Dynamic Reconfiguration
Route Refresh
56
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Route Refresh
• BGP peer reset required after every policy change
Because the router does not store prefixes which are
rejected by policy
• Hard BGP peer reset:
Terminates BGP peering & Consumes CPU
Severely disrupts connectivity for all networks
• Soft BGP peer reset (or Route Refresh):
BGP peering remains active
Impacts only those prefixes affected by policy change
57
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Route Refresh Capability
• Facilitates non-disruptive policy changes
• For most implementations, no configuration is
needed
Automatically negotiated at peer establishment
• No additional memory is used
• Requires peering routers to support “route refresh
capability” – RFC2918
58
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Dynamic Reconfiguration
• Use Route Refresh capability if supported
find out from the BGP neighbour status display
Non-disruptive, “Good For the Internet”
• If not supported, see if implementation has a
workaround
• Only hard-reset a BGP peering as a last resort
Consider the impact to be
equivalent to a router reboot
59
© 2005 Cisco Systems, Inc. All rights reserved.
Session Number
Presentation_ID
Cisco Confidential
APNIC 22
Route Reflectors
Scaling the iBGP mesh
60
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Scaling iBGP mesh
Two solutions
 Route reflector – simpler to deploy and run
 Confederation – more complex, has corner case advantages
13 Routers ⇒
78 iBGP
Sessions!
Avoid ½n(n-1) iBGP mesh
n=1000 ⇒ nearly
half a million
ibgp sessions!
61
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
AS 100
Route Reflector: Principle
A
C
B
Route Reflector
62
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Route Reflector
AS 100
A
B
C
Clients
Reflectors
• Reflector receives path from
clients and non-clients
• Selects best path
• If best path is from client,
reflect to other clients and
non-clients
• If best path is from non-client,
reflect to clients only
• Non-meshed clients
• Described in RFC4456
63
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Route Reflector Topology
• Divide the backbone into multiple clusters
• At least one route reflector and few clients  per cluster
• Route reflectors are fully meshed
• Clients in a cluster could be fully meshed
• Single IGP to carry next hop and local routes
64
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Route Reflectors:
Loop Avoidance
• Originator_ID attribute
Carries the RID of the originator of the route in the local
AS (created by the RR)
• Cluster_list attribute
The local cluster-id is added when the update is sent by
the RR
Best to set cluster-id is from router-id (address of
loopback)
(Some ISPs use their own cluster-id assignment strategy
– but needs to be well documented!)
65
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Route Reflectors:
Redundancy
• Multiple RRs can be configured in the same cluster
– not advised!
All RRs in the cluster must have the same cluster-id
(otherwise it is a different cluster)
• A router may be a client of RRs in different clusters
Common today in ISP networks to overlay two clusters –
redundancy achieved that way
→ Each client has two RRs = redundancy
66
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Route Reflectors:
Redundancy
AS 100
Cluster One
Cluster Two
PoP2
PoP1
PoP3
67
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Route Reflector: Benefits
• Solves iBGP mesh problem
• Packet forwarding is not affected
• Normal BGP speakers co-exist
• Multiple reflectors for redundancy
• Easy migration
• Multiple levels of route reflectors
68
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Route Reflectors:
Migration
• Where to place the route reflectors?
Always follow the physical topology!
This will guarantee that the packet forwarding won’t be
affected
• Typical ISP network:
PoP has two core routers
Core routers are RR for the PoP
Two overlaid clusters
69
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Route Reflectors:
Migration
• Typical ISP network:
Core routers have fully meshed iBGP
Create further hierarchy if core mesh too big
Split backbone into regions
• Configure one cluster pair at a time
Eliminate redundant iBGP sessions
Place maximum one RR per cluster
Easy migration, multiple levels
70
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Route Reflector: Migration
AS 200
AS 100
AS 300
A
B
G
F
E
D
C
• Migrate small parts of the network, one part
at a time
71
© 2005 Cisco Systems, Inc. All rights reserved.
Session Number
Presentation_ID
Cisco Confidential
APNIC 22
BGP Confederations
72
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Confederations
• Divide the AS into sub-AS
eBGP between sub-AS, but some iBGP information is kept
Preserve NEXT_HOP across the
sub-AS (IGP carries this information)
Preserve LOCAL_PREF and MED
• Usually a single IGP
• Described in RFC3065
73
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Confederations (Cont.)
• Visible to outside world as single AS –
“Confederation Identifier”
Each sub-AS uses a number from the private AS range
(64512-65534)
• iBGP speakers in each sub-AS are fully meshed
The total number of neighbours is reduced by limiting the full
mesh requirement to only the peers in the sub-AS
Can also use Route-Reflector within sub-AS
74
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Confederations
• Configuration (rtr B):
router bgp 65532
 bgp confederation identifier 200
 bgp confederation peers 65530 65531
 neighbor 141.153.12.1 remote-as 65530
 neighbor 141.153.17.2 remote-as 65531
Sub-AS
65532
Sub-AS
65530
AS 200
Sub-AS
65531
B
75
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Confederations: AS-Sequence
Sub-AS
65002
Sub-AS
65003
Sub-AS
65001
Confederation
100
G
Sub-AS
65004
C
D
E
B
180.10.0.0/16
  200
180.10.0.0/16
 {65002}  200
A
180.10.0.0/16
 {65004  65002}  200
H
F
180.10.0.0/16
  100  200
76
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Route Propagation Decisions
• Same as with “normal” BGP:
From peer in same sub-AS → only to external peers
From external peers → to all neighbors
• “External peers” refers to
Peers outside the confederation
Peers in a different sub-AS
Preserve LOCAL_PREF, MED and NEXT_HOP
77
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Internet
Connectivity
Multi-Level
Hierarchy
Policy
 Control
Scalability
Migration
Complexity
Confederations
Route
Reflectors
Anywhere
in the
Network
Yes
Yes
Yes
RRs or Confederations
Yes
Anywhere
in the
Network
Medium
Very High
 Very Low
Medium
to High
Most new service provider networks now deploy Route Reflectors from Day One
78
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
More points about confederations
• Can ease “absorbing” other ISPs into you ISP – e.g., if
one ISP buys another
Or can use AS masquerading feature available in some
implementations to do a similar thing
• Can use route-reflectors with confederation sub-AS to
reduce the sub-AS iBGP mesh
79
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
BGP Scaling Techniques
• Route Refresh
Use should be mandatory
• Route Reflectors/Confederations
The only way to scale iBGP mesh
80
© 2005 Cisco Systems, Inc. All rights reserved.
Session Number
Presentation_ID
Cisco Confidential
APNIC 22
Route Flap Damping
Network Stability for the 1990s
Network Instability for the 21st Century!
81
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Route Flap Damping
• For many years, Route Flap Damping was a
strongly recommended practice
• Now it is strongly discouraged as it causes far
greater network instability than it cures
• But first, the theory…
82
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Route Flap Damping
• Route flap
Going up and down of path or change in attribute
BGP WITHDRAW followed by UPDATE = 1 flap
eBGP neighbour going down/up is NOT a flap
Ripples through the entire Internet
Wastes CPU
• Damping aims to reduce scope of route flap propagation
83
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Route Flap Damping (continued)
• Requirements
Fast convergence for normal route changes
History predicts future behaviour
Suppress oscillating routes
Advertise stable routes
• Implementation described in RFC 2439
84
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Operation
• Add penalty (1000) for each flap
Change in attribute gets penalty of 500
• Exponentially decay penalty
half life determines decay rate
• Penalty above suppress-limit
do not advertise route to BGP peers
• Penalty decayed below reuse-limit
re-advertise route to BGP peers
penalty reset to zero when it is half of reuse-limit
85
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Operation
Reuse limit
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
0
1000
2000
3000
4000
Time
Penalty
Suppress limit
Network
Announced
Network
Re-announced
Network
Not Announced
Penalty
86
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Operation
• Only applied to inbound announcements from eBGP
peers
• Alternate paths still usable
• Controllable by at least:
Half-life
reuse-limit
suppress-limit
maximum suppress time
87
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Configuration
• Implementations allow various policy control with
flap damping
Fixed damping, same rate applied to all prefixes
Variable damping, different rates applied to different ranges
of prefixes and prefix lengths
88
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Route Flap Damping History
• First implementations on the Internet by 1995
• Vendor defaults too severe
RIPE Routing Working Group recommendations in ripe-
178, ripe-210, and ripe-229
http://www.ripe.net/ripe/docs
But many ISPs simply switched on the vendors’ default
values without thinking
89
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Serious Problems:
• "Route Flap Damping Exacerbates Internet Routing
Convergence“
Zhuoqing Morley Mao, Ramesh Govindan, George
Varghese & Randy H. Katz, August 2002
• “What is the sound of one route flapping?”
Tim Griffin, June 2002
• Various work on routing convergence by Craig
Labovitz and Abha Ahuja a few years ago
• “Happy Packets”
Closely related work by Randy Bush et al
90
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Problem 1:
• One path flaps:
BGP speakers pick next best path, announce to all peers,
flap counter incremented
Those peers see change in best path, flap counter
incremented
After a few hops, peers see multiple changes simply
caused by a single flap → prefix is suppressed
91
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Problem 2:
• Different BGP implementations have different
transit time for prefixes
Some hold onto prefix for some time before advertising
Others advertise immediately
• Race to the finish line causes appearance of
flapping, caused by a simple announcement or
path change → prefix is suppressed
92
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Solution:
• Do NOT use Route Flap Damping whatever you do!
• RFD will unnecessarily impair access
to your network and
to the Internet
• More information contained in RIPE Routing
Working Group recommendations:
www.ripe.net/ripe/docs/ripe-378.[pdf,html,txt]
93
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
BGP for Internet Service Providers
• BGP Basics
• Scaling BGP
• Using Communities
• Deploying BGP in an ISP network
94
© 2005 Cisco Systems, Inc. All rights reserved.
Session Number
Presentation_ID
Cisco Confidential
APNIC 22
Service Provider use of Communities
Some examples of how ISPs make life easier for
themselves
95
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
BGP Communities
• Another ISP “scaling technique”
• Prefixes are grouped into different “classes” or
communities within the ISP network
• Each community means a different thing, has a
different result in the ISP network
96
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
BGP Communities
• Communities are generally set at the edge of the ISP
network
Customer edge: customer prefixes belong to different
communities depending on the services they have
purchased
Internet edge: transit provider prefixes belong to difference
communities, depending on the loadsharing or traffic
engineering requirements of the local ISP, or what the
demands from its BGP customers might be
• Two simple examples follow to explain the concept
97
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Community Example – Customer Edge
• This demonstrates how communities might be used
at the customer edge of an ISP network
• ISP has three connections to the Internet:
IXP connection, for local peers
Private peering with a competing ISP in the region
Transit provider, who provides visibility to the entire Internet
• Customers have the option of purchasing
combinations of the above connections
98
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Community Example – Customer Edge
• Community assignments:
IXP connection:
community 100:2100
Private peer:
community 100:2200
• Customer who buys local connectivity (via IXP) is put in
community 100:2100
• Customer who buys peer connectivity is put in community
100:2200
• Customer who wants both IXP and peer connectivity is put in
100:2100 and 100:2200
• Customer who wants “the Internet” has no community set
We are going to announce his prefix everywhere
99
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Community Example – Customer Edge
CORE
Aggregation Router
Customers
Customers
Customers
Communities set at the aggregation router
where the prefix is injected into the ISP’s iBGP
Border Router
100
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Community Example – Customer Edge
• No need to alter filters at the network border when
adding a new customer
• New customer simply is added to the appropriate
community
Border filters already in place take care of announcements
⇒ Ease of operation!
101
© 2006 Cisco Systems, Inc. All rights reserved.
APNIC 22
Community Example – Internet Edge
• This demonstrates how communities might be used
at the peering edge of an ISP network
• ISP has four types of BGP peers:
Customer
IXP peer
Private peer
Transit provider
• The prefixes received from each can be classified
using communities
• Customers can opt to receive any or all of the
above