About the Authors

Barry Raveendran Greene is a Senior Consultant in the Internet Architectures Group of Consulting Engineering, Office of the CTO, Cisco Systems. Cisco's CTO Consulting group assist ISPs throughout the world to scale, grow, and expand their networks. The assistance is delivered through consulting, developing new features, working new standards (IETF and other groups), and pushing forward Best Common Practices (BCPs) to the Internet community. Barry's current topics of interests are ISP Operations...

Access

This is the edge of the ISP network facing the customer. The access layer can be made up of all kinds of devices, from a simple router supporting PSTN modem banks to cable-aggregation devices supporting broadband customers. For this reason, ISPs tend to subdivide their access layers into more manageable units. Larger businesses often have different business units running different access services, and it is useful to give these different units management access to the devices they are...

Access Control Lists Turbo ACLs

Turbo ACLs use a technique that takes a standard or extended ACL, creates a set of data tables, and compiles them for runtime processing. For this reason, Turbo ACLs also are referred to as compiled ACLs. Turbo ACLs do not change the first match wins characteristic of all ACLs. Instead, they reduce the number of CPU operations to find a match, allowing for larger ACLs to be used without an increase in packet latency. This provides ISPs with a tool to allow large ACLs without a significant...

Access List Types

Extended 48-bit MAC address access list IP standard access list (expanded range) IP extended access list (expanded range) Enables IP access-list compilation (new from 12.0(6)S) Simple rate limit-specific access list Specifies a dynamic list of permits or denies KA9Q NOS-compatible IP over IP tunneling A single source host (equivalent to a.b.c.d 255.255.255.255) Log matches against this entry, including input interface Matches packets with given precedence value Matches packets with given TOS...

Access Lists on the VTY Ports

It is important to secure the VTY ports used for Telnet access with a standard ACL. By default, there are no access controls on any of the VTY ports. If this is left this way and a password is applied l to the VTY port, the router will be wide open to anyone who attempts a brute-force crack against the password. The following configuration with access-list 3 is typical of a better approach aaa authentication login Cisco-Lab local username Ciscol password 7 11041811051B13 access-list 3 permit...

Access Network Prefixes

Access network prefixes commonly are used in nonpermanent access networks. So customers connecting using cable, ADSL, PSTN, or ISDN dialup tend to be assigned address space on a dynamic basis. When they connect, PPP or DHCP is used to assign them an address for the duration of their session. When they terminate the connection, the address goes back into the pool. Access network prefixes are not carried in the IGP. If they were carried in the IGP, then every time a customer connects to the...

ACL Option added in IOS Software Release 12010S1 ri21

The optional ACL parameter to the command can be used to control the exact behavior when the received frame fails the source IP address check. The ACL can be either a standard or an extended IP access list T131 < 1-99> IP standard access list < 100-199> IP extended access list < 1300-1999> IP standard access list (expanded range) < 2000-2699> IP extended access list (expanded range) If an ACL is specified, when (and only when) a packet fails a uRPF check the ACL is checked to...

Address Space for Customers

Customers are assigned address space by the ISP according to the policies set out by the RIR of which the ISP is a member. As an approximate guide, ISPs are expected to follow the same procedures with their customers as they have followed with the RIR in obtaining address space. So, when the ISP makes first technical contact with the customer, an assessment of address space requirements is made according to need. Need should not be confused with want. Staff members from many organizations who...

Aggregation Router Filters

The minimum inbound filter that any ISP should be applying to the customer-facing interfaces on its aggregation routers is the unicast RPF check. This ensures that all packets coming from the customer are checked to make sure that their origin address comes out of the address block assigned to the customer. This check is much more efficient than applying any inbound filters on the aggregation router, and it is a recommended best practice throughout this book. If the customer has address space...

Aggregation Routers

If a prefix is injected into the iBGP at a gateway router (the standard way of injecting a customer prefix into the iBGP), the next-hop address is the IP address of the point-to-point link between the gateway aggregation router and the customer. This means that the iBGP has a large number of next-hop addresses to resolve from the IGP (not bad in itself) and that the IGP will be larger, resulting in slower convergence and greater potential instability in case of instability or failures in the...

And Impacts on uRPF Deployment

Typical Asymmetrical Routing Example Figure 4-24. Typical Asymmetrical Routing Example Every router on the Internet makes this best-path decision. Because these decisions are done from a point of view of the router's position in the Internet, the resulting best path might not match what the best path is on other routers. Routing protocols are created with the means to ensure that these independent best-path decisions do not cause routing loops. Although routing loops are not a...

Appendix B Cutand Paste Templates

The following are some cut-and-paste templates that you can modify to configure your routers. Make sure that you change any sample IP addresses or AS numbers used in the templates to match your own addressing Do not use the addresses in the examples because they are invalid. As described in the main text, it is considered good practice to set up a configuration template for each class of router running in the network. Use these templates, taken from running configurations in ISP backbones...

Appendix C Example Configurations

This appendix aims to give ISPs learning about the art of constructing an ISP caliber backbone a little guidance on some of the configuration steps and design hints that have been covered throughout the IOS Essentials whitepaper and this book. The common elements of an ISP's network have all been included, including border, aggregation, and dial access routers. Clearly it is almost impossible to give sample configurations suitable for all ISPs. The following aims to give guidance to small or...

Appendix E Traffic Engineering Tools

As a follow-up on how to track where your customers are going on the Internet, this appendix provides a list of publicly available tools that can be used to pull in statistics from your network. Most ISPs do not use things like HP OpenView, Sun NetManager, CiscoWorks, or Spectrum to manage their networks. These network management packages are great for the enterprise LANs, but they do not have the simple scaleable tools needed for ISP networks. Instead, ISPs pull together different, mostly...

Applying to the RIRs or Upstream ISP for Addresses

The documentation and application process for obtaining address space is different for each of the three RIRs, so it is not covered in this text. However, the actual process should be relatively straightforward. The first step in making any application for address space is to approach the upstream ISP. Many are members of one of the three RIRs and so can act on behalf of the RIR to assign address space. Some might charge a small fee for the service. When the upstream ISP cannot offer this...

Background

URPF is a CEF feature that uses the information in the FIB to automatically perform the BCP 38 checks. The original strict mode uRPF was designed for the customer ISP edge of the network (see Figure 4-20). The objective was to design a feature that can easily be automated in the customer provisioning system, scale as new addresses blocks are allocated to the customer, and work with the MTRIE-based CEF switching. uRPF meets these objectives, even when the customers are multihomed to one or more...

Basics

Before pushing ahead and installing a new circuit, some of the basics of multihoming must be understood and remembered The new circuit should go to a different PoP of the upstream ISP. If it goes to the same upstream PoP, site redundancy of the upstream is lost. Complete PoP outages can occur because of electrical, operational, or environmental problems. If the upstream ISP has only one PoP, request or insist that the new connection be terminated in a different router, one with a different...

BCP 38 Implementation with uRPF Strict Mode

URPF's key BCP 38 implementation principles follow A route must exist in the FIB matching the prefix to the interface. This can be done through a connected interface, a static route, a network statement (BGP, OSPF, RIPv2, and so on), or dynamic routing updates. Traffic from the interface must match the prefixes for the interface. If there are multiple entries for the prefix in the route tables, the prefix local to the router implementing uRPF must be preferred (using BGP weight with multihomed...

BGP Conditional Advertisement

Conditional advertisement of prefixes has been introduced into IOS Software Releases 11.1CC, 11.2, 12.0 and more recent versions, in an effort to contribute to the stability of large BGP-based networks (specifically, the Internet). Conditional advertisement usually is configured when an AS has at least two connections to another AS. The inter-AS peering routers watch the links between the autonomous systems. If one link fails, prefixes are advertised out of the other link. This allows ISPs to...

BGP fastexternalfallover

By default, if the physical connection to the eBGP neighbor goes down, the peering relationship is reset immediately. By adding the no bgp fast-external-fallover configuration, the peering is held open for the duration of the BGP keepalive timer. This configuration is desirable, if not essential, in the case of long-distance peering links or unreliable or long-latency connections to other autonomous systems, and when ISPs prefer stability over convergence speed in large networks. router bgp 109...

BGP Filter Processing Order

When constructing BGP filters, most ISPs use prefix lists or distribute lists, AS path filters, and route maps to implement their filtering policies. It is important to realize that these lists are processed in a particular order newer engineers sometimes are unaware of the standard IOS Software sequence. In the following, example the IOS Software configuration generator (NVGEN) has printed the configuration into the previous order 0.1 prefix-list rfc1918-out out Implementation order is...

BGP Flap Damping

A consequence of the churning in the Internet caused by indiscriminate clearing of BGP sessions, by unstable infrastructure, and by antisocial configuration practices is the request for BGP flap damping to be available in IOS Software. Prefixes that appear and disappear from the Internet Route Table cause a CPU hit on the router Withdrawing a prefix means that it has to be withdrawn from the BGP and forwarding tables the prefix also has to be withdrawn from neighbors and so on throughout all...

BGP Flap Damping Configuration

Recommended route flap-damping parameters for use by ISPs were composed into a document by the RIPE Routing Working Group and are available at www.ripe.net docs ripe-229.html. These values are used by many European and U.S. ISPs, and they are based on the operational experienced gained in the industry. The configuration examples are reproduced here for convenience the values have been updated to include recent changes in the locations of the root nameservers. The current address list is...

BGP Flap Statistics

It is possible to monitor the flaps of all the paths that are flapping. The statistics will be lost when the route is not suppressed and stable for at least one half-life time. The display looks like the following cerdiwen sh ip bgp neighbors 171.69.232.56 flap-statistics BGP table version is 18, local router ID is 172.19.82.53 Status codes s suppressed, d damped, h history, * valid, > best, i - Origin codes i - IGP, e - EGP, - incomplete Network From Flaps Duration Sup-time Path *> 5.0.0.0...

Bgp Med Not

When an MED is not set on a route, Cisco IOS Software has always assumed that the MED is 0. Some other vendors have assumed that the MED is 4,294,967,295 (232 -1). This divergence can result in eBGP routing loops between Cisco routers and other vendors' routers. This confusion was the result of a lack of any definition in the BGP standard on what to do if a MED was not set. The most recent IETF decision regarding BGP MED assigns a value of infinity to the missing MED, making the route lacking...

BGP Multipath

The BGP implementation in IOS Software supports three ways of load sharing over parallel circuits. Two of them are applicable to eBGP. eBGP multihop has been used in IOS Software for several years and is the common way of setting up an eBGP peering without using the directly connected peer addresses the common use of this is for load sharing over parallel peering circuits. More recently, eBGP multipath was added to give an alternative mechanism for load sharing without some of the side effects...

BGP Neighbor Authentication

You can invoke MD5 authentication between two BGP peers. This feature must be configured with the same password on both BGP peers otherwise, the connection between them will not be made. Invoking authentication causes the Cisco IOS Software to generate and check the MD5 digest of every segment sent on the TCP connection. If authentication is invoked and a segment fails authentication, a message appears on the console. Configuring a password for a neighbor causes an existing session to be torn...

BGP Neighbor Changes

It is possible to log neighbor state changes to a UNIX syslog server. This is extremely useful for most syslog-based monitoring systems because it gives early warning of problems with iBGP peers, and more especially external BGP neighbors. The logging is enabled by the following commands router bgp 109 bgp log-neighbor-changes Note that, as of IOS Software Releases 12.1(4) and 12.0(12)ST, the default action is to log all BGP neighbor changes (see DDTS CSCdm59903). At time of writing, the...

BGP Neighbor Shutdown

A new feature introduced into 11.1CC and 12.0 software is the capability to shut down a BGP peering without actually removing the configuration. Previously, the only way to disable a BGP peering was to delete the configuration from the router. This was very disruptive to the router's functioning, and it significantly increased the likelihood of making mistakes when reinstating the configuration at a later stage. A neighboring peering is shut down with this command example router bgp 200...

BGP network Statement

A historical limitation was placed on the number of network statements that could be applied to the BGP configuration. In the early days of IOS Software, in which the average router had at most 30 interfaces, the limit of 200 network statements seemed a large number compared with the number of networks that such a router could originate. However, in the last five years or so, routers have grown through having hundreds of interfaces to the thousands of interfaces that we see on some of the...

BGP Peer Groups

The second essential feature is the BGP peer group. This groups the BGP peers with the same outbound policy into one group. The normal situation without using peer groups is for the router to calculate the update to be sent to each neighbor individually. For a low number of peers, there is probably not much impact on the router CPU, but as the number of neighbors increases, so does the burden on the CPU for example, 50 neighbors means that 50 updates have to be computed and sent individually....

BGP Route Flap Damping

Route flap damping (introduced in Cisco IOS Software at Release 11.0) is a mechanism for minimizing the instability caused by route flapping. A route flap occurs when a BGP network prefix is withdrawn and reannounced specifically, this happens when a BGP speaker hears a WITHDRAW followed by an UPDATE for a prefix. (A peering with an eBGP neighbor being reset does not count as a flap.) Whenever a network goes down, the rest of the Internet is told about it. Hence, BGP propagates this state...

BGP Update Source

In the following example, the iBGP mesh is built using the loopback interface on each router. The loopback doesn't ever disappear, which results in a more stable iBGP, even if the underlying physical connectivity is less than reliable. ip address 215.17.1.34 255.255.255.255

Black Hole Routing as a Packet Filter Forwarding to NullO

Another way of implementing destination-based packet filtering on a router is to create a specific list of static host routes and point them to the pseudo-interface NullO. This technique commonly is referred to as black-hole routing. Null0 is a pseudo-interface, which functions similarly to the null devices available on most operating systems. This interface is always up and can never forward or receive traffic. Although Null0 is a pseudo-interface, within CEF it is not a valid interface....

Border Router

The border router is perhaps the most important router in an ISP's backbone network operation. It provides connectivity to the rest of the Internet for the ISP and also protects the ISP's network and customers from the ravages of the Internet. In ISP workshop presentations, we use the analogy of the front door of your house or the garden gate protecting you from the world out there. The configuration of the border router is critical to the correct and reliable operation of the rest of the ISP's...

Caida

The Cooperative Association for Internet Data Analysis (CAIDA) has a very comprehensive web site listing a lot of tools and pointers. This CAIDA effort is supported through sponsorship provided by the United States National Science Foundation (NSF), Cisco Systems, and other organizations. CAIDA Internet Tools Taxonomy

Car

Active defenses are tools, techniques, or procedures executed during attacks. They are used to limit or block the attack in progress. In many instances, these tools will have an effect on other Internet applications and services, yet the trade-off is between no Internet services and limited disruption. It is highly recommended that the ISP document and train staff on the use of these tools. That way, the ISP's NOC can quickly respond to an attack in progress. Cisco IOS Software has security...

Case Study

The following case study implements the techniques described previously for an ISP that recently needed to multihome to the Internet. The scenario is quite simple, but it covers a situation that causes many newcomers to BGP and the Internet quite a lot of trouble. The network in Figure 5-31 shows the network layout. AS 5400 is based in Europe, and AS 2516 is based in Japan this in itself poses a challenge because the multihoming is between entities that are quite literally on opposite sides of...

Operational Practices

So far this book has given specific and detailed advice about router-configuration best practices for ISP backbones. It is important to see these suggestions in the bigger picture of an actual ISP business. This chapter covers some of the operational issues regarding the establishment of an ISP backbone, the choices made, the positioning of hardware, the configuration of software, and the establishment of external relationships. Aspects of ISP business operations have been covered in other...

Choosing an IGP

A general discussion on the appropriate technical choice of IGPs for an ISP backbone is currently beyond the scope of this book. Many good comparisons have been done describing the pros and cons of the different IGPs. A commonly quoted example is the presentation by Dave Katz at the June 2000 NANOG comparing IS-IS and OSPF. The choice of IGP generally seems to be made on the basis of experience because technically there is little way to choose among the three for most practical purposes. Those...

Cisco IOS Software release designations definedSoftware Lifecycle Definitions

Software naming conventions for Cisco IOS Cisco IOS reference guide Cisco IOS roadmap Cisco Resource Manager Private I http www.opensystems.com index.asp Crystal Reports Netforensics http www.netforensics.com NTP RFCs RFC 1128, RFC 1129, RFC 1165, and RFC 1305, all available at http www.ietf.org rfc

CLI String Search

After a considerable number of requests from ISPs, a UNIX qrep-like function (pattern search) has been introduced as a new feature in IOS Software from releases 11.1CC and 12.0. It allows operators to search for common expressions in configuration and other terminal output. Again, only salient points are covered here because the IOS Software documentation now gives more detailed information at 20t1 cliparse.ht . The function is invoked by using a vertical bar , like the UNIX pipe command begin...

Client Server Models and Association Modes

NTP servers can associate with each other in a number of modes. The mode of each server in the pair indicates the behavior that the other server can expect from it. An association is formed when two peers exchange messages and one or both of them create and maintain an instantiation of the protocol machine. The association can operate in one of several modes server, client, peer, and broadcast multicast. The modes further are classified as active and passive. In active modes, the host continues...

Command Group Organization

The commands listed in the first preceding group have no ambiguity, so they all can appear under the router bgp < as> definition. The commands listed in the second group have no ambiguity either, so they can follow the first group of commands under the main BGP definition (with the exception of VPNs). The commands listed in the third group can have potential ambiguity and thus are listed under a new address family submode A neighbor can have different route map or prefix list statements,...

Commentary

Figure 5-6 typifies the PoP design that ISPs generally use in their PoPs. The smaller ISPs obviously won't have as much sophistication and segregation of equipment, and the larger ISPs will have a significantly larger layout. Many types of access layers are possible two have been shown by way of example, although other access layers are very similar in implementation. Having established the principle of subdividing the PoP into units depending on function and management, the next important...

Committed Access Rate CAR

Configuring Committed Access Rate c f qcprt1 qcfcar.ht RFC 1812. Requirements for IP Version 4 Routers. F. Baker (ed). June 1995. (Status Proposed standard.) Also see the update, RFC 2644. RFC 2196 FYI8. Site Security Handbook. B. Fraser. September 1997. (Obsoletes RFC 1244) (Status Informational.) One of the most useful starting places for Internet security. RFC 2827 BCP 38. Network Ingress Filtering Defeating Denial-of-Service Attacks Which Employ IP Source Address Spoofing. P. Ferguson and...

Comparing Router IDs

As part of the standard path-selection process in IOS Software, the router does not switch between two eBGP paths based solely upon the router ID. Consider a situation in which a router hears two announcements of a particular prefix from two different neighbors. If the path-selection process determines that all attributes are identical apart from the router ID, it currently takes the path with the oldest entry in the routing table, not the entry with the lowest router ID. This choice was made...

Conclusion

This section gave an example of how to work out an addressing scheme for a developing ISP network. This is intended to help the growing ISP business work out how to apply addressing to its network and how to allocate assigned address space to its infrastructure. Indeed, following these processes should aid in the application process for address space from the RIRs. One of the hardest questions that a vendor is posed at the time of an RFP is which IGP the new ISP network should choose. As...

Conditional Advertisement Example

Consider the example depicted in Figure 3-8. This shows a dual-homed enterprise network (AS 300) that has received address space from its two upstream ISPs. It announces the 215.10.0.0 22 prefix to ISP 1 (AS 100) and the 202.9.64 23 prefix to ISP 2 (AS 200). These networks are part of the respective upstream ISPs address blocks, so all that the Internet sees are the two aggregates as originated by ISP 1 and ISP 2. This is the steady state situation. Figure 3-8. BGP Conditional Advertisement...

Configuration

Specify communities into community lists (or define AS path lists and so on) that will classify traffic for accounting. ip community-list ip community-list ip community-list ip community-list ip community-list Define a route map to match community lists, and set the appropriate bucket number route-map set_bucket permit 10 match community 30 set traffic-index 2 Look here route-map set_bucket permit 20 match community 40 set traffic-index 3 route-map set_bucket permit 30 match community 50 set...

Configurations

The following Cisco IOS Software configurations work and have been tested in a lab environment. They are based heavily on known current configurations used in the field today, with additions modifications to include as many recommendations from the main text of this book as is feasible. The configurations have been annotated using IOS Software comments (with ) where more explanation is required. This should make it easier to cut and paste configurations into your own test environment. But...

Configuring EIGRP

EIGRP also is used quite extensively in ISP backbones, finding favor with ISPs that have been required to support multiple protocols in the past. It's also accepted that EIGRP is probably the easiest routing protocol to get started with. It found favor (in the form of IGRP) with ISPs that didn't want to use IS-IS in the earlier days of the Internet when implementations of OSPF were still not mature enough for their needs. EIGRP has no area concept the network runs as one large IGP. EIGRP does...

Configuring OSPF

OSPF enforces a fairly rigid design for an ISP backbone. Area 0 is the backbone area and must exist if there are to be more than two OSPF areas in a network. Area 0 provides transit between the other areas, and every other area must be connected to it. OSPF offers a multitude of area types, including backbone, regular, stub, totally stubby, and not so stubby areas. Most ISPs tend to use only backbone and regular areas very few make use of OSPF inter-area summarization capabilities. The reason...

Console Server

The next logical step from using one modem or multiple modems is to install a console server. This is a device to which all the equipment consoles are connected. The most popular console server in recent years has been the Cisco 2511 router it has 1 Ethernet, 2 WANs, and 16 asynchronous serial ports. Originally intended as a dialup router, it has long since been retired from that function in most ISPs and now is used as a console server throughout many ISP backbones. The configuration is...

Core

The PoP core has devices called core routers. These support high-bandwidth links only, interconnections between similar core devices, connections to other units within the PoP, and connections to other PoPs. It is very common for ISPs to install two core routers one device implies a single point of failure, something that high-quality and high-availability networks won't tolerate. The definition of high bandwidth depends on the region of the world. In some countries, intracore connections often...

Creating Your Own Net Police Filter

ISPs that want to create their own net police filters are strongly encouraged to do the following Consider the impact Shutting the door doesn't make the storm go away it shuts out everything, good and bad. Maintain an accurate list Consult each of the RIR's published CIDR blocks for the default allocations. Ensure that you have an up-to-date list, and create a process for validating your filter with future updates of the RIR's list. Consult with colleagues Consult with your peers about the...

Customer Router Filters

As a service to their customers, all ISPs should be supplying sample filters for routers that are used to connect permanently connected networks to the Internet. If the customers are using routers that are incapable of filtering, to quote an ISP overheard recently, that device should be replaced with a real router. It is an unfortunate fact today that too many people equate security with firewalls and completely fail to remember that a router is a very sophisticated first-line security device...

Designing a Test

From the description of the previous test scenario, it should be quite clear what components make up an ISP's test lab. Some ISPs build a replica of one of their PoPs others simply have a few of the major devices used on their backbone connected in a simple network. Preference usually is given to the PoP replica design because it becomes very simple to replicate problems that occur on the live network. Several ISPs even have this test lab as part of their backbones the lab won't take an active...

Designing Flap Damping Parameters

It is important to note that prefixes are suppressed only when they are heard from a neighbor and have a flap history and a penalty value that will cause them to be suppressed. A prefix attracts a penalty of 1000 when the WITHDRAW is heard from the neighboring eBGP peer. However, because the prefix has been withdrawn, it is not present in the BGP table apart from having a history entry, so it will not be damped. The penalty will decay, however. When the prefix is reannounced by the neighbor, if...

Displaying BGP Policy Accounting Statistics

The statistics are stored in a table of packet byte counters per input software interface (with the assumption that each customer is connected to an input software interface). You can display them with show cef interface interface policy-statistics. SNMP support will be added sometime in the near future. The statistics actually are displayed per configured table map match category. Using a route map, you can match against configured community lists, AS paths, and so on, and you can set this to...

Displaying BGP Policy Accounting Status

To inspect which prefix is assigned which bucket and which communities, do the following 196.240.5.0 24, version 21, cached adjacency to POS7 2 packets, 0 bytes, traffic_index 4 Look Here via 14.1.1.1, 0 dependencies, recursive next hop 14.1.1.1, POS7 2 via 14.1.1.0 30 valid cached adjacency BGP routing table entry for 196.240.5.0 24, version 2 Paths (1 available, best 1) Not advertised to any peer 100 14.1.1.1 from 14.1.1.1 (32.32.32.32) Origin IGP, metric 0, localpref 100, valid, external,...

Displaying Prefix List ORF

The command to display the prefix-list ORF received from a neighbor is shown here show ip bgp neighbor x.x.x.x received prefix-filter This displays the received prefix list. Changes to the output of show ip bgp neighbor x.x.x.x now include information about whether ORF is supported or available, as can be seen in the following snippet Capability advertised received Filter sent received (25 entries) BGP policy accounting allows you to account for IP traffic differentially by assigning counters...

Distribution

The distribution layer is one step removed from the core and gets its name from its function of acting as a distribution layer between the core routers and the access part of the network. Indeed, many small- to medium-size ISPs don't have any distribution layer they simply connect the access part of the network to the core. It all depends on the size of the PoP. The distribution layer can be made up of two or more routers quite often there could be considerably more. ISPs conscious of providing...

EBGP Multihop

Diversely Multihomed Stub Network Figure 5-21. Diversely Multihomed Stub Network eBGP multihop is an eBGP peering between the loopback interfaces (or other interface not on the demarcation zone between the two networks) of routers in the two networks. The configuration could be something like the following Router A router bgp 65534 neighbor 1.1.1.1 remote-as 100 neighbor 1.1.1.1 ebgp-multihop 5 ip route 1.1.1.1 255.255.255.255 serial 1 0 ip route 1.1.1.1 255.255.255.255 serial 1 1...

EBGP Multipath

With IOS Software Release 11.1CC came a new BGP feature that allows more than one path to the same destination to be installed in the forwarding table. This feature, called eBGP multipath, is designed to allow ISPs to load-share over external circuits to eBGP neighbors. When the border router has more than one path to the same external network, the BGP path-selection process makes a decision at Step 11, installing the oldest received path into the RIB. So only one of the external paths is used....

Editing Keys

Several keys are very useful as shortcuts for editing the IOS Software configuration. Although these are covered in detail in the IOS Software release 12.0 documentation set, it is useful to point out those used most commonly, shown in Table 1-2. Completes the command being typed in. This saves typing effort and is especially useful when the operator is still learning the IOS Software command set. Lists the available commands starting with the characters entered so far. Allows the operator to...

Effects of CIDRization

One of the most critical issues that could threaten the stability of the Internet is the size of the global Internet Route Table. The table's growth influences scalability, increases operational capital cost, and has posed a security risk to the Internet. For a variety of reasons, ISPs throughout the world have injected all sorts of networks into the Internet, ranging from 8s (old Class A's) to 32s (host routes). The result is rapid growth of the Internet Route Table. Classless interdomain...

End Sites

Two types of end sites exist as far as traffic flows are concerned. The first is the ISP that is connecting customers to the Internet. The traffic flow in this case is mostly inbound a typical volume ratio might be 70 percent inbound and 30 percent outbound. The second is the content provider, and traffic flow levels typically are reversed. This section considers only the first scenario the later section on Outbound Traffic Loadsharing considers the case for the content provider.

Example

An ISP has filtered its IRC server from receiving ICMP echo-reply packets to protect it. Now many attackers are going after the customer's devices to fill some network segments. The ISP chose to use CAR to limit all ICMP echo and echo-reply traffic received at the borders to 256 Kbps. An example follows traffic we want to limit access-list 102 permit icmp any any echo access-list 102 permit icmp any any echo-reply interface configurations for borders interface Serial3 0 0 rate-limit input...

Example 21 Sample Output from Displaying Flow Information on a Net FlowEnabled Router

IP packet size distribution (410772243 total packets) 1-32 64 96 128 160 192 224 256 288 320 352 384 416 448 480 .000 .168 .384 .102 .160 .107 .019 .005 .003 .001 .001 .000 .000 .000 .003 512 544 576 1024 1536 2048 2560 3072 3584 4096 4608 .001 .000 .035 .000 .003 .000 .000 .000 .000 .000 .000 IP Flow Switching Cache, 4456704 bytes 15074 active, 50462 inactive, 125120769 added 369493980 ager polls, 0 flow alloc failures last clearing of statistics 4d05h Total Flows 605 3494 4104 845158 87119 59...

Example 41 Information That Can Be Gained from CDP

Device ID Excalabur Entry address(es) IP address 4.1.2.1 Platform cisco RSP2, Capabilities Interface FastEthernet1 1, Port ID Holdtime 154 sec Cisco Internetwork Operating System Software IOS (tm) RSP Software (RSP-K3PV-M), Version 12.0(9.5)S, EARLY DEPLOYMENT MAINTENANCE INTERIM SOFTWARE Copyright (c) 1986-2000 by cisco Systems, Inc. Compiled Fri 03-Mar-00 19 28 by htseng CDP can be disabled using this global command If CDP is required on an ISP's network, it is possible to leave CDP running...

Example 45 Multihomed uRPF

Interface serial 1 0 1 description Link to Acme Computer's Router C ip address 192.168.3.2 255.255.255.252 ip verify unicast reverse-path no ip redirects no ip directed-broadcast no ip proxy-arp ip route-cache distributed neighbor 192.168.10.3 remote-as 65000 neighbor 192.168.10.3 description Multihomed Customer - Acme Computers neighbor 192.168.10.3 update-source Loopback0 neighbor 192.168.10.3 send-community neighbor 192.168.10.3 soft-reconfiguration inbound neighbor 192.168.10.3 route-map...

Example B1 General System Template

No service finger replaced with ip finger from 12.0 service timestamps debug datetime localtime show-timezone msec service timestamps log datetime localtime show-timezone msec interface loopback 0 description Loopback interface for router XY Enable logging with two loghosts using facility local4 - these commands are default from 12.0 - best included anyway Enable Cisco Express Forwarding

Example B2 General Interface Template

Interface serial 0 0 description BW Connection to XYZ, Circuit ID, Cable ID. bandwidth BW interface hssi 1 0 description BW Connection to ABC, Circuit ID, Cable ID. bandwidth BW interface fastethernet 2 0 description Core link to Router2, X-over Ethernet no ip redirects no ip directed-broadcast no ip proxy-arp no cdp enable

Example B3 General Security Template

Service password-encryption enable secret < removed> no enable password aaa authentication login default tacacs+ enable aaa authentication enable default tacacs+ enable And set up the TACACS+ authentication - two servers ip tacacs source-interface loopback 0 tacacs-server host z.z.z.A tacacs-server host z.z.z.B tacacs-server key < removed> - need to run crypto key generate rsa before applying this template Protect the console ports - list NOC and other permitted addresses in access-list...

Example B4 General iBGP Template

Neighbor ibgp-peer description Internal BGP peers neighbor ibgp-peer remote-as 65280 neighbor ibgp-peer update-source loopback 0 neighbor ibgp-peer next-hop-self neighbor ibgp-peer version 4 ultra paranoid neighbor ibgp-peer send-community send communities neighbor ibgp-peer password < removed> use password on peering

Example B7 Prefix List to Deny RFC 1918 and Martian Networks

Ip Networks which shouldn't be announced ip prefix-list rfc1918-sua deny 0.0.0.0 8 le 32 ip prefix-list rfc1918-sua deny 10.0.0.0 8 le 32 ip prefix-list rfc1918-sua deny 127.0.0.0 8 le 32 ip prefix-list rfc1918-sua deny 169.254.0.0 16 le 32 ip prefix-list rfc1918-sua deny 172.16.0.0 12 le 32 ip prefix-list rfc1918-sua deny 192.0.2.0 24 le 32 ip prefix-list rfc1918-sua deny 192.168.0.0 16 le 32 ip prefix-list rfc1918-sua deny 224.0.0.0 3 le 32 ip prefix-list rfc1918-sua deny 0.0.0.0 0 ge 25

Example C2 Core Router Configuration Example

Version 12.0 service nagle no service pad service tcp-keepalives-in service timestamps debug datetime msec localtime show-timezone service timestamps log datetime msec localtime show-timezone boot system flash slot0 c7200-k4p-mz.120-10.S2 logging buffered 16384 debugging aaa new-model aaa authentication login default tacacs+ enable aaa authentication enable default tacacs+ enable aaa accounting exec default start-stop tacacs+ aaa accounting commands 15 default start-stop tacacs+ ip subnet-zero...

Example C3 Aggregation Router Configuration Example

Version 12.0 service nagle no service pad service tcp-keepalives-in service timestamps debug datetime msec localtime show-timezone service timestamps log datetime msec localtime show-timezone service password-encryption logging buffered 16384 debugging aaa new-model aaa authentication login default tacacs+ enable aaa authentication enable default tacacs+ enable aaa accounting exec default start-stop tacacs+ aaa accounting commands 15 default start-stop tacacs+ clock timezone GMT 0 ip...

Example C4 Service Router Configuration Example

Version 12.0 service nagle no service pad service tcp-keepalives-in service timestamps debug datetime msec localtime show-timezone service timestamps log datetime msec localtime show-timezone aaa authentication login default tacacs+ enable aaa authentication enable default tacacs+ enable aaa accounting exec default start-stop tacacs+ aaa accounting commands 15 default start-stop tacacs+ clock timezone GMT 0 ip subnet-zero no ip source-route no ip finger ip telnet source-interface Loopback0 ip...

Example C5 NOC Router Configuration Example

Service timestamps debug datetime msec localtime show-timezone service timestamps log datetime msec localtime show-timezone aaa authentication login default tacacs+ enable aaa authentication enable default tacacs+ enable aaa accounting exec default start-stop tacacs+ aaa accounting commands 15 default start-stop tacacs+ clock timezone GMT 0 ip subnet-zero no ip source-route no ip finger ip telnet source-interface Loopback0 ip tftp source-interface Loopback0 ip ftp source-interface Loopback0 ip...

Example F2 Antispoofing Configuration Example

Interface hssi 0 1 description 16Mbps link to our upstream provider bandwidth 16384 ip access-group 111 in no ip redirects no ip directed-broadcast no ip proxy-arp access-list 111 deny ip 127.0.0.0 0.255.255.255 any log access-list 111 deny ip 10.0.0.0 0.255.255.255 any log access-list 111 deny ip 172.16.0.0 0.15.255.255 any log access-list 111 deny ip 192.168.0.0 0.0.255.255 any log access-list 111 deny ip 169.223.0.0 0.0.255.255 any log access-list 111 deny ip 211.255.0.0 0.0.31.255 any log...

Examples of the New CLI in

This section gives some examples of the revised CLI. They have been taken from real, live running configurations working in networks today. The first example shows how BGP supporting IPv6 is configured. This router is a part of the 6BONE, the IPv6 experimental backbone no bgp default ipv4-unicast allow more than IPv4 uni bgp log-neighbor-changes bgp dampening neighbor UPSTREAMS peer-group neighbor iBGP-peers peer-group neighbor 2001 200 0 1805 2 remote-as 2500 neighbor 3FFE 800 FFF9 0 0 9...

Exception Dumps by FTP

Cisco routers can be configured to dump core memory to an FTP server as part of the diagnostic and debugging process. However, this core dump should be to a system not running a public FTP server, but one heavily protected by filters (TCP Wrapper even) that allow only the routers access. If the loopback interface address is used as source address from the router and is part of one address block, the filter is very easy to configure. A 200-router network with 200 disparate IP addresses makes for...

Figure 31 Dual Gateway LAN

Interface ethernet 0 0 description Server LAN ip address 169.223.10.1 255.255.255.0 standby 10 ip 169.223.10.254 interface ethernet 0 0 description Server LAN ip address 169.223.10.1 255.255.255.0 standby 10 ip 169.223.10.254 interface ethernet 0 0 description Service LAN ip address 169.223.10.2 255.255.255.0 standby 10 priority 150 standby 10 preempt standby 10 ip 169.223.10.254 The two routers have their LAN IP addresses conventionally defined in the preceding configuration. However, another...

Figure 35 GRE Tunnels at IXPsNAPs

Interface tunnel 0 ip address 221.0.1.1 255.255.255.252 tunnel source 220.0.0.2 ip route 169.223.0.2 255.255.255.255 220.0.0.1 Router B interface tunnel 0 ip address 221.0.1.2 255.255.255.252 tunnel source 169.223.0.2 tunnel destination 220.0.0.2 interface tunnel 0 ip address 221.0.1.1 255.255.255.252 tunnel source 220.0.0.2 ip route 169.223.0.2 255.255.255.255 220.0.0.1 Router B interface tunnel 0 ip address 221.0.1.2 255.255.255.252 tunnel source 169.223.0.2 tunnel destination 220.0.0.2 ip...

Figure 415 Egress Packet Filtering on the Upstream Gateway Router

Allow source address 16S.21.0.0 16 < - - _ -- Allow source address 16S.21.0.0 16 < - - _ -- Ex. IP packeis win n unce adftfew nf 10.1.1.1 Awls btoctad, access-list 110 permit ip 165.21.0.0 0.0.255.255 any access-list 110 deny ip any any log interface serial 0 1 description Upstream Connection to ISP A ip access-group 110 out The last line of the access list determines whether there is any traffic with an invalid source address entering the Internet. If there are any matches, they will be...

Figure 422 uRPF Dropping Packets that Fail Verification

Fddi 2 0 fl 172.15.8,0 attached Fddi 2 0 0 216.SlO.e.a 172.ia.ee. Fddi 2 0 fl 172.15.8,0 attached Fddi 2 0 0 5. If an uRPF ACL is applied, the packet is processed through that feature ACL before final dropping. This ACL could be configured to overrule the uRPF check and pass the packet. 11 6. CEF table (FIB) lookup is carried out for packet forwarding, passing packets that match the FIB + adjacency check or dropping packets that are spoofed sources. For example, if a...

Figure 43 Command Auditing on the Router Through Cisco Secure and TACACS

Configuration control and audit of who has done what and when on the routers is the key objective for using AAA command accounting on an ISP's backbone. aaa authentication login default tacacs+ enable aaa authentication enable default tacacs+ enable aaa accounting command 15 start-stop tacacs+ aaa accounting exec start-stop tacacs+ ip tacacs source-interface Loopback0 tacacs-server host 215.17.1.2 tacacs-server host 215.17.34.10 tacacs-server key CKr3t

Figure 430 How Smurf Uses Amplifiers

Each host on that IP network will take the ICMP echo request and reply to it with an echo reply. This multiplies the inbound traffic by the number of hosts responding. On a multi-access broadcast network, potentially hundreds of machines could be replying to each packet, resulting in what is called an attack from a smurf amplifier network. The systems most commonly hit by these types of attacks are Internet Relay Chat (IRC) servers, specific Web sites, and their providers. Two parties are hurt...

Figure 48 Ingress Filtering

Egress filtering applies a filter for all traffic leaving an ISP's networks (see Figure 4-9). It is applied to information leaving the network to the Internet or customer networks. Be mindful that these terms are relative to the specific network's point of view. For example, ISP B's egress traffic is ISP A's ingress traffic. Ingress egress filters help protect an ISP's resources and its customers' networks, allows it to enforce policy, and minimizes the risk of being the network chosen by...

Figure 511 RIR Areas

It is hoped that this section will encourage ISPs to consider how to design a scalable addressing plan. Conservation and efficient utilization of address space often are seen as problematic and even undesirable by ISPs trying to minimize the number of prefixes carried around in their network. This is only an example of the considerations necessary when designing the addressing plan for an ISP network. It does not advise on how to go through the process of applying for address space from the...

Figure 516 Address Deployment Plan After Second Allocation

InfrHsliudurn Cl Icf-c j sl- a-inm'l'j InfrHsliudurn Cl Icf-c j sl- a-inm'l'j When the 80 percent utilization has been reached, the next application can be made, and so the cycle goes. It might be quite likely that third and future assignments would not be contiguous, so the ISP should not rely on expanding the 19 out into a 18. If it happens, consider it good fortune and take it as encouragement to announce only a 18 to the Internet

Figure 517 Example of Customer Assignments

Some ISPs have written scripts to do this assignment automatically from their address blocks as part of their customer sign-on and provisioning systems. Others simply maintain a large spreadsheet kept up-to-date with each assignment made. Some of the scripts are available on the Internet. A commonly used example is tree, available by anonymous FTP from ISI. 1

Figure 529 Two Upstream ISPs

Again, the common solution to this problem is for both upstream ISPs to provide the full routing table to AS 109, as can be shown in the following router configuration examples router bgp 109 network 221.10.0.0 mask 255.255.224.0 neighbor 222.222.10.1 remote-as 107 neighbor 222.222.10.1 prefix-list rfc1918-deny in neighbor 222.222.10.1 prefix-list my-block out neighbor 222.222.10.1 route-map AS107-loadshare in ip prefix-list my-block permit 221.10.0.0 19 See Appendix B for the RFC1918 list ip...

Figure 533 Router B Upstream Link Load

Over time, this configuration will have to be monitored because traffic patterns do change as networks grow and the clientele on the network come and go. But as a case study for good multihoming practice, this is one of the best efforts seen on the Internet today. Unlike the claims from some of the potential upstream ISPs of AS 17660, all this has been achievable with minor router platforms and minimum memory requirements (the two 2600s each still have around 32 MB of memory spare, plenty of...

Figure 55 PoP Access Network

Most ISPs don't bother too much with the redundancy of the RAS. Several will provision two switches, but it is rare for them to provision two Ethernets out of the RAS. The top-end RAS devices have two Ethernets, so better redundancy can be provided. However, in most cases, ISPs tend to use low-end units and large numbers of these because failures are actually easier to deal with. For other types of access networks, there will be other designs. Cable and xDSL access require a much more...

Figure F1 ISP Network Example

The configuration in Example F-1 is used on all routers in the network. Notice that the AAA authentication used is simplistic it normally would not be recommended for an ISP backbone. Similar configurations should be used on the switches in the network. Any staff workstations servers also should use appropriate tools to limit Telnet access to the workstations server's resources 2 .

Figure F3 Closing Off Access to Everyone Except the NOC Staff

Example F-3 ACLs with Telnet Access Closed to All but the NOC's Network Example F-3 ACLs with Telnet Access Closed to All but the NOC's Network aaa authentication login ISP local username Ciscol password 7 11041811051B13 access-list 3 permit 211.255.1.0 0.0.0.255 line vty 0 4 access-class 3 in exec-timeout 5 0 transport preferred none transport input telnet login authentication ISP history size 256

Further Reference on IOS Software Releases

Figures 1-1 and 1-2 provide a visual map of IOS Software releases up to 12.1 they also show how the different versions and trains interrelate. This has been and still is an often-asked question in the ISP arena and other marketplaces in which Cisco is present these visual roadmaps have been created to show the interrelation of the different IOS Software versions. The current up-to-date roadmap can be seen at Consult the following URLs on Cisco.com for more detailed and up-to-date information on...