Commands and Applications Used to Isolate Problems Occurring at the Transport Layer

To ascertain that a problem is rooted at the transport layer rather than at a layer below it, you must use proper tools and commands. You can test the operability of Layers 1, 2, and 3 by using the Windows commands ping, tracert, netstat, and nbtstat. If the results prove that the lower layers (physical, data link, and network) are in good working order, you can focus your attention on Layer 4-related issues such as filters, policies, quality of service (QoS) tools, and so on. You can use the commands in this section to isolate the problems at the transport layer and identify whether they are TCP or UDP related.

The following command displays protocol statistics and current TCP/IP network connections on an IP host (Windows 9x/2000/XP):

The elements of this command are as follows:

[-a]—Displays all connections and listening ports.

[-e]—Displays Ethernet statistics. You can combine this with the -s option.

[-n]—Displays addresses and port numbers in numerical form.

[-s]—Displays per-protocol statistics. By default, statistics are shown for TCP, UDP, and IP; you can use the -p option to specify a subset of the default.

[-p proto]—Shows connections for the protocol specified by proto; proto can be TCP or UDP. If proto is used with the -s option to display per-protocol statistics, it can be TCP, UDP, or IP.

[-r]—Displays the routing table.

NOTE NetBIOS over TCP (NBT) is a session layer protocol that provides three services: name service, session service, and datagram service. A few years ago, NetBIOS ran on top of the nonroutable protocol NetBEUI. Indeed, NetBIOS was considered an extension of the NetBEUI protocol. Many applications were written to run on top of NetBIOS/NetBEUI. Those applications took advantage of NetBIOS's name, session, and datagram service. Later, when routable protocols such as IPX/SPX and TCP/IP became popular, the NetBIOS protocol was enhanced to run on additional protocols besides NetBEUI. NBT is a result of those efforts.

The following command displays protocol statistics and current TCP/IP connections using NBT on an IP host (Windows 9x/2000/XP):

nbtstat [ [-a RemoteName] [-A IP-address] [-c] [-n] [-r] [-R] [-RR] [-s] [-S] ]

The elements of this command are as follows:

[-a]—Lists the remote machine's name table given its name

[-A]—Lists the remote machine's name table given its IP address

[-c]—Lists NBT's cache of remote (machine) names and their IP addresses

[-n]—Lists local NetBIOS names

[-r]—Lists names resolved by broadcast and via WINS

[-R]—Purges and reloads the remote cache name table

[-RR]—Sends name release packets to WINS and starts refresh

[-s]—Lists sessions table converting destination IP addresses to computer NetBIOS names [-S]—Lists sessions table with the destination IP

The following Cisco IOS command displays all IP access lists that are configured on a router at the present time. Extended access lists, if used, can influence a router's behavior by referencing source/destination port numbers:

show ip access-lists

The following command displays various IP-related statistics. Examples of the displayed statistics are format errors, bad hops, encapsulation failures, unknown routes, and probe proxy requests:

show ip traffic

NOTE This example describes one of the pieces of useful information that the show ip traffic command yields. In one of the Trouble Ticket labs in the previous version of the CIT course, the global configuration command no ip forward-protocol udp was typed on the student router while ip helper-address ip-address was left on the router interface(s). The no ip forward-protocol udp command renders the ip helper-address ip-address interface configuration command useless. As a result, many applications such as DHCP clients failed because the router would not forward requests for leased IP addresses to the DHCP server(s). Most students got stuck on this problem because they had to isolate problems using the Cisco IOS show commands and avoid using the show running-config and the show startup-config commands on the routers. The students would then ask which show command they could have used to help them isolate the problem. The answer was always show ip traffic.

show ip traffic displays various IP-related statistics. One of the statistics shown that can lead to isolating the aforementioned problem is the number of "UDP: no port" cases. When you enter the no ip forward-protocol udp global configuration command on a router, the router always notices a related request, such as a DHCP request. The router fails to forward the request and increment that counter, so the counter continues to grow. The [no] ip forward-protocol udp [port] command gives you the option of specifying the port number; however, if you do not specify a port, this command turns off/on forwarding for eight UDP ports (NetBIOS Name, Network Time Protocol [NTP], Terminal Access Controller Access Control System [TACACS], DHCP).

You can test the functionality of any TCP port by using the Telnet application as follows and referring to a desired port number:

telnet host [port]

You can examine the contents of a router's IP route caching table using the following command. The keyword flow specifies that the output should include the netflow cache:

show ip cache [flow]

Note that if Cisco Express Forwarding (CEF) is enabled on a router (at least on some interfaces), it builds and maintains the IP Forwarding Table. You can display the IP Forwarding Table (some people merely call it the IP CEF table) by using the following command:

show ip cef

The following Cisco IOS command displays the local router's QoS policy maps:

show policy-map

The following IOS command displays the current queueing configuration on the local router (such as custom, fair, priority, and random-detect):

show queueing

Examples Demonstrating Transport Layer Problem Isolation Commands

So that you can practice isolating problems at the transport layer, this section presents a case along with a series of example outputs and results.

You are the second-level network engineer for the AMIRACAN Corporation. You have console access to the router named Toronto and the access switch named Toronto_SW, as well as IP connectivity to all other devices in your division. You are told that users cannot access the database server named CIT DB. The users report that they cannot even connect to the distribution router named Ottawa from their PCs. You know from your base configuration information that the end users connect over Toronto_SW to the Toronto router. Toronto and Toronto_SW are connected via a 100-Mbps FastEthernet link. Figure 11-1 is a simplified network diagram displaying your territory of responsibility, which includes the access layer (Toronto switch and router), the distribution layer router (Ottawa), and the connection to the core layer (Guelph).

Figure 11-1 Baseline Network Diagram for AMIRACAN Corporation

Figure 11-1 Baseline Network Diagram for AMIRACAN Corporation

You connect to the console port on Toronto_SW and attempt to Telnet to the Toronto router. You can connect to Toronto from Toronto_SW, so there appears to be no issues between these two devices. (See the top part of Example 11-1.) You close the Telnet session to Toronto and try to connect to

Ottawa from Toronto_SW. The Telnet session from Toronto_SW cannot be opened on Ottawa. (See the middle part of Example 11-1.) To help isolate the problem, you check to see whether Toronto_SW can ping Ottawa. You see that Toronto_SW can ping Ottawa, so it appears that the physical, data link, and network layers among these devices are working. (See the bottom part of Example 11-1.) This makes you suspect that the issue is with an access list.

Example 11-1 Output for ping and telnet from TorontoSW to the Toronto and Ottawa Routers

Toronto_SW>telnet Toronto

Trying Toronto (172.22.125.1)... Open **********************************************************

Toronto: A Distribution Workgroup Router at AMIRACAN **********************************************************

User Access Verification

Password:

Toronto>

Toronto>exit

[Connection to Toronto closed by foreign host]

Toronto_SW>telnet Ottawa Trying Ottawa (172.22.128.1)... % Destination unreachable; gateway or host down Trying Ottawa (172.22.128.129)... % Destination unreachable; gateway or host down Trying Ottawa (172.22.127.2)... % Destination unreachable; gateway or host down Trying Ottawa (172.22.127.130)... % Destination unreachable; gateway or host down Toronto_SW>

Toronto_SW>ping Ottawa

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 172.22.128.1, timeout is 2 seconds: !!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 16/18/20 ms Toronto SW>

Next, you try to open a Telnet session from Toronto to Ottawa. You see that Toronto can open a Telnet session to Ottawa, so you know that Ottawa is not blocking all inbound Telnet traffic. (See the top part of Example 11-2.) You look for signs of recent configuration changes on Ottawa by entering the show logging and show clock commands. You see that no configuration changes have been made on Ottawa for several days, so you return to the console session on Toronto and hunt for signs of recent configuration changes on Toronto with the show logging and show clock commands. You notice that someone was at least in configuration mode on Toronto in the past few hours. (See the bottom part of Example 11-2.)

Example 11-2 Output for Telnet Attempt from Toronto to Ottawa and for the show logging and show clock Commands on Toronto

Toronto>telnet Ottawa

Trying Ottawa (172.22.128.1)... Open ********************************************************

Ottawa: an AMIRACAN Distribution Workgroup Router ********************************************************

User Access Verification

Password:

Ottawa>exit

[Connection to Ottawa closed by foreign host] Toronto>show logging

Syslog logging: enabled (0 messages dropped, 1 messages rate-limited, 0 flushes, 0 overruns)

Console logging: level debugging, 115 messages logged Monitor logging: level debugging, 0 messages logged Buffer logging: level debugging, 155 messages logged Logging Exception size (4096 bytes) Count and timestamp logging messages: disabled Trap logging: level informational, 186 message lines logged Logging to 172.27.227.9, 139 message lines logged Log Buffer (65536 bytes):

Dec 19 12:19:15: %SYS-5-CONFIG_I: Configured from console by vty0 (172.22.126.2) Dec 19 13:03:06: %SYS-5-CONFIG_I: Configured from console by vty0 (172.22.126.2) Dec 19 13:21:07: %SYS-5-CONFIG_I: Configured from console by vty0 (172.22.126.2) Toronto>

Toronto>show clock

15:53:22.258 EST Thu Dec 19 2002

Toronto>

You did not have upgrades planned, but you need to review the details of the access lists on Toronto. Because pings to Ottawa from Toronto_SW are successful, you suspect the problem is probably with an extended access list filtering too much traffic. You use the show access-lists command to review the current access lists that are configured on Toronto. The show ip access-lists command that was mentioned previously displays the IP access lists only; the show access-lists command displays all access lists. Of course, in the absence of other access lists (such as IPX, AppleTalk, and so on), both commands will yield the same output.

The only extended access list is called Traffic. It explicitly permits Internet Control Message Protocol (ICMP), FTP, World Wide Web (WWW), and Trivial File Transfer Protocol (TFTP) traffic. However, the implicit deny at the end of the list blocks Telnet traffic that comes from Toronto_SW from being forwarded to Ottawa. (See the top part of Example 11-3.) You can also use show ip route to determine which interface is being used to forward traffic to Ottawa. You see that traffic for Ottawa is sent across the interface named Serial0/0:0. (See the middle part of Example 11-3.)

Finally, you must find out whether an access list is actually applied to the serial 0/0:0 interface. Therefore, you enter the show ip interface serial 0/0:0 command. The results (output of the command) are shown in the bottom part of Example 11-3.

Example 11-3 Output for the show access-lists, show ip route, and show ip interface Commands on Toronto

Toronto>show access-lists

Standard IP access list Admin permit 172.22.121.0, wildcard bits 0.0.0.255 (95 matches) permit 172.22.125.0, wildcard bits 0.0.0.255 Standard IP access list ENDJJSERS

permit 172.22.124.0, wildcard bits 0.0.0.255 permit 172.22.122.0, wildcard bits 0.0.1.255 Extended IP access list Traffic

Remark Allow ICMP, Telnet Outbound, FTP, TFTP, and WWW permit icmp any any (15 matches) permit tcp 172.22.0.0 0.0.255.255 any eq ftp-data permit tcp 172.22.0.0 0.0.255.255 any eq ftp permit tcp 172.22.0.0 0.0.255.255 any eq www permit udp 172.22.0.0 0.0.255.255 any eq tftp Toronto>

Toronto>show ip route

Gateway of last resort is 172.22.126.2 to network 0.0.0.0

D EX 172.21.0.0/16 [170/3873280] via 172.22.126.2, 2d00h, Serial0/0:0 D EX 172.23.0.0/16 [170/3873280] via 172.22.126.2, 2d00h, Serial0/0:0

172.22.0.0/16 is variably subnetted, 13 subnets, 2 masks D 172.22.128.0/26 [90/3973120] via 172.22.126.2, 6d00h, Serial0/0:0

D 172.22.129.0/26 [90/3975680] via 172.22.126.2, 6d00h, Serial0/0:0

C 172.22.126.128/26 is directly connected, Serial0/0:1

C 172.22.127.128/26 is directly connected, Serial1/1

D EX 172.22.0.0/16 [170/3873280] via 172.22.126.2, 2d02h, Serial0/0:0 D 172.22.128.128/26 [90/3847680] via 172.22.126.2, 6d00h, Serial0/0:0

C 172.22.122.0/26 is directly connected, FastEthernet0/0.2

C 172.22.123.0/26 is directly connected, FastEthernet0/0.3

C 172.22.121.0/26 is directly connected, FastEthernet0/0.1

C 172.22.126.0/26 is directly connected, Serial0/0:0

C 172.22.127.0/26 is directly connected, Serial1/0

C 172.22.124.0/26 is directly connected, FastEthernet0/0.4

C 172.22.125.0/26 is directly connected, Loopback0

D EX 172.25.0.0/16 [170/3873280] via 172.22.126.2, 2d00h, Serial0/0:0 D EX 172.24.0.0/16 [170/3873280] via 172.22.126.2, 2d00h, Serial0/0:0 D EX 172.26.0.0/16 [170/3873280] via 172.22.126.2, 2d00h, Serial0/0:0

Example 11-3 Output for the show access-lists, show ip route, and show ip interface Commands on Toronto (Continued)

D EX 198.133.219.0/24 [170/3873280] via 172

22.126.2,

2d00h, Serial0/0:0

D*EX 0.0.0.0/0 [170/3847936] via 172.22.126

2, 6d00h,

Serial0/0:0

Toronto>

Toronto>show ip interface serial 0/0:0

Serial0/0:0 is up, line protocol is up

Internet address is 172.22.126.1/26

Broadcast address is 255.255.255.255

Address determined by setup command

MTU is 1500 bytes

Helper address is not set

Directed broadcast forwarding is disabled

Multicast reserved groups joined: 224.0.0

10

Outgoing access list is Traffic

Inbound access list is not set

Toronto>

You have isolated the issue. The outbound access list named Traffic does not include a permit statement for Telnet. All Telnet traffic from the access switch and the end users is being filtered. The remark statement for the access list Traffic states that it should support TCP outbound.

0 0

Post a comment