Aruba troubleshooting

Aruba troubleshooting Cheat-list:

• Troubleshooting+Cheat+Sheet-.pdf
• Useful CLI commands-v1.pdf

wifi commands: (wifi-ctr) #show ap config ap-name AP_NAME (wifi-ctr) #show datapath session table AP-ADDR (wifi-ctr) #show ap debug system-status ap-name AP_NAME (wifi-ctr) #show ap bss-table ap-name AP_NAME

Aruba controller upgrade procedure

WIFI controller Upgrade Procedure:
1. take image backup:  (ny-wifi-master2) # backup flash
2. take logs backup:   (ny-wifi-master2) # tar logs tech-support
3. copy backup flash to ftp: (ny-wifi-master2) #copy flash: flashbackup.tar.gz ftp: [server_ip] rrm *********
4. download image from ftp:

(ny-wifi-master2) #copy ftp: [server_ip] rrm ArubaOS_MMC_6.3.1.6_43301 system: partition 1
Password:*********
Copying file:....................................................
File copied successfully.
Saving file to flash:
................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
The system will boot from partition 1 during the next reboot.
(ny-wifi-master2) # reload
Do you really want to restart the system(y/n): y
System will now restart!
Shutdown processing started
Syncing data....done.
Sending SIGKILL to all processes.
Please stand by while rebooting the system.
1:<7>ide-disk 0.0: shutdown
1:<0>Restarting system.
1:.
1:<2>Performing hard reset...

To check what software is installed, run:

#show image version
----------------------------------
Partition           : 0:0 (/dev/hda1)
Software Version    : ArubaOS 6.1.3.6 (Digitally Signed - Production Build)
Build number        : 36470
Label               : 36470
Built on            : Tue Dec 11 12:51:05 PST 2012
----------------------------------
Partition           : 0:1 (/dev/hda2) **Default boot**
Software Version    : ArubaOS 6.3.1.3 (Digitally Signed - Production Build)
Build number        : 42233
Label               : 42233
Built on            : Tue Feb 11 11:58:33 PST 2014

commit synchronize force

To enforce a commit synchronize on the Routing Engines, log in to the Routing Engine from which you want to synchronize and issue the commit synchronize command with the force option:

[edit]user@host# commit synchronize force

re0:re1:commit complete

re0:commit complete

[edit]user@host#

For the commit synchronization process, the master Routing Engine commits the configuration and sends a copy of the configuration to the backup Routing Engine. Then the backup Routing Engine loads and commits the configuration. So, the commit synchronization between the master and backup Routing Engines takes place one Routing Engine at a time. If the configuration has a large text size or many apply-groups, commit times can be longer than desired.

SRX cluster initial setup

SRX cluster initial setup:

1. Power on both SRX units. Console to the first one.

2. Enable cluster mode and reboot the devices:
    On device A:    >set chassis cluster cluster-id 1 node 0 reboot
    On device B:    >set chassis cluster cluster-id 1 node 1 reboot

3. Remove default configuration:
root> configure shared
 delete interfaces
 delete system services dhcp   
 delete security nat
 delete protocols stp
 set protocols rstp
 delete security policies
 delete security zones
 delete vlans

4. Configure authentication and ssh access on each device:
root# set system root-authentication plain-text-password
root# set system services ssh root-login allow

5. Configure the device specific configurations such as host names and management IP addresses. This is specific to each device and is the only part of the configuration that is unique to its specific node.  This is done by entering the following commands (all on the primary node):

on Node0:
root# set groups node0 system host-name nyc-broadway-451-0                    
root# set groups node0 interfaces fxp0 unit 0 family inet address 172.25.25.1/24   
set apply-groups "${node0}"

on Node1:
root# set groups node1 system host-name nyc-broadway-451-1                    
root# set groups node1 interfaces fxp0 unit 0 family inet address 172.25.25.2/24    
set apply-groups "${node1}"

The 'set apply-groups' command is run so that the individual configs for each node, set by the above commands, are applied only to that node. This command is required.

6. Configure the FAB links (data plane links for RTO sync, etc):
     set interfaces fab0 fabric-options member-interfaces ge-0/0/2
     set interfaces fab0 fabric-options member-interfaces ge-0/0/3
     
     set interfaces fab1 fabric-options member-interfaces ge-5/0/2     
     set interfaces fab1 fabric-options member-interfaces ge-5/0/3

7.  Configure the Redundancy Group 0 for the Routing Engine failover properties. Also configure Redundancy Group 1 (all the interfaces will be in one Redundancy Group in this example) to define the failover properties for the Reth interfaces.
    
set chassis cluster reth-count 3
set chassis cluster redundancy-group 0 node 0 priority 100
set chassis cluster redundancy-group 0 node 1 priority 1
set chassis cluster redundancy-group 1 node 0 priority 100
set chassis cluster redundancy-group 1 node 1 priority 1

-configure switch fabric
set interfaces swfab0 fabric-options member-interfaces ge-0/0/4
set interfaces swfab1 fabric-options member-interfaces ge-5/0/4

8. Configure interfaces
set interfaces ge-0/0/15 gigether-options redundant-parent reth0
set interfaces ge-5/0/15 gigether-options redundant-parent reth0

BGP tools, troubleshooting and monitoring external routing

This will provide a view from the street, it will display exactly what your router knows.

Route servers / Looking Glass –This is your basic  external view. Log in and see what other ASNs know about your routes.  Critically important during those “something is on fire” times mentioned above.  They are maintained by a myriad of entities and positioned all over the globe.

BGPmon –A project by Andree Toonk. Allows for the automatic discovery and monitoring of prefixes, alerting on many, many  attributes such as prefix hijacks.  Free and commercial plans available, but the commercial plans are far more feature rich and well worth it if you monitor large amounts og BGP.

Peermon –Part of the new and improved BGPmon.  Allows for the on-demand monitoring of prefixes within your network. Very useful for viewing as-path changes of destination networks for long term troubleshooting.

RouteViews – Great project out of U of Oregon that was (is?) run by the groundbreaking David Meyer of OpenDaylight (and many other things) fame.  Peers with networks and records routing changes, allows for public query and has vast historical data.

bgplay - Great visualization tool for tracking routing, as-path and prefix announcement changes. This is part of the routeviews project and utilizes their vast historical data.  L=It currently acks IPv6 support and I;m unsure if it is maintained anymore.

Router Proxies – This has been a big thing int he R&E world for quite some time.  Other entities may offer it, it’s similar to a looking glass but more easily configured to allow or disallow different show commands.  The code is open source and pretty easy to hack new commands into or adapt to new platforms (if I can do it anyone can).

Lookup tools such as whois.  I find that looking uo ASNs and networks against the ARIN, RIPE and other RIRs is very handy as a starting point.  using CLI commands such as “whois -h whois.arin.net 1224″ would display useful information.

Very handy for prefixes and ASNs.  There are also service like the Team Cymru whois server that can display date/time based information for forensics and to provide IP to ASN mappings. Also very handy.  I believe this code is also open source.

IRR Toolset.  Extremely handy for automation of routing policy configuration.  I found it a tad painful to set up but it is a useful toolkit.

Notable Mention: NLNog RING.  – This is a trust based unix host that provides a large variety of services to those that qualify for participation.  Very handy when looking for an on-net perspective.

Notable Mention / Shameless Plug: perfSonar toolkit.  In addition to thewell known performance testing tools, PS provides things like reverse traceroute and other handy networking widgets.  It also has a far lower barrier of entry than the NLNog RING.

There are obviously more ways to do this and there are possibly better ones, too.  This is how I’ve done it for a long time and it has mostly worked for me.  I had to learn most of this by trial and error so I thought it maybe useful to throw it all together into one place for future reference.

Bigip mgmt IP configuration

Configuring the cluster IP address using tmsh

You can configure the cluster IP address using tmsh after you connect a blade to a serial terminal console.

1. Connect to the system using the serial console.
2. Type the following key sequence to set the cluster IP address and subnet mask: tmsh modify sys cluster default address <ip_address/mask> Example: tmsh modify sys cluster default address 192.168.217.44/24
3. Type the following key sequence to set the default gateway for the cluster: tmsh modify sys management-route default gateway <gateway_ip> Example: tmsh modify sys management-route default gateway 172.20.80.254
4. Write the running configuration to the stored configuration files. tmsh sys save [base-config | config]

The system saves the new IP address, subnet mask, and gateway address for the cluster. You can now access the browser-based Configuration utility using the cluster IP address you assigned.