It has been quite some time since I built a lab for learning/studying purposes. Recently I have been tasked to work on creating a Method of Procedure or MOP for a High Availability ASA Software upgrade. While studying for the CCNA Security high availability was discussed in a high-level overview. The study material briefly went over the topic and didn’t mention too much. I remember some of the information that was gone over during the ASA training class I took in August was just some small labs to become familiar with the High Availability technology.
After doing research for performing the high availability software upgrade, I came across a great tutorial. It was written by Rowell Dionicio of Packet6.com titled CISCO ASA- Zero Downtime Upgrade. The tutorial was well written and easier to follow than the normal cisco documentation. I was able to create my MOP and test the upgrade paths using two ASA’s in a lab environment. At this time, I feel confident in perform the upgrade during the change window in three weeks.
Since I started doing the research, I decided to create a high availability network lab. My requirements for the network lab are specific.
The requirements are:
- Each ASA will connect to each other using two dedicated failover links.
-
- One link will be used for statefull failover.
- One link will be used for regular failover.
- ASAv1 will be the Primary ASA.
- ASAv2 will be the Secondary ASA.
- Failover switching should be as quickly as possible to minimize traffic downtime.
- ASA management traffic will connect to the management network on VLAN 100.
- Management VLAN is VLAN 100
- Client traffic VLAN is VLAN 1
- The trunk between the Management switch and access switch will only allow VLAN 100.
- Client facing ports will have spanning-tree portfast and BPDU Guard enabled.
The network topology is pretty simple. The ASA’s are connected directly to each other using two interfaces. The management interfaces are connected to a dedicated management switch. That management switch is connected to the distribution/access switch via a trunk and only VLAN 100 is allowed to traverse the trunk link. The inside LAN facing ports of the ASA’s will be connected to the other switch. The WAN Facing ports of the ASA’s are connected to the Cloud and will be used for internet connectivity.
For this Lab I am using both the console and the ASDM software to configure the ASA’s. I find that it is easier to perform some tasks on the ASA using the ASDM while others are performed using the console.
Switch Setup
The first step in getting the lab working was to configure the switch that is directly connected to the client computer and the management computer. I configured the VLANS first, followed by setting up the interfaces, and verifying using the appropriate show commands.
DSW1#config t
Enter configuration commands, one per line. End with CNTL/Z.
DSW1(config)#vlan 100
DSW1(config-vlan)#name management
DSW1(config-vlan)#vlan 666
DSW1(config-vlan)#name blackhole
DSW1(config-vlan)#exitDSW1(config)#interface range g0/0 – 3
DSW1(config-if-range)#switchport mode access
DSW1(config-if-range)#switch access vlan 1
DSW1(config-if-range)#spanning-tree portfast
DSW1(config-if-range)#spanning-tree bpduguard enable
DSW1(config-if-range)#no shutdown
DSW1(config-if-range)#exitDSW1(config)#interface g0/3
DSW1(config-if)#switchport mode access
DSW1(config-if)#switchport access vlan 100
DSW1(config-if)#spanning-tree portfast
DSW1(config-if)#spanning-tree bpduguard enable
DSW1(config-if)#no shutdown
DSW1(config-if)#exitDSW1(config)#int g1/3
DSW1(config-if)#switchport trunk encapsulation dot1q
DSW1(config-if)#switchport mode trunk
DSW1(config-if)#switchport trunk allowed vlan 100
DSW1(config-if)#no shutdown
DSW1(config-if)#exit
DSW1(config)#DSW1(config)#interface vlan 100
DSW1(config-if)#ip address 192.168.100.10 255.255.255.0
DSW1(config-if)#DSW1(config)#int range g1/0 – 2
DSW1(config-if-range)#switchport mode access
DSW1(config-if-range)#switch access vlan 666
DSW1(config-if-range)#shutdown
DSW1(config-if-range)#exit
DSW1(config)#do copy running-config startup-config
DSW1#show vlan
VLAN Name Status Ports
—- ——————————– ——— ——————————-
1 default active Gi0/0, Gi0/1, Gi0/2
100 management active Gi0/3
666 blackhole active Gi1/0, Gi1/1, Gi1/2
1002 fddi-default act/unsup
1003 token-ring-default act/unsup
1004 fddinet-default act/unsup
1005 trnet-default act/unsup
VLAN Type SAID MTU Parent RingNo BridgeNo Stp BrdgMode Trans1 Trans2
—- —– ———- —– —— —— ——– —- ——– —— ——
1 enet 100001 1500 – – – – – 0 0
100 enet 100100 1500 – – – – – 0 0
666 enet 100666 1500 – – – – – 0 0
1002 fddi 101002 1500 – – – – – 0 0
1003 tr 101003 1500 – – – – – 0 0
1004 fdnet 101004 1500 – – – ieee – 0 0
1005 trnet 101005 1500 – – – ibm – 0 0
Primary Secondary Type Ports
——- ——— —————– ——————————————
DSW1#
DSW1#show interfaces trunk
Port Mode Encapsulation Status Native vlan
Gi1/3 on 802.1q trunking 1
Port Vlans allowed on trunk
Gi1/3 100
Port Vlans allowed and active in management domain
Gi1/3 100
Port Vlans in spanning tree forwarding state and not pruned
Gi1/3 100
DSW1#
The second step I need to do is configure the management switch. The setup of this device is similar to the other switch with the only difference is that there are no client devices connected to it. I could have easily had the ASA’s directly connected to the other switch and save on using a management switch but I decided to use a separate switch to make it easier to visualize and in the future I could always add additional devices to the management network.
MGMT_Switch(config)#vlan 100
MGMT_Switch(config-vlan)#name management
MGMT_Switch(config-vlan)#vlan 666
MGMT_Switch(config-vlan)#name blackhole
MGMT_Switch(config-vlan)#exit
MGMT_Switch(config)#
MGMT_Switch(config)#int range g0/1 – 1
MGMT_Switch(config-if-range)#switchport mode access
MGMT_Switch(config-if-range)#switchport access vlan 100
MGMT_Switch(config-if-range)#spanning-tree portfast
MGMT_Switch(config-if-range)#spanning-tree bpduguard enable
MGMT_Switch(config-if-range)#no shut
MGMT_Switch(config-if-range)#
MGMT_Switch(config)#int g0/2
MGMT_Switch(config-if)#switchport trunk encapsulation dot1q
MGMT_Switch(config-if)#switchport mode trunk
MGMT_Switch(config-if)#switch trunk allowed vlan 100
MGMT_Switch(config-if)#no shutdown
MGMT_Switch(config-if)#
MGMT_Switch(config)#int range g1/0 – 3
MGMT_Switch(config-if-range)#switchport mode access
MGMT_Switch(config-if-range)#switchport access vlan 666
MGMT_Switch(config-if-range)#shutdown
MGMT_Switch(config-if-range)#
MGMT_Switch(config-if-range)#int vlan 100
MGMT_Switch(config-if)#ip address 192.168.100.9 255.255.255.0
MGMT_Switch(config-if)#no shut
MGMT_Switch(config-if)#exit
MGMT_Switch(config)#
MGMT_Switch#sh vlan
VLAN Name Status Ports
—- ——————————– ——— ——————————-
1 default active
100 management active Gi0/0, Gi0/1
666 blackhole active Gi0/3, Gi1/0, Gi1/1, Gi1/2
Gi1/3
1002 fddi-default act/unsup
1003 token-ring-default act/unsup
1004 fddinet-default act/unsup
1005 trnet-default act/unsup
VLAN Type SAID MTU Parent RingNo BridgeNo Stp BrdgMode Trans1 Trans2
—- —– ———- —– —— —— ——– —- ——– —— ——
1 enet 100001 1500 – – – – – 0 0
100 enet 100100 1500 – – – – – 0 0
666 enet 100666 1500 – – – – – 0 0
1002 fddi 101002 1500 – – – – – 0 0
1003 tr 101003 1500 – – – – – 0 0
1004 fdnet 101004 1500 – – – ieee – 0 0
1005 trnet 101005 1500 – – – ibm – 0 0
Primary Secondary Type Ports
——- ——— —————– ——————————————
MGMT_Switch#
MGMT_Switch#sh interfaces trunk
Port Mode Encapsulation Status Native vlan
Gi0/2 on 802.1q trunking 1
Port Vlans allowed on trunk
Gi0/2 100
Port Vlans allowed and active in management domain
Gi0/2 100
Port Vlans in spanning tree forwarding state and not pruned
Gi0/2 100
MGMT_Switch#
Now that both switches are configured, I can see the VLANS properly configured, assigned to the correct interfaces and the truck’s being established. I verified the trunk is working and VLAN 100 traffic can traverse by pinging from the management switch to the client switch.
MGMT_Switch#ping 192.168.100.10
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.100.10, timeout is 2 seconds:
.!!!!
Success rate is 80 percent (4/5), round-trip min/avg/max = 3/4/7 ms
MGMT_Switch#ping 192.168.100.10
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.100.10, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 3/3/5 ms
MGMT_Switch#
At this time, the switch bock is fully configured. I can now move onto the ASA firewalls for the configuration.
High Availability ASA Configuration
The ASA configuration is going to be quite interesting. There are two ASA’s connected to each other using two different interfaces. One interface is used for failover communication and the other interface is used to replicate the state information. In addition to those two-failover links, each ASA has a dedicated management link connected to the management switch and each has a link connecting to the client access switch.
After reading the plethora of documentation on cisco’s website in regard to configuring the Active/Standby Configuration, most of the configuration will be performed on the Main ASA unit. Once the main unit is configured to act as the primary failover unit, you would then issue several commands for the failover unit and the primary unit will copy the configuration over. The main reason for this is that the active/standby units will essentially be viewed as one device.
failover lan unit primary
failover lan interface FOLINK GigabitEthernet0/6
failover polltime unit msec 200 holdtime msec 800
failover polltime interface msec 500 holdtime 5
failover link StateLink GigabitEthernet0/5
failover interface ip FOLINK 172.16.101.1 255.255.255.0 standby 172.16.101.2
failover interface ip StateLink 172.16.100.1 255.255.255.0 standby 172.16.100.2
failover
interface GigabitEthernet0/0
nameif ouside
security-level 0
ip address 10.30.0.2 255.255.0.0 standby 10.30.0.3
!
interface GigabitEthernet0/1
nameif inside
security-level 100
ip address 192.168.1.1 255.255.255.0 standby 192.168.1.2
The configuration above on the primary asa is setting the unit up as the primary unit in the HA pair. Interface G0/6 is set up as the failover lan interface. The interface G0/5 is configured as the statelink. Each failover interface is set with the appropriate ip addresses. the polltime unit and interface commands sets the keepalive and holdtown timers. The Lan and Wan interfaces are configured with the appropriate IP addresses and standby addresses. When this configuration is completed, I issued the no shutdown command on all the interfaces and proceed with the configuration of the standby unit.
failover lan unit secondary
failover lan interface FOLINK GigabitEthernet0/6
failover polltime unit msec 200 holdtime msec 800
failover polltime interface msec 500 holdtime 5
failover link StateLink GigabitEthernet0/5
failover interface ip FOLINK 172.16.101.1 255.255.255.0 standby 172.16.101.2
failover interface ip StateLink 172.16.100.1 255.255.255.0 standby 172.16.100.2
failover
On the secondary unit, you would issue the same commands except for Failover lan unit primary, instead we would replace primary with secondary. Once this step is completed, the console is going to show Detected an Active Mate. This message indicates the failover link is communicating. On the Primary unit, there will be a message stating configuration is being copied to the mate.
To verify the failover the command ASA#show failover will show the following output.
ASA# show failover
Failover On
Failover unit Secondary
Failover LAN Interface: FOLINK GigabitEthernet0/6 (up)
Reconnect timeout 0:00:00
Unit Poll frequency 200 milliseconds, holdtime 800 milliseconds
Interface Poll frequency 500 milliseconds, holdtime 5 seconds
Interface Policy 1
Monitored Interfaces 3 of 61 maximum
MAC Address Move Notification Interval not set
Version: Ours 9.6(1), Mate 9.6(1)
Serial Number: Ours 9A1A78HU36F, Mate 9ATQR0R363F
Last Failover at: 00:16:44 UTC Feb 2 2018
This host: Secondary – Standby Ready
Active time: 0 (sec)
slot 0: empty
Interface ouside (10.30.0.3): Normal (Monitored)
Interface inside (192.168.1.2): Normal (Monitored)
Interface management (192.168.100.2): Normal (Monitored)
Other host: Primary – Active
Active time: 1477 (sec)
Interface ouside (10.30.0.2): Normal (Monitored)
Interface inside (192.168.1.1): Normal (Monitored)
Interface management (192.168.100.1): Normal (Monitored)
Stateful Failover Logical Update Statistics
Link : StateLink GigabitEthernet0/5 (up)
Stateful Obj xmit xerr rcv rerr
General 65 0 66 0
sys cmd 65 0 65 0
up time 0 0 0 0
RPC services 0 0 0 0
TCP conn 0 0 0 0
UDP conn 0 0 0 0
ARP tbl 0 0 0 0
Xlate_Timeout 0 0 0 0
IPv6 ND tbl 0 0 0 0
VPN IKEv1 SA 0 0 0 0
VPN IKEv1 P2 0 0 0 0
VPN IKEv2 SA 0 0 0 0
VPN IKEv2 P2 0 0 0 0
VPN CTCP upd 0 0 0 0
VPN SDI upd 0 0 0 0
VPN DHCP upd 0 0 0 0
SIP Session 0 0 0 0
SIP Tx 0 0 0 0
SIP Pinhole 0 0 0 0
Route Session 0 0 0 0
<— More —>
The output of this command is useful for troubleshooting and verifying the failover system. It shows which unit you are on, which unit is the primary/secondary, which interfaces are being monitored, interface states as well as the state statistics. Once I have verified this is working, the next step is for me to get the rest of the primary unit configured for NAT, routing, etc. For this step I am using the ASDM as it is quicker to use that to create the NAT RULES, ACLS, ROUTING etc. I am not going to go over that as this is a H/A set up. However, the final configuration for the running configuration is below.
ASA# show running-config
: Saved
:
: Serial Number: 9ATQR0R363F
: Hardware: ASAv, 2048 MB RAM, CPU Pentium II 2294 MHz
:
ASA Version 9.6(1)
!
hostname ASA
enable password 8Ry2YjIyt7RRXU24 encrypted
names
!
interface GigabitEthernet0/0
nameif ouside
security-level 0
ip address 10.30.0.2 255.255.0.0 standby 10.30.0.3
!
interface GigabitEthernet0/1
nameif inside
security-level 100
ip address 192.168.1.1 255.255.255.0 standby 192.168.1.2
!
interface GigabitEthernet0/2
shutdown
no nameif
no security-level
no ip address
!
interface GigabitEthernet0/3
shutdown
no nameif
no security-level
no ip address
!
interface GigabitEthernet0/4
shutdown
no nameif
no security-level
no ip address
!
interface GigabitEthernet0/5
description STATE Failover Interface
!
interface GigabitEthernet0/6
description LAN Failover Interface
!
interface Management0/0
management-only
nameif management
security-level 100
ip address 192.168.100.1 255.255.255.0 standby 192.168.100.2
!
ftp mode passive
same-security-traffic permit inter-interface
same-security-traffic permit intra-interface
object network 192.168.1.net
subnet 192.168.1.0 255.255.255.0
access-list inbound extended permit icmp any any time-exceeded
access-list inbound extended permit icmp any any unreachable
access-list outbound extended permit icmp any any
access-list ouside_access_in extended permit icmp any any
pager lines 23
logging asdm informational
mtu ouside 1500
mtu inside 1500
mtu management 1500
failover
failover lan unit primary
failover lan interface FOLINK GigabitEthernet0/6
failover polltime unit msec 200 holdtime msec 800
failover polltime interface msec 500 holdtime 5
failover link StateLink GigabitEthernet0/5
failover interface ip FOLINK 172.16.101.1 255.255.255.0 standby 172.16.101.2
failover interface ip StateLink 172.16.100.1 255.255.255.0 standby 172.16.100.2
icmp unreachable rate-limit 1 burst-size 1
no asdm history enable
arp timeout 14400
no arp permit-nonconnected
!
object network 192.168.1.net
nat (any,ouside) dynamic interface
access-group ouside_access_in in interface ouside
route ouside 0.0.0.0 0.0.0.0 10.30.0.1 1
timeout xlate 3:00:00
timeout pat-xlate 0:00:30
timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 sctp 0:02:00 icmp 0:00:02
timeout sunrpc 0:10:00 h323 0:05:00 h225 1:00:00 mgcp 0:05:00 mgcp-pat 0:05:00
timeout sip 0:30:00 sip_media 0:02:00 sip-invite 0:03:00 sip-disconnect 0:02:00
timeout sip-provisional-media 0:02:00 uauth 0:05:00 absolute
timeout tcp-proxy-reassembly 0:01:00
timeout floating-conn 0:00:00
user-identity default-domain LOCAL
http server enable
http 192.168.100.0 255.255.255.0 management
no snmp-server location
no snmp-server contact
crypto ipsec security-association pmtu-aging infinite
crypto ca trustpoint _SmartCallHome_ServerCA
no validation-usage
crl configure
crypto ca trustpool policy
auto-import
crypto ca certificate chain _SmartCallHome_ServerCA
certificate ca 6ecc7aa5a7032009b8cebcf4e952d491
308205ec 308204d4 a0030201 0202106e cc7aa5a7 032009b8 cebcf4e9 52d49130
0d06092a 864886f7 0d010105 05003081 ca310b30 09060355 04061302 55533117
30150603 55040a13 0e566572 69536967 6e2c2049 6e632e31 1f301d06 0355040b
13165665 72695369 676e2054 72757374 204e6574 776f726b 313a3038 06035504
0b133128 63292032 30303620 56657269 5369676e 2c20496e 632e202d 20466f72
20617574 686f7269 7a656420 75736520 6f6e6c79 31453043 06035504 03133c56
65726953 69676e20 436c6173 73203320 5075626c 69632050 72696d61 72792043
65727469 66696361 74696f6e 20417574 686f7269 7479202d 20473530 1e170d31
30303230 38303030 3030305a 170d3230 30323037 32333539 35395a30 81b5310b
30090603 55040613 02555331 17301506 0355040a 130e5665 72695369 676e2c20
496e632e 311f301d 06035504 0b131656 65726953 69676e20 54727573 74204e65
74776f72 6b313b30 39060355 040b1332 5465726d 73206f66 20757365 20617420
68747470 733a2f2f 7777772e 76657269 7369676e 2e636f6d 2f727061 20286329
3130312f 302d0603 55040313 26566572 69536967 6e20436c 61737320 33205365
63757265 20536572 76657220 4341202d 20473330 82012230 0d06092a 864886f7
0d010101 05000382 010f0030 82010a02 82010100 b187841f c20c45f5 bcab2597
a7ada23e 9cbaf6c1 39b88bca c2ac56c6 e5bb658e 444f4dce 6fed094a d4af4e10
9c688b2e 957b899b 13cae234 34c1f35b f3497b62 83488174 d188786c 0253f9bc
7f432657 5833833b 330a17b0 d04e9124 ad867d64 12dc744a 34a11d0a ea961d0b
15fca34b 3bce6388 d0f82d0c 948610ca b69a3dca eb379c00 48358629 5078e845
63cd1941 4ff595ec 7b98d4c4 71b350be 28b38fa0 b9539cf5 ca2c23a9 fd1406e8
18b49ae8 3c6e81fd e4cd3536 b351d369 ec12ba56 6e6f9b57 c58b14e7 0ec79ced
4a546ac9 4dc5bf11 b1ae1c67 81cb4455 33997f24 9b3f5345 7f861af3 3cfa6d7f
81f5b84a d3f58537 1cb5a6d0 09e4187b 384efa0f 02030100 01a38201 df308201
db303406 082b0601 05050701 01042830 26302406 082b0601 05050730 01861868
7474703a 2f2f6f63 73702e76 65726973 69676e2e 636f6d30 12060355 1d130101
ff040830 060101ff 02010030 70060355 1d200469 30673065 060b6086 480186f8
45010717 03305630 2806082b 06010505 07020116 1c687474 70733a2f 2f777777
2e766572 69736967 6e2e636f 6d2f6370 73302a06 082b0601 05050702 02301e1a
1c687474 70733a2f 2f777777 2e766572 69736967 6e2e636f 6d2f7270 61303406
03551d1f 042d302b 3029a027 a0258623 68747470 3a2f2f63 726c2e76 65726973
69676e2e 636f6d2f 70636133 2d67352e 63726c30 0e060355 1d0f0101 ff040403
02010630 6d06082b 06010505 07010c04 61305fa1 5da05b30 59305730 55160969
6d616765 2f676966 3021301f 30070605 2b0e0302 1a04148f e5d31a86 ac8d8e6b
c3cf806a d448182c 7b192e30 25162368 7474703a 2f2f6c6f 676f2e76 65726973
69676e2e 636f6d2f 76736c6f 676f2e67 69663028 0603551d 11042130 1fa41d30
1b311930 17060355 04031310 56657269 5369676e 4d504b49 2d322d36 301d0603
551d0e04 1604140d 445c1653 44c1827e 1d20ab25 f40163d8 be79a530 1f060355
1d230418 30168014 7fd365a7 c2ddecbb f03009f3 4339fa02 af333133 300d0609
2a864886 f70d0101 05050003 82010100 0c8324ef ddc30cd9 589cfe36 b6eb8a80
4bd1a3f7 9df3cc53 ef829ea3 a1e697c1 589d756c e01d1b4c fad1c12d 05c0ea6e
b2227055 d9203340 3307c265 83fa8f43 379bea0e 9a6c70ee f69c803b d937f47a
6decd018 7d494aca 99c71928 a2bed877 24f78526 866d8705 404167d1 273aeddc
481d22cd 0b0b8bbc f4b17bfd b499a8e9 762ae11a 2d876e74 d388dd1e 22c6df16
b62b8214 0a945cf2 50ecafce ff62370d ad65d306 4153ed02 14c8b558 28a1ace0
5becb37f 954afb03 c8ad26db e6667812 4ad99f42 fbe198e6 42839b8f 8f6724e8
6119b5dd cdb50b26 058ec36e c4c875b8 46cfe218 065ea9ae a8819a47 16de0c28
6c2527b9 deb78458 c61f381e a4c4cb66
quit
telnet timeout 5
ssh stricthostkeycheck
ssh timeout 5
ssh key-exchange group dh-group1-sha1
console timeout 0
dhcpd address 192.168.1.10-192.168.1.200 inside
dhcpd dns 8.8.8.8 interface inside
dhcpd enable inside
!
dynamic-access-policy-record DfltAccessPolicy
!
class-map inspection_default
match default-inspection-traffic
!
!
policy-map type inspect dns preset_dns_map
parameters
message-length maximum client auto
message-length maximum 512
policy-map global_policy
class inspection_default
inspect dns preset_dns_map
inspect ftp
inspect h323 h225
inspect h323 ras
inspect rsh
inspect rtsp
inspect esmtp
inspect sqlnet
inspect skinny
inspect sunrpc
inspect xdmcp
inspect sip
inspect netbios
inspect tftp
inspect ip-options
inspect http
inspect icmp
inspect icmp error
class class-default
set connection decrement-ttl
!
: end
ASA#
Beginning configuration replication: Sending to mate.
End Configuration Replication to mate
To test the high availability set up, I ran a continuous ping from the client pc to site on the internet. I shutdown the interface on the client switch going to the primary ASA and after 1 second and 1 dropped ping packet, the secondary unit took over and the connection continued to work. I re-enabled the interface, the primary unit became the primary unit, but continued to be in standby. This is because there is no preempt feature. The only way this would become the active unit is if the secondary unit fails. Overall, I have achieved a network that contained all the requirements for the lab.
I have learned a great deal designing and working on this lab. It took me several days to comb through all of the documentation of on cisco’s sites and on through various message boards. The High available set up on the ASA’s are done quite well. I like how that most of the configuration is done on one device and you can achieve close to sub second failover.
