Tuesday, December 10, 2019

Client-side fix: Cisco AnyConnect reconnects every few minutes

TL;DR If Cisco AnyConnect is disconnecting, reconnecting every few minutes, try blocking UDP in/out ports for the vpnagent executable/service.

Cisco AnyConnect Secure Mobility Client version 4.7.04056

This one drove me nuts for the longest time until I found time to dedicate to troubleshooting it myself. Symptoms were that my AnyConnect client had been disconnecting, reconnecting every few minutes (2:50 to be exact!), which would, in turn, timeout my RDP session. Total reconnect time was only a few seconds, but you can imagine how having your concentration broken every three minutes is a productivity killer!

I had troubleshot this with my ISP, Comcast/Xfinity and my customer (whose site I was connecting to via VPN). Both essentially were pointing fingers at each other. It would be easy to blame the ISP because the problem didn't happen over my hotspot, but I can't help but think that the VPN server wasn't configured to properly handle such situations. Anyway, I decided to live with it (for far too long) until I could do some troubleshooting myself and figure out next steps.

My troubleshooting steps are below, in case anyone is interested.

Wireshark

Wireshark VPN test-2019-12-09-A.pcapng

Wireshark VPN test-2019-12-09-G-Comcast.pcapng

Wireshark VPN test-2019-12-09-F-Hotspot.pcapng

Wireshark VPN test-2019-12-09-E-Comcast-Reconnect at 129 sec.pcapng

Wireshark VPN test-2019-12-09-D-Hotspot.pcapng

Wireshark VPN test-2019-12-09-C-Comcast-Reconnect at 91 sec.pcapng

Wireshark VPN test-2019-12-09-B.pcapng

Noticed that most application traffic happens via DTLS (OpenSSL) over UDP, but every 20 seconds, there's a TLSv1.2 transmission from the client [PSH, ACK], but no response from the server.  Client retransmits the [PSH, ACK] in intervals of 0.3, 0.6, 1.2, 2.4, 4.8, 9.6 seconds, and then sends a RST.

Google search

cisco vpn client tls every 20 seconds no ack

https://community.cisco.com/t5/vpn-and-anyconnect/anyconnect-vpn-session-disconnect-and-reconnect/td-p/2474657

Article above references this, which was the most helpful

https://www.cisco.com/c/en/us/support/docs/security/anyconnect-secure-mobility-client/116881-technote-anyconnect-00.html

As long as DTLS is enabled, the client applies the DTLS MTU (in this case 1418) on the VPN adapter (which is enabled before the DTLS tunnel is established and is needed for routes/filters enforcement), to ensure optimum performance. If the DTLS tunnel cannot be established or it is dropped at some point, the client fails over to TLS and adjusts the MTU on the virtual adapter (VA) to the TLS MTU value (this requires a session level reconnect).

Block UDP (in & out) for VPN client in Windows Firewall

C:\Program Files (x86)\Cisco\Cisco AnyConnect Secure Mobility Client\vpnagent.exe