Cisco AnyConnect Secure Mobility Client version 4.7.04056
This one drove me nuts for the longest time until I found time to dedicate to troubleshooting it myself. Symptoms were that my AnyConnect client had been disconnecting, reconnecting every few minutes (2:50 to be exact!), which would, in turn, timeout my RDP session. Total reconnect time was only a few seconds, but you can imagine how having your concentration broken every three minutes is a productivity killer!
I had troubleshot this with my ISP, Comcast/Xfinity and my customer (whose site I was connecting to via VPN). Both essentially were pointing fingers at each other. It would be easy to blame the ISP because the problem didn't happen over my hotspot, but I can't help but think that the VPN server wasn't configured to properly handle such situations. Anyway, I decided to live with it (for far too long) until I could do some troubleshooting myself and figure out next steps.
My troubleshooting steps are below, in case anyone is interested.
| Wireshark | Wireshark VPN test-2019-12-09-A.pcapng Wireshark VPN test-2019-12-09-G-Comcast.pcapng Wireshark VPN test-2019-12-09-F-Hotspot.pcapng Wireshark VPN test-2019-12-09-E-Comcast-Reconnect at 129 sec.pcapng Wireshark VPN test-2019-12-09-D-Hotspot.pcapng Wireshark VPN test-2019-12-09-C-Comcast-Reconnect at 91 sec.pcapng Wireshark VPN test-2019-12-09-B.pcapng | Noticed that most application traffic happens via DTLS (OpenSSL) over UDP, but every 20 seconds, there's a TLSv1.2 transmission from the client [PSH, ACK], but no response from the server. Client retransmits the [PSH, ACK] in intervals of 0.3, 0.6, 1.2, 2.4, 4.8, 9.6 seconds, and then sends a RST. |
| Google search | cisco vpn client tls every 20 seconds no ack | |
| Article above references this, which was the most helpful | As long as DTLS is enabled, the client applies the DTLS MTU (in this case 1418) on the VPN adapter (which is enabled before the DTLS tunnel is established and is needed for routes/filters enforcement), to ensure optimum performance. If the DTLS tunnel cannot be established or it is dropped at some point, the client fails over to TLS and adjusts the MTU on the virtual adapter (VA) to the TLS MTU value (this requires a session level reconnect). | |
| Block UDP (in & out) for VPN client in Windows Firewall | C:\Program Files (x86)\Cisco\Cisco AnyConnect Secure Mobility Client\vpnagent.exe |
|
Thank you! This seems to have solved the issue for me!
ReplyDeleteHi Joe,
ReplyDeleteI don't seem to be able to find that dialog box of Windows firewall that you have shown on your page. Could you please help me locate it?
Windows 10:
Delete1. Start -> Settings -> Network & Internet -> (Scroll down to) Windows Firewall -> Advanced Settings
or:
Start -> Type "Windows Firewall" and select Windows Defender Firewall with Advanced Security
2. Select Inbound Rules, click New Rule, and create a Custom rule.
Thanks for digging into this. I must be doing something wrong because now that I blocked UDP, I cannot connect to VPN using Cisco Anyconnect at all. I followed the steps of creating Inbound/Outbound rules in the Advanced Security Menu of Windows Defender Firewall. The error pop up message I am getting is "AnyConnect was not able t oestablish a connection to the specified secure gateway. Please try connecting again." I have tried multiple times without success. Any suggestions on what I may have done wrong?
ReplyDeleteI'm sorry to hear that it's not working for you. I don't know much about VPN administration, but I wonder if the administrators are forcing UDP/DTLS. I forgot to mention in my post that I was somehow a unique outlier, connecting with my personal computer (as a consultant). Most others have company-issued PCs. I haven't heard of anyone else who's experiencing this issue with this company; and once they were able to point the finger at my ISP, they gave up on me.
DeleteReally sorry I can't help you further, but here are more things I tried, but failed to fix the problem.
- Waiting for (several) Windows updates.
- Installing a brand new wireless router.
- Wired connection to the router.
- Changing wireless password to kick all other devices off.
- Countless reboots (computer and router).
- Connecting to mobile hotspot. As I mentioned, the problem went away over the hotspot, but this wasn't a viable permanent solution.
- Installing VPN client on another computer in the house.
- Troubleshooting with company's helpdesk.
- Troubleshooting with my ISP.
- Constantly pinging company's servers or 8.8.8.8.
- New/different VPN profile settings (provided by company).
Good luck. Can you please let me know when and how you fix the issue?
Cheers,
Joe
The IT group who manages the headend ASA should unblock UDP 443 to the ASA, so DTLS can properly establish the tunnel. That's the REAL solution, and it will also result in better VPN performance because DTLS has less overhead, and works better than TCP/TLS for latency-sensitive traffic.
ReplyDeleteThis reply about allowing UDP 443) through then firewall fixed my issue. We have a meraki MX at perimeter and an ASA as a VPN gateway. Was having really low throughout and Wireshark pointed to 'something' wrong. Allowing UDP port 443 through sped up the performance greatly. Thanks for the tip so I though i would leave mine in case it helps someone
DeleteListed here you'll learn it is important, them offers the link in an helpful webpage: 900 seconds into minutes
ReplyDeleteWhat a find. This was fabulous. Thank you Joe Zamora!
ReplyDeleteLoved reading this thankk you
ReplyDelete