SonicWall’s 21,000 channel partners had a very long weekend after the company admitted a sophisticated cyberattack against its internal systems had revealed zero-day product vulnerabilities.
Silicon East President Marc Harrison and two of his employees put in 36 hours of work Saturday and Sunday with almost no sleep after the Milpitas, Calif.-based platform security vendor disclosed it was hacked in at 11:15 p.m. ET Friday. The Marlboro, N.J.-based partner has 17 customers with 800 users on versions of the NetExtender VPN client or SMA 100 product that were initially reported compromised.
Harrison said Silicon East spent between four and six hours Saturday turning off SSL-VPN connections for all impacted users, and ended up working until 2 a.m. ET Sunday. Then at 10:45 p.m. ET Saturday, SonicWall updated its guidance to tell customers that NetExtender didn’t have a zero-day vulnerability after all, and that only its Secure Mobile Access (SMA) 100 series product remains under investigation.
As a result, Harrison and his associates returned to work Sunday morning to re-enable SSL-VPN access for the 14 customers and more than 400 employees at organizations using only NetExtender but not SMA 100. But given how extensively SSL-VPN connections have been used for remote work during COVID-19, Harrison needed to help the three clients and 400 users who were being blocked from work.
“This has been extreme pain,” Harrison said. “People are annoyed and upset, but understand it could have been a lot worse if they had been breached.”
For Silicon East’s three SMA 100 customers, Harrison attempted to follow SonicWall’s guidance to use NetExtender for remote access with the SMA 100 series while disabling Virtual Office, but couldn’t figure out how to do it. Harrison tried unsuccessfully to reach SonicWall tech support for 12 hours Sunday, and finally connected with someone Monday who told him they also weren’t aware of any way to do this.
“The workaround SonicWall published Saturday night is not implementable,” Harrison said. He expected SonicWall would provide partners with more information Sunday night like they had the two nights prior, and was surprised when no updates were posted.
SonicWall didn’t immediately respond to a CRN request for comment. A few thousand devices in the SMA 100 series product have been impacted by the zero-day vulnerability, according to a company spokesperson.
Silicon East is looking into providing a limited number of critical employees at affected customers with an IPsec tunnel using SonicWall’s Global VPN Client (GVC) so that they can work remotely, Harrison said. But this would need to be configured separately for each machine, which Harrison said would probably take 45 minutes per user.
Harrison has asked management of the affected clients to provide a list of their most critical users, and Silicon East will discuss what’s possible and start implementing the workaround. For the remaining users, Harrison said they’ll for now need to go into the office if they’re looking to get work done.
“SonicWall fully understands the challenges previous guidance had in a work-from-home environment, but the communicated steps were measured and purposeful in ensuring the safety and security of our global community of customers and partners,” the company said late Sunday.
The company’s initial Friday night bulletin was ominous and created panic among clients of Canaan Technology since the Norwalk, Conn.-based solution provider has several hundred SonicWall firewalls across its client sites, according to owner David Felton.
“Because it appeared to be a massive breach with huge potential impact to our client base, we immediately circulated the notice from SonicWall among our entire team – not just technicians, but anyone who has client-facing contact,” Felton told CRN.
Canaan Technology then begun the process of identifying which clients have the largest number of users who connect using NetExtender VPN client, Felton said. The company doesn’t have any SMA 100’s under management, so Felton was only concerned about the NetExtender vulnerability.
As Canaan Technology began investigating on a sampling of customer machines which version of NetExtender was installed, the company realized that not one of its clients had the affected version 10.x in place. This was concerning since SonicWall was as recently as last week still pushing out NetExtender 9.x with the latest firmware installed even though version 10.x should have been more recent, he said.
Felton thought about what to do next, and realized that cutting off SSL-VPN access for clients would be a major disruption to their business. The move would additionally cause irreversible damage to the SonicWall brand, which Canaan Technology had been advocating all its clients adopt for more than 20 years, according to Felton.
Given there were no indications that any of the company’s customers actually had the affected version of NetExtender, Felton said Canaan Technology opted to focus Saturday communications on clients who proactively reached out to the company. These customers were told Canaan Technology was closely monitoring the situation and that there was a heightened level of caution due to the SolarWinds breach.
Additionally, Felton said clients given instructions on how they could independently confirm that they were using an unaffected version 9.x of NetExtender. And then this morning, Felton said the company proactively emailed its entire client base to let them know the SonicWall products Canaan Technology manages for their environment were not affected by the compromise.