We have been having a lot of discussions about how to better protect against ransomware and other threats with the recent attacks that have happened in area school districts. Recently, I heard a story about a network that was hit with ransomware. The network admins cleaned up the mess and started to restore their "air-gapped" backups. Quickly they found themselves back in the same situation because the backups were compromised and contained ransomware!
The backups were "air-gapped" so the bad guys couldn't delete them as is often the case. Instead, they infected them. Last week I watched a ransomware post-mortem where an ex-NSA offensive hacker said that often the bad guys are figuring out what the site's backup retention time is and then see if they can infect the backups through the entire retention period before launching the major attack. Remember the average statistics for how long a bad guy can exist in a network before being detected are horrific. Most experts easily state 100-200 days. Some reports are longer than that.
So what is one to do?
There is no magic "ensure my backups are ransomware free" feature in any backup program.
Here are a few suggestions:
- Make sure your backups are air-gapped. There must be no access between an infected machine and either the backup server or the actual backup data. That could be a protected backup internally. It could be a remote backup to your BOCES/RIC or across town to a protected VLAN. It could be a cloud backup More and more we are seeing local backups such as Veeam combined with remote backups off-site to a BOCES/RIC. Some have a Veeam Cloud repository behind that.
- Make sure you maintain a backup that is completely, physically separate such as copying vital data to a removable hard drive that is not part of your or any network.
- Audit your backup strategy including what you backup, whether you have all the agents to cleanly back up whatever you need to back up, and what your retention strategy is. If you are not the person or organization doing the off-site backups, ask your provider these questions. Are you comfortable with how much data you are backing up and how long you are saving it for?
- Test restore data and even a server in a sandbox to prove your backups work. This should be done at least monthly and I know some sites do some form of test restore weekly. If your data is being encrypted on a server, when you run a test restore, you should see the encrypted files when you go to select your test restore files.
- Make sure you patch your operating systems and third-party applications on all your servers and endpoints to minimize the attack surface. Every analysis I see of major attacks against networks shows that the attacker exploited a "well known", "long-standing" vulnerability that has long since had a patch available. Day zero attacks do happen, but they are extremely rare.
- Make sure you have a well maintained, centrally managed antivirus/antimalware agent on each and every endpoint. Ideally, you should have advanced endpoint detect and response (EDR) clients vs. traditional antivirus. CSI's CyberSentinel Endpoint Detect & Respond (CSEDR) client is our preferred protection model because it combines the best of antivirus/antimalware, with the next generation, advanced EDR technology PLUS a 24x7x365 Security Operations Center (SOC) monitoring the CSEDR clients.
- If the bad guys have gotten a foothold somewhere in your network, sooner or later they are going to phone home to report they have penetrated the network. If the infection has gotten past traditional defenses - for whatever reason - then we hopefully will see it based upon behavior. They are going to phone home. CSI's Managed Firewall Service is a way to potentially see suspect traffic.
- If you like to tinker and tweak, you can implement Microsoft's free File Server Resource Monitor (FSRM) to actively monitor file types being saved to your servers and attempt to block/alert on common ransomware file types real-time. There are people on the internet maintaining lists of known ransomware file types and file names. If you have the time and the patience to find whatever the latest threat is and add it to all your servers, you can potentially see it and kill it before it takes root.
Remember all these technologies are screen doors. They are not barn doors. But hopefully using multiple technologies, looking at your servers, workstations, endpoints, network and firewall traffic from different perspectives somebody is going to see the bad guys before things get out of hand.
If you want to talk through how to improve your network and endpoint security, give us a call. We are happy to help.