Don’t make it easy for broken code: Untested backups fail when you need them most

One of the first pieces of security advice you probably ever heard, and the one that’s been repeated the most, is to back up your systems. It’s good advice. When something goes wrong, you restore from your copy, and all’s well that ends well.

But it’s not always that simple. What people throw into a backup isn’t always double-checked to be complete, safe, or current. So it’s no surprise when a restore fails miserably and leaves you with no clear way forward, whether you’re restoring an application your own team runs, one your customers depend on, or someone else’s software from an escrow deposit.

This is the last post in our series on 2026’s Cybersecurity Awareness Month themes, and it turns to broken code, in your software and in the backups meant to bring it back. It covers what goes missing from backups without anyone noticing, what untested backups look like in a real outage, and how to know yours will restore.

Backups are often doomed from the start

Say your team takes backups seriously. Someone has done the due diligence and packed everything they could think of to protect your critical software into the backup, from the code itself to the data it runs on. The jobs run on schedule, the reports come back clean, and as far as anyone can tell, the copy is complete.

Thing is, “best practice” uses broad strokes, and almost all software systems rely on several unique configurations. Here’s what people typically miss, even when they follow the rules to the letter.

• Software spread across several repositories won’t come back as a working application if even one was left out of the backup.

• API keys and credentials are kept out of code repositories for good security reasons, but that means an application restored from its code alone can’t connect to anything it depends on.

• Third-party packages get pulled in at build time, and nothing guarantees the exact versions your software was built with will still be available.

• Build scripts, environment files, and deployment configurations hold the instructions for putting the software together, and they’re easy to miss when they live outside the main repository.

• Technical debt means older systems rely on build tools and libraries that are hard to find or install today, so the code is there but the means to build it isn’t.

• Broken code gets backed up as faithfully as working code, so an unvetted AI-generated change or a corrupted update comes straight back with the restore.

And that’s just on day one. From then on, development keeps going as your team ships features, adds dependencies, rotates keys, and changes how the data is structured. A backup that doesn’t keep pace restores the system you were running six months ago, with data that’s just as out of date. That’s why backups, and the vaults holding software escrow deposits, need to sync every day.

A restore is supposed to put your application back together exactly as it was. But add up everything left out from the start and every change it never caught after that. What do you think the puzzle is going to look like with so many missing pieces?

Broken code hits as hard as a cyberattack

When broken code takes a system down, whether through a faulty update, a hidden bug, or a backup that won’t restore, the damage looks remarkably similar to the aftermath of a ransomware attack. Systems go down, data goes missing, and downtime costs start mounting all the same, except there’s no attacker to point to.

And that’s not a stretch of the truth. The aftereffects of broken code and faulty backups are well documented. Research from ENISA, the EU’s cybersecurity agency, shows that across the major telecom outages reported for 2024, faulty software changes and updates cost users more than 500 million hours of service, nearly three times the 184 million hours lost to every malicious action combined.

Let’s take a look at a specific incident. In January 2017, an engineer at code-hosting platform GitLab accidentally wiped its production database. GitLab had five ways to get that data back, and none of them worked, including a daily backup broken by a version mismatch nobody caught. The site was down for about 18 hours, and six hours of its users’ work never came back.

While that happened nine years ago, it’s still a common issue today. This September, a legacy defect corrupted flight data at UK air traffic control provider NATS. A separate copy reloaded within the hour, but like any out-of-sync backup, it came back with duplicate flight plans and mismatched codes that took until nearly 7 p.m. to fix. More than 2 000 flights were disrupted.

Neither outage was an attack, and for the passengers stuck at their gates or the developers missing six hours of work, that made no difference. Downtime and lost data hurt the same whatever causes them. If your recovery plan only prepares for attackers, it’s ignoring a risk that does just as much damage.

» Find out what a complete software backup needs to include to hold up in a real recovery.

Untested backups are a crack in your last line of defense

Those incidents don’t take away from the fact that restoring a full copy, with all your data intact, is the ideal way to recover from almost any disruption. Backups are the last line of defense after an attack, a mass deletion, or data corruption, because they put everything back the way it was. The UK’s National Cyber Security Centre (NCSC) calls up-to-date backups the most effective way to recover from ransomware.

That’s also why security agencies want those copies kept apart from your own systems. The NCSC recommends offline backups that sit separate from your network, ideally offsite or with a service built for the purpose, and the US government’s ransomware guide agrees, because ransomware goes looking for backups to delete or encrypt. Separation keeps attackers out, but it can’t fix what’s already broken inside the copy.

The key comes down to eliminating broken code.

The best way to do that is to put your backups and escrow deposits to the test with regular rebuild exercises. By Codekeeper’s count, 90% of untested escrow deposits fail once someone finally tests them. Regulators have reached the same conclusion, and a growing list of them now write restore testing straight into their rules.

• GDPR expects any organization handling EU residents’ personal data to restore it promptly after an incident and to regularly test the measures protecting it.

• DORA, the EU’s Digital Operational Resilience Act, requires financial firms to periodically test backup and recovery procedures and keep sufficiently tested exit plans for technology supporting critical functions.

• Under NIS2, cloud, data center, and managed service providers, among others, have to test their continuity plans and regularly test that backups can be recovered.

• New York’s financial regulator requires the banks, insurers, and other firms it licenses to test restoring critical data and systems from backups at least once a year.

• The HIPAA Security Rule asks US healthcare organizations and their business associates for backup and disaster recovery plans, along with procedures for testing and revising them.

• In Australia, CPS 230 requires banks, insurers, and superannuation funds to test their business continuity plans every year, including scenarios where a key service provider fails.

• For software vendors, SOC 2’s availability criteria and ISO 27001’s backup control both expect backups and recovery procedures to be tested regularly.

Just throwing all of your critical assets into a backup isn’t enough. If you want to rely on it on a rainy day, you have to stress-test it frequently, so a stray bit of broken code or a missing dependency doesn’t kick you when you’re already down.

» See how a Marquis Software Solutions firewall backup became the way into 74 banks.

Make every backup prove its worth

You need to be able to trust that a backup can bring your systems back, and getting it to that point starts with where it’s kept.

Step one: Secure it in an independent vault

For software you build, Software Backup keeps your code with its full version history, your application data, and the build scripts, environment files, and deployment configurations a restore depends on, all in an immutable vault that ransomware and accidental deletion can’t touch.

For software you rely on, software escrow puts the source code, data, and documentation behind it with an independent party like Codekeeper, released to you under certain conditions agreed on in an escrow agreement. SaaS Escrow adds the deployment infrastructure, credentials, and third-party dependencies cloud software runs on. Both sync daily through 50+ integrations, so the copy matches the software you’re actually running.

Step two: Prove it restores

Software verification is how that copy gets tested before anyone needs it. Rebuild tests work out how the application is put together, rebuild it, and document the whole flow. If anything is missing or broken, that gets flagged now, while you can still do something about it. You end up with compliance evidence and a deployable copy ready in your vault.

Step three: Run an exit test

Verification proves the copy restores. An Exit Exercise takes a white-glove approach and runs a full stressed exit test, proving you could keep running if the provider behind the software disappeared. Codekeeper’s engineers recover the application from the deposit alone, completely independent of the vendor’s input, then port your data and test the features you need. That supports the exit-plan testing DORA and CPS 230 call for.

Every verification and Exit Exercise is documented as software resilience evidence. If you rely on the software, it shows your auditors that recovery has been tested. If you supply it, it gives procurement teams proof that your customers could recover without you.

» Work out which of your vendors need an exit test first.

Add restore testing to your security habits

One of 2026’s Cybersecurity Awareness Month themes was Securing the Next 250, with an eye on building the type of resilience that stands the test of time. That’s a long horizon, and simple daily cyber hygiene habits probably aren’t enough to reach it.

With all the ways software can fail, the rise in cyberattacks, growing compliance pressure, and even your own development pacing, your software is at constant risk of breaking beyond repair.

You need a fail-safe you can turn to no matter what happens. A reset button, if you will. And not just for your own software, for everything your business needs to function from day to day. For that, there’s Software Escrow.

» Put your disaster recovery to the test with Codekeeper’s software verification, while there’s still time to fix what’s missing.