A recent CrowdStrike update has wreaked havoc on Windows-based systems, causing them to crash and display the infamous Blue Screen of Death. Reports indicate that organizations worldwide are grappling with reboot failures. Notably, Sky News has been unable to broadcast due to the outage.
Worried users have flocked to forums like Reddit to voice their concerns, with one user lamenting: “Wow, stuck in a boot loop, and entire org taken out.”
So, if you arrived at work this morning to find utter chaos, rest assured, you are not alone. Here’s a breakdown of the situation and the next steps to take.
As suspected, the root cause of the widespread global disruption is an issue with CrowdStrike. Engineers from CrowdStrike are currently addressing the problem, which impacts their Falcon Sensor product. CrowdStrike describes Falcon as “the platform purpose-built to prevent breaches via a unified set of cloud-delivered technologies that thwart all types of attacks—including malware and beyond.”
The IT outage has affected airports, businesses, and broadcasters, as reported by Sky News. In the U.S., flights have been grounded, trains in the U.K. are impacted, and boarding scanners at Edinburgh Airport in Scotland are down.
Microsoft has announced it is implementing “mitigation actions” following service issues that began around 6 p.m. Eastern Time. The company is investigating issues with its cloud services in the U.S. and several of its applications and services, according to Sky News.
I have reached out to CrowdStrike and Microsoft for comments and will update this article once they respond.
Initial reports focused on a faulty update, but Brody, director of CrowdStrike Overwatch, clarified on X (formerly Twitter) that the issue stems from “a faulty channel file, not quite an update.”
Brody also provided a workaround:
1: Boot Windows into Safe Mode or Windows Recovery Environment (WRE).
2: Navigate to C:\Windows\System32\drivers\CrowdStrike.
3: Locate and delete the file matching “C-00000291*.sys”.
4: Reboot normally.
While this workaround exists, it’s not scalable since it must be manually applied to each system. For large organizations, this could mean hours or more of downtime.
Adam Harrison, managing director at FTI Cybersecurity, emphasized the difficulty of resolving the issue once systems are in a reboot loop. “Manual fixes are going to take time for system admins to apply; CrowdStrike can’t remotely push a new update to fix this. Each system will require manual intervention.”
Harrison noted that while some may be able to roll back to known good states, most will not have that capability. “The fix itself is quick, but scaling it to thousands of servers or workstations makes it a significant challenge.”
The situation is equally challenging for CrowdStrike. What can they do to assist?
“They must communicate the fix as swiftly and broadly as possible,” said Harrison. “My assumption is that the update has already been pulled, so systems that hadn’t updated should avoid receiving the faulty update.”
Ian Thornton-Trump, CISO at Cyjax, asserted that CrowdStrike “will undoubtedly do their utmost to retract the update and instruct old agents not to update until the issue is resolved.”
However, Thornton-Trump acknowledged, “what’s done cannot be undone for those machines already affected. If they can be booted in safe mode, an out-of-band update or patch might be possible. This is time-consuming, and critical machines might need to be restored from backup or a shadow copy (a built-in MSFT recovery feature). Whatever path they choose, they will strive to fix it as quickly as possible.”
Harrison suggested that CrowdStrike might develop a tool to apply the fix at the disk level, such as bootable media. “This could help those with thousands of systems to fix, though it wouldn’t fully solve the problem remotely or at scale, but it could reduce recovery times.”
This story is developing. Stay tuned and check my Forbes page for updates.
Unfolding the Future: A Glimpse into Google's Pixel 9 Pro Fold As the dust settles…
Phil Donahue has passed away: Phil Donahue the creator of a groundbreaking daytime talk show…
No, this is not the new "Tiger King." Let's start there because HBO's new documentary…
Mike Tyson has declared himself "ready" for his upcoming bout against Jake Paul, now rescheduled…
John Aprea, best known for his portrayal of the young Salvatore "Sal" Tessio in "The…
An empty beach house on North Carolina’s Outer Banks, situated along the Cape Hatteras National…
View Comments