I heard that CloudStrike is something that runs on Windows servers, and an error with it caused a bunch of Win Servers to crash. What’s the impact of the issue too?
I’m not a tech person, tho I do use Linux desktop, btw 😉
deleted by creator
Great response, and matches with what I’ve been reading.
Holy shit, someone(s) at CrowdStrike have had a reeeeally bad end of the week!
Agreed. It is odd that CrowdStrike doesn’t seem to?? Please correct me. That they don’t have a test release and a general release.
Weird
It’s definitely weird. They might well throw one out two developers and As under the bus for this event (I’ve seen this happen at other organizations - and such decisions smack of shitty management). But no matter how you look at it this is a company-wide failure. Because they didn’t have the infrastructure and policies in place to test their changes properly.
For context, I do not work in anything remotely close to an IT department. I work in a hospital. This affected my work the other day too. I am a bit more tech savvy than some of my coworkers, so I was attempting to see if I could fix the issue on my own by reverting to the previous windows update in recovery mode.
However, doing so prompted me for a Windows product key, which I obviously didn’t have because I didn’t install Windows on that computer.
The IT department had to come around individually for every single affected computer. They had to manually look up and type out the unique Windows product key for every single affected computer in order to be able to fix the problem.
Not sure if most installs of Windows act that way or not, but it definitely made the process more manual and annoying than it had to be. I have no idea why many of the recovery options required me to look up and enter a Windows product key. Seemed very odd to me and just made the ordeal more manual and time consuming than it had to be.
I believe some hospitals even ended up having to cancel surgeries.
deleted by creator
I suppose that’s possible, but I didn’t see the word “Bitlocker”. Would that not necessarily appear on screen? It just asked for a product key, which I thought was odd.
It wasn’t blocking me from logging into Windows (which would blue screen though). It was instead blocking me from using certain recovery options.
Edit: After some digging, that is likely what it was even thought it didn’t say Bitlocker on the screen. From screenshots, it looks like it occasionally doesn’t say that. Would make sense for security purposes and I’m sure many companies had something like that enabled. It made fixing the whole ordeal a much more slow and manual process though instead of just giving users some instructions!
Also sorry idk who downvoted you!
What a disaster! Thanks for the thorough response 😊
I was wondering why this didn’t happen to me, and I guess it’s because my company uses Carbon Black which seems to be a competitor to CrowdStrike. Phew!
Thanks for the good explanation.
I don’t know the details, but skimming past headlines, looks like it was blue screen boot loop. So it must have involved a low level key like UEFI Secure Boot package or shim… I think.
So, I kind of know what I’m talking about. I’m a software engineer with a passion for cyber security, but I’ve never worked with cloud strike or tools like it. Cloud strike is a Microsoft network security tool that runs on local IT servers, as I understand it, to help keep them safe. Well, they pushed an update last night without user approval that fucked up some file that the entire local computer depends on, which caused blue screen loops. The only way to fix this is to go and manually boot in safe mode, and if the server is secured by bitlocker, then you have to individually look up and enter the boot key for each server. Basically, even though cloud strike fixed the problem, it’s already on all these local systems and can’t be rolled back or repaired by them; each organization affected has to go and fix it themselves. Because it’s a network dependency, it means portions or wholes of these companies’ networks are down until they can manually save boot all 5,000 of their servers or whatever.
I’m guessing the 33% vote rate is due to your intro, not your explanations?
Intrinsic trust in outside actors in the kernel is a bad idea.
State actors wish they could fuck shit up like Microsoft.
Crowd strike is an antivirus program that is installed on servers and laptops/desktops. The update corrupted a file that caused the operating system not to boot. The impact is thousands of hours of manual labor to recover these servers and endpoints. You have to do it in person unless your user is tech savvy enough to get into safe mode and delete a file. And you need admin rights
To add to these guys, what it looked like in hospitals was all computers going blue screen of death on a loop. You would reboot and it would get to the desktop and go BSOD again. Communications with windows servers also went down.
The problem is, the safety plans are contingent on having windows working. There’s little to no contingent on no windows, people just expect it to always be there.
I and my team and all the individuals we ended up responsible for were fine by morning, but it was not a safe time.
911 was affected. Ambulance dispatch was affected. Many medical based institutions rely on windows and these security systems as well as airlines and such.
That said, please check on grandma, your favorite old uncle, single parent living solo, and all the rest today. There is no telling how many people are dead or injured in homes right now who couldn’t get through to emergency services last night. Maybe still can’t today.
My hospital is gonna be so mad I quit I could actually probably have done the computers for my nursing unit it didn’t sound that complicated (but I know the other nurses would have been lost).
There is no telling how many people are dead or injured in homes right now who couldn’t get through to emergency services last night.
Waooooowww. I hadn’t realized how harmful this issue was. It sounds like CloudStrike’s future is bleak.
**Clown Strike
Including Mercedes formula1 team lol.
Big time AV software used by a lot of conpanies issued an update that crashed all windows machines it was installed on. The only way to fix it is to boot into safe mode and then delete a file. On millions upon millions of servers and pcs. One by one.