The Scoop
Software engineers at the cybersecurity firm CrowdStrike complained about rushed deadlines, excessive workloads, and increasing technical problems to higher-ups for more than a year before a catastrophic failure of its software paralyzed airlines and knocked banking and other services offline for hours.
“Speed was the most important thing,” said Jeff Gardner, a senior user experience designer at CrowdStrike who said he was laid off in January 2023 after two years at the company. “Quality control was not really part of our process or our conversation.”
The issues were raised during meetings, in emails, and in exit interviews, ex-employees told Semafor. Almost two dozen former software engineers, managers and other staff described a workplace where executives prioritized speed over quality, workers weren’t always sufficiently trained, and mistakes around coding and other tasks were rising. One former senior manager said they sat in multiple meetings where staff warned company leaders that CrowdStrike would “fail” its customers by releasing products that couldn’t be supported.
Of the 24 former employees who spoke to Semafor, 10 said they were laid off or fired and 14 said they left on their own. One was at the company as recently as this summer. Three former employees disagreed with the accounts of the others. Joey Victorino, who spent a year at the company before leaving in 2023, said CrowdStrike was “meticulous about everything it was doing.”
CrowdStrike disputed much of Semafor’s reporting and said the information came from “disgruntled former employees, some of whom were terminated for clear violations of company policy.” The company told Semafor: “CrowdStrike is committed to ensuring the resiliency of our products through rigorous testing and quality control, and categorically rejects any claim to the contrary.”
Founded in 2011, CrowdStrike quickly rose as an industry leader in cybersecurity with the 2013 launch of its Falcon antivirus package. It went public in 2019, kicking off a massive growth spurt, adding thousands of workers and increasing revenue by more than a thousand percent by the end of fiscal year 2024.
A bad software update by CrowdStrike in July caused what may be the biggest IT outage in history, shutting down 8.5 million computers and costing Fortune 500 companies as much as $5.4 billion in damages. It stranded travelers at airports, locked customers out of online banking accounts, and took emergency call centers offline. The incident cost CrowdStrike about $60 million in deals it had expected to close during the fiscal quarter that ended July 31, Chief Financial Officer Burt Podbere told analysts on its Aug. 28 earnings call, when the company lowered its revenue and profit guidance for the rest of the year. Adam Meyers, senior vice president of counter adversary operations, will testify in front of Congress later this month.
“The magnitude of the July 19th incident will never be lost on me, and my commitment is to make sure this never happens again,” CEO George Kurtz told analysts on the call. “Beyond apologies, I want our actions to speak even louder than our words. We work to recover customers quickly no matter the location or need.”
The former employees Semafor spoke to described a range of issues that long preceded the outage at the company. There’s been no determination that those problems were related to the July incident.
Know More
Some former employees said quality checks on software were rushed at times to get products launched quickly.
“It was hard to get people to do sufficient testing sometimes,” said Preston Sego, who worked at CrowdStrike from 2019 to 2023. His job was to review the tests completed by user experience developers that alerted engineers to bugs before proposed coding changes were released to customers. Sego said he was fired in February 2023 as an “insider threat” after he criticized the company’s return-to-work policy on an internal Slack channel. That’s the company’s designation for employees who present security risks. CrowdStrike declined to comment, saying it does not “discuss individual personnel matters.”
There were other issues. In one incident in the professional services department, one former employee described how a customer’s private information was accidentally uploaded to the wrong client’s folder three times, narrowly escaping sharing private client data with the wrong customer each time. CrowdStrike confirmed the incidents and said they occurred because of a “manual data entry error.” It said the data was “basic information like host names, IP addresses, and domain names,” and “checks are now run” to ensure private customer data isn’t sent to the wrong client.
Multiple people also cited issues with CrowdStrike’s Falcon LogScale service, which uncovers security and reliability issues in a customer’s systems. One recalled at least two instances where bad updates to LogScale briefly turned off its real-time alerts that notify customers of potentially malicious activity, which some of the engineers who built the updates blamed in internal meetings on tight deadlines. CrowdStrike denied the instances, saying it is not aware of any “‘bad update’ where alerts were lost and not received by customers.” The company also said the service isn’t designed to alert customers to potential data breaches in “real time.” It, instead, is designed to “rapidly shut down threats with real-time detection and blazing-fast search,” according to the company website.
A separate ex-employee said CrowdStrike rushed the 2022 launch of its cloud threat hunting service, called Falcon OverWatch Cloud Threat Hunting, where the company’s security professionals look for suspicious behavior that could indicate a breach on customers’ cloud setups, like Amazon Web Services. Engineers and threat hunters were given just two months for work that would normally take a year, according to a former senior manager who worked on the project. When the service launched, he said it lacked the internal tools that threat hunters used to fully monitor customers’ cloud systems for threats; employees ended up responding to alerts from existing security systems until at least last summer, about a full year after it was launched.
The former senior manager said CrowdStrike also used staff who had been trained to monitor customers’ computer systems — like laptops and desktops — and tasked them with looking for threats in cloud setups without mandating new training.
“AWS is a beast, and it takes a very special staff to be able to do that,” he said. CrowdStrike “took people who were like cops, looking for threats on the ground all day, and asked them to fly an airplane and look for threats in the sky.”
CrowdStrike confirmed that it used existing engineers instead of hiring a new team of “cloud threat hunters.” As a new service, it said, “there were no experienced ‘cloud threat hunters’ to be had, and it would not have been possible to hire individuals with specific training in a field that did not exist until CrowdStrike developed it.” The SANS Institute has been teaching courses and giving talks on cloud security since at least 2020, more than two years before the launch of CrowdStrike’s service.
“Any statement implying CrowdStrike employees were not trained to do their jobs is false,” CrowdStrike told Semafor. While the company confirmed that it didn’t mandate new training, it provided it for anyone who wanted it, the company said. “Employees routinely attend training appropriate to their position.”
“This service has worked as intended at all times,” CrowdStrike said. “Even before this novel service offering was launched the Falcon Overwatch team hunted on all public cloud environments and released research into this area.”
CrowdStrike also denied that its systems lacked the tools threat hunters needed and that it rushed the project. The company said the OverWatch product line has been around for more than a decade “and is routinely enhanced to meet the evolving threats and needs of our customers.”
Sego said temporary coding meant to keep projects moving — a common practice at tech companies — was often never improved. One former senior engineer said he asked unsuccessfully to be given time to fix old coding more than 20 times. CrowdStrike said “coding is an iterative process, and it is commonplace in the software industry to release and continuously improve upon code based on real-world experience with the product.”
Ex-employees cited increased workloads as one reason they didn’t improve upon old code. Several said they were given more work following staff reductions and reorganizations; CrowdStrike declined to comment on layoffs and said the company has “consistently grown its headcount year over year.” It added that R&D expenses increased from $371.3 million to $768.5 million from fiscal years 2022 to 2024, “the majority of which is attributable to increased headcount.”
CrowdStrike said it “receives, evaluates, and incorporates a range of feedback from its team,” and that it “focuses on always maintaining a high-performance culture.” The company also noted that it “has been recognized as one of the Fortune 100 Best Companies to Work For for the last four years.”
For the July outage, CrowdStrike has blamed a defect in an update to its Falcon Sensor. The episode has cost the company more than $21 billion in stock-market value and brought on a slew of lawsuits, including one potential suit by Delta Airlines, which pegged its losses at $550 million after thousands of flights were canceled.
At a hacker convention in August, CrowdStrike President Michael Sentonas accepted an award on stage for “Most Epic Fail.” He said it’s “super-important to own it when you do things horribly wrong.”