Why do SIEM products keep failing?

Recent announcements from vendors such as Palo Alto and Crowdstrike highlight that the SIEM market is thriving, with an expected value of $10 billion by 2030. However, some players seem to be struggling to stay relevant despite positive marketing efforts. Why are we seeing this conflicting evidence?

Firstly, it's not as conflicting as it initially appears. With newer entrants and established leaders, it becomes harder for struggling players to stay relevant. A strong market and failing products coexist because the success of the market makes it more competitive, challenging players to keep up.

Despite this, the number of failed or struggling products continues to grow. The count of corpses (or zombies) keeps increasing at an alarming rate. With the risk of showing my age, I can mention several SIEM products throughout the years that have not endured: Intellitactics, Netforensics, Sentinel (not MS, NetIQ), ArcSight, Nitro, Envision, Netwitness, Logrhythm, Exabeam. What is it about SIEM that keeps it a strong and relevant market while so many products fail?

The answer lies in the evolution of SIEMs. SIEMs from 20 years ago were completely different from today's SIEMs. The requirements for a SIEM have rapidly evolved. SIEMs today have to do more while handling a substantially larger amount of data. This evolution pushes vendors to the limit, forcing them to take shortcuts that create huge technology debt in the longer term.

The first "extinction wave" was caused by the data backend. The massive increase in the volume of data to be ingested by the SIEM forced vendors to adopt newer distributed systems, such as the Hadoop stack. Those SIEMs built on top of old traditional RDBMS (ArcSight on Oracle, for example) faced significant rearchitecture challenges, leading to the demise or irrelevance of many.

As the volume of data grew, the impact was felt on the other side of the pipe as well. It caused not only ingestion and retrieval performance problems but also drove up the number of alerts generated by the SIEM. The simple approach of applying correlation rules was not enough anymore. This led to the deployment of UEBA solutions on top of the SIEMs to augment analytics capabilities and prioritize the alerts generated. The natural consolidation of SIEM and UEBA, given their shared data and user base, pushed the most viable UEBA vendors to add SIEM capabilities. The traditional SIEM vendors responded by acquiring standalone SOAR and UEBA vendors, but many of them ended up with poorly integrated capabilities. These vendors started accumulating technical debt related to those integrations and, when the next wave came, it was just too much to handle.

To make things worse, when that next wave came, it wasn't just another wave, but a real tsunami: The cloud. The major pressure on SIEM solutions has always been the data volume to ingest, and with the strong shift of IT environments to the cloud, those volumes not only accelerated in growth, but they also moved in terms of location. The data to be ingested by the SIEM was not on-premises anymore, it was in the cloud. The SIEM needed to scale and to be able to ingest data from the cloud, and the natural way of doing it was to move it to the cloud. But moving things to the cloud is not simple. Surely you can just forklift things and keep them exactly the same as they were on-premises, but that doesn't allow you to leverage the elasticity and scalability of the cloud. Many SIEM vendors took too long to realize it; they initially followed the trend with the same architecture they were using on-premises, but soon realized they would have to go through a major re-architecture exercise. Those who took too long to do it ended up in the same situation as those from the RDBMS times, with tech debt to address and reduced ability to innovate.

The situation for those, such as Splunk, Exabeam, and Logrhythm, was cruel. As they slowly moved to launch their new cloud platforms or to finally integrate those previously acquired pieces, big players such as Microsoft, Google, Crowdstrike, and Palo Alto decided to join the game. At that point, vendors without a cloud-native, unified SIEM-UEBA-SOAR platform were just too far behind to compete. That's the scenario that culminated in the crazy "SIEM red wedding day" (h/t Fernando Montenegro) on May 15th.

This is a long story to find the proper response to my initial question, why do so many SIEM vendors fail? The response is not that they are operating in a struggling market; the challenge is to stay relevant in a (thriving) market that is not only technically challenging, with continuously evolving requirements, but where new entrants and large vendors can quickly take the space of those accumulating technical debt.

And what will the next extinction wave look like? As the famous quote often attributed to Yogi Berra says, "predictions are hard, especially about the future." At this point, I'd say it should be related to vendors' ability to successfully incorporate new AI capabilities and to their ability to integrate into complex, dynamic architectures. Should we start placing our bets?