New Event Streaming Paradigm: NoSQL, NoMQ
Global security provider Palo Alto Networks processes terabytes of network security events each day. They analyze, correlate, and respond to millions of events per second to secure their customers. With that much throughput where latency is crucial, the company’s engineers spent a lot of time figuring out what the best message queuing (MQ) would be to use. The answer? None at all.
This paper explains why and how Palo Alto Networks completely eliminated the MQ layer (Apache Kafka) and decided to use their existing low latency distributed database (ScyllaDB) as an event data store and as a message queue. In addition to providing an in-depth look at the current solution, their engineers explain the pros and cons of all 4 approaches they implemented:
- Relational database
- NoSQL + Kafka message queue
- NoSQL + Cloud-managed Kafka message queue
- NoSQL, no Kafka message queue
