NYSE Adopts Redpanda for Real-Time Cloud Streaming & AI Data
The New York Stock Exchange (NYSE) has adopted Redpanda to power its new Cloud Streaming service, a move designed to deliver real-time market data with unprecedented speed and precision. This strategic integration marks a significant step in the NYSE’s ongoing multi-year cloud migration, enabling customers to gain immediate insights for smarter trading decisions and more effective risk management.
Operating seven exchanges, the NYSE Group represents an immense data engine, processing over 500 billion messages and facilitating the trade of more than 2.5 billion shares daily. Managing such a colossal flow of information in real-time, especially for critical financial decisions, demands a robust and efficient data streaming solution. Redpanda’s platform meets this challenge, delivering data with latencies under a hundred milliseconds.
Sridhar Masam, Chief Technology Officer at NYSE, underscored Redpanda’s unique capabilities in handling this scale. “Redpanda is the only Apache Kafka-compatible data streaming platform that can cost-effectively and reliably manage the massive scale of the NYSE,” Masam stated. He emphasized the critical role of speed, noting, “When milliseconds matter, Redpanda delivers real-time market data so that our customers can make shrewd investment decisions and manage risk.” Masam also highlighted a shared commitment to continuous innovation, expressing anticipation for expanding the partnership, particularly given Redpanda’s support for Apache Iceberg, a foundational technology for building advanced AI applications that can make autonomous decisions.
The adoption of Redpanda is a key component of the NYSE Group’s broader cloud strategy. While historical trade and reference data were already available on AWS Cloud, the exchange sought to significantly enhance its offerings with real-time data streams. This necessitated building a solution capable of handling immense data volumes at very low latencies, all while ensuring seamless integration into customers’ existing applications.
NYSE Cloud Streaming, powered by Redpanda, serves as a new data distribution platform that grants customers immediate access to live market data. This data is delivered in a developer-friendly format, integrating easily with systems designed for Apache Kafka, a widely used data streaming platform. This cloud-native approach eliminates the need for organizations like retail broker-dealers, fintech companies, and other financial institutions to maintain their own physical infrastructure, leading to a more accessible environment and a lower overall cost of ownership.
Alex Gallego, founder and CEO of Redpanda, expressed pride in the partnership and acknowledged the immense responsibility of serving such a critical financial institution. “We are honored to provide the real-time data platform that serves as the foundation of NYSE Cloud Streaming,” Gallego said. He highlighted Redpanda’s proven track record in mission-critical deployments for Fortune 500 global organizations, asserting its ability to “always deliver – even in hyper-scale environments like the NYSE which processes hundreds of billions of messages daily.” Gallego concluded by looking forward to further developing their state-of-the-art data platform for future self-directing AI applications.