We’re excited to share the next chapter of Meroxa and what it means for software engineers to build, test and deploy data applications. Building data-driven applications in today’s world is incredibly complex. Most of the underlying infrastructure and tooling that helps make real-time and event-driven applications possible requires that developers build all sorts of plumbing before they can deliver real to their customers. Being able to follow standard DevOps practices that developers have come to expect when building web apps is almost non-existent in the current data app world.
Today, Meroxa is pleased to introduce a public beta of Turbine. Turbine is a code-first data application framework that engineers can use to build features that respond to and run code against data changes and events, in real-time. The best part about Turbine is that it fits within your current development workflows (e.g. Build, Test, & Deploy) to the point where building a data app will feel a lot like writing a web app. When coupled with the Meroxa platform, Turbine data apps are easily deployed and scaled to meet the velocity of the data changes happening upstream.
Getting Started
Building data apps start with the Meroxa CLI on your operating system of choice (Windows, Mac, & Linux). Once you’ve got the CLI installed, creating the initial data app is a simple command:
$ meroxa apps init customer_360 –lang js
You’ll get a new directory called `customer_360` on your machine with all of the scaffolding needed to start building the app in JavaScript. The app has a small set of conventions that you need to follow. The best part about this approach is that we didn’t create any bespoke DSLs or some YAML to drive the application and the infrastructure. If JavaScript isn’t your thing, you can write Turbine apps in Go, and Python!
Enrich Customer Data As it Comes In
Being able to respond to customers immediately is critical to engagement. At Meroxa, anytime someone creates an account, we take that data and enrich it using the Clearbit API before storing it back in our production database. We’ve taken a simplified version of our code to demonstrate how we do it. You’ll see a simple Turbine app that listens to a production PostgreSQL database (`demo_pg`) for changes, runs custom code via the `Process` function, and writes that data back to PostgreSQL.
Full enrich example code on GitHub
This is the core of the entire Turbine application. While this example is in Go, the same could be written in JavaScript or Python.
Bringing Developer Experience to Real-Time
Infrastructure should be there to support the developer and what they’re trying to accomplish, not the other way around. A lot of the emphasis on real-time architectures is placed on the infrastructure itself without regard to how developers have to code against these new paradigms. This is why real-time data apps have only been available to large organizations that have dedicated teams to develop against the paradigm. Hence, why we built Turbine. But don’t take our word for it:
Calvin French-Owen (Co-Founder and former CTO @ Segment): We processed 1m+ events/second at Segment, so we built a ton of tooling to make processing data both simple and correct. We never open-sourced them, so I’m glad to see Meroxa making it available to the world with Turbine. Simple, and performant.
Fredrik Björk (Co-Founder and CEO @ Grafbase): Finally a code-first approach to real-time applications that lets developers focus on shipping code over infrastructure.
Rob Malnati (COO @ thatDot): thatDot specializes in detecting complex relationships in real-time data via Quine, our open-source streaming graph processor. Turbine & Meroxa makes it almost trivial for any developer to bring these new capabilities to their applications by moving data with ease so that real-time can truly be the default.
Feedback & Learn More
We’ve only scratched the surface of what’s possible with data apps. During this beta period, we want to make sure we make Turbine and the Meroxa platform as reliable as possible before calling them generally available. Our promise is to be open and transparent about the current state of these solutions and build in concert with your feedback. For an in-depth look, checkout out the documentation. If you have any questions or comments, feel free to connect with us on Discord, or email us at support@meroxa.io.
We’re so excited to share this next step in our journey and remove all the barriers to building real-time data applications.