Reimagining Web APIs - Multilingual/ Rusty Web Servers

Last month at my day job, our team faced a formidable challenge. A Python service system, critical for our trading systems, was struggling under heavy loads. The culprit? A complex serialization process—vital for our validation protocols. I remember thinking that Rust could fix it. But, despite the amazing tooling by PyO3, the required setup and switch to a completely different build system (maturin) was too difficult and risky for our situation. While we ended up changing the serialization mechanism and the solution was correct for those constraints, I ended the day feeling that those constraints should not have to exist.

In the world of programming, where languages coalesce and diverge like rivers, Python has long been the Nile: wide, welcoming, and nurturing civilizations of code on its banks. But even the mightiest rivers have their limits.

I've sailed the Python River for years, relishing its beauty and versatility. It's my go-to language for everything from web development to AI, revered for its simplicity and readability. Yet, there's an unspoken truth we often ignore: when it comes to sheer performance, Python can be like a rowboat in a race against speedboats.

The Historical Solutions

Historically, when working on large applications with a Python codebase, if Python's performance became a bottleneck, we had to adopt either a microservice-based architecture or make complex subprocess calls to execute a binary. This meant that services written in faster languages like Rust, C, Go, or C++ would communicate through network calls, whether on the same machine (localhost) or different machines in the same cluster. The first approach would require us to deal with the latency of the network call and the latter approach, managing the infrastructure for a specific binary involved in the subprocess added further complexity. While this approach resolved performance issues, it introduced communication and deployment complexities and missed out on potential performance improvements due to additional network calls.

Then Comes PyO3

In the realm of Python development, the concept of creating extensions is not uncharted territory. For years, developers have deftly woven C libraries into Python's fabric, leveraging its robust capabilities. However, this magic has primarily been restricted to libraries due to various reasons. Enter Rust (with PyO3) – a language that slices through the complexity of programming with unmatched precision and speed. It's a language admired for its fast, safe, and concurrent nature. Yet, it carries the weight of a complex, boilerplate-heavy type system, necessitating extensive recompiles for even minor alterations. This presents a stark contrast to Python's fluid and forgiving nature.

While the integration of Rust and Python has been substantially improved due to PyO3, it has primarily been limited to library development, its application at a broader, application-level code scope remains a road less traveled. The reluctance stems from the daunting prospect of implementing a new build system and managing the intricate infrastructure requirements. Such hurdles often seem insurmountable, overshadowing the potential synergies that could be unlocked by combining Python's simplicity with Rust's raw power.

Robyn: The Confluence

Robyn, a framework not just built but woven from these two languages. Its promise? To let Rust and Python dance together, seamlessly. For the past 2 years, we have tried to squeeze as much performance out of a Python codebase as practically and also somewhat (impractically) possible. But the latest version of Robyn aims to offer a bridge between the two worlds of Rust and Python and break traditional boundaries.

The latest version introduces the ability to use Rust in a Python codebase without needing any setup. Not only that, it introduces the ability to run Rust without doing a manual recompile, still maintaining our beloved interpreted and dynamic nature of a Python codebase. With a simple command robyn app.py --compile-rust-path="." --dev .

• But let's dive deeper into the capabilities of Robyn. When it comes to creating a new Rust file, the process is straightforward. Just use the command robyn --create-rust-file hello_world and a new Rust file named "hello_world" will be generated.
• Running projects with Rust files is also a breeze. All you need to do is import the Rust py03 file, and you're good to go. With the command robyn app.py --compile-rust-path=".", your project will be up and running with the power of Rust seamlessly integrated.
• Robyn doesn't stop there. It also offers a convenient way to run in dev mode. Simply use the command robyn app.py --compile-rust-path="." --dev to enable dev mode and experience the benefits of Rust without the hassle of manual recompilation.

Debunking Myths

"But web frameworks are I/O bound," they say. To those, I ask, "Why settle?" Why settle for a framework that isn't striving to push the boundaries of what's possible? In a world where milliseconds can mean millions, every bit of speed matters.

With the potential of offloading the blocking adapters in Rust, the speed can be improved substantially.

Find an example of a Robyn app using a rust DB connector here

Bridging Two Worlds

Integrating Rust into our Python codebase is more than just bridging two languages; it involves merging two philosophies. It combines the ease and dynamism of Python with the raw power and efficiency of Rust. In the past, this task was daunting and filled with challenges, but it held promises of untapped potential. However, with the latest Robyn release, we hope that harnessing this potential will be less difficult.

A notable advantage of this integration, particularly salient in my mind, is the unlocked potential within the AI domain, especially in working with Large Language Models (LLMs). This capability allows for more powerful applications in the field of AI.

Extending Rust in Python is not just about raw power and efficiency, it's also about having the ability to explore a new ecosystem, which was alluring but required us to climb a monumental stairway. While this integration is still in super early access, it shows us a glimpse of the massive potential in the Python ecosystem.

Conclusion

To my fellow developers, I extend an invitation: join us on this journey. Experiment with Rust in your Python projects. Let's not just write code; let's craft it, with the best tools at our disposal. The future of web development isn't written in a single language; it's a symphony of them.