Introduction to the Tower library

One of the components of my OpenTelemetry demo is a Rust application built with the Axum web framework. In its description, axum mentions:

axum doesn't have its own middleware system but instead uses tower::Service. This means axum gets timeouts, tracing, compression, authorization, and more, for free. It also enables you to share middleware with applications written using hyper or tonic.

-- axum README

So far, I was happy to let this cryptic explanation lurk in the corner of my mind, but today is the day I want to understand what it means. Like many others, this post aims to explain to me and others how to do this.

The tower crate offers the following information:

Tower is a library of modular and reusable components for building robust networking clients and servers.

Tower provides a simple core abstraction, the Service trait, which represents an asynchronous function taking a request and returning either a response or an error. This abstraction can be used to model both clients and servers.

Generic components, like timeouts, rate limiting, and load balancing, can be modeled as Services that wrap some inner service and apply additional behavior before or after the inner service is called. This allows implementing these components in a protocol-agnostic, composable way. Typically, such services are referred to as middleware.

-- tower crate

Tower is designed around Functional Programming and two main abstractions, Service and Layer.

In its simplest expression, a Service is a function that reads an input and produces an output. It consists of two methods:

One should call poll_ready() to ensure that the service can process requests
call() processes the request and returns the response asynchronously

Because calls can fail, the return value is wrapped in a Result. Moreover, since Tower deals with asynchronous calls, the Result is wrapped in a Future. Hence, a Service transforms a Self::Request into a Future<Result<Self::Response, Error>>, with Request and Response needing to be defined by the developer.

The Layer trait allows composing Services together.

Here's a slightly more detailed diagram:

A typical Service implementation will wrap an underlying component; the component may be a service itself. Hence, you can chain multiple features by composing various functions.

The call() function implementation usually executes these steps in order, all of them being optional:

Pre-call
Call the wrapped component
Post-call

For example, a logging service could log the parameters before the call, call the logged component, and log the return value after the call. Another example would be a throttling service, which limits the rate of calls of the wrapped service: it would read the current status before the call and, if above a configured limit, would return immediately without calling the wrapped component. It will call the component and increment the status if the status is valid.

The role of a layer would be to take one service and wrap it into the other.

With this in mind, it's relatively easy to check the axum-tracing-opentelemetry crate and understand what it does. It offers two services with their respective layers: one is to extract the trace and span IDs from an HTTP request, and another is to send the data to the OTEL collector.

Note that Tower comes with several out-of-the-box services, each available via a feature crate:

balance: load-balance requests
buffer: MPSC buffer
discover: service discovery
filter: conditional dispatch
hedge: retry slow requests
limit: limit requests
load: load measurement
retry: retry failed requests
timeout: timeout requests

Finally, note that Tower comes in three crates: tower is the public crate, while tower-service and tower-layer are considered less stable.

In this post, we have explained the what is the Tower library: it's a Functional Programming library that provides function composition. If you come from the Object-Oriented Programming paradigm, it's similar to the Decorator pattern. It builds upon two abstractions, Service is the function, and Layer composes functions.

It's widespread in the Rust ecosystem, and learning it is a good investment.

To go further:

Originally published at A Java Geek on August 20^th, 2023