Phusion Passenger design & architecture

Before we describe Phusion Passenger, it is important to understand how typical web applications work from the viewpoint of someone who wants to connect the application to a web server.

A typical, isolated, web application accepts an HTTP request from some I/O channel, processes it internally, and outputs an HTTP response, which is sent back to the client. This is done in a loop, until the application is commanded to exit. This does not necessarily mean that the web application speaks HTTP directly: it just means that the web application accepts some kind of representation of an HTTP request.

Few web applications are accessible directly by HTTP clients. Common models are:

These descriptions are true for virtually all web applications, whether they’re based on PHP, Django, J2EE, ASP.NET, Ruby on Rails, or whatever. Note that all of these models provide the same functionality, i.e. no model can do something that a different model can’t. The critical reader will notice that all of these models are identical to the one described in the first diagram, if the combination of web servers, application servers, web applications etc. are considered to be a single entity; a black box if you will.

It should also be noted that these models do not enforce any particular I/O processing implementation. The web servers, application servers, web applications, etc. could process I/O serially (i.e. one request at a time), could multiplex I/O with a single thread (e.g. by using select(2) or poll(2)) or it could process I/O with multiple threads and/or multiple processes.

Of course, there are many variations possible. For example, load balancers could be used. But that is outside the scope of this document.

The de-facto standard interface for Ruby web applications is Rack. Rack specifies an programming interface for web application developers to implement. This interface covers HTTP request and response handling, and is not dependent on any particular application server. The idea is that any Rack-compliant application server can implement the Rack specification and work with all Rack-compliant web applications.

In the distant past, each Ruby web framework had its own interface, so application servers needed to explicitly add support for each web framework. Nowadays application servers just support Rack.

Ruby on Rails has been fully Rack compliant since version 3.0. Rails 2.3 was partially Rack-compliant while earlier versions were not Rack-compliant at all. Phusion Passenger supports Rack as well as all Rails 1.x and 2.x versions.

A particularly interesting thing to note is that a lot of the memory occupied by Ruby on Rails applications is spent on storing the program code (i.e. the abstract syntax tree (AST)) in memory. This is observed through the use of the memory statistics function in Ruby Enterprise Edition. Also, a lot of the startup time of a Ruby on Rails application is spent on bootstrapping the Rails framework.

The Apache web server has a dynamic module system and a pluggable I/O multiprocessing (the ability to handle more than 1 concurrent HTTP client at the same time) architecture. An Apache module which implements a particular multiprocessing strategy, is called a Multi-Processing Module (MPM). The single-threaded multi-process prefork MPM had been the default and the most popular one for a long time, but in recent times the hybrid multi-threaded/multi-process worker MPM is becoming increasingly popular because of its better performance and scalability. Furthermore, Apache 2.4 introduced the event MPM which is a hybrid evented/multi-threaded/multi-process MPM and offers even more scalability benefits.

The prefork MPM remains in wide use today because it’s the only MPM that works well with mod_php.

The prefork MPM spawns multiple worker child processes. HTTP requests are first accepted by a so-called control process, and then forwarded to one of the worker processes. The next section contains a diagram which shows the prefork MPM’s architecture.

Nginx is a lightweight web server that is becoming increasingly popular. It is known to be smaller, lighter weight and more scalable than Apache thanks to its evented I/O architecture. That said, Nginx is less flexible than Apache. For example it has no dynamic module system: all modules must be statically compiled into Nginx.

Phusion Passenger’s architecture is a lot like model #2 described in Web application models. In other words, Phusion Passenger extends Apache/Nginx and allows it to act like an application server. This is shown in the following diagram:

Phusion Passenger consists of:

The module is active all Apache/Nginx processes. When an HTTP request comes in, the Phusion Passenger module checks whether the request should be handled by a Phusion Passenger-served application. If so, then the module spawns one or more processes for the corresponding application (if necessary), forwards the request to that application process and forwards its generated response back to the client. This is all done with the assistance of the Phusion Passenger helper agent, which stores state that must be shared among all web server worker processes and handles much of the internal I/O between the web server and the application processes.

It should be noted that applications do not run in the same address space as the web server. This differentiates Passenger from other application-server-inside-web-server software such as mod_php, mod_perl and mod_ruby. If the application crashes or leak memory, it will have no effect on the web server. In fact, stability is one of our highest goals. Phusion Passenger is carefully designed and implemented so that the web server shouldn’t crash because of Phusion Passenger.

A very naive implementation of an application server would spawn an application process every time an HTTP request is received, just like CGI would. However, spawning Ruby applications is typically expensive. It can take a few seconds on a modern computer, and possibly much longer on a heavily loaded server. A less naive implementation would keep spawned application processes alive, similar to how Lighttpd’s FastCGI implementation works. However, this still has several problems:

Both of these problems are very much solvable, and we’ve chosen to do just that.

The first problem can be solved by preloading Rails applications, i.e. by running the Rails application before a request is ever made to that website. This is the approach taken by most Rails hosts, for example in the form of a Mongrel cluster which is running all the time. However, this is unacceptable for a shared host: such an application would just sit there and waste memory even if it’s not doing anything. Instead, we’ve chosen to take a different approach, which solves both of the aforementioned problems.

We spawn Rails applications via a spawn server. The spawn server caches Ruby on Rails framework code and application code in memory. Spawning a Rails application for the first time will still be slow, but subsequent spawn attempts will be very fast. Furthermore, because the framework code is cached independently from the application code, spawning a different Rails application will also be very fast, as long as that application is using a Rails framework version that has already been cached.

Another implication of the spawn server is that different Ruby on Rails will share memory with each other, thus solving problem #2. This is described in detail in the next section.

But despite the caching of framework code and application code, spawning is still expensive compared to an HTTP request. We want to avoid spawning whenever possible. This is why we’ve introduced the application pool. Spawned application instances are kept alive, and their handles are stored into this pool, allowing each application instance to be reused later. Thus, Passenger has very good average case performance.

The application pool is shared between different worker processes. Because the worker processes cannot share memory with each other, either shared memory must be used to implement the application pool, or a client/server architecture must be implemented. We’ve chosen the latter because it is easier to implement. The Apache control process acts like a server for the application pool. However, this does not mean that all HTTP request/response data go through the control process. A worker process queries the pool for a connection session with a Rails application. Once this session has been obtained, the worker process will communicate directly with the Rails application.

The application pool is implemented inside mod_passenger. One can find detailed documentation about it in the C++ API documentation, in particular the documentation about the ApplicationPool, StandardApplicationPool and ApplicationPoolServer classes.

The application pool is responsible for spawning applications, caching spawned applications' handles, and cleaning up applications which have been idle for an extended period of time.

The spawn server is written in Ruby, and its code can be found in the directory lib/passenger. Its main executable is bin/passenger-spawn-server. The spawn server’s RDoc documentation documents the implementation in detail.

The spawn server consists of 3 logical layers:

As you can see, we have two layers of code caching: when the spawn server receives a request to spawn a new application instance, it will forward the request to the correct framework spawner server (and will spawn that framework spawner server if it doesn’t already exist), which — in turn — will forward it to the correct application spawner server (which will, again, be created if it doesn’t already exist).

Each layer is only responsible for the layer directly below. The spawn manager only knows about framework spawner servers, and a framework spawner server only knows about its application spawner servers. The application spawner server is, however, not responsible for managing spawned application instances. If an application instance is spawned by mod_passenger, its information will be sent back to mod_passenger, which will be fully responsible for managing the application instance’s life time (through the application pool).

Also note that each layer is a seperate process. This is required because a single Ruby process can only load a single Ruby on Rails framework and a single application.

As explained earlier, a single Rails application instance can only handle a single request at the same time. This is obviously undesirable. But before we dive into the solution, let us take a look how the “competition” solves this problem. PHP has similar problems: a single PHP script can also process only one HTTP request at a time.

Passenger cannot use the mod_php way because it would force us to spawn a new Rails application for each request, which is — as explained earlier — unacceptably slow. Instead, Passenger uses the PHP-FastCGI approach. We maintain a pool of application instances, and whenever a request is received, we forward the request to one of the application instances in the pool. The size of the pool is configurable, which is useful for administrators of servers that are either heavily loaded or have little memory.

The reader might also be interested in studying the application pool’s algorithm, which is non-trivial. The algorithm is documented in detail in ApplicationPool algorithm.txt.