Setting up Ubuntu with Nginx, Unicorn, ree, rvm

14 Nov 2010

Since reading the Github post about Unicorn, I have been interested in setting up an Nginx and Unicorn server to check it out. The need for a new server setup came up at work, so I decided to give it a go. This is an attempt to catalog that process for myself and anyone else interested in setting it up for themselves.

(Spoiler alert: If you are using EC2, you can just use the public AMI I have posted at the bottom of this article.)

Goals

  1. Use Nginx and Unicorn for the web/app server setup
  2. Use REE as the base ruby install
  3. Use RVM
  4. Make something I could easily use as a base for quick projects or remote pairing sessions

Install Ubuntu

The installation of Ubuntu is outside the scope of this post. The information below pertains to an Ubuntu 10.10 server (Maverick Meerkat) installation. I have not tested this on any other version of Ubuntu or any other distribution, but suspect you could use it as a loose reference at the very least. I used an official Ubuntu 10.10 32-bit ebs image (ami-508c7839).

Note: You can find the list of official Ubuntu 10.10 AMIs here.

Install required dependencies on top of Ubuntu

C/C++ compiler support, Zlib development headers, OpenSSL development headers, GNU Readline development headers

$ sudo apt-get install build-essential zlib1g-dev libssl-dev libreadline5-dev

I saw that PostgreSQL 8.4 was the current version in the list apt-cache was returning, but had remembered hearing that a new version had recently shipped (9.0). I figured I might as well go for the new shiz...

To install PostgreSQL 9.0...do this:

$ sudo add-apt-repository ppa:pitti/postgresql
$ sudo apt-get update
$ sudo apt-get install postgresql-9.0 libpq-dev

We are going to need to have git on there too:

$ sudo apt-get install git-core

While we're at it, compiling Nginx with rewrite support requires the PCRE development package (I totally knew that before I got to that point in this write-up...totally):

$ sudo apt-get install libpcre3-dev

Install REE

Download REE

$ cd /tmp
$ wget http://bit.ly/ree-187-2010-02-tar-gz  # <-- Modify version # if necessary
$ tar xzvf ruby-enterprise-1.8.7-2010.02.tar.gz

Run the installer, passing the flag to avoid gems being installed (we want to use RVM for all gem work)

$ ./ruby-enterprise-1.8.7-2010.02/installer  --dont-install-useful-gems
(Note: I installed in /usr/local/ree-1.8.7-2010.02...)

Update your path to include the newly installed ruby executable:

$ vi /etc/login.defs

Change the following lines:

ENV_SUPATH      PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
ENV_PATH        PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games

to:

After you reload your environment, which ruby should return the ruby executable location. If it doesn't, confirm the path shown above is correct on your system and that your $PATH has been updated.

In case you are wondering why I didn't go with a 1.9.x installation: As much as I wanted to, I plan on using this server for things like remote pairing sessions and potentially build servers as well as for app deployments. We use jasmine for JavaScript testing...and one of the gem dependencies (johnson) is not compatible with 1.9.x...(yet)

Set up RVM

Update: As Brandon mentions in the comments, a 'global' installation is no longer supported. Take a look at the rvm installation page for installation. The updates to the instructions below are minimal...comment if you need help/support when comparing the two.

I went with a system-wide installation (from here):

$ bash < <( curl -L http://bit.ly/rvm-install-system-wide )

If you see the warning about a return statement in your .bashrc file like I did, you can update your ~/.bashrc file to be this instead.

Note: Obviously if you've tweaked your file, you may want to do this manually instead of using that file. I don't believe it was necessary to modify the user's `.bashrc` file because I ended up modifying the global `bash.bashrc` file. However, I did it anyway.

The output from the rvm installation mentions adding this line to your ~/.bashrc file:

 [[ -s "$HOME/.rvm/scripts/rvm" ]] && source "$HOME/.rvm/scripts/rvm"

It also mentions that you should changed it to be the following (since ~/.rvm isn't there in a system-wide installation):

[[ -s "/usr/local/lib/rvm" ]] && source "/usr/local/lib/rvm"

I added it to the end of the global bashrc file instead of ~/.bashrc (/etc/bash.bashrc, gist of file) so it's loaded for "everyone".

Then, you need to add any of your users to the rvm group, so they don't run into permissions issues:

$ usermod -aG rvm -g rvm ubuntu

Note: That command appends the 'rvm' group to the ubuntu user account, and makes that the user's default group

Install some rubies

Before we install any versions of ruby, you may want to modify your global gemset file so you automatically get some standard gems you will always want. I kept the existing (rake) and added bundler because...it's awesome. You can find that file at: /usr/local/rvm/gemsets/global.gems

I only installed ree for now:

$ rvm install ree

I also made that my default interpreter:

$ rvm --default ree

Now, when you log in, and you type rvm info you should see all the environment variables set up to point to the REE installation inside /usr/local/rvm/rubies directory. If not, you may want to stop & figure out why that's not the case...

Skip gem documentation installation

I don't like having the rdoc & ri documentation installed on gem installations either...so I dropped this into my ~/.gemrc

---
:verbose: true
:update_sources: true
:sources:
- http://gems.rubyforge.org/
:backtrace: false
:bulk_threshold: 1000
:benchmark: false
gem: --no-ri --no-rdoc

Nginx

Installation (from source)

The Nginx installation page mentions you can likely install nginx from whatever package management system you have on your system (apt, rpm, etc), but that they are usually out of date. We can't have that...so we'll install from source (0.8.52):

Note: Obviously you can likely grab a packaged version from a reliable source and install a recent build that way. I opted to build it myself.

$ cd /tmp
$ wget [link to gzipped source](http://wiki.nginx.org/Install)
$ tar xzvf nginx-X.X.XX.tar.gz
$ cd nginx-X.X.XX
$ ./configure --prefix=/usr/local/nginx-0.8.52 \
              --with-sha1=/usr/lib \
              --with-sha1-asm \
              --with-http_ssl_module \
              --with-http_gzip_static_module \
              --user=nginx \
              --group=www-data \
              --conf-path=/etc/nginx/nginx.conf \
              --http-log-path=/var/log/nginx/access_log \
              --error-log-path=/var/log/nginx/error_log \
              --pid-path=/var/run/nginx.pid \
              --http-client-body-temp-path=/var/tmp/nginx/client \
              --http-proxy-temp-path=/var/tmp/nginx/proxy \
              --http-fastcgi-temp-path=/var/tmp/nginx/fastcgi \
              --with-http_realip_module \
              --with-http_stub_status_module

Note the path prefix, the user & group names, the pidfile location (you will need to know that later). I got two deprecation warnings, but zero errors (the deprecation warnings were: sys_errlist and sys_nerr)

Assuming you didn't get any errors, now you can build & install it:

$ make
$ make install

I symlinked /usr/local/nginx to point to the install as well:

$ ln -s /usr/local/nginx-0.8.52 /usr/local/nginx

Now, let's add the group & user (and add them to the rvm group):

$ sudo grpadd www-data
$ sudo useradd --system --no-create-home --group www-data --groups rvm nginx

Basic Configuration

To simplify starting & stopping the nginx daemon, we'll add an init script. I started from this one but ended up slighly modifying it (I added a 'configtest' command option, and removed the unnecessary pidfile options for calls to start). You can grab my version here.

Set it up to auto-start on boot:

$ sudo chmod +x /etc/init.d/nginx
$ sudo /usr/sbin/update-rc.d -f nginx defaults

Create the /var/tmp/nginx directory (and update ownership):

$ mkdir /var/tmp/nginx
$ chown nginx:www-data /var/tmp/nginx

Now it should start up:

$ /etc/init.d/nginx start
Starting nginx: nginx.

Let's verify (some columns removed for formatting):

$ ps aux |grep nginx
root     23510  0.0  0.1   0:00 nginx: master process /usr/local/nginx/sbin/nginx
nginx    23511  0.0  0.1   0:00 nginx: worker process

$ netstat -l
...
tcp        0      0 *:www                   *:*                     LISTEN
...

Looking good. Now you should be able to hit your server and see "Welcome to nginx!".

Now run /etc/init.d/nginx stop and make sure those processes die and ports don't show up in netstat output anymore. If so, you are done with your nginx setup (for now).

Unicorn

If you are like me and would like a little more background information about Unicorn, I highly recommend reading the "Everything You Need to Know About Unicorn" post by Tyler Bird on Engine Yard's blog.

Installation

The unicorn installation is pretty straightforward:

$ rvm use default # just to make sure
$ gem install unicorn
Building native extensions.  This could take a while...
Successfully installed rack-1.2.1
Successfully installed unicorn-1.1.4
2 gems installed

Done.

Configuration

This setup is going to be used as a base-server, so the goal is to make it as lean as possible. We will set up the smallest rack application possible and get it configured to work with nginx, start on boot, and that's about it.

Here is my nginx.conf file:

A couple things to note:

  1. include /etc/nginx/sites-enabled/*: sets up an easy way to organize multiple domains (virtual hosts)
  2. The gzip section is automatically gzipping content for you (cargo-culted that, I haven't verified it is correct)

In /etc/nginx/sites-enabled I added a file called example_rack_app:

Things to note here:

  1. We are talking over a unix socket...note the socket file name name and the upstream name...both are used elsewhere.

Deploy an Application

One thing to note on the setup below is that the application is set up to run under the 'nginx' user (and 'www-data' group), which is the same as nginx itself. I did this for simplicity and because I captured this process before I set up a "production application" deployment. I have shifted away from this style of setup and to creating a specific user for each application I deploy. I plan to write up a separate post on that process soon. Basically, I captured all these config files in Gists as I was figuring out how to set everything up...and didn't want to go back and mess up the documentation of getting a "functional" system up & running.

Let's add our application now:

$ mkdir /web_apps && chown nginx:www-data
$ chmod -R 775 /web_apps && cd /web_apps
$ git clone git://github.com/tomkersten/example_rack_app.git

Now, to make this app's unicorn instances boot on startup...and easier to manage manually...we add an init script for it:

Update: Make /etc/init.d/unicorn executable (Thanks, @phlpmn!):

$ sudo chmod +x /etc/init.d/unicorn

...and set it up to run on reboot:

$ sudo /usr/sbin/update-rc.d -f unicorn defaults
/etc/rc0.d/K20unicorn -> ../init.d/unicorn
/etc/rc1.d/K20unicorn -> ../init.d/unicorn
...

Now you should be able to start up unicorn:

$ /etc/init.d/unicorn start
Starting Unicorn app for example_rack_app: unicorn.

Note: 'unicorn' is a general name, but that script is scoped to only start up our one example app).

Let's verify:

$ netstat -l
...
Active UNIX domain sockets (only servers)
Proto  Flags       Type       State         I-Node   Path
unix   [ ACC ]     STREAM     LISTENING     218868   /tmp/example_rack_app.socket
...                                                        ^^^^^^^^^^^^^^^^^^^^^^

$ ps aux |grep unicorn
root  ... unicorn master -c /web_apps/example_rack_app/unicorn.rb -E production -D
nginx ... unicorn worker[0] -c /web_apps/example_rack_app/unicorn.rb -E product...
nginx ... unicorn worker[1] -c /web_apps/example_rack_app/unicorn.rb -E product...
nginx ... unicorn worker[2] -c /web_apps/example_rack_app/unicorn.rb -E product...
nginx ... unicorn worker[3] -c /web_apps/example_rack_app/unicorn.rb -E product...
(Note: I trimmed some columns from the output for readability.)

Nice! Looking good. Now let's fire up nginx and make sure it's happy:

$ /etc/init.d/nginx start
Starting nginx: nginx.

$ netstat -l
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 *:www                   *:*                     LISTEN
...

Lookin' good so far. Now you should be able to hit the IP of your machine (or the domain you have pointing to it...etc) and see "Hello world!" from our simple rack app.

Monitoring

In the event that one of the pieces of our stack falls all over itself, we don't want to have to manually restart all the processes. We'll push that task off to Monit.

$ apt-get install monit

Now we just have to set it up to run and configure it to watch our processes. Here's my monitrc file (/etc/monit/monitrc).

...the nginx monitoring file:

...and the monit config file for our example rack app:

Grab the monit init.d script here and register monit to start on boot:

$ sudo /usr/sbin/update-rc.d -f monit defaults

You can't start monit at all until you edit /etc/default/monit and replace this:

startup=0

with:

startup=1

Now you should be able to start it up, and verify it's running:

$ sudo /etc/init.d/monit start
$ ps aux |grep monit
root ... 0:00 /usr/sbin/monit -c /etc/monit/monitrc -s /var/lib/monit/monit.state
(Note: I trimmed some columns again.)

Nice. Now if you kill either your unicorn or nginx process, monit should start them back up again in a minute or less (you can configure that duration).

Summary

So, now we have a pretty lean Ubuntu 10.10 server set up with Nginx, Unicorn, REE, rvm, monit, PostgreSQL, git, and configured to compress CSS/JS assets (unverified at this point).

Now you just have to write the app you want to deploy to it! ;-)

Bonus

If you are just looking for an EC2 AMI to play with, I have saved this configuration to a public AMI. Feel free to fire up an instance of it if you like (ami-263eca4f). Also, let me know if you find major flaws in the AMI, I will update it and post a new AMI (or better yet...fix it and share AMI ;-)).

Note: For now, the AMI is only listed in the US-East availability zone. If that is a problem for you, let me know and I will register some in other zones.

Oh...and use this at your own risk. It probably has some huge security holes in it that will allow any script kiddie to hack your codes and steal your databases.

Feedback

Drop me a message at @tomkersten if you see any issues with what I've outlined. As I said, I'll do my best to incorporate suggested changes and keeping an up-to-date AMI available.


Resources

I begged, borrowed and stole pieces of this process from a few places on the Internets. Here is a list of the links I referred to: