If you're interested in running your Django projects behind Tornado, you'll want to employ a daemonized approach. The solution: Deploy Tornado as python scripts controlled by supervisord, a pythonic daemon manager.
I'm going to assume you already have your site running on Tornado behind Nginx, but need to solve the issues above.
First, let's install supervisord. If you're running Ubuntu and have python-setuptools installed, it's as easy as sudo easy_install supervisor. Once you've installed supervisor, you need to create the file /etc/supervisord.conf and fill it like so:
user www-data; [unix_http_server] file=/tmp/supervisor.sock ; path to your socket file [supervisord] logfile=/var/log/supervisord/supervisord.log ; supervisord log file logfile_maxbytes=50MB ; maximum size of logfile before rotation logfile_backups=10 ; number of backed up logfiles loglevel=info ; info, debug, warn, trace pidfile=/var/run/supervisord.pid ; pidfile location nodaemon=false ; run supervisord as a daemon minfds=1024 ; number of startup file descriptors minprocs=200 ; number of process descriptors user=nobody ; default user childlogdir=/var/log/supervisord/ ; where child log files will live [rpcinterface:supervisor] supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface [supervisorctl] serverurl=unix:///tmp/supervisor.sock ; use a unix:// URL for a unix socket ; This is where you run individual Tornado instances. ; We run four; one per processor core. ; In development, we ran as many as four per core with no issues. ; If you're looking to minimize cpu load, run fewer processes. ; BTW, Tornado processes are single threaded. ; To take advantage of multiple cores, you'll need multiple processes. [program:tornado-8000] command=/django-projects/project/tornado/t_mugs_8000.py [program:tornado-8001] command=/django-projects/project/tornado/t_mugs_8001.py [program:tornado-8002] command=/django-projects/project/tornado/t_mugs_8002.py [program:tornado-8003] command=/django-projects/project/tornado/t_mugs_8003.py
This prepares supervisord to control your Tornado programs, as well as creates a unix socket for the supervisord control program. This allows you to monitor the status of individual supervisord "programs" as well as restart/stop them. The status command is sudo supervisorctl status
This means you'll need to do some command-line voodoo to prep a log file and a pid file for supervisord. So, you'll want to do something like sudo mkdir /var/log/supervisord/ && sudo touch /var/log/supervisord/supervisord.log && chmod -R 777 /var/log/supervisord/ as well as touch /var/run/supervisord.pid && chmod 777 /var/run/supervisord.pid. Test your supervisord install like this, by running it in the foreground (non-daemon) mode, where ctrl-c will kill it: sudo supervisord -n. Error messages should be instructive.
Next, you'll need to create an init script which will control the supervisord process as if it were an init daemon. You can do touch /etc/init.d/supervisor and then paste in this code:
#!/bin/bash -e
SUPERVISORD=/usr/bin/supervisord
PIDFILE=/var/run/supervisord.pid
OPTS="-c /etc/supervisord.conf"
test -x $SUPERVISORD || exit 0
. /lib/lsb/init-functions
export PATH="${PATH:+$PATH:}/usr/local/bin:/usr/sbin:/sbin"
case "$1" in
start)
log_begin_msg "Starting Supervisor daemon manager..."
start-stop-daemon --start --quiet -p $PIDFILE -x $SUPERVISORD -- $OPTS || log_end_msg 1
log_end_msg 0
;;
stop)
log_begin_msg "Stopping Supervisor daemon manager..."
start-stop-daemon --stop --quiet --oknodo -p $PIDFILE || log_end_msg 1
log_end_msg 0
;;
restart|reload|force-reload)
log_begin_msg "Restarting Supervisor daemon manager..."
start-stop-daemon --stop --quiet --oknodo --retry 30 -p $PIDFILE
start-stop-daemon --start --quiet -p $PIDFILE -x $SUPERVISORD -- $OPTS || log_end_msg 1
log_end_msg 0
;;
*)
log_success_msg "Usage: /etc/init.d/supervisor
{start|stop|reload|force-reload|restart}"
exit 1
esac
exit 0
When it's working, you'll be able to do things like sudo /etc/init.d/supervisor {stop|start|restart} and it might even work. This will stop/start/restart ALL supervisord controlled programs from your supervisord.conf.
This is sufficient for a machine that's not using static media or virtual hosts. If you DO want to use static media and/or virtual hosts, here's the ACTUAL file we're using for one of our lesser-traveled servers. YMMV, dudes.
user www-data;
worker_processes 1;
error_log /var/log/nginx/error.log;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
use epoll;
}
http {
upstream frontends_watch {
server 127.0.0.1:8000;
}
upstream frontends_kyc {
server 127.0.0.1:8001;
}
include /etc/nginx/mime.types;
default_type application/octet-stream;
access_log /var/log/nginx/access.log;
keepalive_timeout 65;
proxy_read_timeout 200;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
gzip on;
gzip_min_length 1000;
gzip_proxied any;
gzip_types text/plain text/html text/css text/xml
application/x-javascript application/xml
application/atom+xml text/javascript;
proxy_next_upstream error;
server {
listen 80;
server_name watch.tampabay.com;
client_max_body_size 50M;
root /var/www/watch.tampabay.com;
location / {
proxy_pass_header Server;
proxy_set_header Host $http_host;
proxy_redirect false;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Scheme $scheme;
proxy_pass http://frontends_watch;
}
}
server {
listen 80;
server_name elections.tampabay.com;
client_max_body_size 50M;
root /var/www/watch.tampabay.com;
location / {
proxy_pass_header Server;
proxy_set_header Host $http_host;
proxy_redirect false;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Scheme $scheme;
proxy_pass http://frontends_kyc;
}
}
}
Of note: The two different upstream frontends_* areas and the fact that there are two server directives listening on port 80, each listening for a different server name. And yes, I know the media directory is the same between them. We do that on purpose. You can vary yours, if you like.