Firefox Sync Server Setup

I have been using Firefox Sync self-hosting for a while now. I think I started in 2010, the project was still called Weave at this time. Since Firefox 29, Mozilla added a new protocol, and is still offering the possibility to selfhost it. I’ve been postponing upgrading for a while since the old protocol was still working, while still making a few unsuccessful attempts to upgrade. The sync stopped working a while ago, not sure if it’s Firefox related or maybe a problem on my server. Anyway, I finally managed to selfhost the new sync server after much head scratching, hours lost on google search, coffee, and headaches.

This small guide is an attempt to explain how, and also a reminder for me next time I want to do it 😄. It will cover selfhosting Firefox Sync with the following requirements :

  • Hosting on ONE server for all components (Firefox Sync + Firefox Accounts)
  • Debian Wheezy.
  • Apache (reverse proxy).
  • SSL. I am using CACert to sign my certificates, but selfsigned should work as well.
  • MySQL.
  • Linux and Android sync.

This guide assumes you are familiar with Linux system administration in general, that you more or less understand what you are doing and know how to do use a search engine in case anything goes wrong 😄.

I actually did all my tests on Debian Jessie (current testing) then put in production on Wheezy (current stable). It’s a bit different :

– Apache version : 2.2 vs 2.4, but in our case it’s mostly the authorizations.
– Node.js & NPM availability. It’s only packaged for Jessie.

There is a Node.js package in Wheezy-backports, but it does not include NPM. However, there is a repository with Node.js & NPM from NodeSource. Follow instructions here to install the repo : https://github.com/nodesource/distributions.

General requirements

The new sync has two parts :

– Firefox Sync server.
– Firefox Accounts server. The accounts server is divided in three parts : auth server, database server, and content server.

You can find enough documentation on the Sync server to get it running by using Mozilla official Firefox Accounts server. However, there is almost no documentation on running the Firefox Accounts server, which is why I’m writing this.

First thing first, set up a special user to run the services :

Every command after that should be run with this user. Unless root is required of course (apt-get, Apache configuration, …).

Install required packages (run as root) :

Configure WSGI properly. Maybe this is needed, maybe not, but it cannot hurt to do it by the book (as root) :

Create a conf file for apache with the following (eg: /etc/apache2/conf.d/modwsgi.conf) :

Sync server setup

Install the syncserver :

Edit synserver.ini, and specify :
– The public URL. This is the URL as seen by Firefox.
– The database in which to store server data. In the example here, it’s a MySQL database running on the local server, user ffsync, password mypass, database ffdb.
– The secret key. Use the method in the comments to generate it.
– Leave force_wsgi_environ to false. You will (I hope) not need it with the configuration I’m describing here.
– Uncomment the browserid part, and set audiences to the same public URL as above. No trailing slash this time !

Make sure the server will load the right configuration file by modifying syncserver.wsgi, putting the full path of the file in ini_file variable :

We now need to configure Apache. I’m not going to do this in details, there are plenty of available documentation for that. You need the following for one of the virtual hosts :

You will also need to edit the following file : /home/ffsync/syncserver/local/lib/python2.7/site-packages/requests/cacert.pem and add the CACert root certificate. Or your own root certificate.

If you want to run both services on the same servers like me, you will most probably have issues with SNI. By default it does not work with Python 2.7 requests module. There is a workaround though. You can find it here : https://stackoverflow.com/questions/18578439/using-requests-with-tls-doesnt-give-sni-support/. It’s quite simple.

First, install yet another package, needed to build PyOpenSSL :

Then install the needed modules in the syncserver :

Do not forget to restart Apache after any change to syncserver settings.

Firefox Accounts server setup

Note that you will have 3 URLs to remember, and I will refer to them in the following examples.

– Auth server URL : https://auth.mydomain.net
– Content server URL : https://content.mydomain.net
– Auth DB URL : http://127.0.0.1:8000

I’m using a DB on the local server, so I left it as 127.0.0.1 since it’s only accessed from the Auth server.

Auth server setup

Edit the configuration file config/dev.json, and do something like the following. All parameters and their default are explained in config/config.js. You need to put the publicUrl, or you will probably have auth failures with « Bad mac » errors in the logs.

Edit the starting script scripts/start-local.sh and comment the line starting an auth db server in memory, since we are using a MySQL auth db we do not want this.

The way to run the auth server will be the following. The NODE_ENV environnment variable is overriden by the npm start (it’s set as « dev »), that’s why what you set does not matter, and you need to edit config/dev.json.

Create an Apache configuration file like this :

Database backend setup

Two backends are available currently : memory and MySQL. This guide will use the MySQL one. A generic backend template is available there : https://github.com/mozilla/fxa-auth-db-server.

Configure the database backend. Create config/prod.json :

Then create the database and patch it. This is done when you run the backend, but checking if it works now is better :

You will be able to run the database backend like this :

Content server setup

I had problems to install it, sometimes npm failed. After a few tries, it finally succeeded.

Copy server/config/local.json-dist to server/config/local.json and edit it. Pay special attention to public_url and fxaccount_url, and the secret key.

You can run the content server with the following command :

Create an Apache configuration file like this :

Running everything

Run the database backend :

Run the auth server :

Run the content server :

You do not have to run the sync server explicitely, it will be run when requested by Apache.

Tips

I struggled over complicated stuff, but also over simple and stupid issues. Those are a few tips that cover most of the issues I got.

Firewall

If you have a firewall enabled, make sure you can reach out for http, https and git. This is needed for installation, the « npm install » command will try to get git repositories using both http and git.

Also, make sure your syncserver can connect to your Firefox accounts server. For some reason, this was not allowed on my server, and it was not easy to find out why it was not working, until I added logs in one of the python source files as someone suggested in a bug report on github and noticed there was a timeout. Then I took a look at my iptables logs… 😄.

Mail

The confirmation mail may end up in your spam folder. My spamassassin is putting it there 😄. There is a discussion about this issue but it has not been solved yet.

Apache

Make sure the following modules are enabled : wsgi, proxy, proxy_http.

It works fine with the regular MPM prefork which I tested on Jessie. However I’m using MPM ITK on my production server, and I had to compile mod_wsgi to enable daemon mode. I followed this guide : https://klette.us/apache2-mpm-itk-and-libapache2-mod-wsgi.

Node.js

Pay attention to .js vs .json extensions.

While I did not get any issue in npm install on Jessie, I got a few when trying to install fxa-content-server on Wheezy. It’s probably a bug in npm according to comments on the interwebz. Just keep trying if it fails.

Restart

I moved the virtual machine hosting the server to another host recently. Trying to start the database backend resulted in a « Not Found » error. Somehow, the patch level had been removed from the metadata table. You can find it in the schema folder in case this also happens to you.

Android setup

There is information on the interwebz about Android configuration. Unfortunately, I was not able to test it, since the Android sync client does not support SNI – it’s actually not really part of Firefox, it’s a separate app. There is a bug opened about it for a while, which has not been fixed yet.

Laisser un commentaire

Votre adresse de messagerie ne sera pas publiée. Les champs obligatoires sont indiqués avec *

one + four =