Your private PyPi on OpenShift

There are a lot of reasons to run your own,  private Cheese Shop and one of these is distributing code and packages you don’t want to publish because either proprietary or for internal use in your company. Does it worth it installing and configuring a server, setup some pieces of software, keep the service up and running? Yes and no: there are other simpler ways to distribute your reserved python stuff. But what if you could deploy and run shuch an application in, let’s say, 10 minutes or so? Let’s do it the OpenShift way.

How do I setup my own PyPi?

Just create an account if you don’t already have one and start an application called, let’s say, “pypi” with a “Python 2.6” web cartidge. Then clone the application repository locally:

git clone ssh://blabla@pypi-yourdomain.rhcloud.com/~/git/pypi.git/

cd into the repository, add the PyPi Example repo as a remote and pull data from it

cd pypi
git remote add upstream -m master git://github.com/masci/pypi-example.git
git pull -s recursive -X theirs upstream master

Now you’re ready to push your app to the gear:

git push

Done. Go to http://pypi-yourdomain.rhcloud.com and enjoy your private Cheese Shop. If you want to login in the admin use admin/admin credentials. The default configuration protects all the pypi related urls with user authentication, performed with HTTP Basic Auth so you can smoothly run pip or easy_install which will ask you for an username and a password.

How do I install packages from my own pypi?

pip install -i http://pypi-yourdomain.rhcloud.com/pypi/ your_secret_package

How do I publish packages to my pypi?

Write these lines on the file ~/.pypirc

[distutils]
 index-servers = mypypi
[mypypi]
 repository: http://pypi-yourdomain.rhcloud.com/pypi/
 username: admin
 password: admin

And register your package

cd yourpackagesource
python setup.py register -r mypypi sdist upload -r mypypi

Advanced use

PyPi Example is, how may I say, an example app, not exactly a production ready system (well, it depends on your needs). Nevertheless it serves media files through authentication with django-sendfiles so you can play a bit with its backend to improve performances. It handles authorizations, so you could add user and groups through the django admin and state who can do what. The application is based  on djangopypi, so you can rely on a rather active opensource project for the pypi part. And even if you don’t need a private PyPi you could always take a look at the code and see how it works an almost unattended django deployment on OpenShift.

Serving Django media files in OpenShift

As you may already know, setting up a Django instance on the OpenShift platform is a matter of less than 5 minutes.

Through the web management console you can create an  application using the “Python 2.6″ web cartridge thus setting up a gear to host (guess!) a Python enviroment. You clone the gear repository, add the Example App code as a template, push the repo and you’re done.

There’s a folder in gear’s filesystem called data designed to contain files you don’t want to lose among application deployments. This storage is perfect for containing files that users upload through your Django application, and the settings file provided by the Example App is aware of this:

# Absolute filesystem path to the directory that will hold user-uploaded files.
# Example: "/home/media/media.lawrence.com/media/"
MEDIA_ROOT = os.environ.get('OPENSHIFT_DATA_DIR', '')

The OPENSHIFT_DATA_DIR environment variable is always available inside your gear and contains the absolute path to the data dir.

File uploads work out of the box but problems arise when you try to access files contained in the MEDIA_ROOT folder -  by default Apache serves the files contained in <app>/wsgi/static but knows nothing about the data dir located at <app>/../, so you will likely get a 404 trying to access files under the MEDIA_URL. To workaround this problem you can redirect requests for MEDIA_URL to STATIC_URL and symlink the data folder from the static folder.

To redirect requests, create a .htaccess file in the <app>/wsgi folder containing exactly these lines:

RewriteEngine On
RewriteRule ^application/media/(.+)$ /static/$1 [L]

If you manually make the symlink in <app>/wsgi/static,  it will be lost on the next deployment (remember? Only data folder won’t change) so you better let OpenShift do it for you during application deployment. Open the file at <app>/.openshift/action_hooks/build and add the following lines:

if [ ! -d $OPENSHIFT_DATA_DIR/media ]; then
mkdir $OPENSHIFT_DATA_DIR/media
fi

ln -sf $OPENSHIFT_DATA_DIR/media $OPENSHIFT_REPO_DIR/wsgi/static/media

That’s it, commit and push the modifications above and grab files from your media folder.

Amico buleano

# condition è un qualche valore booleano
if str(condition) == "True":
    # fai qualcosa

Un pony è rimasto gravemente ferito è morto durante la scrittura del codice…