Your private PyPi on OpenShift

There are a lot of reasons to run your own,  private Cheese Shop and one of these is distributing code and packages you don’t want to publish because either proprietary or for internal use in your company. Does it worth it installing and configuring a server, setup some pieces of software, keep the service up and running? Yes and no: there are other simpler ways to distribute your reserved python stuff. But what if you could deploy and run shuch an application in, let’s say, 10 minutes or so? Let’s do it the OpenShift way.

How do I setup my own PyPi?

Just create an account if you don’t already have one and start an application called, let’s say, “pypi” with a “Python 2.6” web cartidge. Then clone the application repository locally:

git clone ssh://blabla@pypi-yourdomain.rhcloud.com/~/git/pypi.git/

cd into the repository, add the PyPi Example repo as a remote and pull data from it

cd pypi
git remote add upstream -m master git://github.com/masci/pypi-example.git
git pull -s recursive -X theirs upstream master

Now you’re ready to push your app to the gear:

git push

Done. Go to http://pypi-yourdomain.rhcloud.com and enjoy your private Cheese Shop. If you want to login in the admin use admin/admin credentials. The default configuration protects all the pypi related urls with user authentication, performed with HTTP Basic Auth so you can smoothly run pip or easy_install which will ask you for an username and a password.

How do I install packages from my own pypi?

pip install -i http://pypi-yourdomain.rhcloud.com/pypi/ your_secret_package

How do I publish packages to my pypi?

Write these lines on the file ~/.pypirc

[distutils]
 index-servers = mypypi
[mypypi]
 repository: http://pypi-yourdomain.rhcloud.com/pypi/
 username: admin
 password: admin

And register your package

cd yourpackagesource
python setup.py register -r mypypi sdist upload -r mypypi

Advanced use

PyPi Example is, how may I say, an example app, not exactly a production ready system (well, it depends on your needs). Nevertheless it serves media files through authentication with django-sendfiles so you can play a bit with its backend to improve performances. It handles authorizations, so you could add user and groups through the django admin and state who can do what. The application is based  on djangopypi, so you can rely on a rather active opensource project for the pypi part. And even if you don’t need a private PyPi you could always take a look at the code and see how it works an almost unattended django deployment on OpenShift.

Advertisements

Serving Django media files in OpenShift

As you may already know, setting up a Django instance on the OpenShift platform is a matter of less than 5 minutes.

Through the web management console you can create an  application using the “Python 2.6” web cartridge thus setting up a gear to host (guess!) a Python enviroment. You clone the gear repository, add the Example App code as a template, push the repo and you’re done.

There’s a folder in gear’s filesystem called data designed to contain files you don’t want to lose among application deployments. This storage is perfect for containing files that users upload through your Django application, and the settings file provided by the Example App is aware of this:

# Absolute filesystem path to the directory that will hold user-uploaded files.
# Example: "/home/media/media.lawrence.com/media/"
MEDIA_ROOT = os.environ.get('OPENSHIFT_DATA_DIR', '')

The OPENSHIFT_DATA_DIR environment variable is always available inside your gear and contains the absolute path to the data dir.

File uploads work out of the box but problems arise when you try to access files contained in the MEDIA_ROOT folder –  by default Apache serves the files contained in <app>/wsgi/static but knows nothing about the data dir located at <app>/../, so you will likely get a 404 trying to access files under the MEDIA_URL. To workaround this problem you can redirect requests for MEDIA_URL to STATIC_URL and symlink the data folder from the static folder.

To redirect requests, create a .htaccess file in the <app>/wsgi folder containing exactly these lines:

RewriteEngine On
RewriteRule ^application/media/(.+)$ /static/$1 [L]

If you manually make the symlink in <app>/wsgi/static,  it will be lost on the next deployment (remember? Only data folder won’t change) so you better let OpenShift do it for you during application deployment. Open the file at <app>/.openshift/action_hooks/build and add the following lines:

if [ ! -d $OPENSHIFT_DATA_DIR/media ]; then
mkdir $OPENSHIFT_DATA_DIR/media
fi

ln -sf $OPENSHIFT_DATA_DIR/media $OPENSHIFT_REPO_DIR/wsgi/static/media

That’s it, commit and push the modifications above and grab files from your media folder.

Deploy Django on a subdirectory (or a subpath) with mod_python

It doesn’t matter for how long you’ve been using Django – as much as you think you know it, at some point Django will surprise you transforming simple tasks into issues taking hours to be solved. I’ve never had the need to deploy a Django instance anywhere but the root of the domain name (e.g. http://example.com/myapp). Till last weekend, when I spent almost two hours to figure out how to do it without making a mess of the urls.py module (configuring Apache and mod_wsgi to serve a subpath was straightforward and it’s not covered by this article).

Actually Django does a really good job handling URL prefixes and, to be honest, I wouldn’t write these lines if I didn’t make use of the bittersweet Django Admin. AFAIK, problems arise in the following:

  • Your webserver does not set SCRIPT_NAME environment variable
  • Using the django-admin
  • Using {% url %} template tag
  • Using @login_required decorator

For the latter there’s a quick solution: set LOGIN_URL and LOGOUT_URL variables in your settings.py with the desired url path. In all the other cases, you’ve to play with the SCRIPT_NAME environment variable,  defined in CGI standard, which holds the request URL’s path which points at the application target. Well hidden (IMHO) at the bottom of this page you can find a quick and clean solution to manually set the SCRIPT_NAME variable: just define FORCE_SCRIPT_NAME in your settings.py, e.g. FORCE_SCRIPT_NAME=’/myapp’ without the trailing slash. That’s all.

Milliways, the game at the end of the Universe

Milliways is a game developed for the “5 BUTTONS Competition” (follow the link for the details) contest hosted at experimentalgameplay in collaboration with 02L. Games have to support UnitaZero as their one and only input device, and that’s the fun part!

Basically it’s a cooperative puzzle game (2-5 players) in Tetris style: a row of 5 dice fall to the bottom of the screen and each pad change the face of the corrisponding dice when triggered. When all the 5 dice shows the same face the row disappear. If the rows reach the top of the screen the game is over (you know how Tetris works, don’t you?). Game concept and development are very simple but players’ interaction could be interesting (or I hope so).

Tracks played during the game are courtesy of  The Shaidon Effect, the rest of the components is in the public domain or has a CC license.

The game was developed using Python and Pygame, the code is open source and you can find it here: https://bitbucket.org/masci/milliways/

I wrapped-up a Windows executable with PyInstaller, tested only on a Win7 Pro SP1 machine: http://dl.dropbox.com/u/6417369/milliways.zip

The game is named after “The restaurant at the end of the universe” –  I was reading that novel during the early development.

So you want me to follow you on Twitter

If you’re looking for the requirements to meet to be followed by me on Twitter, Jump to the list.

I was quite skeptical the first time I heard about Twitter – why the heck should I post messages about my status? And in a super-awful third person fashion, maybe? I mean, I’ve a blog where posting things, I’ve an RSS reader to grab news and infos all around, why getting rid of images, formatting  and whatever number of chars I want? The answer was so simple: damn speed!

Maybe I am misusing the tool, but scrolling my Twitter timeline searching for news and useful links posted by the right people is ways faster than the most advanced RSS reader you can find. And throwing interesting stuffs in a read-it-later bucket with the help of tools like (guess?) Read It Later or Instapaper while browsing the timeline with my smartphone is the definitive weapon: informations at the speed of light.

Now, if you are a famous twitterer and take the tool as an output-only pipe for your thoughts maybe you don’t even read your home timeline. But I’m a complete unknown lurker greedy for useful informations with little to no time to dedicate to the fact you just woke up (and maybe at that time I’ve been working for a couple of hours already but this is another story). You simply mess my timeline, and my timeline must be clean to carry out infos efficiently.

Now the real purpose of this post – during the past months my timeline got really dirty since I happened to use Twitter as a social tool. Wrong! I appreciate you’re following me and surely we could be friend and get out for a beer or two, but if you tweet crap there’s no way I can follow you back.

Here is the requirement list you have to meet (if not the whole, at least the most) to let me follow you, sorry for swearing:

  • Don’t flood in any case. If it’s not your fault, try revoking your auth to some of the dozens third party services you trusted. If you flood on purpose, I hate you.
  • Don’t say ‘morning and don’t whish me a good night. Never.
  • Don’t link your 4square/gowalla/whatever accounts to your Twitter stream. I don’t bother where you are when.
  • Don’t spread your floody bloody fucking Follow Friday messages, nor try to cheat changing the weekday.
  • Don’t thank for RTs – you could break the Twitter with an infinite loop.
  • Don’t copy&paste news from notorious twitterers. If I’m both following you and @hackernewsbot, there can be only one. Guess who.
  • Don’t retweet commercials. I don’t want to win any iCrappy device.
  • Try using DMs instead of chatting with your friends about your own fucking business.
  • Don’t post multiple enigmatic messages about your private life waiting for someone to ask you details. Say what you want or keep it for yourself (better).

I reserve the right to expand this list.

Installare OpenERP su OSX con virtualenv: tutorial

Indipendentemente dalle scelte che si fanno circa l’ambiente in produzione, in fase di sviluppo è molto comodo avere un’istanza locale del server openerp e del client web. Mettere in piedi l’ambiente per OSX 10.5 non è semplice come con l’installer all in one per Windows ma grazie all’ausilio di tools come virtualenv vedremo come l’operazione si possa completare senza troppe difficoltà.