Unit testing GAE apps within a virtualenv

I recently ran into an issue when trying to run my usual nosetests against a newly created GAE-based app:

ImportError: No module named google.appengine.api

The SDK installs outside of Python shared modules and obviously has no idea about any virtualenv that you've set up in your project directory. The way I have my project set up is:

+ root
  + src
    +- app.yaml
    + [linked pip installed packages]
    + [module]
      +- tests.py 
  + env
  +- [virtualenv]

Adding GAE package directories to a gae.pth file in [virtualenv]/lib/site-packages did the trick for me and I was able to run unit tests without jumping through nose plugin or custom test runner hoops:

gae.pth:

/path/to/google_appengine
/path/to/google_appengine/lib/yaml/lib
/path/to/google_appengine/lib/fancy_urllib

Of course, this list will grow as you add tests that depend on more SDK modules.

Following Google's local unit testing documentation (service stubs, etc.) got everything else set up just fine for me.

Admittedly, I'm running on Windows 7 and not my OS of choice. However, the solution should translate just fine to a Linux/Mac machine as well.

OAuth 2.0 (client) couldn't get any easier

Sanction

I'm pleased to introduce sanction, a Python OAuth 2.0 client in all of its 66 LOC of glory:

https://github.com/demianbrecht/sanction

Why another OAuth 2.0 client?

I found that most implementations were overly complex, sometimes adhering to architecture principles taught in school, sometimes not. All, however, simply felt much more convoluted and difficult to understand than I thought they should be. My initial attempt at writing my own client resulted in a ~450 LOC implementation that I was quite pleased with at the time. However, after a heated discussion with Jack Diederich, watching his talk "Stop Writing Classes", spending some time waist deep in Python in a professional capacity, subsequently buying into the slogan "We're all consenting adults here" and finally realizing that I didn't want to maintain the code unless there were bugs (I wanted to release a fire-and-forget library), I decided I wasn't so happy with it anymore.

So, after spending some time (again waist deep) with the OAuth 2.0 spec, I forged ahead with a rewrite and subsequently, surprised myself when it ended up being a whopping 66 LOC. This library shouldn't need to be updated when new providers are implemented. As long as the providers adhere to the OAuth 2.0 spec, it'll work out of the box. If there are deviations (such as Facebook and StackExchange), client code will need to be aware of it and handle it as shown in the example code (nothing special, 4 LOC to handle Facebook and StackExchange deviations).

Provider Support

As of this writing, sanction has been tested with the following providers:

• Facebook (include the test API)
• Google
• Foursquare
• bitly
• GitHub
• StackExchange
• Instagram
• DeviantArt

Examples for each are included on github. Usage is detailed in the README.

The sanction library is also available and can be downloaded from PyPi.

Installing MongoDB as a service on Windows 7

Note to self:

When installing MongoDB as a service on Windows 7, don't rely on the PATH variable to access the executable. When installing the service via mongod --install, the service path is inferred, so when you attempt to net start, it won't find the executable (an absolute path is required).

In other words,

mongod --install

should be

C:\path\to\mongod --install

There goes 45 minutes of my day. Thank god it's Friday.

Compiling JPype with openjdk-7

JPype is a requirement of neo4j-embedded and is not available through PyPI. When building from source, I ran into the following compile error:

src/native/common/include/jpype.h:45:17: fatal error: jni.h: No such file or directory

First, make sure that you have the JDK installed (sudo apt-get install openjdk-7-jdk if apt-get is available on your machine) and find where jni.h lives (was /usr/lib/jvm/java-7-openjdk-amd64 for me)

Next, make the following change to setup.py in your JPype directory:

def setupLinux(self):
    self.javaHome = os.getenv("JAVA_HOME")
    if self.javaHome is None :
        #self.javaHome = '/usr/lib/jvm/java-1.5.0-sun-1.5.0.08' # was this
        self.javaHome = '/usr/lib/jvm/java-7-openjdk-amd64' # to this
    self.jdkInclude = "linux"
    self.libraries = ["dl"]
    self.libraryDir = [self.javaHome+"/lib"]

Now, your build should complete as expected (barring any other issues local to your own machine).

Multi-processor make jobs

Compile times (especially when not using a distributed build system) are always nice to keep down. Outside of code structure, I ran across this little ditty to help speed up your local build times:

make -j n

The -j parameter in make tells the build system to use n processors while running targets. To find out how many processors your Linux system has (should run on a Mac too), run cat /proc/cpuinfo | grep processor | wc -l.

HTML5 Video Scaling

According to w3c specs:

The dimension attributes are not intended to be used to stretch the image.

So, how do we scale the actual video? In a project that I'm working on (at home), I wanted to have an HTML5 video element that takes up the entire width and height of the browser window and resized whenever the user resized their browser. Here's what I came up with (using Yii as my PHP framework): Page script:

// you could bind to the equivalent event as well, of course
<video onloadeddata="resizeViewport()" id="background">
    <source src="<?php echo Yii::app()->request->baseUrl?>/assets/video/ed.ogv" />
</video>

.css:

#background
{
    position: absolute;
    -moz-transform-origin: 0 0;
}

.js (jQuery):

$(document).ready(function(){
});

$(window).resize(function(){
    resizeViewport();
});

function resizeViewport()
{
    var d = [$('#background').attr('videoWidth'), $('#background').attr('videoHeight')];

    $('#background').css('-moz-transform', 'scale('+
        $(window).width()/d[0]+','+
        $(window).height()/d[1]
        +')');
}

The scale CSS function is meant to change the dimensions of the image (as opposed to width and height). The other trick here is to ensure that the video has actually loaded so I can extract correct dimensions (which is only known once it's been loaded). Note: The code samples here are intended for FF only (each user agent has a different transform method).

Running Django unit tests

Not being a DBA, or someone who's even really familiar with database administration, I've been banging my head against the wall trying to get unit tests running on my dev box. I'd checked a few posts on SO, but it seems that I'd missed the one most important thing: RTFM.

Got an error creating the test database: permission denied to create database

If you're seeing the above error when running ./manage.py test, read on..

As Django creates and destroys a database when running unit tests, the user that you're using to connect to the server with needs to have permissions to create databases. This cannot be done by granting superuser permissions to the database user (within postgres of course) as the user needs to have database creation by default (you have to SET ROLE when using this method). To grant permissions to create a database by default, you should do this when creating the user account within psql:

create user myuser createdb

No more issues running unit tests.