Using the Google App Engine Datastore Remote API with Django

So you have a Django application and you want to inspect the data in your production Google App Engine (GAE) datastore. What are your options? You could use the developers console - it has a query tool - but it's pretty limited, and it doesn't currently provide a way of exporting elements. You could write and deploy a script that collects and prints the data, but who wants to do a deployment every time you want to tweak the script? So what can you do?

Fortunately GAE offers a Remote API for Python for interacting with remote data elements from the comfort of your very own CLI.

For this example, I'll be using a project with the id blhaastest. If you're unsure of your project id, you can find it in your developer console:

First thing to do is verify your Google App Engine symlinks resolves correctly:

> echo $GAE

Next, start the Remote API Shell (Google login required; may require an App Password if 2-factor-authentication is enabled):

$GAE/ -s

If your login is successful, you should get a response that looks like this:

App Engine remote_api shell
Python 2.7.9 (default, Dec 11 2014, 02:36:08)
[GCC 4.2.1 Compatible Apple LLVM 5.1 (clang-503.0.40)]
The db, ndb, users, urlfetch, and memcache modules are imported.

Now that your remote shell is up and running, you can begin to execute commands against your remote datastore.

First thing to do is import your Django settings, so the shell can begin to know about your application:

> import os
> os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings")

Experienced Django developers will recognize this line above as having been plucked from

Next, import the file containing your model definition(s):

> import library.models

In the example above, the model definitions are in a file named, located in the library directory.

You can now begin using QuerySet operations. For example, here's how you can fetch all Books, sorted by title:

> books = library.models.Book.objects.all().order_by('title')

You can print the Book titles like this (Remote API Shell allows multi-line input, terminated by an empty line):

> for book in books:
>     print book.title
> [return]

Results in the following output:

Don Quixote
Invisible Man
Nineteen Eighty Four
The Illustrated Man
Things Fall Apart

Special thanks to GAE documentation for this article, which I used to help get me started. It has more tips and tricks for using the Remote API, so check it out!

> Read More

Be Careful With Default Values for Method Parameters in Python

Today I Learned: python instantiates default values for method parameters only once — at method definition time. This can cause big problems if your method modifies values in these entities, because these modifications will be propagated to subsequent calls to the same method.

I spent the greater part of day yesterday hunting down some weird behavior in a python app, and I thought I'd share. Here is a simplified version* of what I found...

def test_method(self, http=httplib2.Http()):

   # Remember the original request method
   request_orig = http.request

   # Define a closure that executes the original request method
   def new_request():
      resp, content = request_orig()

   # Replace the request method with our own closure.
   http.request = new_request

We're assigning the new_request closure to the http.request method, and never resetting it. A subsequent method call sees this closure as the original method, and everything breaks down.

The fix is to remove the default http value, and always pass in a throwaway http object.

*Note: the real (more complicated version) is in the authorize() method (line 439) here.

> Read More

Installing Python and Django using MacPorts

Use MacPorts to install python (I'm using Python version 2.7 for this demo):
$ sudo port install python27

If successful, MacPorts should give you this informational message: "To make python 2.7 the default (i.e. the version you get when you run 'python'), please run: sudo port select --set python python27" So let's do that:
$ sudo port select --set python python27

If you want to make sure that installation worked, and you have the correct version, check like this:
$ which python
$ python --version
Python 2.7.8

Using your MySQL server command line, create a MySQL database. I'm using the name 'djangodemo' for this example:
> CREATE DATABASE djangodemo

Install pip:
$ sudo port install py27-pip

If successful, MacPorts should give you this informational message: "To make the Python 2.7 version of pip the one that is run when you execute the commands without a version suffix, e.g. 'pip', run: port select --set pip pip27" Let's do that:
$ sudo port select --set pip pip27

Now we can install Django:
$ sudo pip install Django

Verify that Django is installed:
$ python -c "import django; print(django.get_version())"

Add the Python bin directory to your path. Add this line to your .bash_profile:
export PATH="$PATH:/opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin"

If you're using MySQL, install the Python/MySQL connector:
sudo port install py-mysql

Now you can start building your project. The official Django tutorial is a great place to start.

> Read More

Installing node.js on a Mac from scratch

Every other month or so I've had the need to provision a clean Mac with a node.js development environment. I've summarized my steps here for quick reference (mostly for myself, but perhaps other people will find this useful).

1. Install Xcode from the Mac App Store.

2. Activate Xcode license: sudo xcodebuild -license

3. Install command line tools: xcode-select --install. This will bring up a prompt that asks you if you want to install the command line tools. Click "Install" to continue.

4. Download and install MacPorts

5. Use MacPorts to install node.js and node package manager: sudo port install nodejs npm

6. Use npm to install npm install (or npm install -g to install it globally)


What Women in Technology Means to Me: Benjamin Haas, Software Engineer at Control Group

[This post originally appeared on my company blog here. I am re-posting it here for posterity.]

Hi, I'm Benjamin Haas, and I'm a software engineer and manager at Control Group. I am also a developer advocate, which means that in addition to helping build great products and experiences, I try to ensure that developers have a voice in all company-wide initiatives, projects, and processes. Furthermore, I am involved with our developer community outreach initiative, as we recognize that we are part of a larger group of companies and individuals that are actively working toward the technological transformation of New York City.

One aspect of developer advocacy and outreach that is important for me is being an active part of our Women in Technology program. Gender diversity in the workplace is a goal that I am happy to say I am working toward, and I am joined by a pleasingly well-rounded crew of co-workers.

> Read More

ELNYA Event at Talking Transition

My friend Jackie from Only Make Believe recently asked me to speak at the Emerging Leaders of New York Arts (ELNYA) event at the Talking Transition tent in New York. Talking Transition is an initiative to give New Yorkers the chance to take part in a conversation with the incoming DeBlasio mayoral administration. At the ELNYA event, I represented the city's tech needs, as they intersect with needs of the arts and arts administrators community. Here is the gist of what I said.

> Read More

Locally and Remotely Renaming a Branch in Git

Let's say you're busy working on your project, creating feature branches right and left to build out your super sweet app. Then your dev team decides to change their feature branch naming convention. How do you proceed without having an aneurysm? Can you easily accomodate your team's plan and not end up wanting to punch people in the face? Yep! As a matter of fact it's super simple to rename your branches. Here's how.

> Read More

Update to Installing Yii Users and Rights

It has been a year since I originally wrote the "Installing Yii Users and Rights in 5 Steps" tutorial, and both the Yii framework and the modules themselves have had some updates since then. I made some slight tweaks to the tutorial and updated the git repository to match the current builds.

You can view the updated post containing the tutorial here:


Second Yii NYC Meetup

Our first Yii NYC meetup was a big success (I posted a writeup over on the Control Group blog here), and the second one is just a couple days away. Here are the details:

Date: Tuesday, December 11
Time: 6:30pm - casual conversation, 7:30pm - meeting begins
Location: Control Group offices in the Woolworth Building, 233 Broadway, 21st Floor, NYC | map

Sign up here and come join us!



I just started Yii New York City, a meetup for NYC-area developers interested in the Yii Framework. I'm hoping we can get together monthly, or every other month, and talk about the cool stuff we're building in Yii. There will be introductions, roundtable discussions, demos, and most importantly, free food and beer! My company, Control Group, will be hosting the meetups, and you should come check us out even if just for the awesome views of lower Manhattan.

If you're in or around New York City, go sign up for the group now! Otherwise, get involved and organize a meetup of your own!

Tags: ,
Syndicate content