EngineeringFantasy

A Todo app with flask and Pony

Thursday, 09 October 2014

Pony is a new ORM that has some very nice features especially when it comes to querying. It's still in the works, missing features like migration and py3 support (although version 0.6 supports py3 and already has a release candidate), but the creators tell me that both these features will be made available by the end of this year. So, I decided to make a simple todo app with pony, flask and flask-restful. You can get the source code from github.

Installation

First, we need to set up our project and install a couple of requirements:

pip install flask flask-restful pony

Routes

We are going to be making an API for out todo list. A todo item will have three things:

  • id, the primary key
  • data, what the item is about e.g. write a tutorial on pony
  • tags, what type of item it is, e.g. work, important, home

In order to do this, lets create our routes:

  • / will show you all the todo items on a GET request. One can add new items with a PUT request.
  • /<int:todo_id> will allow a GET request, and a DELETE request. that shows you information regarding the todo item with that id.
  • /tags/, will show you all the tags that are available, with links to those tag elements
  • /tags/<int:tag_id> will show you the corresponding tag.

Thus, a tag must have an id and a value, which will likely be a string. Lets get to making our routes then:

# Inside app.py
from flask import Flask
import flask.ext.restful as rest

app = Flask(__name__)
api = rest.Api(app)


# Resource ######################################################################
class Todos(rest.Resource):

    def get(self):
        """Will give you all the todo items"""
        return {}

    def put(self):
        """Payload contains information to create new todo item"""
        return {}


class TodoItem(rest.Resource):

    def get(self, todo_id):
        """
        Get specific information on a Todo item

        :param todo_id: The Todo Item's ID, which is unique and a primary key
        :type todo_id: int
        """
        return {}


class Tags(rest.Resource):

    def get(self):
        """Will show you all tags"""
        return {}


class TagItem(rest.Resource):

    def get(self, tag_id):
        """
        Will show you information about a specific tag

        :param tag_id: ID for the tag
        :type tag_id: int
        """
        return {}

# Routes #######################################################################
api.add_resource(Todos, '/', endpoint='Home')
api.add_resource(TodoItem, '/<int:todo_id>', endpoint='TodoItem')
api.add_resource(Tags, '/tags/', endpoint='Tags')
api.add_resource(TagItem, '/tags/<int:tag_id>', endpoint='TagItem')


if __name__ == '__main__':
    app.run(debug=True)

The get and put methods inside the rest.Resource objects indicate the types of methods that you can use to interact with that route. The api.add_resource(...) function allows you to register the API handler, i.e. a rest.Resource object and a route. the endpoint is optional.

The endpoint is an optional parameter and gives its route a reference, which is useful for redirection.

Using Pony

Right now, all the API will return are empty json objects. Before we actually start returning stuff, lets make our models:

# Inside models.py
from pony import orm  # 1

db = orm.Database()  # 2


class Todo(db.Entity):  # 3

    _table_ = 'Todos' # 4

    data = orm.Required(unicode)  # 5
    tags = orm.Set("Tag")  # 6


class Tag(db.Entity):

    _table_ = 'Tags'

    name = orm.Required(unicode, unique=True)  # 7
    tags = orm.Set("Todo") # 8

The first thing we do is import pony.orm in 1 . In most of the documentation, they import everything via from pony.orm import *, but since I like namespaces, I prefer using orm. pony.orm houses everything you need to create and query objects.

We initialize the database using orm.Database() in 2. This instance of the pony.orm.Database class stores all the data needed to create the tables that your models define.

In 3, we create an Entity object by inheriting from db.Entity. Note that we use the db instance, and not any import from pony.orm. Its similar to SQLAlchemy's sqlalchemy.schema.MetaData object.

In 4, we set the table name to Todos, its a lot like __tablename__ in SQLAlchemy.

In Pony, you can have either Required columns or Optional columns (described here). In 5, we set data to be a Required field of type unicode. Note that this unicode class is not from the pony.orm namespace.

You can also have many-to-many types in pony, using Set in 6 and 8. In 7, we set unique to be True. You have a couple of optional attributes that you can add to Required or Optional fields. (Required and Optional Explained in more detail)

Notices any id anywhere? No? Well, thats because pony implicitly creates the id column for you. (More about implicit ID creation)

Now, in app.py we need to bind a database and then generate mappings:

# Inside app.py
(...)
from models import *

app = Flask(__name__)
api = rest.Api(app)

db.bind('sqlite', 'todo_api.db', create_db=True)
db.generate_mapping(create_tables=True)


# Resource #####################################################################
(...)

We first bind the objects, in this case Tag and Todo to the sqlite database. (More on binding databases)

We set create_tables and create_db to True because this is a new database, and no tables have been created yet. (More information regarding creating tables)

generate_mapping generates all the SQL needed to create the tables. If you actually set orm.sql_debug(True), then all the SQL that is being generated will be logged into your console.

These are what the generated tables look like:

What the generated tables look like in Pony

Querying

Lets get to finding stuff from the database. First, we need to show all the todo items in our database for our Todos.get function:

# Inside app.py
class Todos(rest.Resource):
    def get(self):
        """Will give you all the todo items"""

        with orm.db_session:  # 1
            return {
                item.id: {
                    'task': item.data,
                    'tags': [tag.id for tag in item.tags]  # 3
                }
                for item in Todo.select()  # 2
            }

We are using a context manager, orm.db_session in 1. This treats everything that pony does within the code-block as a transaction, you don't have to worry about committing. We are going to return a list of all the items according to their id, we have the 'task' value in the return dictionary provide the information regarding the Todo item. As for 'tags' we loop through the values in the item's tags and give the id. Note that in 2, we are looping through all the Todo items in the database, Todo.select() is the same as orm.select(tdo for tdo in Todo).

However, the id of the tags alone are not enough information about the tags associated with a certain Todo item, it would be better if we could generate links to the tag pages. We can actually add a property inside Tag as url:

# Inside models.py

(...)
@property
def url(self):
    return "http://localhost:5000/tags/{}".format(self.id)
(...)

We use property because we will be able to access it as if it were any other attribute.

Now that we have added this property, we can call the url property instead of the id:

# Inside app.py
(...)
'tags': [tag.url for tag in item.tags]  # 3
(...)

Now that this is done, lets send a GET request to the / url:

quazinafiulislam@Nafiuls-Mac: ~
 $ http GET http://localhost:5000/
HTTP/1.0 200 OK
Content-Length: 3
Content-Type: application/json
Date: Thu, 09 Oct 2014 10:32:00 GMT
Server: Werkzeug/0.9.6 Python/2.7.6

{}

Creating Objects

So that (probably) works! We won't know until we've added a Todo item. We haven't inserted anything into our database yet so lets do that. We are going to be accepted PUT requests for our / route. In order to do this, we need to import the json module, as well as flask.request:

# Inside app.py
import json  # New import

from flask import Flask, request  # New import
import flask.ext.restful as rest
(...)

Lets start by working on our Todos.put function. This is how the data is going to come in:

{
    "data": "Buy Milk!",
    "tags": ["work", "high"]
}

In order to make this function work, we need to do a couple of things:

  • Extract data and tags from the json request.
  • Create a new Todo item.
  • Create new tags, if tags exist, then you need to use them instead of creating new ones.

In order to fulfill those roles, this is the function I ended up with:

# Inside app.py
(...)
def put(self):
    """Payload contains information to create new todo item"""

    info = json.loads(request.data)  # 1

    with orm.db_session:  # 2
        item = Todo(data=info['data'])  # 3

        for tag_name in info['tags']:  # 4
            tag = Tag.get(name=tag_name)  # 5
            if tag is None:  # 6
                tag = Tag(name=tag_name)  # 7
            item.tags += tag  # 8

    return {}, 200  # 9
(...)

In 1 we are loading the data from request.data, and turning that json string into a python dictionary. After doing so, we start our database session by using the orm.db_session context manager in 2.

We start off in 3 by creating a new Todo item. Note that our arguments are named/keyword arguments. We now loop over all the values in values in info['tags'] using tag_name as the temporary variable in 4.

In 5, we try to retrieve the tag if it exists in the database. If it doesn't 6, then we create a new one in 7. We then append this tag to the item's list of tags in 8.

In 9 we return an empty json object, with a 200 code, to say that everything happened smoothly. Note that in this entire function, it feels like there's no database. I just find this amazing that I feel as if I'm working with native python object stored in memory.

quazinafiulislam@Nafiuls-Mac: ~
 $ http PUT http://localhost:5000/ data="Buy rice" tags:='["work", "high"]'
HTTP/1.0 200 OK
Content-Length: 3
Content-Type: application/json
Date: Thu, 09 Oct 2014 11:36:04 GMT
Server: Werkzeug/0.9.6 Python/2.7.6

{}

Yay! So that worked. Or did it? Lets go back to our home page at / and lets see whats up:

quazinafiulislam@Nafiuls-Mac: ~
 $ http GET http://localhost:5000/
HTTP/1.0 200 OK
Content-Length: 166
Content-Type: application/json
Date: Sat, 11 Oct 2014 02:33:24 GMT
Server: Werkzeug/0.9.6 Python/2.7.6

{
    "1": {
        "tags": [
            "http://localhost:5000/tags/1",
            "http://localhost:5000/tags/2"
        ],
        "task": "Buy rice"
    }
}

Now that we're done with the Todo.get and Todo.put, lets do TodoItem.get:

class TodoItem(rest.Resource):
    def get(self, todo_id):
        """
        Get specific information on a Todo item

        :param todo_id: The Todo Item's ID, which is unique and a primary key
        :type todo_id: int
        """

        try:
            with orm.db_session:
                todo = Todo[todo_id]  # 1
                tags = [{tag.name: tag.url} for tag in todo.tags]  # 2

                return {
                    "task": todo.data,
                    "tags": tags
                }

        except orm.ObjectNotFound:  # 3
            return {}, 404

Since we are getting todo_id from the request itself, we can use it to get the Todo item with that id in 1. This syntax is the same as Todo.get(id=todo_id).

In 2, we are are creating a list of dictionaries from the tags in todo.tags. We are using tag.name and tag.url. If you just wanted to get the names of of the tags, you could do either of these:

tag_names = list(todo.tags.name)
tag_names = [tag.name for tag in todo.tags]

If you think I'm pulling your leg, you can check the source code and run it for yourself. In 3 if we cannot find the object, then we return a 404 error message back.

Now on to Tags.get:

class Tags(rest.Resource):
    def get(self):
        """Will show you all tags"""

        with orm.db_session:
            return {
                tag.name: tag.url
                for tag in Tag.select()
            }

Nothing new here, very similar logic to our / url. Finally in our TagItem.get we have:

class TagItem(rest.Resource):
    def get(self, tag_id):
        """
        Will show you information about a specific tag

        :param tag_id: ID for the tag
        :type tag_id: int
        """

        try:
            with orm.db_session:
                tag = Tag[tag_id]
                todos = list(tag.todos.data)  # 1

                return {
                    "tag": tag.name,
                    "tasks": todos
                }

        except orm.ObjectNotFound:
            return {}, 404

This is similar logic to that of TodoItem.get. The difference here is that in 1, we're getting all the data for the todos in a different way since we only need to know one thing. The code in 1 is the same as:

todos = [tag.data for tag in todo.tags]

Deleting Objects

All that's left is to find a way to delete objects from the database.

def delete(self, todo_id):

    try:
        with orm.db_session:
            todo = Todo[todo_id]  # 1

            if todo:
                tags = todo.tags.copy()  # 2
                todo.delete() # 3

                for tag in tags:
                    if not tag.todos:  # 4
                        tag.delete()  # 5
    except orm.ObjectNotFound:
        return {}, 400

    return {}, 200

In 1 we get the todo item like before. In 2, we copy the tags, because when we delete them in 3, we can no longer reference them through todo.tags, since the todo item is marked for deletion. If you do not do this, then you are going to get a OperationWithDeletedObjectError.

In 4 we check to see if the tag has todo items associated with it, if it doesn't then we delete the tag too in 5.

Now, a demonstration of the API that we've built:

Update

Last Updated on 15 Oct 2014 7:00 PM

I'd like to thank the pony ORM authors for their help in making this tutorial.

You can also take a look at the modularized version of the source code

P.S.

If you want to know what command line tool I used to make requests, I used httpie.