Maximum results with minimum cost - that’s ideal of all business and software development processes. How to get such a results in project with tight deadline and big performance expectations? Unfortunately there is no easy way: quality of software needs time and money, but .. choice of technologies, tools, solutions can be critical for product lifecycle. We can improve total time and cost of development and maintenance by choosing appropriate solutions.

The basis of effective software development is deep research and experience built on previous projects and also mistakes. In our firm we carry out systematic research of technologies dedicated web and mobile development. We create internal projects and test to detect potential problems. Today I wont to present you short review of verified RESTful solutions for Python which really speed up development time.

Technology research and benchmarks

The picture below presents a benchmark of peak JSON response per second for many technologies also Python. There are 3 Python approaches with the following results:

Django-stripped (Django without context processors and middlewares) – 13 269 per sec
Flask – 11 506 per sec
Django – 7122 per sec

You can find full analysis on TechEmpower blog (amazing job, thank you so much!) and source code of applications on TechEmpower Github.

</br>
image source

We analyzed this benchmark and we can offer few simple practice for improve this results thanks to:

database optimalization for PostgreSQL – using pgBouncer and pgPool for connection pool problem,
rendering template with Jinja,
using ujson (ultra json) instead standard serialization library.

The picture below presents comparison performance of using PostgreSQL + Django and pgBouncer+pgPool+jinja templates created by Askthepony.com (look there for more). In sum using this simple practice we can threefold increase performance.

postgres performance chart

image source

Let’s get it started!

Assumptions of simple, example application: web platform and mobile application on iPhone/iPad. Web application written in Django has information about members of Python Group newsletter, but from mobile application level it’s possible to add new and view existing members. API is a “overloaded bottleneck”, so we have to implement efficient RESTful webservice (this example is of course a big grain of salt ;) )

Flask

Firstly, simple model of Member based on SQLAlchemy:

from sqlalchemy import Column, Integer, String
from newsletter.database import Base
class Member(Base):
   __tablename__ = 'newsletter_members'
   id = Column(Integer, primary_key=True)
   last_name = Column(String(50))
   first_name = Column(String(120))
   email = Column(String(120), unique=True)

   def __init__(self, last_name=None, first_name=None, email=None):
      self.last_name = last_name
      self.first_name = first_name
      self.email = email

   def __repr__(self):
      return '%s' % (self.last_name)

Method to handle the APIs requests:

from flask.views import MethodView

class API(MethodView):
   def get(self, member_id):
       if member_id is None:
           return Member.query.all()
       else:
           return Member.query.filter_by(id = member_id).first()

   def post(self, data ):
       member = Member(first_name = data['first_name'], email=data['email'])
       db.session.add(member)
       db.session.commit()
       return 'OK'

app.add_url_rule('/users/&lt;int:user_id&gt;', view_func=API.as_view('user_api'), methods=['GET', 'POST'])

Class MethodView is recognizing each REST methods based on generic dispatch request method.
It was simple and quick, but we can tune this. </li>

17 lines source code of full application, 4 lines for configuration, 2 lines implementing RESTful API and the same result like solution above:

import flask
import flask.ext.sqlalchemy
import flask.ext.restless

app = flask.Flask(__name__)
app.config['DEBUG'] = True
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:////tmp/test.db'
db = flask.ext.sqlalchemy.SQLAlchemy(app)

class Member(db.Model):
    __tablename__ = 'newsletter_members'
    id = db.Column(Integer, primary_key=True)
    last_name = db.Column(String(50))
    first_name = db.Column(String(120))
    email = db.Column(String(120), unique=True)

db.create_all()
manager = flask.ext.restless.APIManager(app, flask_sqlalchemy_db=db)
manager.create_api(Member, methods=['GET', 'POST'])
app.run()

Scripts above was amazing huh? But the main concern is how it works in real project with more requirements and development is dictated by real needs no pretty theory..</br> The following features are useful in real cases:

versioning:

apimanager.create_api(Member, url_prefix='/api/v2')
GET /api/v2/members/

validation:

manager.create_api(Member, methods=['GET', 'POST'], validation_exceptions=[ValidationError])

Example result:

{  validation_errors :
  { email :  Email is not valid.. }
}

specifiy data to REST methods, including/excluding fields of models:

manager.create_api(Member, include_columns = ['last_name', 'email'])

pagination:

manager.create_api(Member, methods=['GET', 'POST'], results_per_page=2)

Example result:

{  validation_errors :
  {
     age :  Must be an integer ,
  }
{
   num_results : 6,
   total_pages : 3,
   page : 1,
   objects : [
    { last_name :  Kovalsky ,  email :  kovalsky@gmail.com ,  id : 1},
{ last_name :  Novak ,  email :  novak@gmail.com ,  id : 2}
  ]
}

pre/post proccessors:

def pre_get_single(instid):
    # do something with single object
    pass

def pre_get_many(params):
    # do something with many objects
    pass

# Create an API for the Member model.
manager.create_api(Person, methods=['GET', 'POST'],
preprocessors={'GET_SINGLE': [pre_get_single], 'GET_MANY': [pre_get_many],})

authentication:

def auth_func(params):
    if not current_user.is_authenticated():
        raise ProcessingException(message='Not authenticated!')
    return NO_CHANGE
manager.create_api(Person, preprocessors={'GET_SINGLE': [auth_func]})

filtering:
example request URL:

GET /api/member?q={ filters :[{ name : email ,  op : like ,  val :  kovalsky }]}

In conclusion using micro framework Flask with RESTless library is good combination. Our company have experience with Flask based REST and it’s very convenient solutions to existing applications.

Assuming that app architecture where Flask responsible for API, Django app for web user interface, iOS and Android app for mobile interactions, we can say more about Flask advantages:

very good performance,
simple and fast prototyping and extensions of API,
separation API server from Django web application,
simple to learn for uninitiated in Flask development,
working with gunicorn (it possible to have Django and Flask on one server),
as default is SQLAlchemy ORM.

Disadvantages are:

distributed documentation of Flask (many, many plugins),
more efford is needed to logic structure of application,
lack of automatically generated admin panel which is very useful for filtering results and generating raports,
no possibilities to use helpers methods and libraries written to handle some buissness logic from Django.

What about Django REST solutions?
It’s many available libraries for REST like: django-rest-framework, django-tastypie, django-piston (not maintained already) and more. I focuse on the first solution, because it’s based on good patterns, very good maintained, with quick support, documentation on good level and reasonable approach.

On the beginning few words of explanation are needed. Name of django-rest-framework is dictated by the ability to quickly and easy find for developers. But.. it’s not correct terminology, because this library is based on wider concept than REST – Hypermedia driven APIs. I skip definition in this blog entry, but more information you can find here and also in books.
Django-rest-framework provide tool to design and develop browsable APIs based on HTML.

Django-rest-framework is very advanced tools and below you can find only the most basic and simple realization of the defined problem in 4 quick steps:

Model definition:

from django.db import models

class Member(models.Model):
    last_name = models.CharField(max_length = 100, verbose_name =  Last name )
    first_name = models.CharField(max_length = 100, verbose_name =  First name )
    email = models.EmailField(max_length = 100, verbose_name =  Email )

    def __unicode__(self):
        return self.email

    class Meta:
        verbose_name =  Newsletter member
        verbose_name_plural =  Newsletter members

Serializer – class describing rules for model and how should it be represented in API:

class MemberSerializer(serializers.ModelSerializer):
    class Meta:
        model = Member
        fields = ('last_name', 'first_name', 'email')

View on Class Based Views architecture:

from newsletter.models import Member
from newsletter.serializers import MemberSerializer
from rest_framework import generics

class MemberList(generics.ListCreateAPIView):
    model = Member
    serializer_class = MemberSerializer

class MemberDetail(generics.RetrieveUpdateDestroyAPIView):
    model = Member
    serializer_class = MemberSerializer

URL:

from django.conf.urls import patterns, url
from newsletter import views

urlpatterns = patterns('',
    url(r'^members/$', views.MeberList.as_view()),
    url(r'^members/(?P[0-9]+)/$', views.MemberDetail.as_view()),

This solutions is also very quick and simple.
Advantages of this solution are:

more advanced and extensible tool,
implementation according to needs, more ways to implement the same feature,
possibility of use Django classes and methods or own helpers from web user interface application,
fast for automatic testing (simple implementation of tests based on Django TestCase).

Disadvantges:

weak support of nested objects, in consequence problems with iOS Restkit integration,
lower performance than Flask (Django is big machinery, but on Dabapps blog you can find useful advices on profiling django-rest-framework and benchmarks)

In summary, development of API can be really fast and simple what is very beneficial for project. in retrospect and developed projects I personally prefer django-rest-framework for smooth cooperation with Django. API project has clean and logic file structure, because of framework patterns. In one place I have full Django power and all my helpers. I can test my API in easy way. But Flask is also very good choice, even better for overhead systems.

Presentation with condensed informations:

Solution4Future company blog

Python RESTful webservices with Python - Flask & Django solutions

Technology research and benchmarks

Let’s get it started!

Flask

Comments