André Kolell » Pylons Web Framework

How to use Ext JS with Pylons and Mako

Andre Kolell — Thu, 10 Sep 2009 04:25:57 +0000

In this little tutorial I will show you how to use the Python web framework Pylons and the Mako template library with the cross-browser JavaScript framework Ext JS.

I will explain how to set up a complex layout (this means an Ext.Viewport) and how to dynamically load its contents and further functionality.

First of all, you need to set up Pylons and a create a new project. Let’s name it “PylonsAndExt”. During setup, mako needs to be selected as our template engine. We do not require SQLAlchemy support…

paster create -t pylons PylonsAndExt

If you like to, verify that everything’s working well by serving the application (paster serve development.ini) and visiting http://127.0.0.1:5000.

Now we need a controller. Since this controller should provide us with our main application, we name it “application”:

paster controller application

To map the application controller’s index() action to the root URL of the project, add a new line to the main route map in config/routing.py just after “# CUSTOM ROUTES HERE” and empty the public directory (to avoid serving static files):

map.connect('/', controller='application', action='index')

Next, we modify the ApplicationController’s index() method so that it returns a mako template. Putting all our Ext JS code into mako files enables us to better internationalize and localize our JavaScript:

def index(self):
   return render('/application.mako')

To create an Ext.Viewport, we first need Ext. After downloading, place all (relevant) Ext JS files in public/ext. Inside application.mako, we can now generate our viewport:



  Getting started with Pylons and Ext JS

Since we use AJAX to load the content of the second tab (“AJAX Tab”), we need to add the ajax() method we specified above to the ApplicationController:

def ajax(self):
   return 'Some AJAX Content'

Instead of returning a simple string, we could have used a mako template to return more sophisticated content again. Ok, that’s all really basic so far. I should continue with some more challenging functionalities within the next days…

Any suggestions?

Problem setting up psycopg2 under Windows

Andre Kolell — Fri, 04 Sep 2009 06:53:24 +0000

When I set up Pylons today in a virtual isolated Python environment on my Windows machine, I had a little fight with psycopg2 (which usually allows me to connect to my PostgreSQL database). At first, everytime I tried to “easy_install psycopg2″, I ended up with the error message “unable to find vcvarsall.bat”.

Luckily I found a solution to get the “unable to find vcvarsall.bat” problem fixed.

But unfortunately there was another error when I wanted paster to serve:

File “C:\Python26\lib\site-packages\psycopg2\__init__.py”,
line 60, in
from _psycopg import BINARY, NUMBER, STRING, DATETIME, ROWID
ImportError: DLL load failed: This application has failed to start because the application configuration is incorrect. Reinstalling the application may fix this problem.

Of course, reinstallation did not fix the problem. I thought the whole challenge might be caused by my virtual environment, because Jason Erickson (who publishes the psycopg Windows binaries) mentioned problems that would occur when using psycopg with virtual environments in his blog. But nothing I tried improved my situation.

Solution: Use an older version (2.0.10) of psycopg2!

Later on I found a post in Google Groups, where someone had a similar problem with psycopg2 under Windows. He figured out that there was something wrong with the build of psycopg2 (version 2.0.12) and that simply using an older version (for example 2.0.10) worked fine… That made my day!

The Pylons Web Framework

Andre Kolell — Sun, 26 Apr 2009 08:16:40 +0000

As described in one of my previous blog posts, I’ve got the chance to introduce Pylons to my web engineering class as part of my studies at the University of Western Sydney. Thus this blog post gives an overview of the Pylons web framework.

Pylons is an open source web framework written in Python. It uses the model view controller architecture, follows the approach of convention over configuration and puts emphasis on loose coupling and clean separation. This helps developers to quickly create sophisticated web applications without hiding what is really going on.¹ In the following paper I will describe the way web applications have been developed in the past and what changes modern web frameworks have led to, especially Pylons.

Developing Web Applications the Old Way

In the past web applications were written as a series of simple CGI scripts. Although these scripts are easy to understand and give developers a lot of power, these scripts have many disadvantages. Some of them are redundant code, no separation of logic and design, a difficult to understand application structure, suboptimal data structures and database access, unreadable URLs and possibly a bad performance since interpreters and modules used by the scripts need to be loaded into memory on each request.²

Web Application Development with Pylons

Modern web frameworks have improved web application development and maintainability by addressing each of the issues described above. Taking a closer look into Pylons internal architecture helps to understand how things have changed.

Compared to other web frameworks such as Django or Ruby on Rails Pylons’ core is remarkably slim and clear, reflecting its focus on loose coupling and clean separation. Instead of using lots of “glue code”, Pylons uses low-level APIs that allow quickly and easy assembling of the desired components. Since Pylons also follows the approach of convention over configuration, a Pylons web application can be easily setup by using some of its carefully chosen default components. These include an http server, session management, template engines, URL rewriting, interactive debugging and so forth.

After a Pylons web application has been built, it is deployed on a server. This can be a Paste HTTP server that comes with Pylons, but other HTTP servers such as Apache work as well. Instead of being initialized on each request again, a Pylons web application is set up only once and held in memory by the server (as a Python object), which makes Pylons very fast.³

Internally Pylons uses the model view controller architecture, leading to a clear separation of the storage and retrieval of data (model), the business logic (controller) and the information representation (view). This separation not only improves the application’s maintainability and expandability, it also helps to increase security and facilitate the development process as a whole (e. g. programmers and designers can work on the application independently).

Every HTTP request is processed by the Pylons application object in the HTTP server’s memory (Fig. 1). Before a controller handles a request, it has to pass Pylons completely customizable middleware. At first the Cascade uses the StaticURLParser to see whether the URL requested matches a (static) file in the project’s public directory. If no file is found “404 Not Found” is returned to the Cascade and from there forwarded to the RegistryManager. Since there is only one central application object, a function is needed that separates the different simultaneous threads, each with duration form the incoming request to the outgoing response, from each other. For this purpose the RegistryManager is set up. It keeps every thread in its scope and provides variables that seem to be global but are in fact limited to their thread. After that the request goes to the StatusCodeRedirect and the ErrorHandler, neither do anything relevant just yet, because they are only needed when it comes to the response after a controller action is called. As the names of the following CacheMiddleware and SessionMiddleware already suggest, they add further information to the thread-variables initialized by the RegistryManager, which automatically offer cache and session functionality in Pylons and its controllers. The last step in Pylons original middleware chain is the RoutesMiddleware, which is an improved version of Ruby on Rails routing system.⁴ The RoutesMiddleware compares the requested URL to a route map and extracts the routing results to the variables that have been initialized by the RegistryManager. After a request passed all these middleware components, Pylons uses the now available information to delegate the request to the controller action identified by the RoutesMiddleware.⁵

Fig. 1: The Pylons Middleware Architecture and HTTP Communication Handling

As shown so far, the default setup of Pylons already solves many of the issues of the early web’s CGI script applications. Pylons has a straightforward architecture that clearly separates logic and design, comes with caching and session functionality, uses easily readable URLs and offers the possibility to customize everything as needed.

But there’s a lot more Pylons can do to facilitate web application development. On the model layer Pylons offers many different ways to store data, e. g. in files in the file system, via web services, in object and XML databases and in relational database management systems (RDBMS).⁶ When choosing to use a RDBMS such as PostgreSQL or MySQL, object-relational mappers such as the famous SQLAlchemy can be used to map the database’s data structures to objects in the Pylons application. As soon as objects in the application are changed, object-relational mappers automatically manipulate the underlying data.⁷

For authentication and authorization several third-party packages are available. AuthKit as well as repoze.who and repoze.what are some popular examples.⁸ These packages are implemented as middleware components and authentication and authorization checks can be easily added to controller actions by using decorators and predicates respectively, i. e. every method knows what (right/role) is required to perform it.

One of the most common responses of web applications is to return a HTML document. Templating systems such as Mako, Genshi or Jinja can be added to a Pylons application to support creating dynamic HTML documents. As always, controller actions pass the data that is necessary for the view to a chosen template. The template then uses this data and let it become part of a good looking design.⁹

There is still more Pylons has to offer: Besides a rapidly growing community and an increasing number of references there are internationalization tools, testing tools and logging facilities inside Pylons. Unicode support and systems for easily deployment in different environments are also included. Because Pylons is open source and has exceptionally open low-level APIs, missing functionality can be developed and added easily without changing Pylons core.

Conclusion

The architecture of modern web applications has dramatically changed since simple separate CGI scripts haven been used. Although modern web frameworks – and Pylons as one of them – may appear to be nontransparent and hard to understand at the beginning, they can, as soon as their general functionality and behavior is understood, facilitate the web application development process enormously. Caching, session management, authentication and authorization, URL mapping, database layer, object mapping and user interface generation are only some issues web frameworks can take care of.

References

Agendaless Consulting. 2009a. repoze.who – WSGI Authentication Middleware.
[WWW] http://static.repoze.org/whodocs/. (April 2009).

Agendaless Consulting. 2009b. repoze.what – Authorization for WSGI applications.
[WWW] http://static.repoze.org/whatdocs/. (April 2009).

Bel-EPA. 2009. AuthKit.
[WWW] http://bel-epa.com/docs/authkit/. (April 2009).

Gardner, J. 2009. The Definitive Guide to Pylons. Berkeley, CA: Apress.

V. Jayaram. 2007. cgi Scripting Language.
[WWW] http://www.hinduwebsite.com/webresources/cgiscripts.asp. (2009).

Footnotes

¹Gardner, 2009, p. 3-5.
²Some advantages and disadvantages can be found in V. Jayaram, 2007.
³Gardner, 2009, p. 404-405.
⁴Gardner, 2009, p. 195.
⁵See Gardner, 2009, p. 408-411 for a detailed Pylons middleware explanation.
⁶Gardner, 2009, p. 127.
⁷Of course a lot of configuration is possible to keep the performance on a good level.
⁸See Bel-EPA, 2009 for AuthKit and Agendaless Consulting 2009a and 2009b for repoze.who and repoze.what.
⁹See http://www.makotemplates.org for Mako, http://genshi.edgewall.org for Genshi and http://jinja.pocoo.org for Jinja.

This paper is also available for PDF-download.

Introduction in Pylons in Web Engineering Class

Andre Kolell — Thu, 09 Apr 2009 04:39:52 +0000

During my studies at UWS I’ve got the chance to introduce Pylons to our web engineering class. I plan to finish a short report, describing the advantages and the architecture of Pylons, until April 17th. The report will be available here after it has been marked. The topics the report will cover are described in a presentation I made with Google Docs:

SSL-Zugriff auf Pylons (auf Paste HTTP Server) via Apache-Proxy

Andre Kolell — Sat, 27 Dec 2008 18:10:40 +0000

Kürzlich stand ich vor der Herausforderung, den Datentransfer mit einer online auf einem Paste HTTP Server laufenden Pylons-Instanz mit SSL zu verschlüsseln.

Ich entschied mich für den Einsatz eines vorgeschaltenen Apache, wie ihn u.a. James Gardner im Kapitel “Authentication and Authorization” seines kürzlich erschienenen Buches The Definitive Guide to Pylons beschreibt. Da Jamens Gardners Ausführungen jedoch nicht umfassend genug sind und ich kein geeignetes Tutorial habe finden können, möchte ich hier knapp zusammenfassen, wie der Zugriff auf Pylons (Version 0.9.7) auf einem Paste HTTP Server bei einem bereits installierten SSL-Zertifikat (hierfür gibt es genügend Tutorials…) durch den Einsatz eines Apache-Proxy auf SSL beschränkt werden kann.

Zuerst gilt es, den SSL-Zugriff auf den Paste HTTP Server über den Apache-Proxy einzurichten. Hierfür ist die Konfiguration der virtuellen Hosts des Apache anzupassen. Diese könnte beispielsweise folgendermaßen aussehen:

NameVirtualHost *:80
NameVirtualHost *:443


    ServerName zu-sichernde-pylons-instanz.de
    RewriteEngine On
    RewriteRule ^/(.*) https://%{HTTP_HOST}%{REQUEST_URI} [R]



    ServerName zu-sichernde-pylons-instanz.de

    SSLEngine on
    SSLCertificateFile /etc/apache2/ssl.crt/crt.crt
    SSLCertificateKeyFile /etc/apache2/ssl.key/key.key
    SSLCACertificatePath /etc/apache2/ssl.crt
    SSLCACertificateFile /etc/apache2/ssl.crt/ca-bundle.crt

    ProxyPass / http://127.0.0.1:5000/ retry=5
    ProxyPassReverse / http://127.0.0.1:5000/
    ProxyPreserveHost On
    
        Order deny,allow
        Allow from all

Alle nicht SSL-verschlüsselten Anfragen (Port 80) auf die “zu-sichernde-pylons-instanz.de” werden durch diese Konfiguration auf SSL (Port 443) umgeleitet. Die SSL-verschlüsselten Anfragen werden wiederrum an den lokal auf dem Port 5000 laufenden Paste HTTP Server weitergeleitet. Normalerweise sollte nun bereits alles funktionieren.

Allerdings sollte noch der bis dato ggf. öffentliche Zugriff auf den Paste HTTP Server auf den Server selbst beschränkt werden, da der Apache möglichst der einzige sein sollte, der mit dem Paste HTTP Server kommunizieren soll. Hierzu ist der Host in der development.ini anzupassen:

[server:main]
host = 127.0.0.1
port = 5000

Das war’s dann auch schon. Eine weiterführende Diskussion, u.a. hinsichtlich des Durchreichens der Information an die Pylons-Instanz, ob es sich (ursprünglich) um eine verschlüsselte oder unverschlüsselte Anfrage handelt, findet sich in der Pylons Google Group.