Pythoneering

Had a need to download and install html5lib the other day, and was most amused to find a module called ihatexml.py lurking there in the source code!…

PyCon UK 2008 has been and gone and, once again, it was an intense and stimulating experience. Our presentation - a follow-up to the one we gave last year - seemed to go well. Once that was over, it was possible to relax and enjoy the other sessions.

The keynotes, from Mark Shuttleworth on Saturday and Ted Leung on Sunday, were both interesting. I picked up a couple of very useful tips from Simon Willison’s demo of advanced Django admin customisations, and it was great to talk to other Djangonauts in Sunday morning’s Django BoF.

Other talks of note included Jonathan Hartley’s demos of pyglet - something I must check out soon as a potential alternative to Pygame - and John Pinner’s entertaining account of how he made his central heating system Python-powered. However, the highlights for me were probably the talks given by Raymond Hettinger. Raymond’s introduction to descriptors and how Python’s dot operator works was truly enlightening, as was his presentation on doing AI with Python. He presented with great enthusiasm and explained the difficult concepts clearly and concisely.

See my PyCon set on Flickr, or the PyCon UK photo pool.

In the past, I’ve used Subversion for the one-person software development projects with which I’m often involved. Just recently, however, I thought I’d give Mercurial a spin, having heard good things about it and seen a lot of people start using it.

So far, I’m impressed - particularly by the ease with which you can clone a repository to create a new branch for development, and then merge changes made there back into the original. Another nice feature is the ability to view repository changelogs with a web browser, courtesy of Mercurial’s built-in web server. And then there’s the fact that large parts of it are written in Python…

I’ll admit to being a little scared of Python generators when I first became aware of their existence. I was fairly sure that I didn’t understand them properly, and I failed to see how useful they really are. A few things have opened my eyes in recent months. First, Wesley Chun’s Core Python Programming helped to clarify a couple of issues, and then there was a session at SPA2008 which generated one or two “Aha!” moments, despite ultimately making my head spin.

Most recently, a few folk have blogged about David Beazley’s PyCon ‘08 presentation, Generator Tricks for Systems Programmers. I concur with the person who said it was one of the best introductions he’d come across. I particularly like the approach David’s taken, of avoiding trivial Fibonacci sequence examples and jumping straight into real problems of the kind faced by sysadmins. The performance stats he’s gathered are striking; I hadn’t realised just how competitive writing generator-based scripts for log file analysis can be with non-generator approaches or the more traditional tools used for this purpose, such as awk.

All this has made me more determined to make much better use of generators in my day-to-day Python programming.

I’ve been doing a lot of thinking lately about Python 3.0 and the impact it might have on our teaching of the language to our first-year students - most of whom have done little or no programming before.

I’ve written a paper about the issues; you might want to check it out if you have an interest in the teaching of programming using Python.

This is very cool. And Python-based, to boot, with a strong Django flavour to it. Needless to say, I wasn’t quick enough to get a beta testing account. I’m on the waiting list, like countless others, no doubt…

Pygments has been on my radar for a while, but I hadn’t looked closely at it until the other day, when I had a need to highlight some code fragments in a reST document that I’m writing. Hacking rst2html.py to do the necessary work was easy - just a copy-and-paste from some sample code in the docs. But then I hit a problem; my document includes code fragments written for both Python 2.5 and Python 3.0, and Pygments can handle the former but not the latter.

I delved into the source code with some trepidation, wondering how long it would take to hack a solution of some kind. I needn’t have worried. Pygments has good docs and a very clean design, so it took just a few minutes of study to figure out what was required. Less than fifteen minutes later, I had a Python3Lexer class plugged in and working!

If you want the code, it’s now been merged into Pygments’ Mercurial repository, courtesy of Pygments lead developer, Georg Brandl (Cheers, Georg!) Check it out with

  hg clone http://dev.pocoo.org/hg/pygments-main

I’ve been playing with Python 3.0 recently. It does some nice things, like fixing the asymmetry between reading from stdin and writing to stdout by making print a function rather than a statement:

print('Square-root of', x, 'is', math.sqrt(x))

There’s also a new (C#-inspired?) syntax for string formatting:

output = 'Square-root of {0:g} is {1:g}'
print(output.format(x, math.sqrt(x)))

It’s a shame that they haven’t closed the loop by providing a printf function of some kind. Still, I suppose it isn’t hard to write one:

def printf(format_string, *args, **kwargs):
    print(format_string.format(*args, **kwargs))

Got back from SPA2008 earlier this evening. The past couple of days have been pretty exhausting, with some good sessions and some that didn’t quite hit the spot. The Erlang session ended up being a little disappointing, as there was insufficient time to get to grips with many of the exercises and hence develop a feel for the language. Another session, on embedding domain-specific languages in Java, turned out to be really thought-provoking. I was familiar with the concept of method chaining, but it hadn’t really registered with me that doing this essentially created a domain-specific language from Java syntax. The final session I attended, on the principles behind MapReduce and Hadoop, also turned out to be very interesting.

I didn’t get to many of the diversions, but I did make time for the Spanish guitar recital by David Harvey on Tuesday - an excellent way of winding down after a long day.

On balance, I’d say this was an even better SPA conference for me than the one in 2005. I’ve come to the conclusion that these are tremendously valuable events, even for folk in academia. I can’t think of many better ways of picking up on current trends in industry. SPA folk are friendly, passionate about their profession, keen to share what they know with you, and equally keen to learn from the things that you have to say. The event is intense, stimulating, but good fun, too. I hope it won’t be another three years until my next visit…

A good day today.  Went to an intense but stimulating three-hour session on Python generators in the morning, which had gone beyond my prior knowledge of them within the first half-hour.  I was struggling to grasp some of the more advanced aspects of what was being demonstrated, but I certainly came out of the room with a better understanding than when I went in.

Other sessions of note included a panel discussion asking whether software practice had actually advanced over the past 15 years, and a thought-provoking presentation on the design of domain-specific languages.  Between this and the evening meal, I had the chance to run a Django BoF (basically just a demo and a discussion).  Four people turned up and we had some good discussion of Django vs. Rails.