Sunday, July 25, 2010

A view from EuroPython 2010

I have just returned from EuroPython which was, like last year, in Birmingham, UK. This post will not go into too much technical detail, email if you need more. I'm guessing that the reader does not want to know how Python implements IEEE 754 floating point format (that's even mentioned in the Programming Foundations course, do try to keep up!). It requires about 4000 lines of C code to convert between float and text - no, you didn't want to know that.
Neither am I going to give a blow-by-blow account of each talk - they are available on the europython 2010 website.
There were just under 400 delegates this year - slightly down on last year. The organisers should not be concerned though: early indications from rival conference YAPC Europe (Yet Another Perl Conference) is that attendance is approximately half of last year.
Organisation was slightly better than 2009, a number of lessons appear to have been learnt. Talk streams were still patchy, but I guess that's inevitable.


Multiprocessing, the GIL, and threading
The conference was opened by Russel Winder who impressed last year as well. The theme was very familiar to me, it told the story that Intel, and others, have been banging on about for a couple of years now - the fact that multi-core CPUs are here to stay and currently the only way to get the increase in speed beloved of developers. Russel went much further though, with my brain screaming ME TOO! Where Intel's solutions are distinctly small scale with core numbers in single figures, Russel spoke of much larger clusters (note the term). His contention was that current hardware solutions with cache are not scalable beyond 16, and neither are software solutions using threads. Message passing systems, like MPI, are a more likely future than threading systems like TBB or OpenMP. And no one in their right mind would be programming native threads. Here, here. A thousand times, here, here. All these lessons were learnt on mainframes in the 1970s: those that ignore history are destined to repeat it. It is a great shame that Russel's talk was rushed.

That did not stop discussion in the conference about the vagaries of the Global Interpreter Lock (GIL), the much maligned excuse for not doing multithreading in Python. Hey guys: multithreading is hard, error prone, and not necessarily all that faster. Let's just accept that and move on. There will be a reworked GIL in Python 3.2, work which has come out of Google's "Unladen Swallow" project. It will be interesting to see how much that helps (?) and what excuses people will use to avoid multithreading in the future.

I was browsing one of the book stalls, looking at an Advanced Python book when it hit me how much Python has moved forward in the past couple of years. The section on Multiprocessing mentioned an out-of-date module, not Subprocess and Multiprocessing. I dismissed the book as "out-of-date" even though the first edition was 2008 - QA's own Python courses have always covered those two modules. Then I realised that we have only had our own courses for about a year, although I have been tinkering with Python for (wow!) over ten years.


Python's progress
Version 2.7 of Python was released on 4th July. OK, I'll update the QAPYTH2 course material as soon as someone gives me the time. (Actually many of the significant changes in 2.7 are back-ports from Python 3.1, so I can flitch the material from my Python 3 course. Just don't tell the boss). Python 2.7 is the last Python 2 major release. Version 3.1 is now the only Python development stream, although 2.7 will continue to be maintained for around five years. When I started writing our own Python course material in late 2008 I was interrupted by the Python 3.0 release literally as I was writing the PowerPoint slides. I decided that a new course on Python 2 was daft, and switched to Python 3. It just so happened that I thought that Python 3 was a vast improvement for the language as well (don't cry for me, Perl 6). Six months later at EuroPython 2009 I realised I might have made a mistake, so I back-ported the course material to Python 2. I ended-up with two courses and "let the market decide". One year on and we have taught many more Python 3 courses than Python 2. and we are in a great position to move forward, unencumbered by legacy Python. Maybe it wasn't such a bad idea to target Python 3, but it was scary to be ahead of the curve.

Python 3.2 should be out around the end of the year. One speaker gave January 2011 and another said "before the end of this year". Take your pick. Christmas? Python 3.3 was mentioned a few times, it should include Google's "Unladen Swallow" (yes Ian, it is from "Monty Python and the Holy Grail") with a target of a 5 times performance improvement (European or Asian?). No dates yet.

My impression is that many developers still have not realised the benefits of Python 3, and have no plans to move over. They will.

It looks like Python will overtake Visual Basic in the Tiobe stakes in the next few months (it passed Perl a couple of years ago). Are developers realising the benefits of Python? They are.


The cheese shop (the Python module repository: PyPi)
(Recent course delegate: "but the Monty Python cheese shop had no cheese in it!". Me: "neither has PyPi")

One of the main benefits of Perl (what?) is CPAN. Dave Cross (well-known Perlmonger) has said that this was the main reason for not moving to Python. Well Dave, Python just had its 10,000th module uploaded (round of applause). Actually I'm not so sure that is such a good thing, duplication and crud makes navigation difficult. Still, it's an indication of popularity and progress.


One of the main benefits of Python 2 against Python 3 is the cheese chop. Here too Python 3 is catching up. The module numbers are still only a few hundred, but there major modules are appearing all the time, for example NumPy (numerical processing) was just released for Python 3.


Wot I learnt
Quite a lot, as always. I learnt the expression for such as __path__ is "dunder path". Nice one. I also learnt about the importance of the new unittest module (that'll have to go in) and changes in the way import works for Python 3.2. I discovered how technicians lie about language comparisons to their managers to justify using a cool product (actually, I already knew that). Speakers were at times less than accurate with their comparisons against Perl.

I learnt how HTML5 is going to make Microsoft Silverlight obsolete. Well the speaker did not actually say that, but I can dream.

There were some fascinating statistics, like there are more people in India with access to a mobile 'phone than to a flush toilet (please don't take the comparison further). I would love to use that in a course if only I could find the derivation.

The portability issues of using ActiveX are well-known to anyone using Microsoft's remote desktop (there are no portability issues - it's not portable), but apparently not to the South Korean government.

The spec. for HTML1 was three pages, for HTML5 is 900. That's progress.

Oracle had a stand at the conference, and they were giving out DVD's with developer's resource on for, let's see, Linux, PHP, Ruby, Python, and Oracle VM. Hummm, spot the missing language. Me: "What about Perl then?"; Oracle chappie: "oh, that's only used by system administrators for scripting". Go figure.

And you can write cool games in Python, drive neat little robots, automate PowerPoint slides (I gotta get me some of that, and the robot stuff). And the Python Software Foundation are as cheerfully disorganised as everyone else. As with all conferences, it is always nice to have opinions confirmed, and to be able to say from time to time "actually, I knew that".

Oh, and I saw a great example for "If then else for the lazy" in our UNIX fundamentals course:
./configure && make


Finally...
Python is self-assured, confident, and looking forward. The community is proud of its product, and itself. So they should be.