Planet TriZPUGPlanet TriZPUGLike just about everyone else, we’ve written our own suite of tools to help with building complex content management systems in Django here at Caktus. We reviewed a number of the existing CMSes out there, but in almost every case the navigation and page structure were so tightly coupled the system broke down when it came time to add additional, non-CMS pages.
We wrote a few little apps, django-pagelets, django-treenav, and django-crumbs, each of which manages different pieces of content (little snippets of content, full CMS pages, navigation, and breadcrumbs). All of the apps are available for free under an open source license on Google Code.
Decoupling was a great move for us, and the ability to plug and play any single part of the system is a huge benefit. Sometimes, however, the completely decoupled architecture was a bit of a pain: If we didn’t provide a link from the pagelets app to the treenav app, how would it be possible to edit a page’s corresponding navigation item on its change form in the Django admin interface?
Enter Generic Relations. Using Django’s content types framework, it’s possible to create admin inlines for generic relations with just a few simple lines of code.
In this case, I’ll show how we allowed users to edit a page’s corresponding navigation item in django-pagelets without requiring everyone (i.e., those who don’t need it) to install django-treenav. First, define the generic inline in the admin.py file of the app that contains the model you want to link to:
from django.contrib.contenttypes import generic class GenericMenuItemInline(generic.GenericStackedInline): """ Add this inline to your admin class to support editing related menu items from that model's admin page. """ max_num = 1 model = treenav.MenuItem
Then, inside the Admin class for the related model in question, dynamically import and add GenericMenuItemInline to the admin’s list of inlines based on whether or not it’s in the project’s INSTALLED_APPS:
from django.conf import settings class PageAdmin(admin.ModelAdmin): # ... inlines = [MyOtherInline] if 'treenav' in settings.INSTALLED_APPS: from treenav.admin import GenericMenuItemInline inlines.insert(0, GenericMenuItemInline)
For more information, see the corresponding pagelets admin.py and treenav admin.py. Thanks for reading and don’t hesitate to post comments if you have any questions!
It just came to my attention that psycopg2, the python driver for PostgreSQL database has a website again! For several months (years?) the site was unavailable, except for an plaintext rant about the author’s ire toward Trac. Now it has several pages of nice-looking sphinx documentation. I haven’t delved into it yet, so I don’t know how good it is, but it’s nice to see a professional looking website providing the public presence for the driver that lets me get at my data. It may be technically irrelevant, but it gives me a little more confidence that the software isn’t as hackish as the site used to be.
If you haven’t seen it yet, I recommend hopping on over to their site, http://initd.org/. It’s well worth a look.
Welcome back psycopg2!
When I create a new browser view following Professional Plone Development, or using ZopeSkel’s ‘paster addcontent view’ local command, I get (basically) the following code:
Products/MyProduct/browser/myview.py:
from zope.interface import implements, Interface
from Products.Five import BrowserView
from Products.CMFCore.utils import getToolByName
from Products.MyProduct import managementMessageFactory as _
class IMyView(Interface):
"""
IterationUserstoryAssociation view interface
"""
def test():
""" test method"""
class MyView(BrowserView):
"""
IterationUserstoryAssociation browser view
"""
implements(IMyView)
def __init__(self, context, request):
self.context = context
self.request = request
@property
def portal_catalog(self):
return getToolByName(self.context, 'portal_catalog')
def test():
""" test method """
return "OK"
Products/MyProduct/browser/configure.zcml
<browser:page
for="*"
name="myview"
class=".myview.MyView"
allowed_interface=".myview.IMyView"
template="myview_view.pt"
permission="zope.Public"
/>
Products/MyProduct/browser/myview_view.pt
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"
lang="en"
metal:use-macro="here/main_template/macros/master"
i18n:domain="Products.MyProduct">
<body>
<div metal:fill-slot="main">Content Here
</div>
</body>
</html>
This creates a view on any piece of content in the site, accessible from @@myview. It looks like this (applied to the default Events collection that comes with Plone):
So this is great, and for most views, it’s probably all you need. But sometimes, I’d say most times, you want more from this view. Often we want to segregate our view from Plone, maybe even to the point of using a different slot in the template to toss out most of what main_template does.
But honestly, I think what we usually want is what Plone already has. Every “regular” view of a content object, and its folder_contents view, has certain user interface elements that are usually really handy to have available to your user when they are using your view.
Specifically, I’m talking about the content actions, or the “green bar dropdowns”. Here’s a close up so you know what I’m talking about:
In our view, we don’t get these. That really sucks.
After much toil, I’ve found out why, and two different ways to fix it.
First, the why. The “content actions” are actually a viewlet within a dedicated viewlet manager. We can see this through the @@manage-viewlets view:
Along for the ride are the “content views”, which are defined via <action> tags in our Generic Setup profile (the profiles/default/types/*.xml files, specifically). This isn’t confusing at all, is it (and the fact that the viewlet manager and content views viewlet are both named “plone.contentviews” is just icing on the cake!)?
Recall that a viewlet is registered such that it can/will only show up on a content object that implements a specific interface. Another potential limiting facotr, it turns out (my Zope 3 book was in another building today), is the specific view that the viewlet is rendered on. This has lots of potential uses, and speaks to the sweetness of the Component Architecture, but was very hard to figure out.
So the culprit in the case of our content-action-less view, is plone.app.layout. In plone.app.layout.viewlets’ configure.zcml file, lies the following code:
<browser:viewlet
name="plone.contentactions"
for="*"
view="plone.app.layout.globals.interfaces.IViewView"
manager=".interfaces.IContentViews"
class=".common.ContentActionsViewlet"
permission="zope2.View"
/>
<browser:viewlet
name="plone.contentactions"
for="*"
view="plone.app.content.browser.interfaces.IFolderContentsView"
manager=".interfaces.IContentViews"
class=".common.ContentActionsViewlet"
permission="zope2.View"
/>
What this is saying, is that the plone.contentactions viewlet will only show up for content objects that implement plone.app.layout.globals.interfaces.IViewView or plone.app.content.browser.interfaces.IFolderContentsView. This is precisely why we don’t get content actions in our view. Our views aren’t implementing either of those interfaces.
The comment a few lines above these declarations very coyly explains things:
<!-- Content actions (menus)
The default version is a blank bar; the one with real menus is
registered for the main view + folder contents only.
-->
So now we get to the two possible solutions. I don’t think one is better than the other, it just depends on how explicit you want to be, and how much you trust plone.app.layout :)
The first method assigns the viewlet, just as if it was a custom viewlet, to your browser view’s interface. In our case, it would look like this:
<browser:viewlet
name="plone.contentactions"
for="*"
view=".myview.IMyView"
manager="plone.app.layout.viewlets.interfaces.IContentViews"
class="plone.app.layout.viewlets.common.ContentActionsViewlet"
permission="zope2.View"
/>
The advantage here is that the viewlet will render no matter what happens with plone.app.layout. It also gives you more flexibility; you can limit the application of the viewlet in lots of different ways. You could combine the view attribute with the for attribute to limit it to only certain views that are registered to certain content types. You could create a special interface (instead of using the standard issue interface that represents your view class) that could be used as a marker interface to turn the viewlet on or off depending on an arbitrary state of your views. It could get really interesting.
The easiest, and most straight-forward way to do this is to just add one of the interfaces, already registered to the viewlet, to your view class.
plone.app.layout specifies plone.app.layout.globals.interfaces.IViewView and plone.app.content.browser.interfaces.IFolderContentsView. After looking at the interfaces, it becomes obvious that plone.app.layout.globals.interfaces.IViewView is probably what we want to use. Here’s what the code looks like:
from zope.interface import implements, Interface
from Products.Five import BrowserView
from Products.CMFCore.utils import getToolByName
from plone.app.layout.globals.interfaces import IViewView
class IMyView(Interface):
"""
IterationUserstoryAssociation view interface
"""
def test():
""" test method"""
class MyView(BrowserView):
"""
IterationUserstoryAssociation browser view
"""
implements(IMyView, IViewView)
def __init__(self, context, request):
self.context = context
self.request = request
@property
def portal_catalog(self):
return getToolByName(self.context, 'portal_catalog')
def test():
""" test method """
return "OK"
Here you can see it in action:
Instead of implementing the interface yourself, you may be able to use ZCML to mark your view class with a registered interface… more on that later.
I mistakenly posted the wrong screen shot in the “The Final Result” section. Fixed 09:00 03/11/2010
UPDATE I thought this was broken, but I think it was just my convoluted setup. If you try this, please leave a comment and let me know how it went for you!
A quick note, if you want to add zopeskel to your development buildout.
All there is to it is to add the following part to your buildout.cfg:
[zopeskel]
recipe = zc.recipe.egg:scripts
eggs = PasteDeploy
PasteScript
ZopeSkel
scripts = zopeskel
paster
Then just enable it by adding zopeskel to the parts parameter of your main [buildout] section.
[buildout]
parts = zopeskel
When you re-run bin/buildout, buildout will grab the necessary packages, and create a paster and zopeskel script in your bin directory.
Note: I wanted the paster command, since I’m used to ZopeSkel’s “standard” user interface. To get this, I had to add the PasteDeploy egg and paster script to the configuration. Feel free to omit it if you just want to use the new “kinder, gentler” interface. I haven’t tried it yet, but I believe the explicit inclusion of the PasteScript egg is unnecessary.
Just wanted to update everyone who showed interest in the new release of GNOME Developer Kit I announced yesterday. Based on some preliminary statistics I collected in the (less than) last 24 hours, it seems that the VMware image type got the most download, followed closely by the installable ISO format. I guess that was due to VirtualBox being able to use *.vmdk files and some people opting for the free virtualization tool.
Here are the preliminary results so far:
About GNOME 2.29.92
Due to the number of downloads and and comments I received, I felt that I should provide with some background on how to install/remove packages and update your system using the conary package management system. So here you go:
The package management system behind the GNOME Developer Kit is called conary and is considered by many as the next generation package management system when compared to some of the popular options out there. One of the reasons behind this claim is the fact that your entire system is actually completely maintained in a versioned state, and conary is always “aware” of what is installed on your system and what files and dependencies make up the entire “set”. This allows for some pretty nifty operations such as rolling back to a specific state of your system.
In order to check for new updates for your system, open a terminal and run the command sudo conary updateall. conary will then check for updates and prompt you to accept the update or not. Please keep in mind that the first time you run conary for the first time, you will experience a delay as your entire system gets analyzed in preparation for the changes that are to take place. All subsequent actions performed will be much faster, I promise. If after a while you don’t feel like waiting for the prompt, add –no-interactive to the update command to have your system updated automatically.
Now, let’s just say that you decided to install something new, such as Banshee. Easy, just run sudo conary update banshee (remember to add –no-interactive for no-hands updates) and voilá!
Want to know what was actually installed on your system? conary q banshee will tell you what version of banshee was installed. How about what files were installed? conary q –ls banshee will give you a list of all the files that were installed and conary q –lsl banshee will give you the long list with file permissions and modes.
Changed your mind and want to remove banshee from your system? sudo conary erase banshee will take care of that. Want to actually roll your system back to the state it was before you installed banshee instead? sudo conary rollback 1 will rollback your system exactly one transaction. Want to go further back? Just increase that number to represent how many transactions to roll back. Want to rollback but don’t remember what point in time you want to go? sudo conary rblist will display a list of all transactions and what was changed. Note that each transaction is preceded by the letter “r“, so if you want to rollback to the point r.15, then use sudo conary rollback r.15 (and don’t forget that “r” or you’ll rollback exactly 15 transactions instead).
How about searching for a package? If it is something that it is already installed on your system, then conary q [package name] will give you the information you want. If the package is not installed on your system yet, then conary rq [package name] is what you need, though since conary does not yet make use of metadata, you’ll need to know the exact name of what you’re looking for. Now, let’s say you want to find out what package provides the command /sbin/service? Use conary q –path /sbin/service to find out that initscripts:runtime=8.81.2-0.11-1 is responsible for providing it (use rq if you want to search the remote repository).
Well, I think this is enough to get you going. You’ll probably want to install Flash and media codecs to enjoy browsing some sites and listening to your media, so let’s apply what we’ve learned so far and run: sudo conary update flashplayer group-codecs
If you’ve stayed with me until now, you may want to read up on what else conary can do or even consider packaging for GNOME Developer Kit. Your help will be greatly appreciated!
Nine More Minutes will be attending the Triangle MLS Tech Fair as an exhibitor this Thursday, March 11, in Cary, NC. The tech fair runs from 9:00am to 4:00pm at 111 Realtors Way, Cary, NC.
We’d love it if you could stop by our booth (either #23 or #22 depending on the diagram you’re looking at) and say “hi”. We have plenty of candy and pens to give away, and would be happy to discuss how our software development and IT consulting services could benefit your small business!
UPDATE: Thanks Alberto Ruiz for pointing out that VirtualBox can use .vmdk files, so the VMware image can be used for that purpose.
Thanks to the incredible work of Zhang “Jesse” Sen and Vladimir Melo, a brand new release of the GNOME Developer Kit has been published! “What’s new”, you may ask? Everything, since all packages are built directly from git.gnome.org! 
But that alone is not what makes this release so cool, but the fact that the final image went through a dramatic “diet”, shedding a lot of its “weight” and going from a 1.4GB monster to less than 700MB of pure GNOME goodness!!!
 |
| From Screenshots |
Firefox was replaced by Epiphany and codecs and fancy-Nancy stuff was scrapped to make room for a lightweight release for developers and translators!
So go ahead and try the new images today:
I wrote my first Nose plugin this weekend and I’ve got to say it was dead simple.
I was looking around for ways to keep Nose from searching for tests in certain directories. You see Nose is a nosey little critter that will scour every nook and cranny of your directory structure looking for unittests to run. Really, it wants your code to work and it loves making dots. But, see I knew better and knew if that little Nose test discoverer ventured down a few rabbit holes it would end up in a world of pain–segfaults or worse. I just wanted to avoid the whole mess and just have Nose exclude a few directories from its massive testhunt.
I asked a few people I know who use Nose regularly about my options. “Write your own plugin,” was the consensus. Nose’s architecture makes it very easy to write plugins for just about every aspect of its behavior…check out its plugin api. The project documentation is also very helpful in this regard.
I spent most of my time trying to figure out how to test my testing plugin. In the end it only took a couple of hours to go from Nose novice to having written a packaged, testable Nose plugin. So, here’s a quick example of how you would use this new nose-exclude plugin:
$ ls test_dir dir_with_tests dir_with_bad_tests there_be_dragons_here more_tests
In this example, I want Nose to ignore a couple directories and not even bother searching them. You would run:
$ nosetests --exclude-dir=test_dir/dir_with_bad_tests \ --exclude-dir=test_dir/there_be_dragons_here test_dir
You could further specify to exclude subdirectories if you wanted that level of control. There is also an –exclude-dir-file= option available that allows you to specify a file containing paths to be excluded.
$ nosetests --exclude-dir-file=exclude_dirs.txt test_dir
nose-exclude is pretty simple and covers a fairly basic use case so hopefully others will find it useful. For now nose-exclude is available on bitbucket, but will be up on PyPI shortly.
The latest URI Template draft is now available: draft-gregorio-uritemplate-04. Warning that the syntax has changed substantially from -03 if you haven't been following along. As always, please direct feedback to the W3C URI mailing list.
We’re always looking for new tools to make our development environment more robust here at Caktus. We write a lot of tests to ensure proper functionality as new features land and bug fixes are added to our projects. The next step is to integrate with a continuous integration system to automate the process and regularly check that status of the build.
After attending Dr. C. Titus Brown’s “Why not run all your tests all the time? A study of continuous integration systems.” talk at Pycon and seeing Django’s Hudson setup, I figured I’d take a look at Hudson CI.
Hudson is very easy to setup. I started with a fresh Ubuntu 9.10 install on the smallest Rackspace cloud instance and had it running after a few commands. I followed the Debian setup instructions, which basically consists of:
$ wget -O - http://hudson-ci.org/debian/hudson-ci.org.key | sudo apt-key add - $ echo "deb http://hudson-ci.org/debian binary/" >> /etc/apt/sources.list $ apt-get update $ aptitude install hudson $ apt-get upgrade
That’s it! It’s already up and running on port 8080 using it’s own web server. Go ahead and pull it up in your browser.
As a test, let’s setup django-crm (a Caktus open-source community project) as our first Hudson job. Click “New Job”, type in a job name, click “Build a free-style software project”, and hit OK. django-crm contains a sample project that we’ll use to run the test suite. On the job configuration page, check Subversion in the Source Code Management section and type in the Repository URL:
Click Save, run the job by clicking “Build Now”, and check out the Console Output:
Started by user anonymous Checking out a fresh workspace because /var/lib/hudson/jobs/django-crm/workspace/sample_project doesn't exist Checking out http://django-crm.googlecode.com/svn/trunk/sample_project A manage.py A site_media A site_media/css A site_media/css/jquery.autocomplete.css A site_media/css/django-contactinfo.css A site_media/js A site_media/js/jquery-ui-1.7.2.custom.min.js A site_media/js/jquery-1.3.2.min.js A site_media/js/django-crm.js A site_media/js/jquery.autocomplete.min.js ... Finished: SUCCESS
Cool, now let’s run some tests. Took keep things simple, let’s grab Django and a few dependencies using aptitude:
$ wget http://www.djangoproject.com/download/1.1.1/tarball/ $ tar xzvf Django-1.1.1.tar.gz $ cd Django-1.1.1 $ sudo python setup.py install $ aptitude install python-dev python-imaging python-setuptools python-pip
To run the tests, add an “Execute shell” build step in the Build section with this command:
#!/bin/bash -ex cd sample_project python manage.py test crm
Run the job again and look for the test results in the console output:
[workspace] $ /bin/sh -xe /tmp/hudson6670261053226891793.sh + cd sample_project + python manage.py test crm ... Finished: SUCCESS
To integrate Hudson with the Django test suite, I used unittest-xml-reporting. Just “pip install unittest-xml-reporting” and add the following lines to your settings file:
TEST_RUNNER = 'xmlrunner.extra.djangotestrunner.run_tests' TEST_OUTPUT_VERBOSE = True TEST_OUTPUT_DESCRIPTIONS = True TEST_OUTPUT_DIR = 'xmlrunner'
Then check “Publish JUnit test result report” in the Post-build Actions section and add the path to the test XML output “sample_project/xmlrunner/*.xml”:
Run the job and you should see a new “Test Result” link in the navigation. Now you can view the test results right in your browser window.
To add coverage reports, I used Ned Batchelder’s coverage.py (pip install coverage). Navigate to Hudson’s plugin manager (Hudson -> Manage Hudson -> Manage Plugins), install the Cobertura Plugin, and restart Hudson when prompted. Then modify your shell script like so:
#!/bin/bash -ex cd sample_project coverage run manage.py test crm coverage xml --omit=/usr/
This will generate an XML coverage report in the working directory, so we just need to tell Hudson where to look for it. Check “Publish Cobertura Coverage Report” in the Post-build Actions section and enter the path to the report:
Run the build again and you should have access to a new “Coverage Report” link.
This was just a simple example of getting Hudson setup with a Django project and I know a lot more can be done with Hudson (check out the large number of available plugins). The top items on my todo list are: see Hudson setup environments with virtualenv and pip, integrate more closely with the test suite (possibly using nose), check for PEP compliance, and setup build failure notifications. I hope to write more as I continue to setup our Hudson environment!
A few useful Hudson/Python/Django links I discovered while running through this setup:
Unless Google also adds a disclaimer of all patents on all the new stuff, I'd be very careful about which ones we adopt.
The Google Data Protocol Patent License:
Subject to the terms and conditions of this License, Google hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this License) patent license for patents necessarily infringed by implementation (in whole or in part) of this specification. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the implementation of the specification constitutes direct or contributory patent infringement, then any patent licenses for the specification granted to You under this License shall terminate as of the date such litigation is filed.
I am not a lawyer, but I think that pretty much covers it. This bit of FUD slinging from Dave is particularly infuriating since I wrote that patent license.
Python Testing: Beginner's Guide
I was looking for some material on proper python testing in order to improve my QA skills and after some “Googling” came across “Python Testing: Beginner’s Guide“. My first impulse was to hit Amazon and see if they had it and if I could buy an ebook version. Sadly, they only had the “dead tree” version, so I decided to check if the publisher, PackT Publishing, had an alternative.
Boy, was I glad I hit their web site! Not only there was an ebook version of that book, but they do not password protect them, giving you complete control over your purchase!!! You can also copy text from it, which makes your life really easy when you’re following along and want to copy some of the code being described!
Needless to say I purchased the ebook version and have already allocated my weekend to read it, so you can expect a review next week. In the meantime, here’s a free chapter (PDF) for you to get a taste for the book: Chapter 5: When Doctest isn’t Enough: Unittest to the Rescue
Following the tradition of releasing simultaneously with the Transifex project, I’m pleased to present you the Transifex “Magneto” Appliance 0.8! There are just too many cool features to mention here… so I won’t! Just go ahead and read the release notes instead.
 |
| From Transifex v8.0 featutes |
As far as the appliance goes, the most important thing to know is that I dropped MySQL and replaced it with Postgresql, so if you’re thinking of updating an existing deployment, you’ll have to backup your data and handle the restoration process. If you’re installing for the first time, choose from the following image types:
The appliance is pre-configured with 2 unique users: editor and guest (with passwords editor and guest respectively) and several projects for you to play! To keep it up to date, log in to the web based administrative interface by connecting to your appliances url using https and adding port 8003 at the end. Then, login as admin (the initial password is password but you’ll be prompted to change it during the initial wizard). I can proudly say that the Transifex Appliance has been downloaded several hundred times in the last 2 months and is currently being used by several companies and projects that are either test driving Transifex or decided to host their own instance like the Xfce project for their translations!
As always the development branch of the appliance will follow the development code line of Transifex and provide a playground for anyone who wants to help out the project, such as the tasks created ahead of the upcoming Google Summer of Code. Download the appliance today and see why projects such as Meego, LXDE, Xfce, Fedora, and many more chose Transifex to manage their translations!
With the GNOME 2.30 release just around the corner, translators are feverishly working hard to get the desktop completely translated into a multitude of different languages! But unless you’re comfortable building the application you’re trying to translate on your own (or perhaps the entire desktop), you’re pretty much doing what I call “blind translations.”
|
| From Screenshots |
The good news is that you don’t have to do any compiling to play with the very latest GNOME applications! Just download the GNOME Developer Kit and start translating knowing that you can actually see what you’re translating!
Borrowing from a previous post I wrote, just what is the GNOME Developer Kit? It is a continuous build of GNOME packages all bundled up into a distribution (in this case, Foresight Linux) and distributed in a few different formats that you can either install or run in a virtual environmen.
So if you’re a translator or writing docs, imagine being able to see the application you’re trying to translate running right in front of you! As the GNOME Developer’s Kit already comes with a lot of tools such as gettext, intltool and poEdit, you got your work cut out for you!
So don’t just sit there! Go download your GNOME Developer Kit today!
Many Plone adopters take advantage of Google Analytics for web analytics. Nowadays, it is as easy as pasting a simple code snippet into your Plone site’s ‘Site Settings’:
This will provide you a whole host of usable data for metrics. One exception, and common request, that I’ve seen is how to track the number of downloaded files/images/objects. Google suggests that this is easily done, with a small onClick snippet.
Here is a sample link tag with the required onClick snippet:
<a href="http://www.example.com/files/map.pdf" onClick="javascript: pageTracker._trackPageview('/downloads/map'); ">
Out of the box, this is not possible with the simple implementation, as Plone and it’s visual editor, Kupu, both filter our Javascript for security reasons.
For deployments where content authors are trusted users, the following steps will provide a means to easily track number of downloads on an object by object basis.
Modify safe_html Transform
Disable Kupu’s JS event filter
After doing some research, it appears that the only way to disable Kupu’s filtering mechanisms are by patching the code. There is no configuration available for this, which is unfortunate.
vi /buildout/path/eggs/Products.kupu-[some-version]-py2.4.egg/Products/kupu/common/kupucontentfilters.jsthis.events = []; // 'onclick|ondblclick|onmousedown|onmouseup|onmouseover|onmousemove|onmouseout|onkeypress|onkeydown|onkeyup'.split('|');this.events = ['onclick',];
this.events = 'onclick|ondblclick|onmousedown|onmouseup|onmouseover|onmousemove|onmouseout|onkeypress|onkeydown|onkeyup'.split('|');Now you should be able to add the aforementioned onClick snippet to downloadable objects and accrue data in Google Analytics.
So what exactly is a python iterator, and how is that different from an iterable?
An iterable is an object that implements a method __iter__(), which, when called, returns an iterator. The __iter__() method can be called by the iter() function, and is also called behind the scenes in a for loop.
An iterator is a specific kind of iterable. It is an iterable which also implements a next() method, which returns a value each time it is called, until it runs out of values, at which point it raises a StopIteration exception. (And raises a StopIteration every time thereafter). Iterators also as a general rule return self when __iter__() is called.
Why does this matter?
The important thing to note here is that iterables do not have to perform their own iteration. To take a very simple case, list objects are iterables but not iterators. That is, they define an __iter__() method, but not a next() method. Rather than returning the list itself, __iter__() returns a listiterator object.
>>> names = ['Tom', 'Dick', 'Muhammad']
>>> iter(names)
This means that the list doesn’t have to store state information about the iteration process. A list doesn’t need to know that you just looked at the second object, so next time you iterate over it, you should see 'Muhammad'. In fact, it can’t know this, because a list might be in use in two iterations at once.
Looping over an iterable twice
What if you do the following?
>>> def get_pairs(seq):
... for x in seq:
... for y in seq:
... if x is not y:
... print(x, y)
The list would have to store its index in the x loop, but then it would have to store a different index in the y loop. It’s far easier to create a separate iterator for each, that just has to handle its own loop. Just to make it clear, if verbose, the following code does approximately the same thing:
>>> seq = ['Tom', 'Dick', 'Muhammad']
>>> iterator_x = iter(seq)
>>> iterator_y = iter(seq) # First pass
>>> x = iterator_x.next() # x == 'Tom'
>>> y = iterator_y.next() # y == 'Tom'
>>> if x is not y: # x is y: skip
... print(x, y)
>>> y = iterator_y.next() # y == 'Dick'
>>> if x is not y: # x is not y
... print(x, y) # prints "Tom Dick"
>>> y = iterator_y.next() # y == 'Muhammad'
>>> if x is not y: # x is not y
... print(x, y) # prints "Tom Muhammad"
>>> y = iterator_y.next()
StopIteration
>>> x = iterator_x.next() # x == 'Dick'
>>> iterator_y = iter(seq) # Restart the inner loop
>>> y = iterator_y.next() # y == 'Tom'
>>> if x is not y: # x is not y
... print(x, y) # prints "Dick Tom"
>>> y = iterator_y.next() # y == 'Dick'
>>> if x is not y: # x is y: skip
... print(x, y)
>>> y = iterator_y.next() # y == 'Muhammad'
>>> if x is not y: # x is not y
... print(x, y) # prints "Dick Muhammad"
>>> y = iterator_y.next()
StopIteration
>>> x = iterator_x.next() # x == 'Muhammad'
>>> iterator_y = iter(seq) # Restart the inner loop
>>> y = iterator_y.next() # y == 'Tom'
>>> if x is not y: # x is not y
... print(x, y) # prints "Muhammad Tom"
>>> y = iterator_y.next() # y == 'Dick'
>>> if x is not y: # x is not y
... print(x, y) # prints "Dick Muhammad"
>>> y = iterator_y.next() # y == 'Muhammad'
>>> if x is not y: # x is y: skip
... print(x, y)
>>> y = iterator_y.next()
StopIteration
>>> x = iterator_x.next()
StopIteration
Note that each loop has its own iterator. This is not the case with, for instance, file objects. File objects implement the iterator protocol directly, which is to say, they have both an __iter__() method and a next() method. As a result, you can only use a file object in one loop at a time.
Looping over an iterator twice
If we try to run the same get_pairs() function with file objects, it doesn’t give us the expected results.
File 'names.txt':
Tom
Dick
Muhammad
Result of get_pairs(open('names.txt')):
Tom
Dick
Tom
Muhammad
So two things have happened here:
- We're getting newlines from each line of our file.
- The only value being used in the
x loop is 'Tom'.
The reason for (1) is pretty obvious and not particularly interesting, so lets look a little closer at (2). When you enter the outer loop, python calls f.__iter__(), which, for an iterator like a file object, returns a copy of the file object itself. It then calls f.next() on the iterator, and stores the value of that ('Tom\n') in x.
It then enters the inner loop, and calls f.__iter__(), returning another copy of the same file object. Now we have two loops working on copies of the same object. So when the inner loop calls f.next(), it immediately returns 'Dick\n', because *it is the same iterator* which just returned 'Tom\n' in the outer loop. The next time through, it returns 'Muhammad\n', and finally, it raises a StopIteration because it has exhausted the file, and returns control to the outer loop.
We’d like the outer loop to loop over 'Dick\n' and 'Muhammad\n' now, but remember that it is operating on the same object as the inner loop was, so when it calls f.next(), it just raises another StopIteration instead, and exits the outer loop.
Restarting an iterable
Another benefit of creating an iterable which does not perform its own iteration is that it can be restarted. Once an iterable raises a StopIteration exception, it is required to raise a StopIteration every time next() is called from then on.
An iterable, on the other hand, doesn’t implement next() itself, so it never raises a StopIteration. If you try to start a new loop with your iterable, you get a new, fresh iterator, which isn’t stuck in StopIteration mode.
For instance, with an iterable:
>>> names = ['Tom', 'Dick', 'Muhammad']
>>> for x in names:
... print x
Tom
Dick
Muhammad
>>> for x in names: # Looping over a new iterator.
... print x
Tom
Dick
Muhammad
And with an iterator
>>> f = open('names.txt'):
>>> for x in f:
... print x.strip()
Tom
Dick
Muhammad
>>> for x in f:
... print x.strip() # Immediately raises StopIteration
>>>
An iterable can be looped through over and over again, and an iterator, only once.
How can I take advantage of this?
These very different usage patterns are simply implemented by returning different things from the __iter__() method. If you create an object which returns itself, you need to also implement next(), and have it raise StopIteration once it is exhausted. If you return a different object from __iter__(), you can use it over and over again.
Note that you cannot just write an iterable and expect it to work on its own. It has to return a legitimate iterator, which has both __iter__() and next() methods. Thus writing an iterable is a little more work, but the added flexibility is often worth it.
See my follow-up post: Tricks with Iterators. In it I discuss various ways you can take advantage of iterators and iterables.
People are often asking me how and why my department shifted from an ASP.NET environment to Django. I’ve finally gotten around to writing about the process leading up to our decision. I hope people out there find it useful in their own development groups and discussions.
Almost two years ago I was in a rather unlikely situation in that I was running a software engineering department containing both a C# team and a Python team. The Python group was focused on building scientific computing and NLP-type applications, whereas the C# team was focused on building web applications.
A few of us Python folks in the department had already started playing around with Django–building internal web applications and projects outside of work. It did not take long for us to realize the power of Django and how quickly we were able to produce high-quality applications with little effort. This was my (strong) impression, but in order to propose a corporate platform shift I was going to need some data to support my claims.
It slowly dawned on me that I had a perfect test bed. Here we had two teams using different technology stacks within the same department. The same department. That means they shared the same development processes, project management tools, quality control measures, defect management processes. Everything was the same between these groups except for the technologies. Perfect! So like any good manager I turned my teams into unwitting guinea pigs.
We can accomplish more with Python + Django than with C# + ASP.NET given the same amount of time without sacrificing quality
For the sake of this study, I defined productivity as a normalized team velocity: how many story points were completed / developer / week. I record the normalized team velocity for each team’s sprint for later analysis.
For those of you unfamiliar with the concept story points I highly recommend Mike Cohn’s Agile Estimation and Planning.
I hear this a lot. Yes, you can. The problem is that most people do not bother creating a common scale or continually calibrate their estimations (within or between groups). Generally, it’s way more work than most groups need to deal with and it doesn’t deliver much utility to most groups so it isn’t often discussed or practiced.
The methods described below should outline the additional calibration work that was performed to ensure a common estimation scale between the two teams.
Both teams continued business as usual working on projects in parallel. Each sprint consisted of 3-4 developers. It is worth noting that Team ASP.NET did not make use of MS MVC Framework, but they did use Linq-to-SQL for its ORMy powers.
Special care was taken to maintain linkage between the two team’s effort estimates. During sprint planning, each team would use a common story point calibration reference when making estimates. In order to detect any potential deviations in calibration, during several planning poker sessions I included stories that had already been estimated during previous sprints or by the other team; no significant deviations were found.
At the end of each sprint I would calculate the normalized developer velocity ( # of completed story points / developer / week ). These values were recorded for both teams. It should be noted that only Django-based sprints were used in analysis for Team Python.
I recorded results for approximately 6 months.
Normalized Developer Velocities: C# + ASP.NET and Python + Django
The above histogram shows the distribution of normalized velocities associated with each completed sprint. The table below summarizes the distribution of velocities associated each team.
| units: story points / developer / week |
C#/ASP.NET | Python/Django |
|---|---|---|
| mean | 5.8 | 11.6 |
| stdev | 2.9 | 2.7 |
| min | .3 | 8.5 |
| max | 9.3 | 15.8 |
The distribution of velocities between the two samples are similarly shaped, but have clear differences in their mean. The average velocity of a C#/ASP.NET developer was found to be 5.8 story points/week. A Python/Django developer has an average velocity of 11.6 story points/week. Independent t-tests reveal these differences as being statistically significant (t(15) = 4.19, p<7.8e-4).
Given our development processes we found the average productivity of a single Django developer to be equivalent to the output generated by two C# ASP.NET developers. Given equal-sized teams, Django allowed our developers to be twice as productive as our ASP.NET team.
I suspect these results may actually reflect a lower bound of the productivity differences. It should be noted that about half of the Team Python developers, while fluent in Python, had not used Django before. They quickly learned Django, but it is possible this fluency disparity may have caused an unintended bias in results–handicapping overall Django velocity.
The productivity differences quantified by our findings were then included as part of an overall rationale to shift web-based development platforms. Along with overall velocity differences, the costs associated with maintaining each environment were considered: OS licensing and database licensing for development and production environments, as well as costs associated with development tools. I’m happy to say we are now a Python and Django shop.
Updated:
Several good questions over at Hacker News
Here are my slides (pdf) (OpenOffice) from my presentation "Threading is not a model" from PyCon. Be warned that they aren't very useful, there are no speaker notes and the slides themselves are pretty light on content, except for near the end of the deck where it gets into Stackless Python, Go and IO code examples. If you weren't actually in my talk you should wait for it to appear on pycon.blip.tv.
I have made the journey to the mecca of nerdery, at least my kind of nerds. Nerds who write Python. I’m at PyCon 2010 Atlanta.
The morning begins with Guido’s keynote, answering questions posted via Twitter: #python.
More to come…
The schedule isn't up yet, but I just found out today that my submission got accepted for this year's Triangle Game Conference. The really great news is that Christopher's submission also got accepted. This will be the first time I've presented at the same conference as my son!
Python and Django are tools we use on a daily basis to build fantastic web apps here at Caktus. I’m pleased to announce that Caktus is sending five developers–Colin, Alex, Mike, Mark, and myself–to PyCon 2010! PyCon is an annual gathering for users and developers of the open source Python programming language. This year the US conference is being held in Atlanta, GA. We’ll be driving down tomorrow (Thursday) from Chapel Hill, NC and staying for the conference weekend plus one day of the sprints.
Hope to see you there!
Posted here because I got hung up on this. If you add a link to your profile, but it doesn't show up when you try to force a recrawl, you need to make sure the "This is a profile page about me" checkbox is checked in your profile.
Thanks to Ade, the Python version of mimeparse now has pirate tests.
So, Buzz is finally out. We've been dogfooding it internally for a while and it's been great.
One of the best aspects of Buzz is the use of open standards. It's just feeds. And if you want to hook up a new data source to Buzz, it is just a matter of adding a link/@rel='me'. That two way connection between your profile page and the source is the proof that it is a source that you truly own. That's a pretty easy thing to do if you are a service provider, adding a little markup to allow integration with Buzz. And it's not like that information gets locked up; you can always pull the site connectivity information back out via the Social Graph API. Here is a demo site for querying the Social Graph API.
Once you get that working you can always go back and add support for fast updates by pinging a PubSubHubbub hub. Again, this is a really simple thing to do, and I've done it for Piccolo.
See DeWitt's post for where this is all headed and the other open protocols that are going to be integrated.
Four years ago today I decided I wanted to do something a little different for Valentine's Day, so I found some red card stock in the house and began cutting out hearts. Lots of hearts. 2 inches across, about 50 in total. I then began sticking them up around the house, not randomly, but in places where Lynne would find them as she went about her daily routine. One on the bathroom mirror, one on the shower door, one on the rear-view mirror of the van, etc. She enjoyed it and so did the kids, watching her discover each new heart.
That was four years ago, but the kids were so enthralled with it that they've demanded I do it every year since, and I've complied. Last year was the toughest year since I was supposed to be traveling on Valentine's Day, which meant that I had to cut out 50 hearts in advance, pre-attach sticky gum to the back of each one, and break them into batches for the kids to distribute around the house when they got up on Valentine's Day morning.
This year is no different; it's 2am and I just finished distributing 50 hearts around the house. It's no longer surprising for Lynne, and on its fourth year is well on its way to becoming a tradition, but I'm going to keep on doing it, because our relationship isn't hearts and flowers once a year, but all the little things we do each day, every day, together.
At a previous employer we made tensile testing equipment. Now for tensile testing a material you cut it into a dogbone shape and then pull at each end of the dogbone, plotting out the change in length of the specimen against how much load is on the specimen. Once you have all that data you can plot stress-strain curves and learn all sorts of interesting things about said material.
Now if you are developing a new material, or in general working in a laboratory setting, the amount of material to test isn't large, and the amount of work of preparing specimens and loading them into a tester and reading off the results falls into that's-what-we-have-interns-for.
If, on the other hand, you are in a manufacturing environment and the results of your tensile testing are used as inputs into your SQC/SPC process to control your manufacturing line then the amount of testing falls into that's-why-they're-unionized.
For those special folks willing to shell out enough money, we offered a robotics 'package'. By which, I mean, that our intrepid sales folks would offer a robotic automation system to a customer, they would feign interest, my boss the engineering manager would pull a ridiculously large number out of thin air in an attempt to dissuade the customer, the sales guy would 'negotiate' with them, and the price would drift down low enough that the customer would accept, and the VP of the week for our division would drool, and then we'd be forced into building one. The only problem was that we didn't have a robotics package, and so every time we got an order we had build up the entire system from scratch.
Every time the delivery date would slide closer and closer, and Joel, the engineer put in charge of designing these systems, would be out on the manufacturing floor assembling a Medusa looking nest of wires and robotics, testing and tweaking. We would all help out, but invariably as the time got closer we'd joke that he should really get it working, because if it wasn't done in time, we could always ship them a Joel-in-a-box until we did get it working. And we could, you know, because being an engineer, he wasn't unionized.
This all came rushing back to me the other day when I was reading Ian Hickson's draft of the Web Socket protocol. It was written in a very clear and precise manner, but also very clearly directed at someone that is going to be implementing the protocol, either on the client or the server end. It was refreshing to read, and in sharp contrast to many of the specifications I've had to read recently. Somewhere between now and Jon Postel we lost something in spec writing. Over time specifications have drifted towards being written in a stiff and officious manner, laden with a sense of, but no actual, rigour; as if they were the Doings of Serious Men. In the old days RFCs were written more along the lines of "A letter to the implementer", with directions on how to implement the protocol. Heck, some of them even went so far as to include code. Actual code! The faux rigour was something that I fought against, and largely lost, in the writing of RFC 5023. Reading the Web Sockets protocol specification did give be that flash back to better spec writing. And it also brought back memories of Joel. So here's the rub: Any spec is a good spec if it doesn't require a Joel-in-a-box to get it up and running.
The NASA budget request, which actually included a slight budget boost up to $19 billion, would cut out the Constellation completely, extend the International Space Station's mission through at least 2020, and set aside $6 billion over five years to support commercial spacecraft development.
It would also increase funding for fundamental research, Earth science and development programs for ground-breaking technology for space exploration.
There's obviously a lot of hand-wringing about this, which is understandable, for many people it's an unthinkable change because NASA has been the sole provider of U.S. manned space flight for the past 50 years. Personally I'm thrilled.
http://a9.com/-/spec/opensearch/1.1/
http://a9.com/-/spec/opensearchrss/1.0/
http://activitystrea.ms/spec/1.0/
http://backend.userland.com/blogChannelModule
http://backend.userland.com/creativeCommonsRssModule
http://base.google.com/ns/1.0
http://base.google.com/ns/1.0
http://conversationsnetwork.org/rssNamespace-1.0/
http://earth.google.com/kml/2.0
http://earth.google.com/kml/2.1
http://feedsync.org/2007/feedsync
http://gdata.youtube.com/schemas/2007
http://geourl.org/rss/module/
http://hacks.benhammersley.com/rss/streaming/
http://madskills.com/public/xml/rss/module/trackback/
http://media.tangent.org/rss/1.0/
http://my.netscape.com/rdf/simple/0.9/
http://my.theinfo.org/changed/1.0/rss/
http://ns.opensocial.org/2008/opensocial
http://openid.net/xmlns/1.0
http://portablecontacts.net/ns/1.0
http://postneo.com/icbm
http://purl.org/dc/elements/1.1/
http://purl.org/dc/terms/
http://purl.org/net/rss1.1#
http://purl.org/rss/1.0/
http://purl.org/rss/1.0/modules/aggregation/
http://purl.org/rss/1.0/modules/annotate/
http://purl.org/rss/1.0/modules/company
http://purl.org/rss/1.0/modules/content/
http://purl.org/rss/1.0/modules/email/
http://purl.org/rss/1.0/modules/event/
http://purl.org/rss/1.0/modules/image/
http://purl.org/rss/1.0/modules/link/
http://purl.org/rss/1.0/modules/reference/
http://purl.org/rss/1.0/modules/richequiv/
http://purl.org/rss/1.0/modules/rss091#
http://purl.org/rss/1.0/modules/search/
http://purl.org/rss/1.0/modules/servicestatus/
http://purl.org/rss/1.0/modules/slash/
http://purl.org/rss/1.0/modules/subscription/
http://purl.org/rss/1.0/modules/syndication/
http://purl.org/rss/1.0/modules/taxonomy/
http://purl.org/rss/1.0/modules/threading/
http://purl.org/rss/1.0/modules/wiki/
http://purl.org/syndication/history/1.0
http://purl.org/syndication/thread/1.0
http://rssnamespace.org/feedburner/ext/1.0
http://schemas.google.com/acl/2007
http://schemas.google.com/codesearch/2006
http://schemas.google.com/g/2005
http://schemas.google.com/gCal/2005
http://schemas.google.com/gdata/batchbatch
http://schemas.google.com/photos/2007
http://schemas.google.com/photos/exif/2007
http://schemas.google.com/spreadsheets/2006
http://schemas.google.com/spreadsheets/2006/extended
http://schemas.xmlsoap.org/soap/envelope/
http://search.yahoo.com/mrss/
http://web.resource.org/cc/
http://webns.net/mvcb/
http://wellformedweb.org/CommentAPI/
http://www.bloglines.com/about/specs/fac-1.0
http://www.georss.org/georss
http://www.itunes.com/dtds/podcast-1.0.dtd
http://www.microsoft.com/schemas/rss/core/2005
http://www.opengis.net/gml
http://www.opengis.net/kml/2.2
http://www.opml.org/spec2
http://www.usemod.com/cgi-bin/mb.pl?ModWiki
http://www.w3.org/1998/Math/MathML
http://www.w3.org/1999/02/22-rdf-syntax-ns#
http://www.w3.org/1999/xhtml
http://www.w3.org/1999/xlink
http://www.w3.org/2000/01/rdf-schema#
http://www.w3.org/2000/svg
http://www.w3.org/2003/01/geo/wgs84_pos#
http://www.w3.org/2005/Atom
http://www.w3.org/2007/app
http://www.w3.org/XML/1998/namespace
http://xmlns.com/foaf/0.1/
urn:atom-extension:indexing
xri://$xrd*($v*2.0)
xri://$xrds
I was in California earlier this week, but on Lynne's advice I moved up my return flight as the threat of snow grew.
The snow began to fall on the cab ride home, leaving me just an hour or two window to test the generator and refill all the gas tanks.
That was Friday night.
It's now Sunday and we are full into the North Carolina system of snow removal, "wait until it melts".
School is already cancelled for tomorrow, and at the rate it's melting, they may not go back to school until Wednesday.
This past Thursday I gave a presentation on Fabric for TriZPUG. I want to thank everyone for their comments and appreciation. We had a great attendance and good discussions.
I presented Fabric as a tool that I use for automated deployment and maintenance of servers. I can publish and update multiple servers with differential instructions based on their defined roles (e.g. Apache web server, MySQL database server, Nginx server, etc.). We briefly touched on some other cool things you can do like database maintenance, database dumps, backups, Django contribs for remotely working with Model objects, and even toggling maintenance modes for your website.
The next part of the presentation dealt with another use I’ve found for Fabric: bootstrapping a server. The ease of automating distributed systems made Fabric an ideal tool for lightweight configuration of a newly installed OS. Go ahead and have it apt-get/yum/emerge whatever packages are going to be required–again as specified by its defined role. For instance, I have a few fabfiles that will prep an Ubuntu or a RHEL servers for serving Django apps with Nginx, Apache+mod_wsgi, and MySQL.
So, the most common questions and comments I’ve received have been along the lines of “Why not just build packages and meta packages for each server configuration?” This is absolutely an approach that one could take and a powerful one at that. I’ll be the first to admit I am not a packaging whiz so the thought of building a slew of .debs and then maybe parallel .rpms if we have to migrate to a new platform based on the client or project is not super appealing, but luckily I work with people smarter than myself who enjoy that sort of thing.
The sweet spot I’ve found for bootstrapping servers with Fabric is where your server configuration may not be final, the project maybe short term, you may still have to switch between distros, and you do not have access to VMs that can be cloned. Fabric is lightweight and available for your more ephemeral projects that may not necessitate the effort required to build custom packages.
For instance, one place Fabric is handy is prepping Amazon EC2 servers for quick compute jobs. Now I could go ahead and create custom AMIs for every one-off project, but Fabric makes it easy to spin a server up and then bootstrap it with the required packages for a particular project. Nice and lightweight.
I would love to hear about other tools people are using out there. It was nice that after my presentation a bunch of people shared their tools and methods for bootstrapping servers. Og Maciel and other rPathers demonstrated rPath’s rBuilder Online and how it can be used to easily build packages and entire system images or VMs based on sets of packages and a few clicks. Tobias McNulty of the Caktus Consulting Group also demonstrated his work on Mallet, which can be used to quickly build and configure Debian-based servers. Mallet is also available through PyPI.
Overall a great turn out and I learned a lot. Thanks folks!
Yesterday I attended my first TriZPUG meeting to check out Kurt Grandis‘ talk on Fabric, “a Python library and command-line tool for streamlining the use of SSH for application deployment or systems administration tasks.”
It was pretty cool to see a bunch of guys who share the same interests take some time on a Thursday to hang out, drink beers, and chat about python, django, zope, and other stuff. After the original talk was over and some of the other lightening talks that succeeded it was over, a couple of things became very clear to me:
Having been using rBuilder Online to manage and maintain my Transifex Appliance, and being somewhat “spoiled” with the ability of having fine grained control over the entire software stack and having the option of deploying my final “product” on several different cloud environments, I couldn’t help but offer to speak a bit about my experience. I sure hope my impromptu presentation didn’t come across as being “just a sell’s pitch” and I definitely tried my best not to sound like I was selling something. I truly feel that the technology developed here at rPath can solve many of the typical issues that people have getting their product through the many different life cycles and eventually out the door and into the hands of their customers!
Today I started going through Fabric’s documentation and am already making plans to include it in some of the test automation tools we’re developing here!
Anyhow, after my presentation there was a quick intro to epdb, the “Extended Python Debugger”, a very cool python debugger developed by an ex-rPathian and something I use on a daily basis! Turns out that the epdb currently packaged for Foresight Linux was outdated, so I spent a few minutes during my lunch today to update it. If you’re running Foresight, just run conary update epdb=:2-devel or wait for it to make its way to the stable label. If you’ve never heard of epdb, I strongly suggest you give it a try!
I just received a copy of Plone 3 for Education from Packt. I’m planning a review once I’ve put it through its paces, but in the meantime I wanted to send along my first impressions.
The general focus of the book is on course creation, but it appears to cover a lot of the “eventualies” of using Plone in an educational environment (“eventually, you’ll need a staff directory”… “eventually you’ll want to embed media”). Very promising.
The book is on the thin side. From skimming through, it doesn’t seem to be a problem though. The sections are arranged in a fairly logical order, content creation up front and skinning, then deployment at the end.
The book is comprised of (relatively) short, cookbook-style, “if you want to do this, here’s how” sections. Right to the point. This is very promising. It should make the book a good reference as well as a tutorial.
I an very interested in getting into the technical aspects of the book, but I’m especially anxious to see how well it works from an even less-than-anticipated level of technical ability. The book’s target audience includes “makers of school websites… tech-savvy teachers… and seasoned software developers” (from the back of the book), and “web masters… and enterprising faculties” (from the preface), but it shows real promise as being suitable as a next-step for people who already have some basic plone skills and want an easy-to-follow guide for doing something practical. Especially for those of us in an educational environment.
I’ve got my fingers crossed :)
I’ll post a full review soon. Erik has a good reputation in the community, and a knack for explaining things well, so I’ve got high hopes.
There’s a sample chapter available on calendaring, if you’d like to read more of the book than I have at this point ;)
One side note: the cover of the book has a yellow rose on it. Classy. I guess it’s there because of Erik’s last name. I wish Packt would take a cue from O’Reilly and explain the covers of their books in a colophon, but it’s a fair assumption.
This is cool, but one hopes we’re looking at a special case and not a standard practice.
The mind wonders to strange places when contemplating what they’d put on a book authored by someone with a last name like, say, Johnson. :P
Just wanted to tease you guys out there about a new feature that the Transifex guys are working on these days: Translation Reviews! Have you ever wandered if your translations conform to the standard vocabulary that your team uses? Have you ever wanted someone to take a look at what you’ve done before sending in your final work for commit approval?
 |
| From Transifex v8.0 featutes |
Now, mind you this is still very alpha code but that is probably a good thing since you can play with it and give your feedback on how to improve it. As always, you can get this in an easy to consume format by using the Transifex Appliance Developer edition… or you can join the Xfce translators who are already enjoying Transifex latest code!
I just checked in a port of mimeparse to Go.
The list of supported languages for mimeparse is now: Erlang, JavaScript, Perl, PHP, Python, Ruby, Java, and Go.
The Go code size is a little larger than the original Python, 187 lines of code for Go
versus 123 for Python. Now 27 of those added lines are lines just containing
a single '}', so the difference isn't even that large.
Some of the increase is size is also probably me not producing very idiomatic Go code,
so take all these observations, and the code itself, with a grain of salt, as I'm a
Go newbie. One of the things I missed from Python is list comprehensions, and
it was one of the things that I was going to point to
as making a big difference in the length of the code. I'm not going to
claim that now, because when I look at the difference between the Python parse_mime_type
where the parsing of the mime type parameters is done, and the equivalent
in the Go ParseMimeType you can see that while the list comprehension in Python is
smaller, it is also more difficult to read, and it has a bug. I still thing there
are places where it would be useful, for example, in ParseHeader().
Python
params = dict([tuple([s.strip() for s in param.split("=")])\
for param in parts[1:] ])
Go
params := make(map[string] string);
for _, s := range parts {
subparts := strings.Split(s, "=", 2);
if len(subparts) == 2 {
params[strings.TrimSpace(subparts[0])] = strings.TrimSpace(subparts[1]);
} else {
params[strings.TrimSpace(subparts[0])] = "";
}
}
One of the surprising things about the Go code is the strong typing; that is, while Go is strongly statically typed, the code isn't cluttered with types. The type inferencing makes a tremendous amount of code noise disappear. For the most part you see types explicitly in the function declarations, but in the body they are mostly absent.
One of the other things that keeps the code size small is functions. Yes, functions, because sometimes that's all you need, without being forced to put everything in a class, if you know what I mean.
Unfortunately with the simplicity of mimeparse, I didn't get to use some of the coolest features of Go, such as channels and interfaces. Of all the things that Go introduces, interfaces are one of the biggest changes and also one of the most important. Russ Cox has a good overview of them, and delves into the implementation details in the compiler.
In case you’ve missed it, the Xfce project has been using their own installation of Transifex to manage their translations online! Translators can now visit http://translations.xfce.org and keep up with the action!
 |
| From Transifex v8.0 featutes |
I’ve been contributing with translations for the Brazilian Portuguese language for quite some time now, and have been a strong supporter for the Transifex project as well, so I was thrilled to learn they were “working together”! But there is a second reason why I’m mentioning this on my blog:
Turns out that Nick Schermer, maintainer for http://translations.xfce.org, is using my Transifex appliance too!!! Moreover, he chose to use the appliance built from the development branch to get the very latest bits being committed to the development branch of Transifex. It has been a win-win-win (yes, 3 times!) relationship so far for all parties involved, for:
 |
| From Transifex v8.0 featutes |
Some of the cool features that you can expect from the next version of Transifex (and that the over 200 registered Xfce translators are already enjoying) are:
 |
| From Transifex v8.0 featutes |
The Transifex Appliance (developer image) has been updated almost on a daily basis, so those out there already using it can keep it updated using the web based appliance management tool or running conary updateall. As always, you can expect a stable release the very same day that Transifex releases the upcoming 0.8 version! I’d love to hear from all of you appliance users out there. Just drop me a line or a comment here and I’ll do my best to improve your experience.
Geek: (to self) WTF? SRSLY?
Geek: (to Non-geek) Hey, can I borrow your phone number?
Non-geek: My phone number? What's wrong with yours?
Geek: I want to give your number to Google so they'll turn up my Google App Engine access.
Non-geek: You want to give my phone number to Google for a whatsa-who?
Geek: It's no big deal, they'll just send you an access code via SMS...
Non-geek: Access code. NSS. Riiight. You want my Social Security number too?
Geek: No, no! It's just a...
Non-geek: Uh-huh. Um... fuck off, R2D2.
Welcome to "What's Next in HTML," where I'll try to summarize the major activity in the ongoing standards process in the WHAT Working Group. Wait... what happened to This Week in HTML5? Hell, what happened to HTML5? Well, nothing. It took over five years to create, but it's in Last Call now.
In other news:
$ hg pull pulling from https://go.googlecode.com/hg/ searching for changes adding changesets adding manifests adding file changes added 632 changesets with 2498 changes to 1100 files
No doubt it's been a while since I did a pull, but really, it looks like someone could write a 'This Week in Go' column.
One of the python/plone developer positions I posted about before is still open.
It’s a mid-tier, mid-level position, developing Plone/Python-based applications for a research center at the University of North Carolina School of Medicine. Specifically, the job is for the Lineberger Comprehensive Cancer Center. It’s located in Chapel Hill, North Carolina.
This means it’s a state job and has great benefits. Being a mid-tier, mid-level position means that there is a lot of room for advancement over time.
My original post explains the job in detail.
It’s essentially building applications to meet research needs in a team environment. Data management, bioinformatics, image analysis and more. The goals are short, productive development cycles, innovation, and ultimately publication of our work.
See the job posting for application details.
If you have any questions, I can answer them off the record in a purely unofficial way via e-mail at josh underscore johnson at unc dot edu.
Six bugs have already been filed against rst2rfc, so I'm going to take that as a good sign. The source has been moved into the repostory and I've created a mailing list for discussions. If you have patches, please put them up on codereview.appspot.com and make sure to inlucude me (joe@bitworking.org) as a reviewer.
Triangle Game Conference is currently accepting submissions to the 2010 Triangle Game Conference (TGC). All submissions must be received no later than close of business Friday, January 15, 2009.
I've already submitted a proposal, and so has my son Christopher. Even if I'm not speaking I'll still attend. Hope to see you there!
Because, you know, I don't need to know about lxml preserving the case of my doctype every single time I run piccolo.
import warnings
from html5lib.constants import DataLossWarning
warnings.simplefilter("ignore", DataLossWarning)
from lxml import etree
On his latest post titled “Foresight Linux is dead?“, Thilo Pfennigs rightly asks the question that many of the current Foresight Linux users may be asking themselves. With the current stable release dated as of May 2009 and no explicit roadmap stating when the next release will be published, is it really safe to say that Foresight Linux is indeed dead?
In order to properly answer this question, one must first take a look at what the year of 2009 reserved for this young distribution. Born out of Ken Vandine’s desire to follow the GNOME project as close to the upstream source as possible and introduce all the latest and coolest applications out there to the desktop before anyone else, Foresight was for a while synonymous to bleeding edge Linux done right!
Powered by the revolutionary Conary package management system and a small but talented and determined crew of developers, Ken was able to ship a new version of the distribution the same day that a GNOME release was published, a feat that no other distribution was able to keep up, even those enjoying of large hordes of developers and user base. Foresight was the first distribution to include several trend setter applications out there to the default installation, such as Banshee, F-Spot, Tomboy, Gwibber, Pulse Audio, PackageKit, among many others! And since the distribution followed a rolling release cycle, users did not have to wait for a major release in order to get the very latest bits.
Even if DistroWatch’s numbers weren’t impressive, those who took the time to test drive the distribution fell in love with the community, package selection, and most likely the possibilities that the underlying Conary technology provided for those inclined to do a little packaging or package maintenance. If you were a GNOME user/fan and didn’t mind the small sized, hand picked repository of supported packages, then you’d probably feel right at home! Sure there were KDE, Xfce, Fluxbox, Openbox packages available but those were mostly supported by some of the core users who didn’t mind doing the heavy lifting.
Then came 2009 and with it the major financial crisis the shook many companies around the world, creating a massive layoff wave for most of the first quarter. Sadly, approximately 75% of the active developers that comprised Foresight’s core developer base were part of the many casualties, including Ken Vandine, the heart and soul of the distribution! By late February these developers had already joined the ranks of companies such as Red Hat and Novell to do package and kernel management. Ken himself was quickly nabbed by Canonical to join their Desktop Experience Team, concluding then the completely dismemberment of the seasoned Foresight team!
Deprived of its core developers who were now devoting their time to working for their respective new companies, Foresight’s run at being a bleeding edge distribution and being able to keep up with the release schedules of GNOME (and all of its dependencies) quickly spiraled down toward what looked like certain doom. António “Doniphon” Meireles, second in charge of the distribution and holder of all the knowledge related to how all parts worked together became the sole guardian and maintainer of all packages. Have you ever tried to sync up and maintain all the modules that make up the X.org stack by yourself? How about making sure that every single package in the repository is properly compiled and linked to a newer version of Python?
Unfortunately for many of our loyal users expected point releases stopped from happening on time and deadlines were never met. Having been using Foresight Linux as my primary and only distribution for the last 3 years, I myself started to wonder if 2009 would mark the end of it all.
It took a few months for the remaining developers and users to get over the deep scars left from the massive exodus suffered early last year, but our user base proved to be very resilient and new developers stepped up to fill in the gaps. António was still doing the heavy lifting but this new crop of developers took upon themselves to bring the distribution closer to its former shape.
Slowly but surely milestones were achieved and the development branch eventually caught up with the latest GNOME packages. As of 2 weeks ago the development branch was pretty stable and I believe that only a few minor issues with PolicyKit were blocking a new release. Some massive work has also been done to pave down the way for Foresight 3.0, a major move that will allow for a more modular platform that can be used to derive other distributions, leveraging the flexibility and functionality provided by Conary. Moreover, the “Boots” project was kicked off to bring a Fedora based distribution completely managed by Conary, which should free up the time our developers spend maintaining some of the more complex stacks of the operating system and let them focus on making your desktop “freaking cool!”
So to answer the original question posted by Thilo, “is Foresight Linux dead?” I can gladly say “Far from it!” I predict that the Foresight community will rally together in 2010 to get back to being the most GNOMEic and bleeding edge distribution out there! As the Foresight Community Manager I can honestly say that we have always been and will always be a niche distribution! We don’t have the man power that distributions such as Ubuntu, Fedora, Mandriva or OpenSuse have to provide the same level of documentation or user support. We obviously cannot afford to have the same depth of package variety in our repositories or claim to have the expertise and time to resolve all issues that manage to get filed in our tracking system. But I can guarantee one thing: Foresight is here to stay!
If you want to try a revolutionary package management system and want to be part of a an exciting crew, come hang out with us on #foresight at Freenode. We will help you get started and I promise you that you’ll be able to contribute in no time.
Expect great things from Foresight Linux this 2010!
Caktus Group Chris Church David Ray Frank Wierzbicki Gary Poster J. Cliff Dyer Joe Gregorio Joseph Tate Josh Johnson Kurt Grandis Og Maciel Paul Smith TriZPUG Events TriZPUG News ZopeSkel BBQ Sprint