Portrait of Edd Dumbill, taken by Giles Turnbull

Subscribe to updates

Feed icon Atom or RSS

or get email updates

What I make

a conference management web application

XTech Conference
a European web technology conference

Using sparklines to aid conference proposal selection

In a recent article I explained some of the mechanics behind putting together the schedule for XTech 2007. We've just added new visualization features to Expectnation that make proposal choice easier, and I thought I'd show some examples from the XTech 2007 review voting.

Sparklines, devised by Edward Tufte, are thumbnail sketches of data that succinctly convey patterns. When Nat Torkington used sparklines to show reviewer voting patterns for OSCON program committee, it seemed a helpful addition to make to Expectnation.

The profile of each reviewer is interesting — are they a harsh marker, or liberal with the top grades? — but sparklines really come into their own when drawn per proposal. Here are some small screenshots from this year's XTech voting, anonymized of course.

High flyers

High scoring chart

Reviewers are pretty unanimous about this proposal, as 4.0 is the top grade in our scoring system. The lack of variance suggests that I ought to ensure that all the assigned reviewers put in their scores — this could be just one person voting. 

High scoring chart

High scoring chart

These next two are the pretty typical distribution for highly graded proposals: mostly top marks and a few average ones. 

Middle ranking

Some of the more interesting trends show up when the scoring isn't biased to either the top or bottom ends of the scale. 

Medium scoring chart

The reviewers are unanimous about this paper's soundness. It's not made anyone go "wow", but there's nothing to grumble about either. 

Medium scoring chart

This proposal clearly polarises opinion, and is such stands a better chance than the unanimous 3.0 above. Perhaps the subject matter or approach is controversial or timely.

Medium scoring chart

Another proposal that divides the reviewers' opinions. It's also worth me checking here that I don't either have a pathologically strict reviewer, or at the other end, a reviewer with a wild passion for the cause this paper advocates. 

Low scoring

The quality of submissions is usually so good that I don't give low graded proposals much attention, but the sparklines could alert me to potential oversights. 

Low scoring chart

This proposal seems to polarise opinion between "rubbish" and "ok", so it's interesting to me to check out the subject matter and see if I'm missing something with potential. 

Low scoring chart

Low scoring chart

These proposals were not received at all well.

In summary, I'm delighted to find a solid practical application for sparklines. My thanks to Nat Torkington for the inspiration. 

blog comments powered by Disqus

You are reading the weblog of Edd Dumbill, writer, programmer, entrepreneur and free software advocate.
Copyright © 2000-2012 Edd Dumbill