Software Engineers Who Don't Value Software

Software engineering researchers can be bozos.

Software engineers love to sit around tables and complain about how nobody values their work. It's true, too: people will happily pay for physically tangible resources but always expect software to be cheap or free, even if it's much more complex and difficult to engineer than the physical object. So you would think that a software engineering conference, of all things, would appreciate the value of software and be willing to spend a little money to save a lot of time and effort down the road.

As you can guess by the very existence of this entry, you would be spitting-in-your-soup wrong. They've saved themselves a pocketful of change by using the free version of CyberChair (not a great conference manager even in its commercial version, but that's a story for another day), but in the process they've bought the program committee a right-royal headache.

Let's take the bidding interface. I would love to show you some screen-shots, but I can't do it without violating the confidentiality of the authors (or my own bids). This makes me sad, because words cannot do justice to how bad this program is. (I tried to include a redacted version of the HTML here, but Blogger will not accept a frameset tag, more's the pity. But in the process of generating this I had a look at the page source. Not only do many of the tags not nest, there are even closing tags with no corresponding opening tag. Wonderful Perl code, circa 1997.)

The idea is that you bid on all the submitted papers to indicate whether or not you'd like to review them. The bidding interface consists of two frames (yes, frames, in 2007). The left frame is some indifferent text, while the right frame is a long vertical list of all the papers, consisting of a paper number and a drop-down where you select your preference. That's right, paper number: like you know or care.

In the left frame you can select the list of all paper titles, with or without abstracts. So now you have two lists of 419 entries each...in two different frames. Guess who is responsible for keeping them in synch while scrolling through each one independently?

Let's say you've made a dozen bids. You'd hate to lose them because you accidentally killed your browser or something like that. But is there a way of saving bids? No, not unless you scroll down another 407 entries to find a save button (which is cleverly disguised under the name Update Bids, which has exactly the wrong meaning). And if you do save, you'll have to scroll your way all the way back up. (I later found that there's another copy of this button at the very top, which still leaves open a Texas-sized wasteland for the middle 395 papers.)

By the way, be sure to click on the correct button. For reasons that beggar comprehension, the system also offers a Reset button which, presumably, restores the defaults—not, I think, because there is any good reason for it (there absolutely is not), but solely because HTML offers a way of doing this. This is brain-dead user-interface design. And it's about to get worse.

For each paper you can enter a bid, which is an indication of how much you'd like to review it. There's a special category for papers with which you have a conflict-of-interest (a standard term-of-art in this area). Concretely, the bidding categories are that you'd love to review it (A), that you're willing to do so (B), that you'd rather not do so (C), or that you're conflicted with the paper (D). Now which letter would you think goes with Conflict?

As you're looking down the list of all papers (left frame), you might notice that there are two little columns labeled A and C. These are the counts of how many people have already bid A or C on that paper. Let me explain why this is about the stupidest information to provide. You want people to bid bidding independently of each other; you don't want someone to look at a paper and think, “Hmm, I could go either way on this; oh look, we've had ten people bid C and nobody bid A, which suggests nobody really wants to read this paper...I'd be crazy to get stuck with it, so I'll go with C”. I can't imagine a system better designed to make life maximally hard for a program chair.

Anyway, after well over two hours of this last night, I submitted my bids (my hand poised shakily over Update Bids, fearful of what would happen when I clicked). I then sent mail to Kathi, who is also on the committee, warning her to look for any other interface than the list-of-all-papers I'd just been through.

Happily, there is another interface: you can select just an area at a time and view all the papers in that area. Kathi happily marched off to use that, little knowing her life would be worse than mine.

First, the list of papers in an area only changes your view in the left frame. The right frame is still the list of all papers. So if you pick an area and its papers are paper numbers 11, 17, 89, 103, and so forth, guess how you enter your bids for those papers?

Second, you can't choose a single bid for all the papers in an area. That means it's not enough that you don't want to read a single paper on the Semantic Web; you have to find every paper in the category and manually decline (C) each of them. (Hmm; perhaps an Ontology would be able to help with that.)

Third, curiously, the list of papers does not include the A and C columns. In other words, they didn't reuse code! The good software engineering practices just keep on coming.

Finally, many papers fall under multiple categories. But if you bid on paper 11 under one category, it still shows up in sorted numeric order in all its other categories, without even indicating that you've already bid on it. (In other words, you would definitely see far more than 419 papers by the time you're done.)

This is bloody-minded software development in the extreme, a process of mindless, thoughtless, soulless application of algorithm to data structure. It is difficult to imagine that a living, breathing combination of human sinew, blood and fiber could have created an interface so awful.

The real moral, though, lest we lose sight of it, is the waste of it all. There are 34 regular program committee members, 16 people on the “Expert-Review Panel” (including Kathi and me), and two program chairs. Suppose each of us wastes just two hours of our time because of bad software design (seeing as I believe I've alredy wasted one hour, the odds of this are fairly high). Some of these people are fancy consultants, so on average say our time is worth USD 100/hour. That's a collective USD 10,400 just in terms of wasted billable time (never mind the frustration, the lost goodwill, etc). That's far more than the cost of a better conference management package. What does that tell you about what the committee thinks of our time? I can't wait to meet one of these people and hear them complain at the conference about the popular unwillingness to pay for quality software.

Maybe I should send them a bill. Maybe we should all send them bills, so they would come to their senses.