Since I've been doing research related to human subjects (specifically, evaluating user expectations in computer security), I needed to complete Brown's IRB (Institutional Review Board) certification. The certification course and exam is conducted by CITI, an organization that runs these services for a host of institutions. Presumably they charge a reasonable sum for their service.
I think IRB is a valuable practice. Studies with human subjects are fraught with difficulty; it's not so much that even seemingly innocent projects can expose subjects to risks (though they can), so much as the very act of forcing experimenters to think about these issues is valuable, and can help them re-think their study to be less intrusive, risky or harmful. Of course, there is still a slightly surreal air to this kind of training, which is meant to apply to everyone, including those who work with children, with prisoners, the whole lot. But let's just assume that's the common case (even though I don't believe that), so it's important everyone goes through training in all those aspects.
The bigger issue I have is with the kind of reductionist tests that accompany such training. I went into the (on-line) course hoping for an interesting, educational experience; what I got was a quick return to high-school history class, an absurd exercise in rote memorization and the recitation of slogans. Surely this could be tested in an interesting way, even on the Web (more on that in a bit), but you got the sense they weren't even trying.
It reminded me of not only high school history but also of driving tests. Every driving test has a de rigeur question about blood alcohol or about drunk driving penalties (but not—and this is a pet peeve—about the rights of bicyclists). But the answers are not instructive; instead, they're petty. Suppose the penalty for drunk driving is three months in jail; the options are never (1) three days, (2) three months or (3) three years, answers with an order of magnitude difference. They will, instead, be (1) one month, (2) two months, or (3) three months. You, reading the rules, and being the kind of person who is too sensible to drive drunk in the first place, probably thought “Okay, so the penalty is some number of months”, figuring you'd captured the high-order bit; instead you'd only captured the low-order one.
You could design a smart test. You could put people in situations and ask them what course of action they would take. Sometimes, the reasonable course of action would prove to be the wrong one according to the law, and understanding that difference would be instructive. But designing such a test requires the designer to actually understand testing, which is (guess what) a subtle and rare talent. A good tester, for instance, understands that most questions should be meaningful to administer even in an open-book exam. And so on.
Well, the CITI test was a lot like a typical computerized driving test. I had to work very hard at memorizing key phrases on the assumption that they might show up later (and some did). Only one question that I got wrong was actually instructive. And then I got to a question along these lines (actual wording not reproduced, so that the CITI doesn't come after me):
The purpose of SSL is to secure data:
- True
- False
Well, I thought! A question right in my wheelhouse! Here was an eminently debatable proposition...and then I remembered this exchange from one of my favorite Simpsons episode (cribbed from here):
Proctor: What was the cause of the Civil War?
Apu: Actually, there were numerous causes. Aside from the obvious schism between abolitionists and anti-abolitionists, economic factors both domestic and inter—
Proctor: Hey, hey... just say slavery.
Apu: Slavery it is, sir.
So I played it straight.
And now I'm certified.
Posts
