SoftestPawn’s Weblog

Oh no, not another blog!

Peer Review in the Real World

Posted by softestpawn on August 10, 2008

Some academics seem to think peer review is a suitable way to establish knowledge, with a nod to “it’s not perfect”. But imagine applying it in the real world:

“We need to know more about Wodgets to feed the Widget project”

“OK, so what’s the plan for the investigation?”

“We’ll I’m going to get the team together, explain the problem and let them get on with it”

“Heh. So, how are you going to monitor progress?”

“Oh we’ll have a regular review document where they can publish what they’ve found out. It’ll need to be fairly brief, we don’t want everyone wasting time reading about everyone else’s work. We’ll get colleagues in the same field to select them by how interesting they are, but I don’t think we need to enforce any other requirements. It’ll take too long to see if it’s correct for example, no point in duplicating effort. And we’ll give credits to those who get articles published that will contribute to promotion. They’re all professionals, it won’t affect the areas they work in, the results, or team relationships at all”.

“Heh. Ha ha. What then?”

“Oh nothing extra. We’ll just keep doing that. If people read the review documents they might get around to critically examining some of the articles, but there’s no real need as it’ll get in the way of their work. If someone does happen to find something wrong, they can write an article for the next review document, and if it gets published then everyone who happens to read it will know about the correction. If they notice this affects some other work they can update that when they get around to it, and see about publishing that correction and so on”

“Ha ha ha ” (Collapses in gales of laughter). “And how will we know when we’ve established a reasonable knowledge base of Wodgets?”

“Ah when experts – those who have lots of credits – feel that the articles in the journals all say roughly the same thing, they’ll tell us that they all agree with how a Wodget works. We might see if some researchers on the Wadget project can drop in occasionally and agree with them.”

“Hoo brilliant! That’s fantastic!” (wipes away tears, snot and the coffee sprayed over the table) “Anyway what are you really going to do?”

“Follow the department’s standard quality process … ”

“Oh how dull. Well carry on. Keep me informed as usual”


Postscript: Academics are working on it


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: