Materials and Methods

A couple of days ago, I endorsed Trisha Greenhalgh's advice that you can safely ignore any scientific paper which has a flawed Materials & Methods section.  So M&M is important: it's a record of what you used and how you used it with sufficient detail for someone else to replicate your work and see if they get the same results.  The Economist just recently had an intelligent lay-person's view of the worrying findings by Begley and Ellis, John Ioannidis and others of a widespread failure to replicate sexy, high-impact papers.  The failure in replication is probably because the original findings were cutting edge, novel, of substantive biomedical importance and . . . wrong.  There's a whole supplement in Nature on the matter from April this year which appears to be open access.  And a chunky article in The Atlantic about Ioannidis.  So nobody can claim that they know nothing about these issues and that SS Science is steaming ahead without an iceberg in sight.

In The Institute, we're training the next generation of scientists and one of the ways we are doing this is insisting that each student keeps a lab-book for each course and writes up each experiment or investigation in the particular and peculiar style in which scientists keep their records.  Teachers of science will each have a particular bee in the bonnet about some aspect of these reports.  Me, I get all hot and bothered about people writing down spuriously accurate numbers which have been lifted wholesale from their calculators.  Some deprecate the first person and want everything written up in the passive voice.  Others want students to write micrometer as μm, not as um which is far easier to find on the keyboard and, in context, is unmistakable.  There are excellent reasons for insisting about each of these elements of style.  If you inculcate correct, professional looking reports early in the career, it will pay off later, so we should probably be a bit dogmatic for absolute beginners even if we let the older students have some slack.  

So I told my two 3rd year classes that I didn't see any point in them transcribing the M&M for each experiment from the nicely typed and neatly typeset manual into a crabby scrawl of ball-point pen in their lab-books.  For the first couple of weeks, several in each class couldn't shake the habit and, accordingly took 20 minutes longer on the writing-up than their peers who wrote "Materials and Methods - see page 28-29 of the Manual".  On the third week, I blew up because one of my students had found a gross typo in the M&M (a row of figures which didn't add up correctly in a table describing the volumes required to make up a particular concoction).  Is there, I asked, any value in transcribing this stuff with all these errors and internal inconsistencies without any sort of processing by the higher mental centres?  Are you, I added, human being or automata?  This is the second or third year that the manual has been printed and used by a couple of hundred students and a good handful of lecturers and all of them have dutifully followed the protocol and written it up without any of them noticing that one of their test-tubes has short measure. Typos are forgivable - I keep finding them in The Blob - but I try to make sure that tables are internally consistent, that the row and column totals sum correctly.  Life is much easier now with spreadsheets to keep track of these things.