I was digging through a pile of old Natures, still in their mail-wrappers since 2014! In the 4th December 2014 issue, there's an article and a News&Views on the cultural dishonesty of bankers. It had a certain resonance because I've just written a bit about codes-of-conduct whereby a group of people formally write down what they stand for and what is beyond acceptable limits. The primary research was carried out by a trio from the Dept Economics at the University of Zurich. It's the sort of study that has a good chance of making the cut at the weekly editorial meeting at Nature. As a general science publication, they have to include some copy that is accessible to a wide tranche of their readers - which is a very broad church from Astrophysics to Zoology. A neat little paper which plays on what everyone knows to be true - either to confirm our bias or to counter-intuitively rubbish it - has a chance.
It's a psychometrical study, such as economic researchers have been doing both before and after Freakonomics was published. They recruited 128 bankers from a multinational financial institution and split the sample into two groups: experimental and control. Each group was asked a series of questions and then put in a closed room and asked to toss a coin 10x and report how many heads came up. To encourage them to be bothered with such a trifling task the researchers incentivised everyone by saying they would be given $20 for every H that came up . . . provided that the total was more than the average score of a [fictional] pilot study. The punters were on a honour system to report the result accurately. The experimental group's priming/prior questionnaire included questions about their work and work-practice in the banking industry while the control group were asked more neutral questions about TV and soccer. Here are the results:
The study was very widely picked up and reported by the world media: Les banquiers, tricheurs par culture (Le Monde); Banks breeding dishonesty (Sydney Morning Herald); Folk med dette job snyder mere end andre (avisen.dk); Bankers think they have to behave badly (Grauniad). This reportage will doubtless have pleased the publicity department at U.Zurich and launched young Alain Cohn's career and Nature can be a bit of a whore for news-coverage, so they're happy. And everyone who likes to put the boot in for bankers, which is pretty much everyone, is happy because their prejudice is confirmed.
But reflect. In science we use 1/20 or p<0.05 as a purely conventional level of statistical significance. If the chance of getting your result by chance is less than 1/20 then you may be on to something. But the corollary is that for every 20 such studies you carry out you can expect one result to be that extreme. p=0.033 would certainly not be enough to convince your line manager that the company should invest $1 billion in developing a new cure for cancer based on your weeny pilot-study N=61+67 experimental result. If you were Brian Nosek you'd make the graduate student do the whole thing again in a different bank before rushing such a marginal result into print. Or you could ask my friend Tony, who is a leader in his field because he has a good crap-detector to say v e r y s l o w l y "extraordinary claims require extraordinary levels of proof".
Explanatory note for those not brought up in Greater London (the entire readership?): Porkie is from cockney rhyming slang: pork pie = lie. Similarly tit for tat = hat; so a hat comes a titfer.
Post a Comment