Wednesday 27 May 2020

Law of small numbers

Well I'm very sorry to report that the Corona Contentment and Coping with Cocooning survey coordinated by NUIG has been shuttered for lack of interest. It launched at the beginning of April and got a very respectable Citizen Science response. More than 100,000 residents in Ireland were prepared to share how they were coping with weirding weeks of lockdown. That's important. The Man needs to know the impact of his actions, 100,000 people is data which can provide evidence to inform policy.  Without evidence, policy is shaped by grass skirts and maracas, nameless fears and the weaselling of lobbyists. It was a bit of a worry that 63% of the respondents had a university degree, which far outweighed that sort of person in the general population where about 63% of ordinary folks don't have a degree. Insofar as a degree = a government job / currency trader and "working" from home on full salary, that will bias the results towards the happy end of the scale.

Also the median age of respondents was 48, full ten years higher than the median age for Ireland. The 2nd survey a fortnight later, 'they' uttered a call for more young people to respond - because they were more likely to adversely affected and/or just to capture the whole demographic. I agreed and sent a call-to-contribute to all the students [N = 100] I taught this year. The surveyors were hoping to top out the 100k sample size as word got out but actually secured a miserable fall off to about 1/3 of the original numbers. Survey fatigue, I guess. There were some interesting and important results from that survey about the negative impact of putting all our resources in the Corona-basket.

Third survey went out on 6th May, I responded in the same relentlessly upbeat fashion as before, and before.
Anxiety? Poof!
Conversation? For/with the birds!
Dry cough, fever? Nada, nunca!
Indeed I was so busy being Corona-content that I didn't even look at the results of Survey III. Yet I was fully expecting there to be a Survey IV on Weds 20th May; but when I got there it was like the Mary Celeste: echoing corridors and empty rooms. This worthy Citizen Science project had been closed down. No adequate explanation but the numbers told the story. Only 8,700 people had answered the call to Survey III another 2/3rds collapse in responses. Even those small numbers are data to a wonk like me and I plotted them out to see whither they extrapolated for Survey IV.  It looks like [below] an exponential decay which is much easier to handle if you transform the data to a logarithmic scale.

Than strongly suggests that Survey IV would garner only 2,300 responses. Although that is still twice what Gallup will predict the results of an election form, it is worrying small for data anonymity.

What fresh hell is this? We are assured that the survey couldn't be tracked back to us by The Man. Nobody wants to be taken away in shackles because they spoke to the postman in the previous three days and neither of you were wearing an Acme Virucidal Mask © ® €8.99 from Varadkar Holdings. But even at N = 8,700 that assurance is far from being evidence based:
65 yo
56 K
1,347 K
I R 65 and identify as he/him/his, but I was also asked where I was calling from which is a micro-county in the Irish midlands. All other things being equal only ~100 people responded to the 3rd survey from my home county in which only about 0.5% are 65 years old. Things aren't equal because the survey was bought into much more in Dublin that down the country. I am, to the nearest whole number (and it's hard to get fractional people without Madame Guillotine) the only person my age in the whole county and it will be a cinch, for example, to see if I am responding consistently between surveys . . . not least because they asked if we'd completed either of the two previous surveys. In the not-happened Survey IV individual people of pretty much any age would be have identifiable whatever county they hail from.

It don't make nobody no differ if one identifiable 65 y.o. is happier than the old bugger deserves to be. But similar arguments attach to anonymous DNA samples [Science paper 2018] which everyone and his dog is submitting to 23andMe and So many people (<2% of those who identify-as-white in North America) have uploaded their genetic data, anonymised to state or county, that it will be a small hack to find the address of an identify-as-jewish identify-as-bloke 60 y.o. in Baltimore who has a predisposition to stroke and Alzheimer's and is thus a poor risk for a medical insurance company. Even if he didn't send his spit into 23andMe, his second cousin in Arizona might have done. And if you're minded to make an anonymous semen donation for your neighbourhood fertility clinic; forgettit, your offspring will find you and want a part of your estate. Every sperm is sacred!

No comments:

Post a Comment