The stunt will be familiar to most: a petition calling for urgent regulation of a widespread and potentially dangerous chemical known as dihydrogen monoxide (DHMO). It’s a greenhouse gas and the primary component in acid rain. In some cases skin contact with DHMO causes serious burns. Will you support a ban on DHMO?

It’s no secret that ‘DHMO’ is water. Nonetheless, TV and YouTube pranksters continue to find new signatories to DHMO petitions. Stunts like this are held up as anecdotal evidence for the sorry state of scientific literacy in the general population. While these pranks are nothing more than (barely) amusing stunts, they do typify an enduring concern that people don’t know enough basic science.

Many surveys attest to the public’s seemingly lamentable levels of scientific knowledge. A 2010 survey by the National Science Foundation in the U.S found that just 52% of respondents correctly answered that the Earth takes one year to orbit the sun. A similar proportion wrongly agreed that antibiotics were effective against viruses. While performance on individual questions has varied, the average total score has hardly changed since the survey was first administered in the late 1980s. The results of a nationwide survey released by the Australian Academy of Science last week tell a similar story.

Surveys designed to measure scientific knowledge can have serious flaws. Consider a statement found in one such survey: “sunlight can cause skin cancer.” Is this statement true or false? This may depend on what you understand by the term ‘sunlight’. While exposure to UVA and UVB radiation increases the risk of skin cancer, exposure to the visible component of sunlight does not. It might sound like nit-picking, but the ambiguity of statements like this means that even (or perhaps especially) someone who knows the science can get the question ‘wrong’.

A study by ANU researchers Sue Stocklmayer and Chris Bryant found that even scientists answer these questions ‘incorrectly’. Scientists actually performed worse than the general public on two questions and in other cases they expressed disapproval and uncertainty. Findings like this cast serious doubt on the utility of knowledge surveys.

It’s not just a matter of improving survey instruments either. Such surveys operate on the assumption that measuring public science knowledge is indeed useful. They are guided at least in part by what has been termed the ‘deficit model,’ a view that supplying people with more science information will lead to greater support for scientific research and resolve debates on scientific issues. A typical deficit argument reads something like this: If people understood more about climate science then they would support action on climate change. Research suggests that this is not the case.

In a study of climate change attitudes in the UK, Cardiff University researcher Lorraine Whitmarsh found that educational attainment and self-reported knowledge were not associated with climate change scepticism. Instead, scepticism was strongly predicted by political and environmental values. Similarly, Yale University researcher Dan Kahan and his colleagues found that climate change attitudes in the U.S were most polarised among respondents with the highest scores for numeracy and general science knowledge. In each case the message is clear: it’s not just what you know but what (and who) you value.

Albert Einstein also seemed to recognise that scientific debates may not be about science at all. In a letter to mathematician Marcel Grossmann, Einstein famously remarked that “every coachman and every waiter is debating whether relativity theory is correct. Belief in this matter depends on political affiliation.”

Improving general science knowledge is not a panacea for socio-scientific issues and debates. Of course, recognising a disconnect between knowledge and attitudes doesn’t make scientific knowledge redundant – far from it. What the research tells us, however, is that improving scientific understanding will only ever be part of a much larger conversation about values and relationships. Now that’s something worth knowing.