The NSF’s Science and Engineering Indicators report came out not too long ago, and the bulk of it is, as usual, spent on quasi-quantitative measures of scientific productivity– numbers of degrees granted, numbers of patent applications for various countries, etc. I find all of those things pretty deeply flawed, so I tend to skip past them and go straight to the stuff about public knowledge and understanding (chapter 7, available as a PDF at the link above).
This doesn’t get much press, probably because the results are depressing. They’ve asked a bunch of factual knowledge questions of people every so often for going on thirty years now, and the results are essentially the same year in and year out. There are two questions in particular that I like to look at, and because I am a shameless traffic whore, I’ll reproduce them here so we can see whether people who read science blogs are smarter than the general public:
These are both true/false questions of a factual nature, so there are no silly options added. I also tend to think that they’re pretty obvious. But what do the survey results show?
Here’s a clip from the results graph in the report, comparing various countries:
The type is a little small, but the graph on the upper left shows the percentage of people in various countries who correctly answer the true-false question “Lasers work by focussing sound waves” and the graph on the lower right is the percentage correctly answering the true-false question “Electrons are smaller than atoms.”
49% of Americans correctly answered the laser question, and 53% of Americans correctly answered the electron question. That’s roughly what you would expect if people tossed a coin to determine the answer.
And here’s the depressing thing: we lead the world in correct responses to those questions. The Europeans are close (and might actually edge us out, though we’d be blowing them away if not for the huge gender split on the laser question– only 34% of women got that right. Come on, women, you’re letting down the side!). I have no idea what the hell they’re doing in Japan, China, and South Korea, where 30% or less of the population got these right.
These aren’t hard questions, these aren’t questions whose answers offend moneyed interests, these aren’t questions with any religious component. Oil companies aren’t running slick ad campaigns to confuse the issue of whether lasers use light or sound, and preachers are not pushing Large Electron Theory on their congregations. And still, a coin toss would beat most of the world’s citizens in answering these.
This is, to put it bluntly, pretty pathetic. And it’s been consistent over decades. The table showing the time series in this report doesn’t go back past 2001, but I’ve seen earlier versions that went back to 1988 or 1982, and the results are the same.
On the bright side, of course, the results haven’t gotten any worse, despite well-funded and politically connected attempts to weaken science education across the board. That’s pretty small comfort, though.
The fundamental problem here, in my opinion, is that we live in a society where it is considered perfectly ok to not know anything at all about scientific issues. People ought to be embarrassed not to know the answers to these questions, and yet around half of the population feel fine exposing themselves as idiots when asked a survey question. (I would never be able to collect data for this, because I don’t think I’d be able to hide my contempt.) This isn’t limited to one end of the political spectrum– I’ve heard leftist college professors say mind-bogglingly silly things about science– or one side of the science and religion conflict. Across the board, people know next to nothing about science, and are perfectly content with the fact.
The usual response to this complaint is “These are really just trivia questions. What matters is the process, not the facts.” I disagree that these are insignificant trivia– the idea that electrons orbit the nucleus of atoms is pretty central to modern physics and chemistry– but this year’s report included a bunch of new questions about the scientific process. The results are in Table 7-6 and 7-7, and aren’t really any better than the factual knowledge questions. It’ll take several years of these questions (and, ideally, asking them of international populations) to see if this provides any hope, but the initial results are not too encouraging. (40% of US students think watering crops with seawater would be a good idea. If you want proof that we’re no longer an agricultural society, that’s it…)
Much to my chagrin, I have to confess to erroneously answering the electron question. I had “false” in my mind from answering the laser question. Sigh, oh well.
Plants crave electrolytes!
It doesn’t help that news stories jump from either desalination or developing more salt-tolerant crops to “so we don’t need to worry about a shortage of fresh water.” Some of those students are saying we can water crops with seawater because they’re reading or seeing science/tech “news” that suggests we’ll be able to do that by the next growing season.
These are the type of questionnaires that are always so difficult to make inferences with. Looking at the results caused me to do a preliminary victory dance and shout, “Ha! Americans aren’t as dumb as everyone says!” However, on closer reflection I would argue that those who answered from the US actually did worse than the other countries. Assuming that they polled a large number of people, we would expect roughly a 50/50 split for any group which is making random guesses, which is the result the US had. On the other hand, some other countries clearly had a a bias towards one of the two answers. Perhaps these countries have a bit more knowledge that the US but had a tendency to conflate the notion of “light waves” with “sound waves”, or was making some other fundamental mistake. If that were the case it certainly would be the wrong answer but in my mind would demonstrate a better knowledge in physics than “I dunno, I’ll guess A?”
I may know too much about this stuff: on the first poll, I sat there for a while wondering whether anyone had ever tried to pump a laser cavity with ultrasound-induced cavitation or something like that.
Are you serious? Do you think the correct answer to these things helps anyone in their daily lives? Understanding these things is more important that understanding good finance practices, U.S. Civics and History or proper child care and social skills?
Get out of your ivory tower and see what life skills are truly essential when you have two jobs and children demanding your time… your outrage seems ridiculous and does nothing to dispel the tweed jacketed, educated elitist stereotype.
Slight mistake: the graph on the left of that screenshot is labeled with “Antibiotics kill viruses as well as bacteria. (False)”, not the electron question.
Phaedrus, given the number of lasers and the common presence of electronics in Western daily life, I’d say “yes.” Given that this is “scienceblogs.com”, I agree that there’s a bias towards viewing science education as important… but looking at the importance of science and technology to modern daily life I can’t say it’s unwarranted.
Besides, I understand that testing shows students are no better informed about good finance practices, US Civics and History, and proper child care and social skills as they are in science… though I don’t have a citation, so I could be wrong on that.
— Steve
In what context have these questions been posted internationally? Were they in the participant’s native language in all cases? Hmmm. I smell an American monolingual rat somewhere. @Skribb Absolutely correct, a closer vicinity to 50% only suggests a clear case of randomness. Is it not interesting how America is openly losing their grip on their leading position in an international society. This post only strengthens this notion. While pondering the size of an electron, the East is dewesternising (yes, UK spelling) at an astonishing rate, and preparing to take the wheel. Wake up, and start practising Ni Hao. Chad, you silly man.
For the most part, this kind of thing doesn’t really bother me. What fraction of scientists understand double entry bookkeeping? Or can say what common law is? Or answer any of hundreds of other really basic questions about modern civilization? My guess based on admittedly anecdotal evidence is that a large fraction of scientists do know the answer to these questions, but a large fraction also do not. Yet these sorts of questions are at least as important as whether lasers use sound or light.
Well just how do YOU mode lock YOUR lasers, mister smarty pants? 😉
As a physicist, I’m inclined to think too hard about the electron question. The fact is the electron has a smaller scattering cross-section than an atom but it doesn’t have a fixed “size” any more than a whirlpool eddy has a size; its influence simply falls off with distance. Moreover, the “size” of an atom is mostly the size of the electron cloud that surrounds the nucleus.
Re: #4
“Assuming that they polled a large number of people, we would expect roughly a 50/50 split for any group which is making random guesses, which is the result the US had.”
Your entire premise is severely flawed. Are you arguing that everyone in the US poll guessed and that no one knew the correct answer? Really? Because that is what would have to happen to get it completely random – and it would mean no scientists, engineers, or almost anyone that took physics in college (or high school) got it right. (i.e. not a random sample) Hard to believe. And to guess a language barrier without proof doesn’t hold water. The truth is, this isn’t that surprising a result because the university system in the US is by far the best in the world with the most access for its citizens.
And it is an important question, for all those who think it doesn’t matter how lasers work. Its a modern world with a modern, technical economy. These things matter to people with jobs and children at least as much as good finance practices. Now, civics questions are important too, and remember, scientists have had a college education too, so it can be kind of insulting to say that they don’t know anything other than science (without data anyway, then I might believe it).
Slight mistake: the graph on the left of that screenshot is labeled with “Antibiotics kill viruses as well as bacteria. (False)”, not the electron question.
Shit. You’re right. In addition to getting old (next post), I’m going blind, and misread the tiny little type.
For the most part, this kind of thing doesn’t really bother me. What fraction of scientists understand double entry bookkeeping? Or can say what common law is? Or answer any of hundreds of other really basic questions about modern civilization? My guess based on admittedly anecdotal evidence is that a large fraction of scientists do know the answer to these questions, but a large fraction also do not. Yet these sorts of questions are at least as important as whether lasers use sound or light.
I disagree, not because I think the laser question is intrinsically important, but because it also shows a breakdown of reasoning ability. It’s a rare person in this day and age who hasn’t at least seen a laser, even on tv, and the fact that they produce a visible spot should lead you to conclude that they use light not sound, given that you rarely if ever see sound waves.
The electron one is a better example of the problem. You don’t need to know that experiments show the electron is a point particle (or near enough as makes no difference). If you have even the most rudimentary idea of the atoms-as-solar-system model, you should be able to work out that electrons must be smaller than atoms.
And I would argue that the basics of the atomic model are important information. People should know that matter is made of atoms, which contain electrons, and combine to form molecules. That’s the backbone of pretty much our entire understanding of chemistry and biology, not to mention a key starting point for much of physics. Trillions of dollars of technology are built off understanding the atomic model of matter, and people who have the right to vote on issues relating to technology and science policy should at least have that minimal level of knowledge.
Feynman famously said that “matter is made of atoms” is the one fact he would bequeath to a future society that needed to reconstruct modern technology. I think the least that members of modern society can do is to know the facts in that sentence.
Yes, and Brawndo’s got what plants crave!
Re #13
I apologize if what I meant wasn’t conveyed very well in what I said. What I am suggesting is that the more people you have guessing (to be more specific the larger the ratio is of guessers to people who gave it some thought) the more your results will tend towards a 50% success rate. Say I flip a coin 100 times and I use a fair coin in 95 of the flips and a trick coin (always comes up heads) 5 times. Despite the fact that the five trick coins will contribute to the average, the average will still tend to be around 50%. I’m not suggesting that all Americans guess at physics questions, but that the ratio of Guessers to Thinkers could be higher in the US than in the other countries.
Please also note that I’m not suggesting that this is what is going on. I’m merely outlining one of the possible problems with coming to a conclusion with this data. Orzel was suggesting that the best results of the bunch was a ~50% success rate, but as I tried to explain above, the U.S. result around 50% could actually indicate a very poor result.
I don’t know where I said that there was a language barrier anywhere in my previous post and I’m not sure why you are accusing me of having said such a thing. I did suggest that some people might be conflating EM radiation with sound waves based on an ignorance of the distinction, but I wasn’t suggesting that this was a language barrier.
Consider, for example, radio waves and visible light. The two things are fundamentally the same phenomenon and I have seen and had students who, after learning this fact, asked the question “So are sound waves a form of electro magnetic radiation”. What I think is going on this that they recognize we have created some sort of unified theory involving waves and that the things that we call waves are given that name because they are fundamentally related in the same way. Of course there is a major distinction between sound waves and Radio/Micro/Visible/IR/UV etc but this isn’t always immediately apparent.
Perhaps even more alarming than ignorance of basic physical principles is a general overconfidence in unsupported claims that might guide our daily actions or voting patterns– e.g., that children who have been abused or neglected routinely grow up to become criminals, that it’s important to get trauma victims to talk about their experiences, that it’s necessary to cleanse the body of toxins, etc. ad infinitum.
A leftist professor (see above)
This scientist, at least, understands the basic principle both of double-entry bookkeeping and of the common law/legislative law distinction. As a general rule, scientists are quite well informed on these things, because they are just plain well-read and intelligent compared to the general population.
That said, the electron question is kind of silly; an electron can reasonably be considered to have any size you like, from radius zero up to the size of the Universe. The correct response, which alas does not appear on the poll, is “Mu”.
Figure mix-up corrected. I was overzealous in my cropping of the original pic, but have caught the right graphs this time.
But wait – if you count the extent of the “wave function”, then a free electron might be “bigger” than an atom … heh.
But the questions asked about science here would also have been covered in even basic level science classes that any highschool (and indeed, most middle school) graduates would know. On the other hand, it took until I got to the AP level accounting course before we covered double entry bookkeeping, and my elective law class before we discussed common law.
Thanks for posting this. I’ll have to make one of the “outcomes” goals for my summer gen-ed class that 100% know those two.
So knowledge of basic scientfic facts is poor. Knowledge of the scientific method is more important (that science works by making claims which can be independently and repeatedly tested and possibly confirmed). Also, I’d rather learn more about the connection people make between basic science and improvements in their lives. Or the principle that double the car’s speed means four times the distance to stop.
I know that an electron is smaller than an atom. I even know that quantum mechanics can make that question seem a bit uninformed. I have my limits: shown a variety of antibiotics that could potentially treat an infection, I didn’t know which were penicillin-related and which weren’t. That are families of antibiotics I didn’t know about! So there will be always be facts not known. I think you’re overweighting the significance of fact memorisation, which is less and less important the more someone knows how to use Google. So it’s not just the number of patents which doesn’t tell you much.
I don’t like the biology question — it’s sloppy writing to say that genes “decide” anything. The father’s genes don’t decide; they don’t have minds. I’d probably say false in annoyance. It’s like saying that electrons can choose whether to be particles or waves. You know, depending on their mood.
Maybe the rest of the world is more sarcastic than Americans.
I don’t agree with the people who argue that these facts are unimportant trivia that only scientists should be able to know; I’m a high school senior and I know this is all already old hat to any high school student. We learned about lasers in elementary school, and they show up all the time on TV and in movies. The structure of a atom (at least the Bohr model) is covered in basic junior high science.
I guess you could account for people forgetting what they learned in high school, but all the same…this is basic, and it gets rehashed every year. Especially the structure of atoms; we devoted significant time to that in both physics and chemistry classes, hell, even biology.
Granted this could say something about science education in general. I know that my school district only mandates 2 years of science in high school, so I guess if you’re a student who just wants to get over with your science requirement as quickly as possible in freshman and sophomore year, you probably wouldn’t retain much of what you learned.
“I don’t like the biology question — it’s sloppy writing to say that genes ‘decide’ anything. The father’s genes don’t decide; they don’t have minds.”
Yeah, it’s sort of like saying computers add numbers when they do no such thing. They manipulate electrons, and we interpret the results of that manipulation as addition.
The fact that the US is actually beating Japan/Russia/South Korea on basic science lit is counter to my previous belief. On the other hand, the statistical curiosity of multiple participants doing worse than coin-flip makes me wonder about the thought processes involved.
The persistence of the phenomenon makes wonder about our methods of attack; though the fact that it has persisted without notable dire consequences suggests that perhaps altering our approach is not necessarily a productive choice.
(When I come across that many short, clear, new thoughts from a report? This is a good blog post.)
Okay, smarty pants. 🙂 Here’s a trick question for you.
“Lasers work by focusing light waves.” True, or False?
Why don’t you point a laser pointer at your eye and turn it on?(0) Since it works by focusing sound waves, it should be harmless. But don’t point it at your ear.
(0) Please do not point a laser pointer at your eye. The light from it can damage your retina.
“49% of Americans correctly answered the laser question, and 53% of Americans correctly answered the electron question.” That looks more like 63% on the graph to me. I couldn’t find the raw figures in the PDF file you mention – did they print the graph wrong, or did you read it wrong?
I’m generally depressed about surveys like this. But then, how do you reckon people would do on a test which included questions like “How did Anna Karenina die?” I’ve always found it instructive to watch “University Challenge” (“College Bowl” to you), and see the incredulity of the presenter when the scientists try to guess the answers to “obvious” arts questions. Especially when I thought their guess was more credible than mine!
We just live in a world where there’s too much to learn, certainly too much to learn in ten or twelve years of full time education, and not enough of it immediately and obviously impacts on one’s survival chances. As a result people (scientists, artists, and cannon fodder alike) will tend to learn only that which they find “interesting”.
What really puzzles me is not that only 64% of men got the laser question right, but that only 34% of women did. How could they do so much worse than random?
No, the equivalent legal question would be ‘Judges are smaller than courtrooms: true or false”, or for arts, “Bach was a painter: true or false”. People know what a violin is, or a judge, but they don’t necessarily have even the most basic idea of the simplest science concepts.
As a science teacher, I work with many teachers and administrators with backgrounds outside of science. I never test them for their knowledge of “trivia” from my subject, and I concede that knowing some of these facts may not be essential to their daily lives. (I also recognize that they know more about their fields than I do.) However, I am amazed at how my coworkers react to science. When my first principal would ask what I was teaching, her reaction would often be “I’m not a science person; I don’t understand.”
I don’t expect everyone to love science, but the strong aversion to the subject I see in many people is troubling. What happens when this person has to make a health decision? Can they look past the pseudoscience in advertising and make good choices as consumers? Can they fully participate in questions about policy regarding science and technology such as stem cell research, environmental issues, genetic testing or engineering?
I will continue teaching, not so that my students can recite facts, but with the primary goal of making them comfortable with science. At the same time, when our population is more comfortable with science and sees its usefulness, I believe that more of us will be able to correctly answer basic science questions.
The responses to the laser question from the US population really makes me think that there is something wrong with the survey or, at least, this question. Lasers are so ubiquitous in our society, I really doubt that anyone thinks the laser beam is composed of sound. So, my suspicion is that the people that were answering this question know that laser beams are light but were guessing as to how that light was produced. People know that laser light is different from regular light (but probably don’t know the details), and they know that regular light is not produced by ‘focusing sound waves’. So question itself primes people to guess that maybe sound waves are why laser light is different than regular light. In other words, perhaps the problem has more to do with test taking abilities than with knowledge about laser light.
The truth is most people do not retain or acquire knowledge they do not use in day to day circumstances. Yes, people use lasers and electronics everyday, but they also live in frame houses and drive cars and still have no idea how those are built or work. Even in more mundane things like these most people (include academics here) feel there are specialists to call on when things need to be fixed and so they need not concern themselves with them. I remember how amazed a group of geochemists were when they discovered I knew how to do rough carpentry.
IMHO This does not excuse scientific illiteracy in general, but instead reflects how isolated and overspecialized we’ve become.
What I find pretty interesting is that more people in China and India know which parent is responsible for the gender of the child correctly than anything else..
I had a very interesting conversation with my 86 year old father a few months ago. I asked him how high the temp in my engine could get without seriously damaging it. He said 212 because that’s what water boils at.
I said that didn’t make sense because of the antifreeze… to which he responded that antifreeze wasn’t going to change the boiling point of water.
Technically true – until you mix the two! Anyway, the same man who methodically checked the coolant levels in every vehicle he owned ever since I can remember didn’t really know the science behind why he was doing it.
Nor did he know why he had to add salt to the ice to make homemade ice cream freeze.
He understands now. And it was a really great conversation.
btw – I replaced the radiator before I found out the answer to my question the hard way.
Unless for some inexplicable reason people in countries other than the US and EU are exceptionally bad guessers, a large factor of their low rates of correct answers must be due to significantly higher rates of “Don’t Know” responses. Assuming, say, 20% know the right answer in each country, a country with 20% “Don’t Know” responses will register about 50% Correct (20% plus about half of the 60% who guess), while a country with 40% “Don’t Know” responses will register only about 40% Correct (20% plus about half of the 40% who guess). Part of the story here may be that in non-western cultures it may be more acceptable to admit (at least to a pollster) that you “don’t know” what you don’t know.
From the footnotes to the NCF link:
“Don’t know” responses and refusals to respond counted as incorrect.
@ phaedrus
“Understanding these things is more important that understanding good finance practices, U.S. Civics and History or proper child care and social skills?”
I’d say americans dont know d*ck about those subjects either. avg. CC debt $9,900 anyone? watch “Jay-Walking” with Jay Leno to get an idea of how stupid people are about your “useful stuff.”
Without knowledge, process has no just guide, nor valid feedback.
The latter builds out of the former, else wanders aimlessly.
Just came back to this, and looked at the details of the report for some other research I’m doing. I’m amused by the “Earth goes round the sun, true or false” question. In Newtonian physics you can make a good argument that the Earth goes round the sun – the centre of mass is arguably a preferred inertial frame. But in general relativity there’s no preferred frame, and it really doesn’t make sense to say the Earth goes around the sun. Oh well. Regardless, the whole report is really interesting.