Quantitative Analysis of Bullshit in Physics Abstracts

Via Bee, we have the BlaBlaMeter, a website that purports to “unmask without mercy how much bullshit hides in any text.” Like Bee, I couldn’t resist throwing it some scientific text, but rather than pulling stuff off the arxiv, I went with the abstracts of the papers I published as a grad student, which I wrote up on the blog as part of the Metastable Xenone Project a few years ago. The abstracts, their scores, and some comments below:

Paper 1: Optical Control of Ultracold Collisions:

Near-resonant light is used to modify the collision dynamics of magneto-optically trapped metastable xenon atoms. Enhanced collisional ionization occurs for a “control laser” tuned below resonance, greatly exceeding the predictions of existing models of trap loss very close to resonance. With the trapping laser off, control light tuned above resonance suppresses ionization by a factor of 8. With the trap light on, a suppression factor >30 is observed. Increases in the number and density of trapped atoms and in the trap lifetime attest to the utility of optical control of collisions.

This gets a score of 0.1, “Your text shows only a few indications of ‘bullshit’-English.” The passive construction of that first sentence kind of makes me cringe, now, but other than that, it’s fine.

Paper 2: Suppression and Enhancement of Collisions in Optical Lattices:

In a three-dimensional optical lattice for metastable xenon we observe the dynamical effects of the optical potential on Penning ionizing collisions between atoms. Enhancement of collisions over that for free atoms is observed at short times after the atoms are loaded into the lattice. After the atoms thermalize and localize into the potential wells, we observe a suppression of the collisions by as much as a factor of 2. From our measurements and a simple model we are able to extract an estimate of the rate at which atoms “hop” between wells.

This one comes in at 0.31, “Your text shows indications of ‘bullshit’-English. It’s still ok for PR or advertising purposes, but more critical audiences may be skeptical.” I suspect it’s the “as much as,” and “are able to” that do it. I’m a little surprised that the latter survived in the abstract, actually– that sentence could easily be reworded to be shorter and more direct.

Paper 3: Time-Resolves Studies of Ultracold Ionizing Collisions:

Using 40 ns laser pulses, we probe the real-time dynamics of ultracold ionizing collisions in metastable xenon. We time resolve both shielding and enhancement effects, and observe the production of Xe2+ molecular ions through associative ionization. We estimate the rate of molecule formation in excited-state collisions, and directly measure the role of both flux enhancement and excited state survival in the collisional enhancement process. Conceptually simple theoretical models are used to predict the dynamics of the collisional shielding.

This gets a score of 0.51, “Something’s fishy. Obviously you want to sell something, or you’re trying to impress somebody. Are you sure that you have a real message, and if so: who would understand it?” Amusingly, this was the paper that sailed through refereeing with the least trouble– it made it into PRL on the strength of a single extremely enthusiastic referee report. It’s probably got the greatest science content of any of the papers I wrote as a student– the system turned out to be amazingly rich, and it was hard to cram all of the data into a single paper.

I’m not sure what would flag this as having higher bullshit content than any of the others. Playing around with it suggests that the last sentence is largely to blame– deleting it drops the score to 0.30. And, OK, that is a little weaselly, but with reason.

Paper 4: Spin polarization and quantum statistical effects in ultracold ionizing collisions:

We have measured ultracold ionizing collision rates for three bosonic (132Xe,134Xe, and 136Xe) and two fermionic (129Xe and 131Xe) isotopes of xenon in the 6s[3/2]2 metastable state, for both spin-polarized and unpolarized samples. For unpolarized samples at temperatures above the p-wave centrifugal barrier (∼39 μK), we find that collision rates for all isotopes are identical. Quantum-statistical effects forbid s-wave collisions for spin-polarized fermions, giving rise to significant differences between bosonic and fermionic isotopes below the p-wave barrier. We present a technique for measuring collision rates at temperatures below 1 μK, and find that the ratio of polarized to unpolarized collision rates for fermions decreases by a factor of 2 at low temperatures, while the ratio for bosons increases by 50%. We find no evidence of an overall reduction in the collision rate for spin-polarized samples, as has been observed in metastable helium. These results are explained using a simple theoretical model of transmission and quantum reflection off long-range interatomic potentials.

This comes in at 0.13, with the same message as the first paper. This is the longest paper of the lot, with the longest abstract. Maybe that makes it easier to nail down, I don’t know.

Paper 5: Creation of an Ultracold Neutral Plasma

We report the creation of an ultracold neutral plasma by photoionization of laser-cooled xenon atoms. The charge carrier density is as high as 2×109cm-3, and the temperatures of electrons and ions are as low as 100 mK and 10μK, respectively. Plasma behavior is evident in the trapping of electrons by the positive ion cloud when the Debye screening length becomes smaller than the size of the sample. We produce plasmas with parameters such that both electrons and ions are strongly coupled.

This also get a score of 0.1. This is probably the most significant of the papers I was on in grad school, in terms of impact on the field as a whole (it’s the most-cited of them, according to the Abstract Data Service). Nowadays, there are whole sessions at DAMOP on this subject, while the other areas have faded somewhat. That shouldn’t be obvious to the bullshit algorithm, though.

And, just for kicks, here’s the abstract of the Science paper I did as a post-doc:

We report manipulation of the atom number statistics associated with Bose-Einstein condensed atoms confined in an array of weakly linked mesoscopic traps. We used the interference of atoms released from the traps as a sensitive probe of these statistics. By controlling relative strengths of the tunneling rate between traps and atom-atom interactions within each trap, we observed trap states characterized by sub-Poissonian number fluctuations and adiabatic transitions between these number-squeezed states and coherent states of the atom field. The quantum states produced in this work may enable substantial gains in sensitivity for atom interference-based instruments as well as fundamental studies of quantum phase transitions.

That scores 0.18, only a few indications of bullshit. This is far and away the most cited thing I’ve done, with 484 citations recorded by the ADS.

What does all this mean? I have no idea. But it let me postpone editing the glossary for the book-in-progress for a little while, so that’s all to the good.

12 thoughts on “Quantitative Analysis of Bullshit in Physics Abstracts

  1. I played around a bit with the BlaBlaMeter, trying the abstracts from several of my published papers and from several recent papers I happened to have open. I had one abstract score 0.31, a paper that was also an outlier in topic; the rest of mine were between 0.05 and 0.15 (however, I couldn’t check several of my early papers because the OCR software that publisher used in scanning them was FUBAR). Most of the recent papers I checked had scores of 0.2 or higher, with the highest scoring 0.41. There was no obvious correlation between BlaBla score and whether the lead author is a native English speaker–the low scorer (0.07) among the recent papers was a Russian, but another Russian author scored 0.33.

  2. It’s a little hard to take this website seriously (even for fun) when the first sentence after the title is “PR-Experts, politicians, ad writers or scientists need to be strong here!”

    Does that sentence make sense to anyone else? Maybe the German version is better…

  3. Ok, I’ve figure out how this works. Certain words are penalized, and penalty scores are weighted by the length of the text. “Conceptually” is quite a bad word, which explains your .51 score. Changing it reduces the score by almost half. Changing other words like “process” reduce the score by smaller amounts. Of course, the game should really be played to get the highest score… so far my best tries are just above 6.0. “Conceptualized” is the most penalized word I’ve found so far. “Strategies” is another good one, but it doesn’t know about “Synergies”.

  4. I was curious about the Wholly Babble.
    Genesis (JPS1917) 1:1-5 gets 0.05.
    Romans (NIV) 1:18-32 gets 0.09.
    I’ll try some other narratives and speeches, though this tool is probably not calibrated for such texts.

  5. I posted the following phrase: “I cuddle unicorns always when magic comes out of my ass. Also, super powers allow me to transmute into a chimera. Another phenomenon is that I am 800 billion years old.” That paragraph received a score of 0. I’m skeptical of the Blablameter.

  6. Chunks of Dickens’ Great Expectations come in around 0.09 as do chunks of Conrad’s Lord Jim. That’s pretty impressive.

  7. I posted different lengths of text from Cosmopolitan and the majority received scores of 0.1 or less, and of course some famous articles like the Watson-Crick get 0.2 or more.

    I guess the idea of the blah blah meter is:

    “OK, this does nothing serious, get back to work and stop wasting time”

  8. Aren’t abstracts suppose to ‘sell’ something? Isn’t an abstract both a summary of the paper and a sales pitch to get people to read the whole thing? Also ‘conceptual’ and ‘strategies’ are both words that could be PR-speak but could also be legit sciencey stuff.

  9. Interestly, Lewis Carroll’s “Jabberwocky” comes in at 0.03. So obviously words that it doesn’t recognize are ignored rather than given a penalty. I’d argue that using words not recognized should be penalized, as they are likely too erudite for common readers, or obstructive.

  10. The abstract from my own dissretation:
    “The purpose of this project is to evaluate the comparative feasibility of various aero engines for use in the atmosphere of Titan. Combustion calculations are run on several engines, and a nuclear concept is briefly introduced as a comparison- either concept being fair game for research funding at the time being. Finally aerodynamics is considered in order to establish what form a future hydrocarbon powered meteorological aircraft might take.”
    a whopping 0.71, which as a prospective engineer, brings a tear of pride to my eye.

Comments are closed.