Monday, 15 May 2017
Reviewing peer review.
Fleming's discovery of penicillin couldn't get published today. That's a huge problem
Updated by Julia Belluz on December 14, 2015, 7:00 a.m. ET
After toiling away for months on revisions for a single academic paper, Columbia University economist Chris Blattman started wondering about the direction of his work.
He had submitted the paper in question to one of the top economics journals earlier this year. In return, he had gotten back nearly 30 pages of single-space comments from peer reviewers (experts in the field who provide feedback on a scientific manuscript). It had taken two or three days a week over three months to address them all.
So Blattman asked himself some simple but profound questions: Was all this work on a single study really worth it? Was it best to spend months revising one study — or could that time have been better spent on publishing multiple smaller studies? He wrote about the conundrum on his blog:
Some days my field feels like an arms race to make each experiment more thorough and technically impressive, with more and more attention to formal theories, structural models, pre-analysis plans, and (most recently) multiple hypothesis testing. The list goes on. In part we push because want to do better work. Plus, how else to get published in the best places and earn the respect of your peers?
It seems to me that all of this is pushing social scientists to produce better quality experiments and more accurate answers. But it’s also raising the size and cost and time of any one experiment.
Over the phone, Blattman explained to me that in the age of "big data," high-quality scientific journals are increasingly pushing for large-scale, comprehensive studies, usually involving hundreds or thousands of participants. And he's now questioning whether a course correction is needed.
Though he can't prove it yet, he suspects social science has made a trade-off: Big, time-consuming studies are coming at the cost of smaller and cheaper studies that, taken together, may be just as valuable and perhaps more applicable (or what researchers call "generalizable") to more people and places.
Do we need more "small" science?
Over in Switzerland, Alzheimer's researcher Lawrence Rajendran has been asking himself a similar question: Should science be smaller again? Rajendran, who heads a laboratory at the University of Zurich, recently founded a journal called Matters. Set to launch in early 2016, the journal aims to publish "the true unit of science" — the observation.
Rajendran notes that Alexander Fleming’s simple observation that penicillin mold seemed to kill off bacteria in his petri dish could never be published today, even though it led to the discovery of lifesaving antibiotics. That's because today's journals want lots of data and positive results that fit into an overarching narrative (what Rajendran calls "storytelling") before they'll publish a given study.
"You would have to solve the structure of penicillin or find the mechanism of action," he added.
But research is complex, and scientific findings may not fit into a neat story — at least not right away. So Rajendran and the staff at Matters hope scientists will be able to share insights in this journal that they may not been able to publish otherwise. He also thinks that if researchers have a place to explore preliminary observations, they may not feel as much pressure to exaggerate their findings in order to add all-important publications to their CVs.
Smaller isn't always better
Science has many structural problems to grapple with right now: The peer review system doesn't function all that well, many studies are poorly designed so their answers are unreliable, and replications of experiments are difficult to execute and very often fail. Researchers have estimated that about $200 billion — or about 85 percent of global spending on research — is routinely wasted on poorly designed and redundant studies.
A big part of the reason science funders started emphasizing large-scale studies is because they were trying to avoid common problems with smaller studies: The results aren't statistically significant, and the sample sizes may be too tiny and therefore unrepresentative.
It's not clear that emphasizing smaller-scale studies and observations will solve these problems. In fact, publishing more observations may just add to the noise. But as Rajendran says, it's very possible that important insights are being lost in the push toward large-scale science. "Science can be small, big, cure diseases," he said. "It can just be curiosity-driven. Academic journals shouldn't block the communication of small scientific observations."
Updated by Julia Belluz on December 14, 2015, 7:00 a.m. ET
After toiling away for months on revisions for a single academic paper, Columbia University economist Chris Blattman started wondering about the direction of his work.
He had submitted the paper in question to one of the top economics journals earlier this year. In return, he had gotten back nearly 30 pages of single-space comments from peer reviewers (experts in the field who provide feedback on a scientific manuscript). It had taken two or three days a week over three months to address them all.
So Blattman asked himself some simple but profound questions: Was all this work on a single study really worth it? Was it best to spend months revising one study — or could that time have been better spent on publishing multiple smaller studies? He wrote about the conundrum on his blog:
Some days my field feels like an arms race to make each experiment more thorough and technically impressive, with more and more attention to formal theories, structural models, pre-analysis plans, and (most recently) multiple hypothesis testing. The list goes on. In part we push because want to do better work. Plus, how else to get published in the best places and earn the respect of your peers?
It seems to me that all of this is pushing social scientists to produce better quality experiments and more accurate answers. But it’s also raising the size and cost and time of any one experiment.
Over the phone, Blattman explained to me that in the age of "big data," high-quality scientific journals are increasingly pushing for large-scale, comprehensive studies, usually involving hundreds or thousands of participants. And he's now questioning whether a course correction is needed.
Though he can't prove it yet, he suspects social science has made a trade-off: Big, time-consuming studies are coming at the cost of smaller and cheaper studies that, taken together, may be just as valuable and perhaps more applicable (or what researchers call "generalizable") to more people and places.
Do we need more "small" science?
Over in Switzerland, Alzheimer's researcher Lawrence Rajendran has been asking himself a similar question: Should science be smaller again? Rajendran, who heads a laboratory at the University of Zurich, recently founded a journal called Matters. Set to launch in early 2016, the journal aims to publish "the true unit of science" — the observation.
Rajendran notes that Alexander Fleming’s simple observation that penicillin mold seemed to kill off bacteria in his petri dish could never be published today, even though it led to the discovery of lifesaving antibiotics. That's because today's journals want lots of data and positive results that fit into an overarching narrative (what Rajendran calls "storytelling") before they'll publish a given study.
"You would have to solve the structure of penicillin or find the mechanism of action," he added.
But research is complex, and scientific findings may not fit into a neat story — at least not right away. So Rajendran and the staff at Matters hope scientists will be able to share insights in this journal that they may not been able to publish otherwise. He also thinks that if researchers have a place to explore preliminary observations, they may not feel as much pressure to exaggerate their findings in order to add all-important publications to their CVs.
Smaller isn't always better
Science has many structural problems to grapple with right now: The peer review system doesn't function all that well, many studies are poorly designed so their answers are unreliable, and replications of experiments are difficult to execute and very often fail. Researchers have estimated that about $200 billion — or about 85 percent of global spending on research — is routinely wasted on poorly designed and redundant studies.
A big part of the reason science funders started emphasizing large-scale studies is because they were trying to avoid common problems with smaller studies: The results aren't statistically significant, and the sample sizes may be too tiny and therefore unrepresentative.
It's not clear that emphasizing smaller-scale studies and observations will solve these problems. In fact, publishing more observations may just add to the noise. But as Rajendran says, it's very possible that important insights are being lost in the push toward large-scale science. "Science can be small, big, cure diseases," he said. "It can just be curiosity-driven. Academic journals shouldn't block the communication of small scientific observations."
They may lose on every sale but they know how to close.
Darwin’s Finches: An Icon Gets Retouched
Evolution News | @DiscoveryCSC
Some scientists at Georgia Tech collected bacteria from a lake near their campus and “helped” them evolve in a test tube. From their results, they made major pronouncements about the tempo and mode of evolution on the faraway Galápagos Islands. For this, they got funding from the National Science Foundation.
That pretty much sums up John Toon’s write-up for the Georgia Tech News Center, “‘First Arrival’ Hypothesis in Darwin’s Finches Gets Some Caveats.”
What exactly do lake bacteria have to do with Darwin’s finches, you ask? Well, the evolutionary scientists wanted to follow up on the famous study by David Lack in 1947. He had proposed a theory about “first arrivals” in a new ecosystem, but his study lacked some important details.
Among his hypotheses was that the birds were successful in their adaptive radiation — the evolutionary diversification of morphological, physiological and behavior traits — because they were early colonizers of the islands. The finches filled the available ecological niches, taking advantage of the resources in ways that limited the ability of later-arriving birds to similarly establish themselves and diversify, he suggested. [Emphasis added.]
If the early bird gets the evolutionary advantage, what happens if the late bird is a better competitor? It changes the iconic story somewhat. “Being first in a new ecosystem provides major advantages for pioneering species, but the benefits may depend on just how competitive later-arriving species are.” The late bird might just peck the lights out of the early bird.
To find out what happens, the Darwin team under Jiaqi Tan put their lake bacteria into test tubes to watch the games play out.
Tan and other researchers in the laboratory of Georgia Tech Professor Lin Jiang tested that hypothesis using P. fluorescens, which rapidly evolves into two general phenotypes differentiated by the ecological niches they adopt in static test tube microcosms. Within the two major phenotypes — known as “fuzzy spreaders” and “wrinkly spreaders” — there are additional minor variations.
The researchers allowed the bacterium to colonize newly-established microcosms and diversify before introducing competing bacterial species. The six competitors, which varied in their niche and competitive fitness compared to P. fluorescens, were introduced individually and allowed to grow through multiple generations. Their success and level of diversification were measured by placing microcosm samples onto agar plates and counting the number of colonies from each species and sub-species.
Not surprisingly, the results were complicated. The bacteria reproduce much faster than finches, but they reproduce asexually. Their mutation rates are also probably much different. “Still, Jiang and Tan believe their study offers insights into how different species interact in new environments based on historical advantages.”
“If the diversifying species and the competing species are very similar, you can have a strong priority effect in which the first-arriving species can strongly impact the ability of the later species to diversify,” said Jiang, a professor in Georgia Tech’s School of Biological Sciences. “If the species are different enough, then the priority effect is weaker, so there would be less support for the first arrival hypothesis.”
David Lack’s famous book lacked this crucial realization: the first arrival hypothesis works, except when it doesn’t. Evolutionists will now “need to think about the surrounding ecological context of the evolutionary process.”
The paper in Evolution shows that the trajectory of evolution is contingent upon the historical context, which seems to say that anything can happen when the first bird lands on an island:
In the first-arrival hypothesis, David Lack emphasized the importance of species colonization history for adaptive radiation, suggesting that the earlier arrival of a diversifying species would allow it to radiate to a greater extent. Here, we report on the first rigorous experimental test of this hypothesis, using the rapidly evolving bacterium Pseudomonas fluorescens SBW25 and six different bacterial competitors. We show that the earlier arrival of P. fluorescens facilitated its diversification. Nevertheless, significant effects of colonization history, which led to alternative diversification trajectories, were observed only when the competitors shared similar niche and competitive fitness with P. fluorescens. These results highlight the important role of species colonization history, modified by their ecological differences, for adaptive radiation.
Interesting phrase: “alternative diversification trajectories” are possible. The possibility for storytelling just grew exponentially.
It seems unbelievable that nobody performed a rigorous test of David Lack’s hypothesis for seventy years. But it doesn’t matter, because neither study — David Lack’s or Georgia Tech’s –provides any help for Darwinian evolution. The birds hybridize. No origin of species has occurred. The varieties of finches are “trapped in an unpredictable cycle of Sisyphean evolution,” according to McKay and Zink, quoted by Jonathan Wells in his new book Zombie Science (pp. 69-70).
This means that, like the old king Sisyphus of Greek mythology, condemned by the gods to roll a stone up a hill that always escapes and rolls back down, requiring him to repeat the cycle forever, Darwin’s finches are going nowhere.
So here’s what we know in 2017 about Darwin’s finches, one of Jonathan Wells’ original ten Icons of Evolution.
Some finches wound up on the Galápagos Islands sometime.
Darwin captured some on his visit, but never used them to promote his theory.
The finches can freely hybridize.
The only major difference between them is the size and shape of the beak.
When the weather is dry, bigger-beaked birds do better.
When the rain returns, smaller-beaked birds return to previous levels.
No speciation has occurred. (This is called “adaptive radiation.”)
There exists a nebulous idea called “fitness,” measured by number of offspring.
Fitness changes from year to year, as circumstances change.
Varieties of finches exchange places as “fittest” from year to year.
The first arriver gets priority, unless a fitter bird arrives later.
Nobody can know in advance what bird will stay the fittest for how long.
The NSF will let you use bacteria as a proxy for birds.
Yes, “Adaptive radiation is an important evolutionary process,” just like the paper begins. Thanks for the money, NSF!
In Zombie Science, Wells points out that in a 1999 pro-evolution booklet for schools, the U.S. National Academy of Sciences called Darwin’s finches “a particularly compelling example of speciation.” He also points out that after 17 years since he exposed the flaws in this evolutionary icon, the “zombie” keeps coming back from the dead, reviving and stalking in biology textbooks. Here, the zombie makes another appearance: research that goes nowhere, proves nothing, and yet pretends that Darwin’s finches, despite “some important caveats,” provide insight into the origin of species.
It’s hard to kill a zombie when the federal government funds its handlers.
Evolution News | @DiscoveryCSC
Some scientists at Georgia Tech collected bacteria from a lake near their campus and “helped” them evolve in a test tube. From their results, they made major pronouncements about the tempo and mode of evolution on the faraway Galápagos Islands. For this, they got funding from the National Science Foundation.
That pretty much sums up John Toon’s write-up for the Georgia Tech News Center, “‘First Arrival’ Hypothesis in Darwin’s Finches Gets Some Caveats.”
What exactly do lake bacteria have to do with Darwin’s finches, you ask? Well, the evolutionary scientists wanted to follow up on the famous study by David Lack in 1947. He had proposed a theory about “first arrivals” in a new ecosystem, but his study lacked some important details.
Among his hypotheses was that the birds were successful in their adaptive radiation — the evolutionary diversification of morphological, physiological and behavior traits — because they were early colonizers of the islands. The finches filled the available ecological niches, taking advantage of the resources in ways that limited the ability of later-arriving birds to similarly establish themselves and diversify, he suggested. [Emphasis added.]
If the early bird gets the evolutionary advantage, what happens if the late bird is a better competitor? It changes the iconic story somewhat. “Being first in a new ecosystem provides major advantages for pioneering species, but the benefits may depend on just how competitive later-arriving species are.” The late bird might just peck the lights out of the early bird.
To find out what happens, the Darwin team under Jiaqi Tan put their lake bacteria into test tubes to watch the games play out.
Tan and other researchers in the laboratory of Georgia Tech Professor Lin Jiang tested that hypothesis using P. fluorescens, which rapidly evolves into two general phenotypes differentiated by the ecological niches they adopt in static test tube microcosms. Within the two major phenotypes — known as “fuzzy spreaders” and “wrinkly spreaders” — there are additional minor variations.
The researchers allowed the bacterium to colonize newly-established microcosms and diversify before introducing competing bacterial species. The six competitors, which varied in their niche and competitive fitness compared to P. fluorescens, were introduced individually and allowed to grow through multiple generations. Their success and level of diversification were measured by placing microcosm samples onto agar plates and counting the number of colonies from each species and sub-species.
Not surprisingly, the results were complicated. The bacteria reproduce much faster than finches, but they reproduce asexually. Their mutation rates are also probably much different. “Still, Jiang and Tan believe their study offers insights into how different species interact in new environments based on historical advantages.”
“If the diversifying species and the competing species are very similar, you can have a strong priority effect in which the first-arriving species can strongly impact the ability of the later species to diversify,” said Jiang, a professor in Georgia Tech’s School of Biological Sciences. “If the species are different enough, then the priority effect is weaker, so there would be less support for the first arrival hypothesis.”
David Lack’s famous book lacked this crucial realization: the first arrival hypothesis works, except when it doesn’t. Evolutionists will now “need to think about the surrounding ecological context of the evolutionary process.”
The paper in Evolution shows that the trajectory of evolution is contingent upon the historical context, which seems to say that anything can happen when the first bird lands on an island:
In the first-arrival hypothesis, David Lack emphasized the importance of species colonization history for adaptive radiation, suggesting that the earlier arrival of a diversifying species would allow it to radiate to a greater extent. Here, we report on the first rigorous experimental test of this hypothesis, using the rapidly evolving bacterium Pseudomonas fluorescens SBW25 and six different bacterial competitors. We show that the earlier arrival of P. fluorescens facilitated its diversification. Nevertheless, significant effects of colonization history, which led to alternative diversification trajectories, were observed only when the competitors shared similar niche and competitive fitness with P. fluorescens. These results highlight the important role of species colonization history, modified by their ecological differences, for adaptive radiation.
Interesting phrase: “alternative diversification trajectories” are possible. The possibility for storytelling just grew exponentially.
It seems unbelievable that nobody performed a rigorous test of David Lack’s hypothesis for seventy years. But it doesn’t matter, because neither study — David Lack’s or Georgia Tech’s –provides any help for Darwinian evolution. The birds hybridize. No origin of species has occurred. The varieties of finches are “trapped in an unpredictable cycle of Sisyphean evolution,” according to McKay and Zink, quoted by Jonathan Wells in his new book Zombie Science (pp. 69-70).
This means that, like the old king Sisyphus of Greek mythology, condemned by the gods to roll a stone up a hill that always escapes and rolls back down, requiring him to repeat the cycle forever, Darwin’s finches are going nowhere.
So here’s what we know in 2017 about Darwin’s finches, one of Jonathan Wells’ original ten Icons of Evolution.
Some finches wound up on the Galápagos Islands sometime.
Darwin captured some on his visit, but never used them to promote his theory.
The finches can freely hybridize.
The only major difference between them is the size and shape of the beak.
When the weather is dry, bigger-beaked birds do better.
When the rain returns, smaller-beaked birds return to previous levels.
No speciation has occurred. (This is called “adaptive radiation.”)
There exists a nebulous idea called “fitness,” measured by number of offspring.
Fitness changes from year to year, as circumstances change.
Varieties of finches exchange places as “fittest” from year to year.
The first arriver gets priority, unless a fitter bird arrives later.
Nobody can know in advance what bird will stay the fittest for how long.
The NSF will let you use bacteria as a proxy for birds.
Yes, “Adaptive radiation is an important evolutionary process,” just like the paper begins. Thanks for the money, NSF!
In Zombie Science, Wells points out that in a 1999 pro-evolution booklet for schools, the U.S. National Academy of Sciences called Darwin’s finches “a particularly compelling example of speciation.” He also points out that after 17 years since he exposed the flaws in this evolutionary icon, the “zombie” keeps coming back from the dead, reviving and stalking in biology textbooks. Here, the zombie makes another appearance: research that goes nowhere, proves nothing, and yet pretends that Darwin’s finches, despite “some important caveats,” provide insight into the origin of species.
It’s hard to kill a zombie when the federal government funds its handlers.
Subscribe to:
Posts (Atom)