Time to rethink statistical significance, Nature

People often joke about how 95% means 100% to biologists who rely heavily on data analysis. However, it's even more horrifying to see scientists taking 94% as 0%. A recent editorial on Nature, signed by more than 800 scientists, urges the science community to reconsider how p-value is used in research. To many people, p>0.05 means there is no effect and p<0.05 means one step closer to publication. But can we really have a clear cut? The belief in p-value as a categorization tool considerably harms the progress of science. 

"We are not calling for a ban on P values. Nor are we saying they cannot be used as a decision criterion in certain specialized applications (such as determining whether a manufacturing process meets some quality-control standard). And we are also not advocating for an anything-goes situation, in which weak evidence suddenly becomes credible. Rather, and in line with many others over the decades, we are calling for a stop to the use of P values in the conventional, dichotomous way — to decide whether a result refutes or supports a scientific hypothesis."

We must embrace the uncertainty in experiments to remain critical in our scientific inquiries. To do so, the authors give four advice:

  1. "confidence interval" is actually a compatibility interval. "Just because the interval gives the values most compatible with the data, given the assumptions, it doesn’t mean values outside it are incompatible; they are just less compatible. In fact, values just outside the interval do not differ substantively from those just inside the interval. It is thus wrong to claim that an interval shows all possible values." 
  2. Discuss the point estimate as well as the limit of the interval. Interpreting the point estimate, while acknowledging its uncertainty, will keep you from making false declarations of ‘no difference’, and from making overconfident claims.
  3. 0.05 is arbitrarily determined by conventions. P-value threshold should be adjusted depending on the application. 
  4. Be humble. Accept the uncertainty. Compare different theoretical models and thresholds and report all results. That is the right way of doing science.

"What will retiring statistical significance look like? We hope that methods sections and data tabulation will be more detailed and nuanced. Authors will emphasize their estimates and the uncertainty in them — for example, by explicitly discussing the lower and upper limits of their intervals. They will not rely on significance tests. When P values are reported, they will be given with sensible precision (for example, P = 0.021 or P = 0.13) — without adornments such as stars or letters to denote statistical significance and not as binary inequalities (P < 0.05 or P > 0.05). Decisions to interpret or to publish results will not be based on statistical thresholds. People will spend less time with statistical software, and more time thinking.

Our call to retire statistical significance and to use confidence intervals as compatibility intervals is not a panacea. Although it will eliminate many bad practices, it could well introduce new ones. Thus, monitoring the literature for statistical abuses should be an ongoing priority for the scientific community. But eradicating categorization will help to halt overconfident claims, unwarranted declarations of ‘no difference’ and absurd statements about ‘replication failure’ when the results from the original and replication studies are highly compatible. The misuse of statistical significance has done much harm to the scientific community and those who rely on scientific advice. P values, intervals and other statistical measures all have their place, but it’s time for statistical significance to go."

The physics behind tearing a packing tape, Physical Review Letters

Packing tape is always a pain to use, especially when peeling off the tape. The motion was neither smooth nor steady. In fact, most of the times the tape will stubbornly stick, then it will finally give, with a length of tape peeling all at once, and then it will stick again—a motion aptly called stick–slip. Now, in a new study published by Stéphane Santucci and his colleagues, the stick-slip motion was observed on both macroscopic and microscopic scale, and the motion was characterized by a mathematical model.
The researchers attached two layers of acrylic film tape, such as you might use to wrap a gift, and peeled off the top layer at a constant velocity and an adjustable angle using a motor. Using a high-speed camera to monitor the position of the peel front, the team found that the slip distance scaled as the cube root of the duration of the stick regardless of the tape’s peel angle or bending modulus. During the peeling, the increasing elastic bending energy in the stick phase turns into kinetic energy in the slip phase and these microscopic slips combined into a macroscopic slip at the millimeter scale.
One aspect not yet understood is the motion of the peel front. Future research will seek to explain why at the microscale, the tape seems to peel sideways, which is similar to the motion of dislocations in crystalline materials.

This celebrity cat has broken the Internet. Now, we have its genome. bioRxiv

Very few cats have achieved the online fame of Lil Bub, a feral kitten with many peculiarities. While Lil Bub has extra toes, shorter-than-usual limbs, and a tongue that perpetually hangs out of her mouth, these abnormalities just made her more lovable. Now, Lil Bub is not just a YouTube celebrity, but she’s also helping scientists too. In a new paper published on bioRxiv, scientists have successfully extracted the genome of the famous kitten.
Scientists have discovered two tell-tale abnormalities in the genome. One of them is a common genetic variation in a stretch of DNA that acts as an “on-off switch” for the Sonic hedgehog gene, which is responsible for the extra toes of Lil Bub and Ernest Hemingway’s six-toed cat. The other abnormal key gene was more surprising: a variation of TNFRSF11A, a gene known from mice and humans to be associated with a rare condition called osteopetrosis, which causes unusually dense bones and short stature.
According to Leslie Lyons, a co-author of the paper, the findings could have great significance to our understanding of genome of both cats and human. “In human genetics, there’s a still a huge number of variants of unknown significance, which basically means you don’t know if they’re benign or not. That’s one of the things our project is trying to do. If you look at the genetic information in cats and dogs, that might tell you a particular mutation is found all the time in cats and they’re perfectly fine, so it probably is a benign mutation.” said Leslie.
ISP Sci. Rev. 13 (2019)
Editor: Shiwei Wang, Rossoneri Jing
Integrated Science Program
Northwestern University

This email was sent to <<Email Address>>
why did I get this?    unsubscribe from this list    update subscription preferences
ISP Science Review · 616 Noyes St · Evanston, IL 60208-0860 · USA

Email Marketing Powered by Mailchimp