Twitter transcript of #sciwri10 stats talk

 (Posting this here so I can link to it from a post I’m going to make on the official ScienceWriters2010 blog)

BethSkw:        Next up: Get the Numbers Right: a workshop on reporting statistics http://bit.ly/bXrIH9 #sciwri10 #stats

DanielleVenton:        Are you in the right? Get the Numbers Right session about to start. #sciwri10

BethSkw:        Some stuff from this session will be posted on www.stephenornes.com #sciwri10 #stats

BethSkw:        SO on fact checking: marks up article, makes a report. Pet peeve: even good writers have hard time w odds ratios #sciwri10 #stats

BethSkw:        SO: OR=odds ratio 2.38 does NOT mean 138% inc risk. Instead: 2.38x the odds. Rel risk intuitive, this not. #sciwri10 #stats

BethSkw:        More on odds ratio vs. relative risk (my googling) http://bit.ly/aVA6V2 #sciwri10 #stats

BethSkw:       rel btw OR and a % risk not straightforward to calculate - ask a researcher (& hope they know what they’re doing) - Gelman #sciwri10 #stats

BethSkw:        SO www.stats.org “are you a journalist?” link - ask stats questions #sciwri10 #stats

daviddespain:        Are you a sci jorno who wants to ask a question of a statistician? http://www.stats.org/ #sciwri10

BethSkw:        Here’s the direct link for stats help for journalists: http://stats.org/journalist.htm #sciwri10 #stats

DanielleVenton:        Odds ratios don’t translate into percents. See stats.org to get help from a stats expert #sciwri10 #stats

BethSkw:        SO: almost impossible to find absolute risk. Rel risk compares eg smokers to nonsmokers but does not tell ACTUAL risk. #sciwri10 #stats

BethSkw:        SO: when RR=1, no difference. RR=2 means 100% increase in one group vs. the other #sciwri10 #stats

BethSkw:        SO: Comparing events with 80% and 67% prob - odds ratio is 2 but 80% is not twice the risk of 67%! #sciwri10 #stats

BethSkw:        SO: When in doubt, ask the investigator AND THEN ask an independent statistician, or stats.org. #sciwri10 #stats

BethSkw:        SO: Befriend a statistician. When you talk to one, ask if you can come back with Qs later, many will say yes. #sciwri10 #stats

DanielleVenton:        Relative risk is more accurate, double check your stats! #sciwri10 make friends with a statistician

BethSkw:        Andrew Gelman: statisticians are not alike, hate when reporters ask one about sth outside their field. Ask their bg. #sciwri10 #stats

BethSkw:        2000 Bush v. Gore case used stats expert who was smart guy, not qualified in the area, confident enough to show up anyway #sciwri10 #stats

BethSkw:        AG: Statisticians often like to be helpful, we want to see our names in the paper! #sciwri10 #stats

jamesian:        Beware trusting experts in stats who lack specific credibility - Gelman of Columbia #sciwri10

BethSkw:        AG: Silly studies like that influence how ppl think about sex stereotypes, meaning theyre important to discuss #sciwri10 #stats

DanielleVenton:        AG: look at studies in context, not isolation, what did you expect to see? #sciwri10 #stats

BethSkw:        AG: 51% of babies are boys, more boy babies die, at age 20 the numbers are 50/50 #sciwri10 #stats

BethSkw:       AG: extr poverty/famine: sex ratio effects as high as 3%. Smaller effects (race, age, season) < 1%. ergo expect beauty <1% #sciwri10 #stats

BethSkw:        AG: some studies “more vampirical than empirical - unable to be killed by mere evidence” - Freese 2007 http://bit.ly/9g4zVM #sciwri10 #stats

DanielleVenton:        AG: sex ratio studies can be more ‘vampirical’ than ‘empirical’ unable to be killed by mere evidence #sciwri10 #stats

BethSkw:        AG compares results to People’s Most Beautiful lists. Can’t find small effects with this sample size #sciwri10 #stats

Deb_acle:        RT @BethSkw: More on odds ratio vs. relative risk (my googling) http://bit.ly/aVA6V2 #sciwri10 #stats

jamesian:        Scientific method is “machine for exaggeration” says Gelman #sciwri10

DanielleVenton:        AG: be suspicious when something is 10 - 100x larger than expected effects. #sciwri10 #stats

rachelsklar:        Pls blog it!!! RT @arikia: This stats session is amaaazing. @fivethirtyeight you’ve turned me into a total stats head… #sciwri10

BethSkw:        AG: in peer reviewed lit, there is a bias toward overestimating effects. Need to know topic bg to spot red flags #sciwri10 #stats

DanielleVenton:        AG: most effects reported in peer-reviewed lit are overestimated. #sciwri10 #stats

BethSkw:        AG: many “politically incorrect” studies essentially random results b/c no statistical power #sciwri10 #stats

BethSkw:        Next speaker: Tom Siegfried of Science News (prev article of his here: http://bit.ly/d5CjYw) #sciwri10 #stats

davemosher:       Andrew Gelman, statistician at Columbia: Journal system = little black box of overestimation (in regards to reporting sig results) #sciwri10

BethSkw:        TS: stat significant means 95% prob effect isn’t chance? NO! doesn’t even mean that it’s “significant”. #sciwri10 #stats

arikia:       The woman sitting next to me in the stats talk just yelped when Tom Siegfried called the commonly accepted def of stat sig wrong #sciwri10

arikia:        ”Evidence that you’ve seen something unlikely is not evidence that the opposite is true.” #sciwri10

davemosher:        ”Evidence that I don’t own the house is not evidence that the house owns me.” -Tim Siegfried #sciwri10

BethSkw:        TS: Stat significant = less than 5% chance of seeing effect of this magnitude if there really is no effect. #sciwri10 #stats

arikia:        This session is obviously ruffling some feathers but this is stats 101 stuff. Excellently delivered, I might add. #sciwri10

BethSkw:        Wish I could tell the Tom Siegfried’s swimming-in-winter example in 140 chars. Explained the idea nicely! #sciwri10 #stats

BethSkw:        RT @arikia: This session is obviously ruffling some feathers but this is stats 101 stuff. Excellently delivered, I might add. #sciwri10

BethSkw:        TS: If result isn’t statistically significant, can say “did not establish a link” or “…but could have been due to chance” #sciwri10 #stats

BethSkw:        TS: huge confidence interval means result is uncertain. Ex phrasing: “in the range of about 10 to 40%” #sciwri10 #stats

DanielleVenton:        TS: reporting elevated risk? State the comparison point. #sciwri10 #stats

BethSkw:        TS: When reporting Relative Risk, say relative to WHAT. #sciwri10 #stats

BethSkw:        TS: Reader may not know how to interpret simple math like “50% increase” so think carefully when describing numbers #sciwri10 #stats

BethSkw:        TS: Red flags for wrong results: 1st report, hot field, contrary to prev belief (why say prev belief wrong?) #sciwri10 #stats

daviddespain:        Recipe for wrong science news: 1 first report of something 2 advance in hot research field 3 contrary to previous belief #sciwri10

BethSkw:        TS: recipe for wrong science == recipe for hot news. So have to be careful. #sciwri10 #stats

bohemianone:        RT @BethSkw: TS: Reader may not know how to interpret simple math like “50% increase” so think carefully when describing numbers #sciwri10 #stats

arikia:        The recipe for bad science is the same recipe as what’s often sought after for science news. #sciwri10

BethSkw:        Gelman: Maybe drop idea of effect vs. no effect - effect is rarely zero, Q is how much. #sciwri10 #stats

arikia:        There are situations where single events are newsworthy, eg: if you find a talking dog… #sciwri10

BethSkw:        Q: single case study newsworthy? TS: Can be. Stats are just to sort out complicated situations. #sciwri10 #stats

daviddespain:        Forgotten frequently (or conveniently) by reporters RT @BethSkw: TS: When reporting Relative Risk, say relative to WHAT. #sciwri10 #stats

BethSkw:        Andrew Gelman’s blog: http://www.stat.columbia.edu/~gelman/blog/ #sciwri10 #stats

TomLevenson:        3 flags for bad science/science news-first report of something/in a hot field/contrary to prior belief. Amen and amen. #sciwri10

marynmck:        RT @arikia: The recipe for bad science is the same recipe as what’s often sought after for science news. #sciwri10

BethSkw:        AG: psychologists have research on how to communicate risk/probability. TS: “risk communication” field #sciwri10 #stats

BethSkw:        A few risk communication links: http://bit.ly/ak4som http://bit.ly/bFjqh4 http://bit.ly/bNsm02 #sciwri10 #stats

DanielleVenton:        Get comfortable with the numbers in the study before you trust them. No snap judgments. #sciwri10 #stats

arikia:        Untapped resource on finding statistical flaws in research: ppl who don’t like the people who published it. #sciwri10

DanielleVenton:        Stats resource: other researchers who don’t like the author, good for explaining study flaws. #sciwri10 #stats

TomLevenson:        @caribbeanscot: three reasons to question new science (news): it’s a first report in a hot field contradicting prior belief. #sciwri10

maggiekb1:        RT @arikia: “Evidence that you’ve seen something unlikely is not evidence that the opposite is true.” #sciwri10

BethSkw:        SO: OR=odds ratio 2.38 does NOT mean 138% inc risk. Instead: 2.38x the odds. Rel risk intuitive, this not. #sciwri10 #stats

1 Comment

Other Links to this Post

1. Alexander7 — July 21, 2011 @ 2:21 pm

RSS feed for comments on this post. TrackBack URI

Leave a comment