Nutrition Studies Are Just Terrible 

I find a significant association between watching health-related youtube channels and farting. It's science. You can't dispute it.

If you enjoy the BS finding aspect, you may want to listen to Quackcast or Persiflagers Infectious Disease podcast.

Keep on Truckin' Nattosoup , I can’t keep on trucking, Pennsylvania keeps putting on weather restrictions every time there’s a threat of precipitation.

Patrick yes absolutely true! We get away in healthcare research by running the same study repeatedly until we find significant results, while the repeated null results go unpublished! It abuses the assumptions of the statistical models we use to support our findings!

Veritasium's channel is awesome.

In my opinion, it's a tie between nutrition and mental health.

Yeah, I don't know how many more of these I can... stomach!

Even that consensus is hard because there's no single concrete definition on what "whole foods" or "processed foods" are. Is bread a processed food? How much processing is required to turn a "whole food" into a "processed food"? It's like obscene material, it's "I know it when I see it" scenario, and that's not good for scientific study.

"Eat food, not too much, mostly plants" is Michael Pollan's summary, where food is in contrast to "food-like products", probably all we can definitively say at this point.

Actually a pretty spot summary.

Austin Chiang MD MPH It certainly isn’t cheap

Imagine doing that in US with water quality of flint, not you have to adjust for that.

Unless you want to lock people inside a facility for a few years and monitor everything they eat. Even then things won't be accounted for.

Could this apply to public health studies too?

Why do you need nutrition studies? Eating healthy isn't that hard.

happens a lot in youtube videos. Apparently, sound equalization is not a thing for them.

Done!

Not a scientist or a statistician, but my understanding is that by looking at so many different food and health factor combinations, there is a very high chance of one combo producing statistically significant results through pure random chance even if there's no real relationship of note. They reported the results of one combination, and nothing else. Would a scientist withhold exciting findings? No. That indicates that the combination of soda/juice and cancer was that one random pairing that returned significant results, whereas all the others weren't notable. Odds are good that an experiment looking only at sugary drinks and cancer would fail to find significance, and a reproduction of this giant crapshoot study would find random significance in some other food/condition combination instead

@@Emmasama240 Thanks for the thoughtful response. I see what you're saying, especially in relation to the preregistration. Reading your comment and watching veritasium's "why most research is wrong." helped me understand. So basically by making it so broad, the scientist can handpick the outcomes they want to enhance the likelihood of their paper getting published.

That's the Bonferroni adjustment, yes?

@@Roll587 Yeah, for ANOVA and the like. Cost me a few significant results in my dissertation research!

Keelia Silvis Tamar not Tamara, that was my issue

twitter.com/TamarHaspel

@@Ou8y2k2 HaspEL, not Hasp!!! Thanks much!

@@KeeliaSilvis np

I guess they were thinking about people who may start shrinking from things like osteoporosis? (shrugs)

Height is most likely used with weight to calculate BMI.

@@paielinelli Thank you;

Not at all, you can only adjust for confounders you are aware of and accurately measured, that would require absolute knowledge

Not a scientist but I have a bit of background knowledge (I might make some minor mistakes due to how long ago it is that I learned this stuff). They test for whether the outcomes were significant by using a measure of how likely it is that this result happened because of random chance and not an actual cause. These measures are based on the assumption that you are measuring one thing at a time. This measure is written approximately like this: this correlation between x and y has z% chance of being a coincidence. This measure can however be gamed by measuring dozens or hundreds of things at a time because if you measure for hundreds if not thousands of illnesses and then search for a cause among 3300 foods then to use a comparison it would be like flipping a coin a thousand times and pointing out that at some point you flipped tails 20 times in a row and then saying that you "flipped 20 times in a row and got 20 tails so that couldn't be a coincidence" completely ignoring the 9980 other times you flipped the coin. This study is doing the same thing by flipping a coin for every illness and every food they measure and after flipping it so many times you are bound to get some weird / random chance occurrences somewhere.

That's exactly what is happening.

Added!

the issue is that these scientists didn't actually choose what they were studying, all they did was say 'we're going to study everything and whatever gets us the best result, we're publishing.' the whole point of experiments is to have an idea and test specific variables but they didn't really do that.

He is nothing like Alex Jones, and there is no such thing as "Alex Jones, but in a good way". The very phrase is oxymoronic.

It's been awhile, but just in case anyone's curious still: Basically, these researchers collected a huge data set, with information on many different dietary choices and many different health conditions, and looked for associations between foods consumed and negative health outcomes. This means that they (in essence; this is a simplification) ran probably hundreds (thousands?) of individual analyses, seeing if consuming food A was associated with a higher risk of condition A, then condition B, then condition C, and so on. The problem with that is as follows: if you run hundreds of tests, looking for statistical significance at the .05 level, as the sheer number of tests mounts, so too does the probability of getting a spurious statistically significant result, just by chance. This is kind of "meta," but think of it like this - when you do a regression analysis, or ANOVA, or whatever, and your standard for statistical significance is .05, that means that if you get a positive result, there's a 95% chance that the two variables are in fact associated, but a 5% chance that there is no underlying association (but the sample you tested just happens to be structured such that a spurious association appears). When you're just doing a couple of statistical analyses on your data set, .05-level results are usually (depending on the field) considered strong enough to at least support further investigation on the link between the variables. Sure, the result MIGHT be in error, but you ran a tight, well-constructed study, so let's continue that line of investigation. BUT, if you're subjecting that data set to dozens of simultaneous analyses, basically going on a fishing expedition, those two significant results out of 50 comparisons are no longer necessarily impressive. In fact, if you run enough statistical tests on a data set, you expect to get some positive results just by chance (even if there are no associations AT ALL in the data), for the same reason that when running a single test at the .05 level, we expect that 5 times out of 100 we'll get an incorrect positive result. It's never a good idea to do what the people responsible for this study seem to have done; hypotheses and studies really should be much more focused than appears to have been the case here. Sometimes, though, it's unavoidable to do a larger number of comparisons. In such cases, you run what are called post-hoc tests for multiple comparisons (Bonferroni is a commonly used one); in very simple terms, these post-hoc tests "devalue" the significance of your results according to how many simultaneous comparisons you're doing. If any of the statistically significant results survive that process, you're good to go, but often they don't. In this case, I'm wondering if those post-hoc tests were done. In any event, the reason that the results concerning cancer and beverages wouldn't be reliable *unless it still held after post-hoc corrections* is because these significant results may very well be simply due to chance - with this many comparisons, we'd expect a few to be positive even if there were, in reality, no associations among any of the variables.

It's p-hacking and cherry-picking in steroids. I can't understand how authors get away with it

Just get a specialist not internet jeez

This video is about identifying bad research methodology which doesn't mean that all research is bad. Anyway it's about scientists not doctors. Getting your info from RU-vid is likely to get more bad science / bad research. Doctors who study healthcare are knowledgeable about basic conditions like yours and if you have a crap dr just change (research reviews of drs and ask for recommendations)

Statistically significant results may be a result of a large sample size rather than a "real" underlying connection. For example, if you get a random sample of 100,000 people and ask them what icecream they like, and find there's a small difference between men and women in flavor preferences, it could be that there is some genetic, psychological, or social factor that causes that difference, or it could have been that you collected a large sample, so minor deviations are picked up by the test and reported as significant.

If you look for one association and find a 95% chance that it's real, that's one thing. If you look for 1000 associations among data with no true associations, and each one has a 95% of the result being correct, there will be on average 50 false positives that look like associations but aren't. That's effectively what happened in this case, so it's impossible to determine anything from their results.

Yeah this definitely feels like ground level data science exploratory analysis to me. Maybe that’s the problem, that such things would as be published in a medical journal? I’d have to read the paper to truly judge (and I’m not doing that), but it feels like an indicator for further exploration or something to include as a data point in a meta analysis rather than something that holds strong on its own.

​@@cookicha So you think they should ignore any and all results but the ones they expect and can explain? If you know what the results are it is not a experiment it is a processor, ware you fallow a formula. Get the expected results every time wile this is vary handy you don't learn anything from it. How it that good science? Science gets results there not expecting and or are unable to explain all the time. and yes they report that they saw X and no they do not know why it did that so further study is needed. if you only accept the results that are expect and can be explained. nothing is learned and the hole mess is a huge waste of time and money.

Meat is real food and it'll still overload you with saturated fat and cholesterol. There is far more nuances to be had if you plan to be in top shape past 40 years old.

@@DrummerDucky Meat is a great food, depending of the animal, the cut of meat and the way you cook it. Please don't talk nonsense

@@johnspartan12 Legumes are superior in every way - even price - to any form of meat, with a substantially lesser impact on the environment. Bonus : it also allows for a consistent ethical approach to animal welfare.

@@DrummerDucky Meat is superior in the bioavailability of it's proteins, and also in content of B12, so i wouldn't say that legumes are superior in every way, you are right about the environment impact and ethical implications.

If you look for one association and find a 95% chance that it's real, that's one thing. If you look for 1000 associations among data with no true associations, and each one has a 95% of the result being correct, there will be on average 50 false positives that look like associations but aren't. That's effectively what happened in this case, so it's impossible to determine anything from their results. Hopefully that helps clarify the issue.

This study is merely an example. I tried to study up on nutritional science in order to better my diet but it is absolutely infested with studies like these. Not all of them are just as bad as this one but I found it very hard to find even a single good study and even then it is hard to come to any conclusions based on just a single study so generally the nutritional sciences are pretty bad and unreliable.

for real though I'm really glad ya'll are looking at these reports in detail. somebody has got to

If you look for one association and find a 95% chance that it's real, that's one thing. If you look for 1000 associations among data with no true associations, and each one has a 95% of the result being correct, there will be on average 50 false positives that look like associations but aren't. That's effectively what happened in this case, so it's impossible to determine anything from their results.

it isnt scientific in any way , so , no.

@@andyp8464 It is scientific but it's just shitty science.