Classic: Type III Error

You know that eating fruits and vegetables is good for you, right? I tell you that all the time because that’s what the research indicates. But according to an editorial piece in the Journal of the American Medical Association (JAMA), maybe we’ve been wrong all this time (1)—at least as it relates to reducing our risk of cancer. Here’s a recap of the editorial and my take on it.

The Editorial

Mike Mitka is a well respected senior writer for JAMA; he’s published numerous articles that are well researched and contain interviews with the authors and other experts. In a 2010 JAMA article, he writes that an article from the European Prospective Investigation into Cancer and Nutrition (EPIC) clearly demonstrates that high intakes of fruits and vegetables do not reduce the risk of cancer, at least not very much.

Based on the study published in the Journal of the National Cancer Institute, increasing fruit and vegetable intake by six ounces per day reduced the risk of cancer only 3% (2). A couple of well-known researchers supported the notion that research does not support plant intake reducing the risk of cancer, including the director of nutrition for the American Cancer Society.

The problem is that it’s just not true, at least not in that study. The way it’s written, the study suggests that when subjects increased their intake of vegetables and fruits by six ounces, the reduction of cancer risk was a paltry 3%. This was an observational study, not an interventional study. No one kept track of who increased or decreased their plant material intake, and there’s no way to know that from the data collected.

The way I read the article, the EPIC study is central to this new belief about plant consumption not being related to decreased cancer risk. The first articles from the study were published in 2003, and that’s when scientists suggest the evidence grew weaker supporting reducing cancer by plant consumption.

I’ve written about EPIC before; it’s one of the largest observational studies ever conducted with over 520,000 subjects from 23 different centers in 10 European countries. While I suggested that there was strength in numbers when you want to try to tease out subtle effects, I mentioned that the researchers gathered nutritional information at the beginning of the study with a Food Frequency Questionnaire; subjects were asked as many as 256 questions about what they had eaten in the past year. There are two problems with that when you’re looking at the rate of cancer.

First, you have to assume that this one-time questionnaire on diet applies to the patterns of the way the subjects ate and will continue to eat. By the validation studies that were done for the EPIC study itself, that certainly was not true (3). Correlations between what people ate one year apart were as low as 36% in a sub-sample of the subjects. Researchers chose to adjust levels statistically, but that just isn’t the same as actually collecting the data. But when you’re dealing with a half million subjects, you just can’t collect dietary histories on every subject.

Second, the assumption is that the levels that the subjects ate were actually adequate to reduce the rate of cancer. In the study, the average fruits and vegetable intake for men was 17 ounces per day and about 19 ounces per day for women (4). Using an example from the editorial, a medium apple is 10 ounces and a serving of broccoli is about 3.5 ounces. That’s just two servings and that’s close to what the subjects actually ate.

The fact is that most subjects in the study did not eat very many vegetables and fruits and it got worse the further north the country was located. That should have been the overwhelming conclusion of the study to begin with before any other analysis was conducted.

My Take

Trying to explain statistics is not my strong suit, but I adapted this from a blog by Satoshi Kanazawa, an evolutionary psychologist in London and a contributor to Psychology Today (5).

There are two types of errors in statistics. There’s the error of a false positive: you think that the data reflect your hypothesis when it doesn’t. Then there’s the error of false negative: you think the results do not support your hypothesis when it does. Statisticians call the former type of errors “type I errors” and the latter type of errors “type II errors.”

My feeling is that Mitka and researchers who are using the EPIC study to make suggestions about the relationship between plant intake and cancer risk are committing a “type III error.” What’s that? An unwavering belief in statistics: you don’t look at anything but the numbers, and that rules what you believe. Maybe a better way of describing a type III error is hubris, an overbearing pride in statistics. These researchers are very smart people. Did no one consider that the way the data were collected didn’t make sense when tracking diet’s effect on cancer risk? That’s hard to believe unless pride gets in your way.

But when you get right down to it, if you eat an apple and some broccoli on a typical day and someone told you that adding a banana or a tomato would reduce your risk of cancer by 3%, doesn’t it seem like a great idea to add that banana or tomato? Imagine what you could do if you actually ate the recommended eight to ten servings!

The Bottom Line

This will not be the last time I write about EPIC because there are over 500 articles published to date. I still think that eating more vegetables and fruit reduces the risk of all disease including cancer based on the thousands of other studies that say that it does. It seems like the larger the study, the less beneficial effects anything seems to have whether it’s diet, exercise, or supplementation. Maybe someone should start considering whether the effects are being washed out by regressing to the mean. But that’s a message for another day. Time for some berries!

What are you prepared to do today?

Dr. Chet

 

References:
1. JAMA. 2010; 303 (21): 2127-9.
2. J Natl Cancer Inst. 2010 Apr 21;102(8):529-37.
3. International Journal of Epidemiology 1997; 26 (Suppl. 1): S26–S36.
4. Public Health Nutrition: 5(6B), 1179–1196.
5. www.psychologytoday.com/blog/bloggers/satoshi-kanazawa