Thursday, April 21, 2011

New Fatty Liver Study Shows that Carbohydrate Restriction Causes Statistical Anomalies

by Chris Masterjohn

A new study claims to show that carbohydrate restriction is superior to calorie restriction at improving fatty liver disease:

Browning JD, Baker JA, Rogers T, Davis J, Satapati S, Burgess SC.  Short-term weight loss and hepatic triglyceride reduction: evidence of a metabolic advantage with dietary carbohydrate restriction.  Am J Clin Nutr. 2011

In this study the authors randomly assigned 18 people to spend two weeks consuming either a low-calorie diet (~1325 kcal/day) or a low-carb, (<20 g/day), high-protein (33% kcal) diet. The low-carb, high-protein diet led to a greater reduction in the amount of fat stored in the liver.

The authors begin their manuscript by stating as a matter of fact that insulin resistance causes fatty liver, without even acknowledging the competing hypothesis that fatty liver causes insulin resistance. 
 
After the holidays, I'll provide a more comprehensive post addressing whether it makes any sense to single out "carbohydrates" as causing fatty liver (hint: it doesn't), but for now let's take a brief look at this study.



As with any carbohydrate restriction study, there are several competing hypotheses that would be able to explain the results:

  • Restriction of total carbohydrate.
  • Restriction of wheat.
  • Restriction of fructose.
  • Restriction of refined carbohydrates.
  • An altered micronutrient profile.
As can be seen in the figures from my post on meeting the choline requirement, choline is found most abundantly in liver and egg yolks, and with the exception of wheat germ all the foods richest in choline are animal products.  All of the plant foods with half-decent amounts of choline are pretty low in carbohydrate.

Most researchers probably do not think choline plays a major role in fatty liver disease because the most commonly used choline-deficient animal model is one of extreme restriction of choline and methionine, a restriction so extreme that it leads to wasting away instead of obesity.  As I've detailed in my choline posts, however, choline protects against fatty liver in all dietary animal models regardless of whether such an extreme deficiency is present.

Thus, if this study had provided any convincing results, we would have to ask whether these results would have been so impressive if members of the low-calorie group had been given a little liver or egg yolk to boost their choline status.

But just how convincing are these results?

Before randomizing these folks to one or the other group, the authors had already measured hepatic triglyceride (liver fat), and over three quarters of the subjects had biopsies taken as part of their initial diagnosis.

When the authors randomized the subjects, they took into account age, sex, weight, and body mass, but unfortunately didn't consider it a major priority to make sure there was an equal degree of fatty liver between the subjects.

Here is the nonalcoholic fatty liver disease (NAFLD) biopsy score in the two groups at the beginning of the study:

With a p value of 0.16, I suppose we could say that there is "no difference" between the groups, but 16% confidence that there is no difference translates, for me, to 84% worry that there is a difference.  Of course, strictly speaking, there can't possibly be a "true" difference resulting from randomly distributing people to one or another group, so this is really 84% worry that randomization wasn't successful.

(Yes I realize I'm using some liberality in language; statisticians do not quantify "worry.")

And what might I worry about?  Yep, you guessed it: regression to the mean.

The authors, of course, were not looking at the change in biopsy score because biopsies are invasive and they were only taken once.  They were looking at the change in liver fat (hepatic triglyceride).

Lucky for the authors, the relative difference in liver fat at the beginning of the study was only half the relative difference in biopsy score.  Thus the liver fat was similar enough that the authors could state that there was "a similar degree of elevation in both treatment groups."

Nevertheless, let's take a look at the change in liver fat:





If we look back at my regression to the mean post, this looks a lot like the third of the three graphs marked "Change in Cholesterol Over Six Weeks," which I described as "ambiguous."

In the example I gave, I stated the following:
If the difference at the end of the trial is statistically significant, that would suggest the drug was indeed effective, but the fact that random chance produced a similar difference at the beginning of the trial would certainly cast doubt on just how effective the drug really is.
As in that case, the difference between the two groups at the end of the trial (four percentage points) is pretty similar to the difference at the beginning of the trial (three percentage points).  

In this case, however, the difference at the end of the trial is not significant.  The P value for differences between groups is 0.998 which means that if there is no effect whatsoever, we would observe similar differences over 99% of the time.

Now, we could make this look much more impressive if we adjust for the difference at the beginning and present a "change from baseline graph," but if we are to take Newell and Simpson seriously that "the post-treatment difference in means gives the real benefit of new treatment, unbiased by regression to the mean," then we would have to conclude that there is no real benefit of carbohydrate restriction over calorie restriction.  Or, if there is, this study was not large enough or long enough to show it.

Does this study show that both treatments are effective?  Not really.  If patients are diagnosed mainly based on elevated liver fat concentrations, then we would expect any "treatment," regardless of how ineffective it is, to "reduce" liver fat because of regression to the mean.  With an untreated control group, or a control group randomized to a dietary treatment we know to be completely ineffective, we would be able to show the true effect of a real treatment.

I'd love to say that this study shows that consuming more choline-rich animal products decreases liver fat, but in all honesty I don't see how it shows much of anything.  It deserves to be published though.  Maybe some day the raw data can be included in a meta-analysis and we can derive some meaning from it.

Read more about the author, Chris Masterjohn, PhD, here.

11 comments:

  1. I agree that it is flakey, nevertheless it does show a relative trend in favor of low carbohydrate diet regardless of the uneven baseline issue. It is less probable to be purely by chance for the LC (red) as it would be for the CR (blue).

    Without a control group on a null treatment it is harder and more error prone to interpret the statistics, but it is still possible to estimate probabilities - and those are in favor of the LC treatment.

    It is interesting to ask why they allowed the baseline average to be different and why there were no null treatment contorls? An oversight? That kind of mistakes are made and published so frequently that it makes me question an integrity and mathematical qualifications of the scientists in general, medical in particular.

    Regards,
    Stan (Heretic)

    ReplyDelete
  2. Stan,

    I like your fair-handed analysis, but in all honesty the probability in favor of the low-carb group is quite low -- approximately 50%.

    I say this based on comparing between groups both before and after by t test, and making a single adjustment for making two comparisons. 50% confidence in favor of the low-carb group relative to no effect is indeed favoring the hypothesis that low-carb is better over the hypothesis that low-calorie is better, but it is not at all favoring the hypothesis that low-carb is better over the hypothesis that there is no difference whatsoever.

    Thus, I think saying a 50% chance of low-carb being better relative to no effect is favoring the efficacy of low-carb is being much too generous to the hypothesis than the data justify.

    Chris

    ReplyDelete
  3. To briefly speculate about the answer to your question, I don't think these things are often given much consideration. A university will often have the stats dept give free consulting to the research depts, but this takes the form of researchers getting consulting about how to "make this significant" after they have performed the experiment, rather than how to design it. Graduate programs do not require enough stats courses in PhD programs. Older researchers were steeped in research in a time where stats were often not even used, and do not have a high opinion of their necessity. Publish-or-perish-oriented researchers view statisticians as "antagonistic" to research. Thus, stats is a separate, isolated field in most cases, and most research is consequentially rubbish. That's my opinion.

    Chris

    ReplyDelete
  4. More proof you're not a homer. That's why I read your stuff. Happy Easter.

    ReplyDelete
  5. "I'll provide a more comprehensive post addressing whether it makes any sense to single out "carbohydrates" as causing fatty liver (hint: it doesn't)"

    My guess is that your choline series already provided the answer implicitly.

    Copy-paste of part of a comment I posted at Stephan's Fluid-in Fluid-out post:
    -----
    My lipostat/liver hypothesis was inspired by Chris Masterjohn's choline series (Nov 2010).
    From that, the criteria for fatty liver seem to be:
    1) Provide the liver with high energy.
    2) Do NOT provide the liver with nutrients it needs to process that energy.
    It's a bit like burning lots of gasoline in an engine, without providing oil or coolant.
    -----

    Most carbohydrates end up as glucose in the blood stream. I speculate that glucose will be disposed of mostly outside of the liver. So the liver will hardly be presented with energy from glucose, i.e., criterion 1 not met → no fatty liver.

    The story is different for "liver exclusives" like fructose or alcohol: they seem to be disposed of almost entirely by the liver, i.e., criterion 1 easily met, so if criterion 2 is met as well → fatty liver.

    I'm looking forward to your post after the holidays.


    John

    ReplyDelete
  6. Either Cooling Inflammation or Hyperlipid posted about a rat study that found alcohol plus saturated fat = no fatty liver, alcohol without saturated fat = fatty liver. It seems the LC group probably was eating more saturated fat. Whether this works in humans I wouldn't know.

    ReplyDelete
  7. Chris,
    Great post. Once again you are keeping the science real!

    ReplyDelete
  8. About the alcohol plus saturated fat study: I believe Chris blogged about this.

    If I remember correctly, the saturated fat was coconut oil, which of course is special: it contains about 60% medium-chain fatty acids. These are "liver exclusives" as well. So the energy of both alcohol and medium-chain fatty acids is provided to the liver.

    Note that I don't have a biochemistry education, so I don't know if the following reasoning is realistic: before medium chain fatty acids can go through β-oxidation, they apparently have to go through ω-oxidation first.
    Step one of ω-oxidation requires NADPH. And here's the catch: NADPH seems to also be required for fatty acid synthesis. No NADPH means that fatty acid synthesis is replaced by ketone formation. Ketones are water soluble, so in that case the liver doesn't require choline to export the energy into the blood.

    So my uneducated guess would be that the medium-chain fatty acids in coconut oil deplete NADPH, thereby causing most of the alcohol energy to be exported as ketones. This would mean little to no fat accumulation in the liver.


    John

    ReplyDelete
  9. Two weeks of study. Humans live between 70 and 80 years. A complex biological system of any sort does not simply find stasis in a couple of weeks, does it?

    A two week study on any biological level seems to render the result statistically irrelevant. I know that long term studies are notoriously difficult, but geeze - sometimes I wonder if the researchers grew with video games or ADD ...

    Nice Blog, by the way. I like your humility.

    ReplyDelete
  10. 'Nice Blog, by the way. I like your humility.'

    Ditto! Exactly what I was thinking. I do think this blog is excellent.

    ReplyDelete
  11. Thank you for posting this research. I think restriction of refined carbohydrates is the major factor. Refined carbohydrates are the primary source of "fast blood glucose" excess of which causes serious metabolism issues.

    ReplyDelete

To create a better user experience for everyone, comments are now moderated. Please allow up to one business day for your comment to post. In order to avoid the appearance of spam, please avoid posting links, especially to commercial destinations, and using all-caps.