The Journal of the American Veterinary Medical Association (JAVMA) recently published an article written by two veterinarians about homemade and commercially made raw meat-based diets. (A similar article, based on the same data, was published in the AKC Gazette.) The article has been regarded by many proponents of homemade diets as a warning shot fired by the commercial-food -producing community and their traditional veterinary medical footsoldiers over the bow of the raw feeding ship.
The authors – who are both faculty members at prominent veterinary schools – stated that many arguments exist to support both sides of the issue – that, on one side, raw food diets are healthful, and on the other, that raw food diets can put dogs at risk of nutritional imbalances, complications from ingesting raw bones, and bacterial contamination. Further, they suggested that “neither side has provided evidence to prove that their argument is correct.” Generously, they proposed that they would “objectively assess whether raw food diets prepared by typical pet owners are nutritionally balanced and safe.” So far, so good. Some objective, scientifically based research studies about raw diets would benefit everyone.
Inadequate design
However, in our opinion, the study the authors designed and set up was woefully inadequate for the purposes they intended – unless the purpose was, as some raw diet proponents suspect, to simply publish an article that was critical of raw diets, without regard for the actual results of the study.
To begin with, the authors performed a single analysis of five different diets, including a single sample of three homemade meals that included raw meats and other foods, and two commercial raw meat-based diets.
The authors suggested that each of the homemade diets was prepared according to a raw-diet expert’s outline – one based on Dr. Ian Billinghurst’s BARF (Bones and Raw Food) diet, one based on Wendy Volhard’s “Natural Diet,” and one based on Kymythy Schultze’s “Ultimate Diet.” The commercial diets included Steve’s Real Food for Dogs (a frozen raw-meat-based food that includes fruits, vegetables, and supplements) and Sojourner’s Farm, a grain-based supplement that is added to fresh raw meat. The authors said they prepared this diet to the manufacturer’s directions.
Poor sampling
Before we talk about the analysis itself, a word about the samples tested by the authors.
While the authors proposed that their study would enable them to draw conclusions about the nutritional adequacy and balance of the raw food diets, they knew what the ingredients were in only the two commercial diets. They asked three people they knew who regularly fed a raw-food diet to their dogs to prepare a sample of the diet; however, the authors admit that “Exact recipes of the diets were not provided to us.” Therefore, even though these three samples were taken to be representative of the feeding theories forwarded by Dr. Ian Billinghurst, Wendy Volhard, and Kymythy Schultze, they could not even know whether the owners were preparing the diets correctly, according to instructions given by the originators of the diets.
In fact, it’s fairly clear that the diets were not precisely representative of the raw-food diet experts. For example, the article mentions that all the diets contained “entire breasts or legs of chickens.” However, none of the three homemade diet programs recommend these cuts – Billinghurst and Schultze both recommend chicken wings, necks and backs, which have less meat and more bone, to get the proper calcium/phosphorus balance. Volhard’s basic diet uses beef plus a calcium supplement. All three diets also include occasional feeding of organ meats, but we do not know whether these were included.
Complicating the analysis even more, the samples of homemade diets may have been even less representative of the total diet because of the way they were collected. The person who provided the authors with a sample of a diet based on Kymythy Schultze’s “Ultimate Diet” told Schultze that, for two weeks, every time he fed his dog, he would throw a little bit of whatever he was feeding into a bag in the freezer.
So, if he fed the dog a banana as a snack, he put a chunk of banana in the bag, and if he fed two pounds of meat, a chunk of meat was added. Food items were not added to the bag in proportion to the amount of the item actually being fed to the dog.
Surely the authors must have been aware that a sample collected in this manner could not fairly represent any diet. But because they did not insist that all participants provide them with an exact recipe for their samples, they have no way of knowing whether the samples are prepared according to the diet originator’s directions.
Too many conclusions from too little data
The first and most obvious problem is that the authors drew conclusions about complex feeding programs from analyzing a single sample – and one that was not guaranteed to resemble the diet it was supposed to represent. Unfortunately, the next set of problems are even more egregious.
The authors submitted the five samples to a testing laboratory for analysis of various nutrients, such as protein, fat, vitamins, and minerals, as well as contaminants like bacteria. They then compared these results against the standard for commercial dog food, the AAFCO Nutrient Profile. The result? According to the article, “All the diets tested had nutrient deficiencies or excesses that could cause serious health problems when used in a long-term feeding program.”
The authors concluded, “There are clearly nutritional and health risks associated with feeding raw food diets.”
This study, with its accompanying tables indicating the various “nutrient deficiencies and excesses” of the raw diets, published as it was in a peer-reviewed, reputable veterinary publication, undoubtedly convinced many readers (most of them veterinarians) that raw food diets are harmful.
Doing the math
Fortunately, the article was also read – closely – by people who refused to accept the tables and text at face value, and actually “did the math” to double-check the authors’ allegations.
Steve Brown, President of Steve’s Real Food for Dogs (the maker of one of the two commercial diets studied by the authors) challenged the figures published in the JAVMA article. Brown has had his diet analyzed by commercial laboratories many times in order to be able to provide a “Guaranteed Analysis” on the label of Steve’s Real Food for Dogs that proved the nutritional completeness of his product; he was convinced that, in at least the case of his food, the authors’ numbers could not be correct.
Brown was subsequently provided with “corrected” data by the authors’ attorneys. Sixty percent of the values were different the second time around. No, there was no new testing done, but all the values were recalculated and the majority – as published in JAVMA – were wrong. In more than a dozen instances, the original values were “off” by a factor of 10 or more – in five instances, the decimal point had been put in the wrong place. For at least seven values, when the initial error was corrected, the food then met those requirements. For the two commercial raw diets, only one value out of 28 stayed the same; the magnitude of the errors ranged from 4 to 91 percent.
In a letter to the editor of JAVMA, one of the authors apologized for their “inadvertent computational errors.” Pressed by Brown and others, JAVMA later published a set of “corrected tables;” Brown has found yet more flaws within these tables.
However, the authors do not seem to be in a frame of mind to let facts get in their way. In the May 15 issue of JAVMA, one of the authors wrote, “Even considering these corrections, nutritional imbalances were present, and the overall reservations about raw food diets expressed in our article remain unchanged.”
Real science
One of the major tenets of a “scientific” study is that it must be replicable. Precise methodology should be followed so that other scientists can duplicate the experiment, to see whether the same results occur. However, no one can replicate this data – the authors don’t even know what was in the food! No recipe, not even a list of ingredients, nothing.
Another philosophically vexing problem with this entire issue is the authors’ discernible prejudice. They acknowledge that the arguments in favor of raw food diets are “plausible,” but then seem to spend the rest of the paper trying very hard to prove how terrible such diets are. This is hardly an example of “scientific” thinking, which should be open-minded, unbiased, and rational.
The fact is, no studies have been done to compare the relative healthfulness of raw versus commercial diets – and, conducted in a scientific fashion by a neutral party, this is going to be what it takes to put this controversy to rest.
The tragedy of this flawed study is that the damage has already been done. There are 65,000 veterinarians in the AVMA, and most of them receive this journal. All but about 3,000 of them practice conventional medicine, recommend commercial dog foods, and prescribe commercial therapeutic diets. This article has given them seeming justification for their skepticism about raw food diets. Even if corrections are published, these readers’ opinions have already been influenced. Although JAVMA has printed a series of small corrections and letters to the editor on the topic, without a highly publicized retraction, correction, and/or apology – given about the same space and prominence as the original article – it is unlikely that many will feel it necessary to reconsider their newly solidified positions.
As far as we’re concerned, one of the oddest parts of this whole tale is the report from one participating owner who said that, in all the time he was dealing with the authors, they never once asked him if his dog was healthy. Wouldn’t that be the first question to ask?
-By Jean Hofve, DVM
Dr. Jean Hofve is the Companion Animal Program Coordinator for the Animal Protection Institute, located in Sacramento, California.
