11 ms·

How Big Data Transformed the Dairy Industry

79 points by altrus 14y ago

edhallen 14y ago
This is a great addition to the unfortunately too limited category of articles about how big data / analysis changed actual decision-making. That said, the definition of big data used by this article doesn't strike me as what I would think of as big data. Based on what it says, the dairy industry was changed by analysis and data collection but nothing that couldn't be stored on your typical phone. The article hints at big data (via greater genetic analysis) transforming the dairy industry in the future, but the massive changes in cow DNA so far are seemingly due to "small" data. This confusion of big data with just solid analysis and decision-making happens a lot, but does a good job of highlighting how much progress there is to be made in using data to drive decisions (independent of how much data we use).
- huggyface 14y ago
  You've written what I was planning on posting. This is classic data aggregation and analysis. while Big Data is a trendy term, I see and perceive nothing in it that qualifies as even a particularly large set, much less what most see as Big Data.
Falling3 14y ago
I realize I'm coming at this from a different angle than a lot of other HNers, but for me this is good science gone wrong. Dairy cows are some of the most mistreated and unhealthy animals in the agriculture system. This push for efficiency has created a large group of animals with very little genetic diversity. They are kept on a stream of antibiotics because one bad outbreak can take down the entire group. They often develop mastitis and cancer is common by the they are 5 or 6 ("retirement age") due to poor nutrition, over-milking, and living conditions. This kind of science really excites me, but the applications are almost always depressing and we should absolutely not be lauding squeezing out every last "inefficiency" in an animal based industry.
- JoeAltmaier 14y ago
  Lots of worse inbreeding problems - show dogs, fish etc. I live in Iowa, sorry I haven't seen the mistreatment you mention. I'm sure its a problem, any time large amounts of animals become dollars. Puppy mills, fish farms. But compared to beef cattle, dairy have it made. I'm not even sure humans are immune. Talk about a constant stream of antibiotics, low genetic diversity, inhumane living conditions, cancer rates.
  - Falling3 14y ago
    Not to run off on a tangent, but you're wrong about the dairy beef comparison. Before heading to a feedlot, most beef cattle spend around a year of their lives grazing often allowed to stay with their mothers. Beef cattle are killed much sooner, but have something resembling a natural life for a time. Newborn dairy calves are removed almost immediately so no milk is wasted on them. Male dairy cows are used for veal (and I'm sure you know how that goes). The females are placed into small "igloos" with no chance for grazing or physical contact until they are old enough to reproduce. Then it's 4 or 5 years of constant pregnancy and milking until they are "spent" at the ripe ole age of 6. I'm honestly not trying to be preachy here as I know this isn't the appropriate venue, but this is just not an area I feel anyone should be commending for their efficiency.
    - randomdata 14y ago
      I grew up on a dairy farm, and while I'm sure some farmers operate exactly as you describe, I'm not sure you should paint the whole industry with one brush. Every farmer has their own opinions on how their operation should be operated, so every farm will be vastly different. > They are kept on a stream of antibiotics This was never true on our farm. Cows would be treated if sick, but that is it. I don't even remember them being sick all that often. The biggest problems were always related to calving complications (see below). > They often develop mastitis This largely came down to choices made by the animal, like laying in the muck even when there's a nice fresh bed of straw nearby. No farmer wants to see a cow with mastitis, even if just for profit reasons so you always do your best to try to avoid the problem. > The females are placed into small "igloos" with no chance for grazing or physical contact until they are old enough to reproduce. You mean until they are old enough to feed on their own. The reason they are individualized in early life is because otherwise the stronger calves will eat all of the food and who wants to see a calf starve? Even while in the hutch ("igloo"), the calves are free to interact with the nearby calves in other hutches. > Male dairy cows are used for veal (and I'm sure you know how that goes). I'm not sure the general population does know how that goes. I've heard all kinds of horrific stories about how veal calves, specifically, are mistreated. The only differentiator for veal is the age of the animal, otherwise they live like any other beef calf. > Then it's 4 or 5 years of constant pregnancy and milking until they are "spent" at the ripe ole age of 6. I remember plenty of cows that were milked for far longer, but we definitely lost a lot during calving, and as a result of that 4-5 years probably isn't far off the average milking span. Sadly, calving is hard on the animal. To be fair, this is something humans equally struggled with until the advent of modern medicine.
Volscio 14y ago
Just a note: published May 1, 2012
fghh45sdfhr3 14y ago
A high degree if genetic selections is a high risk thing. For a long time you just get a lot more of what ever you want, milk, eggs, etc. And then a virus appears and kills 99.9% or more of your animals because they were all so closely related. The free market rewards the up side of closely related species but does not cost you anything for the potential catastrophe. Day to day the probability of a catastrophic disease is near 0, but in the very long term it is near certain. Our free markets have no smart way of pricing that kind of stuff. That's why we've already lost several varieties of banana to diseases.
- Falling3 14y ago
  Oh yeah that's particularly scary (if you happen to like bananas). Most commercial bananas plants are clones. This leads to strict regulations on commercial bananas and it's still pretty likely we'll see them disappear one day.
  - Qworg 14y ago
    Woah, wait a moment. One variety of commercial banana is mostly clones (the traditional yellow Chicquita/Dole type). There are many other kinds of bananas - less familiar, but also different genetically. Bananas won't disappear - only one type will.
    - Falling3 14y ago
      Yes, but doesn't that one commercial banana account for most of the (US) sales?
      - Qworg 14y ago
        Yes, but we already had to switch once (as per the links above) - we'll do it again if we're forced to.
- pm90 14y ago
  Hey, I found your comment on bananas interesting. Where can I read up more about it?

79 points by altrus 14y ago

This is a great addition to the unfortunately too limited category of articles about how big data / analysis changed actual decision-making. That said, the definition of big data used by this article doesn't strike me as what I would think of as big data. Based on what it says, the dairy industry was changed by analysis and data collection but nothing that couldn't be stored on your typical phone. The article hints at big data (via greater genetic analysis) transforming the dairy industry in the future, but the massive changes in cow DNA so far are seemingly due to "small" data. This confusion of big data with just solid analysis and decision-making happens a lot, but does a good job of highlighting how much progress there is to be made in using data to drive decisions (independent of how much data we use).

huggyface 14y ago
You've written what I was planning on posting. This is classic data aggregation and analysis. while Big Data is a trendy term, I see and perceive nothing in it that qualifies as even a particularly large set, much less what most see as Big Data.

Falling3 14y ago

I realize I'm coming at this from a different angle than a lot of other HNers, but for me this is good science gone wrong. Dairy cows are some of the most mistreated and unhealthy animals in the agriculture system. This push for efficiency has created a large group of animals with very little genetic diversity. They are kept on a stream of antibiotics because one bad outbreak can take down the entire group. They often develop mastitis and cancer is common by the they are 5 or 6 ("retirement age") due to poor nutrition, over-milking, and living conditions. This kind of science really excites me, but the applications are almost always depressing and we should absolutely not be lauding squeezing out every last "inefficiency" in an animal based industry.

JoeAltmaier 14y ago
Lots of worse inbreeding problems - show dogs, fish etc. I live in Iowa, sorry I haven't seen the mistreatment you mention. I'm sure its a problem, any time large amounts of animals become dollars. Puppy mills, fish farms. But compared to beef cattle, dairy have it made. I'm not even sure humans are immune. Talk about a constant stream of antibiotics, low genetic diversity, inhumane living conditions, cancer rates.
- Falling3 14y ago
  Not to run off on a tangent, but you're wrong about the dairy beef comparison. Before heading to a feedlot, most beef cattle spend around a year of their lives grazing often allowed to stay with their mothers. Beef cattle are killed much sooner, but have something resembling a natural life for a time. Newborn dairy calves are removed almost immediately so no milk is wasted on them. Male dairy cows are used for veal (and I'm sure you know how that goes). The females are placed into small "igloos" with no chance for grazing or physical contact until they are old enough to reproduce. Then it's 4 or 5 years of constant pregnancy and milking until they are "spent" at the ripe ole age of 6. I'm honestly not trying to be preachy here as I know this isn't the appropriate venue, but this is just not an area I feel anyone should be commending for their efficiency.
  - randomdata 14y ago
    I grew up on a dairy farm, and while I'm sure some farmers operate exactly as you describe, I'm not sure you should paint the whole industry with one brush. Every farmer has their own opinions on how their operation should be operated, so every farm will be vastly different. > They are kept on a stream of antibiotics This was never true on our farm. Cows would be treated if sick, but that is it. I don't even remember them being sick all that often. The biggest problems were always related to calving complications (see below). > They often develop mastitis This largely came down to choices made by the animal, like laying in the muck even when there's a nice fresh bed of straw nearby. No farmer wants to see a cow with mastitis, even if just for profit reasons so you always do your best to try to avoid the problem. > The females are placed into small "igloos" with no chance for grazing or physical contact until they are old enough to reproduce. You mean until they are old enough to feed on their own. The reason they are individualized in early life is because otherwise the stronger calves will eat all of the food and who wants to see a calf starve? Even while in the hutch ("igloo"), the calves are free to interact with the nearby calves in other hutches. > Male dairy cows are used for veal (and I'm sure you know how that goes). I'm not sure the general population does know how that goes. I've heard all kinds of horrific stories about how veal calves, specifically, are mistreated. The only differentiator for veal is the age of the animal, otherwise they live like any other beef calf. > Then it's 4 or 5 years of constant pregnancy and milking until they are "spent" at the ripe ole age of 6. I remember plenty of cows that were milked for far longer, but we definitely lost a lot during calving, and as a result of that 4-5 years probably isn't far off the average milking span. Sadly, calving is hard on the animal. To be fair, this is something humans equally struggled with until the advent of modern medicine.

Volscio 14y ago

Just a note: published May 1, 2012

fghh45sdfhr3 14y ago

A high degree if genetic selections is a high risk thing. For a long time you just get a lot more of what ever you want, milk, eggs, etc. And then a virus appears and kills 99.9% or more of your animals because they were all so closely related. The free market rewards the up side of closely related species but does not cost you anything for the potential catastrophe. Day to day the probability of a catastrophic disease is near 0, but in the very long term it is near certain. Our free markets have no smart way of pricing that kind of stuff. That's why we've already lost several varieties of banana to diseases.

Falling3 14y ago
Oh yeah that's particularly scary (if you happen to like bananas). Most commercial bananas plants are clones. This leads to strict regulations on commercial bananas and it's still pretty likely we'll see them disappear one day.
- Qworg 14y ago
  Woah, wait a moment. One variety of commercial banana is mostly clones (the traditional yellow Chicquita/Dole type). There are many other kinds of bananas - less familiar, but also different genetically. Bananas won't disappear - only one type will.
  - Falling3 14y ago
    Yes, but doesn't that one commercial banana account for most of the (US) sales?
    - Qworg 14y ago
      Yes, but we already had to switch once (as per the links above) - we'll do it again if we're forced to.
pm90 14y ago
Hey, I found your comment on bananas interesting. Where can I read up more about it?