8 ms·

I've been following this story for a while. When people say the soft serve machine is "broken", they really mean "unavailable to serve ice cream to customers".

I've been following this story for a while. When people say the soft serve machine is "broken", they really mean "unavailable to serve ice cream to customers". That could mean something as innocuous as performing a pasteurization cycle during the day (because it failed to run at night for some reason, perhaps the machine's clock is set wrong). It could also mean there is a physical problem with some part of the machine. The word "broken" implies the latter, but I'm pretty sure the former situation is far more common. In other words, the machine has determined it's unsafe to serve the contents (because soft serve needs to be pasteurized at certain intervals), not that it's physically incapable of doing so. Therefore it's "broken". (Disclaimer: I've never used one of these soft serve machines, just read through manuals and stories about them.)

runnerup 5y ago
Also it can mean that the first half of the lunch rush used up all the frozen ice cream and it can’t freeze new ice cream instantly. So the later part of a rush period will often have unacceptable soft ice cream. When I ran frozen daiquiri machines it took a lot of finesse to make it through a busy night. Have to constantly manage the temperatures and levels to a nearly absurd precision while doing the rest of your job as well. But the only parameters you can manipulate are material input and material output, there’s no “freeze it faster” button or “store it colder” dial.
- BenjiWiebe 5y ago
  Are you sure that's ever an issue with McDonald's? I've been under the impression that McDonald's Taylor ice cream machines are freeze-on-demand.
  - runnerup 5y ago
    If that were the case I would never get soggy/wet/melted McFlurries, but I do sometimes.
mod 5y ago
Honestly, in my area they usually say the machine is "down." It's always down.
- StillBored 5y ago
  The one near my house removed it from their menu due to menu simplification during the pandemic, and despite bringing the menu back, not to long after, the ice cream products never returned. Its just missing from the menu, and they tell you they don't sell ice cream.
chongli 5y ago
It also needs to be said that similar machines provided by Taylor to McDonalds' competitors do not have anywhere close to the amount of downtime that McDonalds' machines have. There's also ample evidence that software updates to Taylor's McDonalds machines have made the error codes more cryptic and the machines more unreliable over time. I think there's a pretty strong legal case to be made that these machines are deliberately designed to break down in order to drive repair service revenue for both Taylor and McDonalds at the expense of franchisees. Unfortunately, franchisees seem to be locked up under binding arbitration clauses in their franchise agreements. Luckily for consumers, 3rd party vendors aren't subject to these contracts.
- gruez 5y ago
  >There's also ample evidence that software updates to Taylor's McDonalds machines have made the error codes more cryptic and the machines more unreliable over time. Source? >It also needs to be said that similar machines provided by Taylor to McDonalds' competitors do not have anywhere close to the amount of downtime that McDonalds' machines have >I think there's a pretty strong legal case to be made that these machines are deliberately designed to break down in order to drive repair service revenue for both Taylor and McDonalds at the expense of franchisees. AFAIK mcdonalds machine are also self-cleaning, which makes them more complicated and prone to failure. There's also advantages though (ie. you don't have to rely on a minimum wage worker to adequately clean the machine to prevent food poisoning), so absent some sort of evidence showing intent, it's at least plausible that there isn't some sort of conspiracy to drive up repair costs.
  - 14 5y ago
    I don’t know the source but definitely did read an article here on HN last year reporting this. Call me a conspiracy theorist but I would bet McDonald’s is paying a pr firm to get reports like that taken away off the net.
    - registeredcorn 5y ago
      If you do find the article, please let us know.
ojosilva 5y ago
> the machine has determined it's unsafe to serve the contents Wouldn't that mean, though, that the Kytch device could be working around the issue in detriment of the patron's, or "end user", safety? The franchisees certainly were very happy with Kytch's reduction in their maintenance and downtime cost, caused by very temperamental, ill-designed machines. Franchisee's are clearly victims of a tightly controlled McD ecosystem (sounds so Apple...). But if the machine is operating in a way that could harm users, even if harmful workarounds is just 1 or 2 out of 30 other harmless issues Kytch monitors, fixes or clarifies, still, it would absolutely justify the take down! Imagine a batch of people who suddenly get a bacteria from unpasteurized ice cream from a Kytch-hacked McD machine. How could McD justify knowing their franchisee's were installing a device that could put their patrons in risk of eating a spoiled dairy products? It's a backdoor for a health crisis, a PR nightmare and endless lawsuits. Even if it doesn't know for sure the device can be blamed, the liability is there.
- tfvlrue 5y ago
  I don't have any first-hand knowledge of what the Kytch device did. But nothing I've read about it ever gave me the impression that it was overriding lockouts. It was intended as a remote monitoring device and troubleshooting tool. For example, instead of an employee having to read a pattern of blinking lights on the machine itself, a franchise owner might receive an email that says "soft serve machine cleaning required" and send somebody to take care of it.
  - ojosilva 5y ago
    Regardless, like I said, the Kytch box (a Raspberry Pi) is overriding the code that accesses the secret menu of the ice-cream machines [1]. That's outside the intended operation. And that's liability if McD knows about it and says nothing. The machine produces food that goes into people's bodies. Ie, if it were to produce a poisonous glycol ice-cream (glycol one of the pasteurization ingredients) all of a sudden, even if it has nothing to do with what Kytch is intended to do, could you really 100% discard in front of a judge that a glitch or a hack in the Kytch's linux computer, which is btw connected to the internet, didn't order the ice-cream machine to do that which, as a consequence, killed a hypothetical 5-year old kid from food poisoning? Imagine McD is now Boeing and the Kytch box is something airlines found that make 737-MAX fly safer by plugging into and overriding the system codes so that it can warn pilots of possible 737-MAX issues... Does it make sense for Boeing to allow such thing even if it makes planes safer? [1] https://www.wired.com/story/they-hacked-mcdonalds-ice-cream-makers-started-cold-war/ https://www.wired.com/story/they-hacked-mcdonalds-ice-cream-...
wronglebowski 5y ago
I've commented on this previously as I worked as a jack of all trades for a McDonald's franchisee for years. The machines tanks and shaving cores must be disassembled and cleaned every two weeks. They have a very intelligent series of sensors and internal mechanisms that make it impossible to defeat or cheat. This is not a low skilled job. I'd say it would be easier to change your cars oil than complete this. There's multiple gaskets and seals which need to be rotated quarterly in a tune up kit. While we were hiring at minimum wage the shake machine trained people were making 15$. I can only imagine what they're making now. I don't find the "cryptic messages" issue to be valid. New machines had new systems and codes, it wasn't malicious. Everyone had access to the manual and the Taylor provider in my region did some phone troubleshooting free of charge to prevent service calls. They even did free annual trainings. Mechanically older machines can have issues, and some franchisees can be stubborn about replacing them. However I believe the vast majority of the time the issue is the machine not being cleaned.
- TedDoesntTalk 5y ago
  Why are these machines so complicated? I thought you just need a few ingredients to make ice cream?
  - wronglebowski 5y ago
    It's optimized for high capacity, high throughput, low planning, production. Do you sell 1 milkshake or 100 a day? Doesn't matter, the tank holds the mix and freezes it on the fly. Taylor machines can output hundreds of units an hour. Or none. It's always "fresh", there's waste when you clean the machine but other than that the ice cream you get has only been frozen seconds before being served. It's highly profitable, when the machine is operating correctly the base food cost is much lower than the other food categories(sandwiches and chicken) but not as good as soda and cookies.
  - tapland 5y ago
    It's a pretty clunky machine and anything in the machinery making or dispensing the ice cream needs to be cleaned thoroughly for food safety reasons. A lot of people would find the disassembly and correct reassembly difficult. And there are huge incentives to just not do it.
s_dev 5y ago
How come these issues aren't existent in McDonald's in Europe?
- grouchomarx 5y ago
  there's a more reliable Italian machine that is also available to franchisees. most (all) american locations use the Taylor. I assume the opposite is true in europe
amelius 5y ago
So "broken" -> "mismanaged".
kube-system 5y ago
"Broken" is simply the colloquial term used to avoid arguments with customers. Clarity is a much higher customer-service priority than precision.
selfhoster11 5y ago
Machine clock being wrong? IDK about the US, but in the UK we have at least 2 different radio-clock signals to choose from for machine synchronisation (one is domestic, one is based in Germany so you can still get it if reception is good). Neither requires internet access or any setup, and is cheap enough to integrate into sub-$30 electronics.
rascul 5y ago
Broken could also mean that nobody has bothered to refill the machine. In my experience working at McDonald's many years ago, it was actually incredibly rare for the ice cream machine to be down for any reason besides nobody has filled it. When cars are wrapped around the building, line is out the door, and you're short two people, nobody is going to take the five minutes to fill the ice cream machine.
- loudtieblahblah 5y ago
  I'm always torn between my disdain for swinging down at the working class and my experience from the time I was 14 until I was 26, My politics have always aligned with something far more left wing than the Democrats. Pro-private unions, pro m4a, pro welfare assistance, higher minimum wages, I wish people had guaranteed sick and vacation time, better work safety protections, better paternity/maternity leave. And a culture that saw jobs that didn't take a degree as something that could still be worked with a since of dignity. But in my actual experience working... I worked just about anything you can think of.. Starbucks, Applebee's, TGIFridays,a BBQ restaurant, Sallie Beauty Supply, CVS, Rite-Aid, CompUSA, video rentals, grocery stores, I worked construction, painted, did lots of odd jobs. And the types of people I met along the way.... The adults in restaurants, pharmacies and retail stores.. who eclipsed me in years anyways, to be frank, were losers. Lazy, shiftless people who had to be told, constantly, to do the basics even when you had entire days when there was little customizer traffic or stock to take care of People who would openly lie about "oh my back" and sit on a stool not doing shit all day, then go off to party at night. People who wouldn't see a customer for hours and couldn't be fucked to "front and face" or do some basic cleaning of the store without being told to.. At 40, 50, 55 years old. I can't tell you how many people at various jobs who got arrested bc they were jacking shit from the store and selling hit items. I worked a toy store in a mall during the Xmas season once and the manager was cleaning the inventory out by 30% and flipping it every week. These were always people with kids before they were 18, or had multiple kids from multiple partners, drug use was way more open and regular than anything you see in the office. You go to McDonald's and you see more tattoos and "grills" than you see any sense. Do you think they give af about an ice cream machine? But the bigger question, is why in 2022 is anyone still buying toxic crap from fast food restaurants? Buy a bag of beans, live a couple years longer.

runnerup 5y ago

Also it can mean that the first half of the lunch rush used up all the frozen ice cream and it can’t freeze new ice cream instantly. So the later part of a rush period will often have unacceptable soft ice cream. When I ran frozen daiquiri machines it took a lot of finesse to make it through a busy night. Have to constantly manage the temperatures and levels to a nearly absurd precision while doing the rest of your job as well. But the only parameters you can manipulate are material input and material output, there’s no “freeze it faster” button or “store it colder” dial.

BenjiWiebe 5y ago
Are you sure that's ever an issue with McDonald's? I've been under the impression that McDonald's Taylor ice cream machines are freeze-on-demand.
- runnerup 5y ago
  If that were the case I would never get soggy/wet/melted McFlurries, but I do sometimes.

mod 5y ago

Honestly, in my area they usually say the machine is "down." It's always down.

StillBored 5y ago
The one near my house removed it from their menu due to menu simplification during the pandemic, and despite bringing the menu back, not to long after, the ice cream products never returned. Its just missing from the menu, and they tell you they don't sell ice cream.

chongli 5y ago

It also needs to be said that similar machines provided by Taylor to McDonalds' competitors do not have anywhere close to the amount of downtime that McDonalds' machines have. There's also ample evidence that software updates to Taylor's McDonalds machines have made the error codes more cryptic and the machines more unreliable over time. I think there's a pretty strong legal case to be made that these machines are deliberately designed to break down in order to drive repair service revenue for both Taylor and McDonalds at the expense of franchisees. Unfortunately, franchisees seem to be locked up under binding arbitration clauses in their franchise agreements. Luckily for consumers, 3rd party vendors aren't subject to these contracts.

gruez 5y ago
>There's also ample evidence that software updates to Taylor's McDonalds machines have made the error codes more cryptic and the machines more unreliable over time. Source? >It also needs to be said that similar machines provided by Taylor to McDonalds' competitors do not have anywhere close to the amount of downtime that McDonalds' machines have >I think there's a pretty strong legal case to be made that these machines are deliberately designed to break down in order to drive repair service revenue for both Taylor and McDonalds at the expense of franchisees. AFAIK mcdonalds machine are also self-cleaning, which makes them more complicated and prone to failure. There's also advantages though (ie. you don't have to rely on a minimum wage worker to adequately clean the machine to prevent food poisoning), so absent some sort of evidence showing intent, it's at least plausible that there isn't some sort of conspiracy to drive up repair costs.
- 14 5y ago
  I don’t know the source but definitely did read an article here on HN last year reporting this. Call me a conspiracy theorist but I would bet McDonald’s is paying a pr firm to get reports like that taken away off the net.
  - registeredcorn 5y ago
    If you do find the article, please let us know.

ojosilva 5y ago

> the machine has determined it's unsafe to serve the contents Wouldn't that mean, though, that the Kytch device could be working around the issue in detriment of the patron's, or "end user", safety? The franchisees certainly were very happy with Kytch's reduction in their maintenance and downtime cost, caused by very temperamental, ill-designed machines. Franchisee's are clearly victims of a tightly controlled McD ecosystem (sounds so Apple...). But if the machine is operating in a way that could harm users, even if harmful workarounds is just 1 or 2 out of 30 other harmless issues Kytch monitors, fixes or clarifies, still, it would absolutely justify the take down! Imagine a batch of people who suddenly get a bacteria from unpasteurized ice cream from a Kytch-hacked McD machine. How could McD justify knowing their franchisee's were installing a device that could put their patrons in risk of eating a spoiled dairy products? It's a backdoor for a health crisis, a PR nightmare and endless lawsuits. Even if it doesn't know for sure the device can be blamed, the liability is there.

tfvlrue 5y ago
I don't have any first-hand knowledge of what the Kytch device did. But nothing I've read about it ever gave me the impression that it was overriding lockouts. It was intended as a remote monitoring device and troubleshooting tool. For example, instead of an employee having to read a pattern of blinking lights on the machine itself, a franchise owner might receive an email that says "soft serve machine cleaning required" and send somebody to take care of it.
- ojosilva 5y ago
  Regardless, like I said, the Kytch box (a Raspberry Pi) is overriding the code that accesses the secret menu of the ice-cream machines [1]. That's outside the intended operation. And that's liability if McD knows about it and says nothing. The machine produces food that goes into people's bodies. Ie, if it were to produce a poisonous glycol ice-cream (glycol one of the pasteurization ingredients) all of a sudden, even if it has nothing to do with what Kytch is intended to do, could you really 100% discard in front of a judge that a glitch or a hack in the Kytch's linux computer, which is btw connected to the internet, didn't order the ice-cream machine to do that which, as a consequence, killed a hypothetical 5-year old kid from food poisoning? Imagine McD is now Boeing and the Kytch box is something airlines found that make 737-MAX fly safer by plugging into and overriding the system codes so that it can warn pilots of possible 737-MAX issues... Does it make sense for Boeing to allow such thing even if it makes planes safer? [1] https://www.wired.com/story/they-hacked-mcdonalds-ice-cream-makers-started-cold-war/ https://www.wired.com/story/they-hacked-mcdonalds-ice-cream-...

wronglebowski 5y ago

I've commented on this previously as I worked as a jack of all trades for a McDonald's franchisee for years. The machines tanks and shaving cores must be disassembled and cleaned every two weeks. They have a very intelligent series of sensors and internal mechanisms that make it impossible to defeat or cheat. This is not a low skilled job. I'd say it would be easier to change your cars oil than complete this. There's multiple gaskets and seals which need to be rotated quarterly in a tune up kit. While we were hiring at minimum wage the shake machine trained people were making 15$. I can only imagine what they're making now. I don't find the "cryptic messages" issue to be valid. New machines had new systems and codes, it wasn't malicious. Everyone had access to the manual and the Taylor provider in my region did some phone troubleshooting free of charge to prevent service calls. They even did free annual trainings. Mechanically older machines can have issues, and some franchisees can be stubborn about replacing them. However I believe the vast majority of the time the issue is the machine not being cleaned.

TedDoesntTalk 5y ago
Why are these machines so complicated? I thought you just need a few ingredients to make ice cream?
- wronglebowski 5y ago
  It's optimized for high capacity, high throughput, low planning, production. Do you sell 1 milkshake or 100 a day? Doesn't matter, the tank holds the mix and freezes it on the fly. Taylor machines can output hundreds of units an hour. Or none. It's always "fresh", there's waste when you clean the machine but other than that the ice cream you get has only been frozen seconds before being served. It's highly profitable, when the machine is operating correctly the base food cost is much lower than the other food categories(sandwiches and chicken) but not as good as soda and cookies.
- tapland 5y ago
  It's a pretty clunky machine and anything in the machinery making or dispensing the ice cream needs to be cleaned thoroughly for food safety reasons. A lot of people would find the disassembly and correct reassembly difficult. And there are huge incentives to just not do it.

s_dev 5y ago

How come these issues aren't existent in McDonald's in Europe?

grouchomarx 5y ago
there's a more reliable Italian machine that is also available to franchisees. most (all) american locations use the Taylor. I assume the opposite is true in europe

amelius 5y ago

So "broken" -> "mismanaged".

kube-system 5y ago

"Broken" is simply the colloquial term used to avoid arguments with customers. Clarity is a much higher customer-service priority than precision.

selfhoster11 5y ago

Machine clock being wrong? IDK about the US, but in the UK we have at least 2 different radio-clock signals to choose from for machine synchronisation (one is domestic, one is based in Germany so you can still get it if reception is good). Neither requires internet access or any setup, and is cheap enough to integrate into sub-$30 electronics.

rascul 5y ago

Broken could also mean that nobody has bothered to refill the machine. In my experience working at McDonald's many years ago, it was actually incredibly rare for the ice cream machine to be down for any reason besides nobody has filled it. When cars are wrapped around the building, line is out the door, and you're short two people, nobody is going to take the five minutes to fill the ice cream machine.

loudtieblahblah 5y ago
I'm always torn between my disdain for swinging down at the working class and my experience from the time I was 14 until I was 26, My politics have always aligned with something far more left wing than the Democrats. Pro-private unions, pro m4a, pro welfare assistance, higher minimum wages, I wish people had guaranteed sick and vacation time, better work safety protections, better paternity/maternity leave. And a culture that saw jobs that didn't take a degree as something that could still be worked with a since of dignity. But in my actual experience working... I worked just about anything you can think of.. Starbucks, Applebee's, TGIFridays,a BBQ restaurant, Sallie Beauty Supply, CVS, Rite-Aid, CompUSA, video rentals, grocery stores, I worked construction, painted, did lots of odd jobs. And the types of people I met along the way.... The adults in restaurants, pharmacies and retail stores.. who eclipsed me in years anyways, to be frank, were losers. Lazy, shiftless people who had to be told, constantly, to do the basics even when you had entire days when there was little customizer traffic or stock to take care of People who would openly lie about "oh my back" and sit on a stool not doing shit all day, then go off to party at night. People who wouldn't see a customer for hours and couldn't be fucked to "front and face" or do some basic cleaning of the store without being told to.. At 40, 50, 55 years old. I can't tell you how many people at various jobs who got arrested bc they were jacking shit from the store and selling hit items. I worked a toy store in a mall during the Xmas season once and the manager was cleaning the inventory out by 30% and flipping it every week. These were always people with kids before they were 18, or had multiple kids from multiple partners, drug use was way more open and regular than anything you see in the office. You go to McDonald's and you see more tattoos and "grills" than you see any sense. Do you think they give af about an ice cream machine? But the bigger question, is why in 2022 is anyone still buying toxic crap from fast food restaurants? Buy a bag of beans, live a couple years longer.