4 ms·

If you have an image of the fridge you could have GPT vision interpret it and offer you recipes or what is missing. I personally like the idea of smart home th

If you have an image of the fridge you could have GPT vision interpret it and offer you recipes or what is missing. I personally like the idea of smart home things and everything connected. It feels like it opens up so many possibilities or ideas. Do I need it for survival? No. But I do like the idea and how it triggers my imagination.

arter4 3y ago
>you could have GPT vision interpret it and offer you recipes or what is missing What is missing and what I should buy are different things. If my fridge has yogurt, bacon and some hamburgers, you might say "well, you are missing eggs". Except maybe I don't want eggs. I may be even allergic. Or maybe I like eggs but I'm in some low-protein diet for whatever reason. The goal of a fridge is not to have literally everything, is to store everything you think you need for some days. As for recipes, yes, that's more creative, but I guess I would just actively Google that (or even use an old fashioned recipe book!) if I were looking for original recipes. Of course, you can also ChatGPT it, but the direct fridge -> ChatGPT interaction is a bit weird because maybe I don't want original recipes or maybe I'm going to eat out. So a hypothetical (?) app that sends me notifications about what recipes ChatGPT suggests by looking at my fridge wouldn't be very appealing unless you're always looking for new recipes. If you're mostly cooking ordinary things, continuous "You could cook X" would count as spam, much like a lot of people skip (or try to skip) ads on Youtube and ignore random "buy this at 70% discount" emails. And if you're that passionate about new recipes I guess you're probably interested in cooking and don't need the fridge to tell you what's missing.
- midasuni 3y ago
  “You are missing Evards brand milk, buy Evards for that fresh taste, I’ve added it to your list for you” Any potential benefit from such a device would be destroyed by the cancerous advertising industry.
  - arter4 3y ago
    Exactly. This would turn a fridge into yet another thing serving ads. No thanks.
    - mewpmewp2 3y ago
      There would be alternative market for ads free for sure or open source alternatives where you could connect with your own selected LLMs.
      - 4RealFreedom 3y ago
        And major brands would force you to pay a fee to disable advertising.
        mewpmewp2 3y ago
        To me paying to disable ads is a fair deal.
        thfuran 3y ago
        I think that if the manufacturer is going to continue using the appliance for their own business purposes, they shouldn't be permitted to describe the transaction that causes you to possess it as a "sale", because you don't own it free and clear.
        mewpmewp2 3y ago
        I don't get why it matters so much to people. It's all just incentives aligning and how the deals work. Having a subscription can actually be beneficial in terms of incentive to keep up supporting the product in my view. If it's a huge problem psychologically for you, maybe you can just take the following 5 years or whatever time you estimate you will be using the product and calculate that into the base price to make your decision.
- mewpmewp2 3y ago
  I mean more interactive way, not notifications being sent. E.g. you start interaction by asking the fridge - I am a bit tired, what is an easy thing to cook based on what I have in the fridge and I am stopping by the store, is there anything I should buy to have a few more options. So GPT vision takes image of what is in the fridge and then tries to solve for that question. If user is allergic to something that would be in GPT's prompt.
  - arter4 3y ago
    Ok, but if it's interactive I can open a fridge, take a picture of it and upload it to ChatGPT. You wouldn't need a smart fridge for that. You would need a smart fridge if you wanted to directly connect the fridge with ChatGPT. Which is what I assumed in the previous comment.
    - mewpmewp2 3y ago
      You may be at work, and store is on the way to work. Also what you described is quite many steps, and the images have to be taken at certain angles to capture everything. It would be directly connected to GPT and you can either use it as mobile app or with other devices or fridge itself directly. So for example you are at work, about to leave home, store is on the way and then you ask this question from the app. Fridge takes photo, forwards it with certain prompts to OpenAI APIs and then gives you the response.
      - arter4 3y ago
        So you're saying a fridge takes a photo (of itself, essentially... a selfie?), forwards it with certain prompts to a GPT and then gives you the response. But as I said earlier I might not like certain foods, or I may actively avoid certain foods for medical or religious reasons. Now, if this was an explicit user-GPT interaction, you could prompt the GPT with "hey, I'm Muslim, what can I cook with this stuff?" or "hey, I'm lactose intolerant, what else could I buy?". You do need to trust the OpenAI provider, but not the fridge. Instead, the fridge automatically talks to GPT. So if you want to avoid "dumb" suggestions you wouldn't be able to follow, you would have to tell your fridge you're Jew or Muslim or lactose intolerant or diabetic. Don't you see an issue with this? I'm not "tinfoil hat" paranoid, I don't think people are out there to get me. But a certain dose of skepticism when it comes to data usage should be healthy, especially when many companies have been known to mess up this aspect (from Cambridge Analytica to Roomba employers sending pictures of a woman on the toilet to Facebook, and much more). You cannot avoid this 100% unless you go to extreme lengths, but if you can avoid sending data to one more company, why not?
rglullis 3y ago
It also (literally) opens up your home for many ideas and possibilities for attackers. Not just for the "haxxor" types, but the far more sneaky, sweet-talking ones. It should not trigger just your imagination, but your paranoia.
- mewpmewp2 3y ago
  Yes of course, but for example you already have your phone, laptops and computers with many things built in them. If you are ok with those, then it should be possible to build those smart devices with similar security and potential for attack.
  - arter4 3y ago
    Yes but while you can install Linux on your own, or even build your own computer if you're interested, you don't get to build a fridge or its operating system, and you most likely don't even have a valid way to see how it's configured (you may be able to hack it somehow, but it would probably void any warranty). So... do you upgrade your fridge OS? Maybe, maybe not. What happens if you do and an upgrade fails? does it talk to the Internet? If it has to (by design), then all bets are off. You could try to mirror traffic and take a packet capture, but a fridge OS could easily use HTTPS over a "smart.myfridge.com" domain and you wouldn't know what's going on. How does it even authenticate to whatever remote server they have? If it uses tokens or client certificates, what happens when they expire? I'm a sysadmin, I know things fail. Even laptops and phones. But, while I can see valid use cases for laptops and phones, I still fail to see the actual appeal of most "smart home" appliances, outside of security systems. And I don't want to have nightmares about my fridge certificate expiration or not being able to turn your light on because my fiber connection went down.
    - mewpmewp2 3y ago
      > Yes but while you can install Linux on your own, or even build your own computer if you're interested, you don't get to build a fridge or its operating system, and you most likely don't even have a valid way to see how it's configured (you may be able to hack it somehow, but it would probably void any warranty). You could install open sourced things to that fridge. E.g. the fridge could either come with sensors immediately available or just have convenient mounts to place your own cameras. Maybe the fridge could have a simple linux box, raspberry PI or whatever set up. There could be a market where you can flexibly pick how immediately available solution you want or if you want to put in everything yourself and control, depending on how concerned you are about privacy and how technical you are. If you want to have full control, you can just do that. > do you upgrade your fridge OS? Maybe, maybe not. What happens if you do and an upgrade fails? Most smart home devices in my experience update through a mobile app connected to your WiFi. > What happens if you do and an upgrade fails Fridge itself would still work, just the picture taking and ai guidance wouldn't. > How does it even authenticate to whatever remote server they have? If it uses tokens or client certificates, what happens when they expire? Through mobile app can update, reset everything. > And I don't want to have nightmares about my fridge certificate expiration or not being able to turn your light on because my fiber connection went down. These basic things like light and other base features you make sure can work even if smart software fails.

arter4 3y ago

>you could have GPT vision interpret it and offer you recipes or what is missing What is missing and what I should buy are different things. If my fridge has yogurt, bacon and some hamburgers, you might say "well, you are missing eggs". Except maybe I don't want eggs. I may be even allergic. Or maybe I like eggs but I'm in some low-protein diet for whatever reason. The goal of a fridge is not to have literally everything, is to store everything you think you need for some days. As for recipes, yes, that's more creative, but I guess I would just actively Google that (or even use an old fashioned recipe book!) if I were looking for original recipes. Of course, you can also ChatGPT it, but the direct fridge -> ChatGPT interaction is a bit weird because maybe I don't want original recipes or maybe I'm going to eat out. So a hypothetical (?) app that sends me notifications about what recipes ChatGPT suggests by looking at my fridge wouldn't be very appealing unless you're always looking for new recipes. If you're mostly cooking ordinary things, continuous "You could cook X" would count as spam, much like a lot of people skip (or try to skip) ads on Youtube and ignore random "buy this at 70% discount" emails. And if you're that passionate about new recipes I guess you're probably interested in cooking and don't need the fridge to tell you what's missing.

midasuni 3y ago
“You are missing Evards brand milk, buy Evards for that fresh taste, I’ve added it to your list for you” Any potential benefit from such a device would be destroyed by the cancerous advertising industry.
- arter4 3y ago
  Exactly. This would turn a fridge into yet another thing serving ads. No thanks.
  - mewpmewp2 3y ago
    There would be alternative market for ads free for sure or open source alternatives where you could connect with your own selected LLMs.
    - 4RealFreedom 3y ago
      And major brands would force you to pay a fee to disable advertising.
      - mewpmewp2 3y ago
        To me paying to disable ads is a fair deal.
        thfuran 3y ago
        I think that if the manufacturer is going to continue using the appliance for their own business purposes, they shouldn't be permitted to describe the transaction that causes you to possess it as a "sale", because you don't own it free and clear.
        mewpmewp2 3y ago
        I don't get why it matters so much to people. It's all just incentives aligning and how the deals work. Having a subscription can actually be beneficial in terms of incentive to keep up supporting the product in my view. If it's a huge problem psychologically for you, maybe you can just take the following 5 years or whatever time you estimate you will be using the product and calculate that into the base price to make your decision.
mewpmewp2 3y ago
I mean more interactive way, not notifications being sent. E.g. you start interaction by asking the fridge - I am a bit tired, what is an easy thing to cook based on what I have in the fridge and I am stopping by the store, is there anything I should buy to have a few more options. So GPT vision takes image of what is in the fridge and then tries to solve for that question. If user is allergic to something that would be in GPT's prompt.
- arter4 3y ago
  Ok, but if it's interactive I can open a fridge, take a picture of it and upload it to ChatGPT. You wouldn't need a smart fridge for that. You would need a smart fridge if you wanted to directly connect the fridge with ChatGPT. Which is what I assumed in the previous comment.
  - mewpmewp2 3y ago
    You may be at work, and store is on the way to work. Also what you described is quite many steps, and the images have to be taken at certain angles to capture everything. It would be directly connected to GPT and you can either use it as mobile app or with other devices or fridge itself directly. So for example you are at work, about to leave home, store is on the way and then you ask this question from the app. Fridge takes photo, forwards it with certain prompts to OpenAI APIs and then gives you the response.
    - arter4 3y ago
      So you're saying a fridge takes a photo (of itself, essentially... a selfie?), forwards it with certain prompts to a GPT and then gives you the response. But as I said earlier I might not like certain foods, or I may actively avoid certain foods for medical or religious reasons. Now, if this was an explicit user-GPT interaction, you could prompt the GPT with "hey, I'm Muslim, what can I cook with this stuff?" or "hey, I'm lactose intolerant, what else could I buy?". You do need to trust the OpenAI provider, but not the fridge. Instead, the fridge automatically talks to GPT. So if you want to avoid "dumb" suggestions you wouldn't be able to follow, you would have to tell your fridge you're Jew or Muslim or lactose intolerant or diabetic. Don't you see an issue with this? I'm not "tinfoil hat" paranoid, I don't think people are out there to get me. But a certain dose of skepticism when it comes to data usage should be healthy, especially when many companies have been known to mess up this aspect (from Cambridge Analytica to Roomba employers sending pictures of a woman on the toilet to Facebook, and much more). You cannot avoid this 100% unless you go to extreme lengths, but if you can avoid sending data to one more company, why not?

rglullis 3y ago

It also (literally) opens up your home for many ideas and possibilities for attackers. Not just for the "haxxor" types, but the far more sneaky, sweet-talking ones. It should not trigger just your imagination, but your paranoia.

mewpmewp2 3y ago
Yes of course, but for example you already have your phone, laptops and computers with many things built in them. If you are ok with those, then it should be possible to build those smart devices with similar security and potential for attack.
- arter4 3y ago
  Yes but while you can install Linux on your own, or even build your own computer if you're interested, you don't get to build a fridge or its operating system, and you most likely don't even have a valid way to see how it's configured (you may be able to hack it somehow, but it would probably void any warranty). So... do you upgrade your fridge OS? Maybe, maybe not. What happens if you do and an upgrade fails? does it talk to the Internet? If it has to (by design), then all bets are off. You could try to mirror traffic and take a packet capture, but a fridge OS could easily use HTTPS over a "smart.myfridge.com" domain and you wouldn't know what's going on. How does it even authenticate to whatever remote server they have? If it uses tokens or client certificates, what happens when they expire? I'm a sysadmin, I know things fail. Even laptops and phones. But, while I can see valid use cases for laptops and phones, I still fail to see the actual appeal of most "smart home" appliances, outside of security systems. And I don't want to have nightmares about my fridge certificate expiration or not being able to turn your light on because my fiber connection went down.
  - mewpmewp2 3y ago
    > Yes but while you can install Linux on your own, or even build your own computer if you're interested, you don't get to build a fridge or its operating system, and you most likely don't even have a valid way to see how it's configured (you may be able to hack it somehow, but it would probably void any warranty). You could install open sourced things to that fridge. E.g. the fridge could either come with sensors immediately available or just have convenient mounts to place your own cameras. Maybe the fridge could have a simple linux box, raspberry PI or whatever set up. There could be a market where you can flexibly pick how immediately available solution you want or if you want to put in everything yourself and control, depending on how concerned you are about privacy and how technical you are. If you want to have full control, you can just do that. > do you upgrade your fridge OS? Maybe, maybe not. What happens if you do and an upgrade fails? Most smart home devices in my experience update through a mobile app connected to your WiFi. > What happens if you do and an upgrade fails Fridge itself would still work, just the picture taking and ai guidance wouldn't. > How does it even authenticate to whatever remote server they have? If it uses tokens or client certificates, what happens when they expire? Through mobile app can update, reset everything. > And I don't want to have nightmares about my fridge certificate expiration or not being able to turn your light on because my fiber connection went down. These basic things like light and other base features you make sure can work even if smart software fails.