4 ms·

Perhaps a more common question: "How many calories do men need to lose weight?" Google AI responded: "To lose weight, men typically need to reduce their daily

• warner25 2y ago
  I think even when the answer is "right" in some sense, it should probably come within the context of a bunch of caveats, explanations, etc. But maybe I'm just weird. Oftentimes when my wife or kids ask me a question, I take a deep breath and start to say something like "I know what you're asking, but there's not a simple or straightforward answer; it's important to first understand ____ or define ____..." by which time they get frustrated with me.
  • hombre_fatal 2y ago
    > I think even when the answer is "right" in some sense, it should probably come within the context of a bunch of caveats, explanations, etc. Funnily enough, this is exactly what the LLM does with these questions. So well that people usually try to tweak their prompts so they don't have to wade through additional info, context, hedging, and caveats.
  • layer8 2y ago
    So you are saying that Google should provide responses that are more likely to frustrate its users? ;)
• sroussey 2y ago
  > Which is obviously dangerous advice. Same advice as my trainer gives me.
  • simoncion 2y ago
    Your trainer advises you to reduce your calorie intake to between 200 and 500 calories per day? [0] That sounds very, very hazardous for anything other than very short term use, and (given the body's inbuilt "starvation mode") probably counterproductive, even then. [0] Note that the robot suggested to reduce calorie intake by 1,500->1,800 calories, and the recommended calorie intake is 2,000.
    • hinkley 2y ago
      People losing weight are probably eating more than 2000 per day to begin with. But if you go from 2800 down to 1500 you’re already likely to exceed 3 lbs of weight loss per week that is recommended without doctor supervision. If you need to lose more than 150 lbs in a year because you’re well past morbid obesity then you need staff, not just a food plan.
      • sroussey 2y ago
        If you’re eating out at Chilies, you could easily be eating 3000 calories per meal.
        • hinkley 2y ago
          I recall when Krispy Kreme came out with a donut shake that was 1800 calories for the large size. It’s crazy out there.
  • Retric 2y ago
    Did they say: reduce their daily calorie intake to 1,500 - 1,800 calories or reduce their daily calorie intake by 1,500 - 1,800 calories These are very different answers, unless you’re consuming ~3,300 calories per day. These kinds of ‘subtle’ phrasing issue often results in AI mistake as both words are commonly used in advice but the context is really important.
    • sroussey 2y ago
      Oh yeah! No, reduce to not reduce by. Though at the time I was eating a few things that had high calories that I didn’t realize so it would have been the same.
• hombre_fatal 2y ago
  I don't see the problem with the answer, and the question is already garbage. Plus, the LLM hedges its advice with precautions. I get a pretty good summary when I paste the question into Google. It comes up with a ballpark but also gives precautions and info on how to estimate what caloric restriction makes sense for you within the first 3 sentences. And all in a format someone is likely to read instead of clicking on some verbose search result that only answers the question if they read a whole article which they aren't going to do. This seems like really lame nit picking. And I don't think it passes the "compared to what?" test.
  • Jaxan 2y ago
    The question is garbage. But people will ask it with their best intentions and not know it’s garbage.
  • beejiu 2y ago
    The basic problem is it says reduce "by 1,500 - 1,800" rather than "to 1,500 - 1,800" (not that that answer is much better). Yes, it's a garbage question, but the first answer is unsafe in all circumstances. The simplest solution here is to show nothing.
• trollbridge 2y ago
  That explains why I haven't been losing weight!
  • Aeolun 2y ago
    I think that would explain why you’re starving, not how you’re not losing weight.
• griomnib 2y ago
  Funny thing is you can train a small BERT model to detect queries that are in categories that aren’t ready for AI “answers” with like .00000001% of the energy of an LLM.
  • hansvm 2y ago
    That's (obviously) a bit of an exaggeration. BERT is just another transformer architecture. Cut down from ~100 layers to 1, ~1k dimensions to ~10, and ~10k tokens to 100, and you're only 1e6 faster / more efficient, still a factor of 10k greater than your estimate and also too small to handle the detection you're describing with any reasonable degree of accuracy.
    • griomnib 2y ago
      I literally have DistilBERT models that can do this exact task in ~14ms on an NVIDIA A6000. I don’t know the precise performance per watt, but it’s really fucking low. I use LLM to help with training data as they are great at zero shot, but after the training corpora is built a small, well trained, model will smoke an LLM in classification accuracy and are way faster - which means you can get scale and low carbon cost. In my personal opinion there is a moral imperative to use the most efficient models possible at every step in a system design. LLM are one type of architecture and while they do a lot well, you can use a variety of energy efficient techniques to do discrete tasks much better.
      • hansvm 2y ago
        Thanks for providing a concrete model to work with. Compared to GPT3.5, the number you're looking for is ~0.04%. I pointed out the napkin math because 0.00000001% was so obviously wrong even at a glance that it was hurting your claim. And, yes, purpose-built models definitely have their place even with the advent of LLMs. I'm happy to see more people working on that sort of thing.
        • griomnib 2y ago
          I applaud you doing the math! Proves you aren’t an LLM :-D
• nuancebydefault 2y ago
  If your calorie intake is just 1500 today, it is bad advice. If your calorie surplus is 1800, it is good advice. But I wonder, were those few words the full response? Information hiding to prove a point is too easy.
  • hansvm 2y ago
    A calorie surplus of 1800/day is ~190 lbs/yr. Is that something people actually do?
    • tredre3 2y ago
      Yes. You can't just say that someone eating 1800 over the recommended 2000 will perpetually gain weight. Weight maintenance calories will depend on the weight of the person. A 500lbs man will need to consume 4000kcals/day to not lose weight. Cutting 1800 of that is realistic and might be good advice on the LLM's part, so it really depends on how GP asked the question.
• zaroth 2y ago
  It’s not even advice, and it’s not wrong.
• hinkley 2y ago
  If you’re aiming to lose weight safely the rule of thumb is 3 lbs a week. 3000kcal per pound works out to an average deficit of about 1280 calories per day. Max. Obese people can lose a bit more under doctor supervision. My understanding is that it’s tied partially to % of body weight lost per week and partly to what your organs can process, which does not increase with body mass.
  • layer8 2y ago
    I don’t think absolute numbers are very useful here. You need around 5–10% reduction in calorie intake to get any weight-loss effect going, and I wouldn’t reduce by more than 20% (relative to weight-maintaining intake — it’s different if you’ve been seriously overeating) if you want it to be sustainable longer-term. So for example if your weight is stable at 2500 kcal per day, I would start by reducing the intake by 250–500 kcal, but not more. If this works well for a month or two and then you want to lose weight faster, you can still reduce your intake further. You generally have to do that anyway even just to maintain the velocity, because weight loss also tends to reduce calorie expenditure. First and foremost, you need to monitor your calorie intake against weight. Here is a useful text about that: https://www.fourmilab.ch/hackdiet/ https://www.fourmilab.ch/hackdiet/
    • hinkley 2y ago
      Your body will get more efficient at whatever exercise you do to make the calories work out. So over time you’ll either have to increase your exercise or rein in the calories a bit more to achieve a sustained result.
      • layer8 2y ago
        That is assuming that you do any exercise. But yes, and the method I linked to explains how to handle any such variation in calorie expenditure regardless of its cause.