4 ms·

This seems like an expression of The Bitter Lesson: https://www.cs.utexas.edu/~eunsol/courses/data/bitter_lesson.pdf https://www.cs.utexas.edu/~eunsol/courses/

• greymalik 1y ago
  How so?
  • chaos_emergent 1y ago
    Author advocates for building general purpose systems that can accomplish goals within some causal boundary given relevant constraints, versus highly deterministic logical flows that are created from priors like intuition or user research. Parallel with the bitter lesson being that general purpose algorithms that use search and learning leveraged by increasing computational capacity tend to beat out specialized methods that exploit intuition about how cognitive processes work.
• adidoit 1y ago
  No it isn't...the author is talking about products and building with a completely different mindset to deterministic software. The bitter lesson is about model level performance improvements and the futility of scaffolding in the face of search and scaling.
  • hodgehog11 1y ago
    It isn't clear to me why these are so different? The alternative mindset to deterministic software is to use probabilistic models. The common mentality is that deterministic software takes developer knowledge into account. This becomes less effective in the big data era, that's the bitter lesson, and that's why the shift is taking place. I'm not saying that the article is the bitter lesson restated, but I'm saying that this is a realisation of that lesson.
    • adidoit 1y ago
      Ah ok yes that makes more sense to me. Thank you for clarifying - I agree this new philosophy of building on probabilistic software is an outcome of the bitter lesson. And we will over time have even more capability in the model (that is more general purpose) than our deterministic scaffolds...
      • ath3nd 1y ago
        > And we will over time have even more capability in the model (that is more general purpose) than our deterministic scaffolds... Who is "we" in that case? Are you the one building the model? Do you have the compute and data capacity to test every corner case that matters? In a deterministic system you can review the code and determine what it does under certain conditions. How do you know the ones that do build non-deterministic systems (because, let's face it, you will use but not build those systems) haven't rigged it for their benefit and not yours? > And we will over time have even more capability in the model (that is more general purpose) than our deterministic scaffolds... Our deterministic scaffolds sounds so dramatic, it sounds like you think of them like the chains that keep holding you, if only those chains were removed, you'd be able to fly. But it's not you who'd be able to fly, it's the ones building the model and having the compute to build it. And because of its non-deterministic nature, a backdoor for their benefit is now simply plausible deniability. Who is we. You are a user of those models, you will not be adding anything to it, maybe only circumstantially by your prompts being mined. You are not we.
        • hodgehog11 1y ago
          This is a genuine concern, which is why it is a very hot topic of research. If you're giving a probabilistic program the potential to do something sinister, using a commercial model or something that you have not carefully finetuned yourself would be a terrible idea. The same principle applies for commercial binaries; without decompilation and thorough investigation, can you really trust what it's doing?
          • ath3nd 1y ago