4 ms·

In this sense, it's more hacking than crareful and well specified engineering, and that could lead down a path of instability in the product where some features

In this sense, it's more hacking than crareful and well specified engineering, and that could lead down a path of instability in the product where some features get better while others get worse, without understanding exactly why.

runnerup 4y ago
I mean pretty much all real engineering started with that time periods “hacking”/“tinkering” before thorough models and equations were derived. We had 200 years of tinkering with relatively modern steam engine technology before Carnot and Watt started just barely scratching the surface of the first principles of thermodynamics and engine efficiency. Even the eponymous Carnot cycle wasn’t rigorously defined mathematically during Carnot’s life. That being (T1−T2)/T1 as the temperature delta part of the equation, because absolute temperature hadn’t been accepted and defined by Lord Kelvin yet. Some decade later the first law of thermodynamics was finally invented. Hundreds of years of experimentation until the first principles. Machine learning has lots of control systems theory and information theory to help with analysis but we barely have an “engineering” in “software engineering” today, let alone in “machine learning engineering”. We’ll get there, but it’ll be awhile before there are proven design equations with rigorous derivations from first principle that allow us to design and build a precise AI model as surely as we can design and build a precise bridge or levee or distillation column. Let the hacking continue, let’s not worry too much about the future “engineering” that will follow in its own time. Unless you want to discover it yourself or fund its discovery.
- localplume 4y ago
  [dead]
- ChatGTP 4y ago
  It's fine to be hacking, if you're not making billions off the service which people expect some type of stability or baseline performance from, at least that's how I interpret what the parent is saying. Maybe it's easy enough for them to just copy the model, tweak, hack and play with it from there with little interruption. No one really knows at the moment.
  - ChatGTP 4y ago
    Sorry, my writing was crap... I meant to say that now the model is in production, it definitely needs to maintain and or improve performance...
    - pffft8888 4y ago
      Yes but how will they do that if they don't have a clear understanding. When we build software, we have (or should have) a clear understanding of the various components and, in some cases, like with distributed and mission-critical/military systems, a formal verification/simulation of the system when needed. When we're dealing with emergent behavior, as we have with these large transformers, but no exact understanding of how the behavior is produced and only a limited way of refining/controlling it, I don't think we're in a position to guaratee that refinements in one area won't lead to regressions in other areas or a change in the global characteristics of the system. I mean... we're dealing with complex emergent behavior, at a different scale of complexity than what we have had to deal with so far (in traditional software development) and no mature verification/analysis tools.
      - ChatGTP 4y ago
        I don't know what to say but it apparently can detect sarcasm from IMDB reviews if that helps? I have to say it's all really beyond me what to think / believe about it anymore.

runnerup 4y ago

I mean pretty much all real engineering started with that time periods “hacking”/“tinkering” before thorough models and equations were derived. We had 200 years of tinkering with relatively modern steam engine technology before Carnot and Watt started just barely scratching the surface of the first principles of thermodynamics and engine efficiency. Even the eponymous Carnot cycle wasn’t rigorously defined mathematically during Carnot’s life. That being (T1−T2)/T1 as the temperature delta part of the equation, because absolute temperature hadn’t been accepted and defined by Lord Kelvin yet. Some decade later the first law of thermodynamics was finally invented. Hundreds of years of experimentation until the first principles. Machine learning has lots of control systems theory and information theory to help with analysis but we barely have an “engineering” in “software engineering” today, let alone in “machine learning engineering”. We’ll get there, but it’ll be awhile before there are proven design equations with rigorous derivations from first principle that allow us to design and build a precise AI model as surely as we can design and build a precise bridge or levee or distillation column. Let the hacking continue, let’s not worry too much about the future “engineering” that will follow in its own time. Unless you want to discover it yourself or fund its discovery.

localplume 4y ago
[dead]
ChatGTP 4y ago
It's fine to be hacking, if you're not making billions off the service which people expect some type of stability or baseline performance from, at least that's how I interpret what the parent is saying. Maybe it's easy enough for them to just copy the model, tweak, hack and play with it from there with little interruption. No one really knows at the moment.
- ChatGTP 4y ago
  Sorry, my writing was crap... I meant to say that now the model is in production, it definitely needs to maintain and or improve performance...
  - pffft8888 4y ago
    Yes but how will they do that if they don't have a clear understanding. When we build software, we have (or should have) a clear understanding of the various components and, in some cases, like with distributed and mission-critical/military systems, a formal verification/simulation of the system when needed. When we're dealing with emergent behavior, as we have with these large transformers, but no exact understanding of how the behavior is produced and only a limited way of refining/controlling it, I don't think we're in a position to guaratee that refinements in one area won't lead to regressions in other areas or a change in the global characteristics of the system. I mean... we're dealing with complex emergent behavior, at a different scale of complexity than what we have had to deal with so far (in traditional software development) and no mature verification/analysis tools.
    - ChatGTP 4y ago
      I don't know what to say but it apparently can detect sarcasm from IMDB reviews if that helps? I have to say it's all really beyond me what to think / believe about it anymore.