4 ms·

Language is the tool we use to codify a heuristic understanding of reality. The world we interact with daily is not the physical one, but an ideological one con

• elif 2y ago
  To put this in coding terms, why would an LLM use rails to make a project when it could just as quickly produce a project writing directly to the socket. To us these are totally different tasks and would actually require totally different kinds of programmers but when one language is another language is everything, the inventions we made to expand the human brain's ability to delve into linguistic reality are no use.
  • WD-42 2y ago
    except LLMs are trained on higher level languages. Good luck getting you LLM to write your app entirely in assembly. There just isn’t enough training data.
    • xienze 2y ago
      But in theory, with what training data there IS available on how to write in assembly, combined with the data available on what's required to build an app, shouldn't a REAL AI be able to synthesize the knowledge necessary to write a webapp in assembly? To me, this is the basis for why people criticize LLMs, if something isn't in the data set, it's just not conceivable by the LLM.
      • Jerrrrrrry 2y ago
        Yes. There is just no way of knowing how many more watts of energy it may need to reach that level of abstraction and depth - maybe on more watt, maybe never. And the random noise in the process could prevent it from ever being useful, or it could allow it to find a hyper-efficient clever way to apply cross-language transfer learning to allow a 1->1 mapping of your perfectly descriptive prompt to equivalent ASM....but just this one time. There is no way to know where performance per parameter plateaus; or appears to on a projection, or actually does... or will, or deceitful appears to... to our mocking dismay. As we are currently hoping to throw power at it (we fed it all the data), I sure hope it is not the last one.
      • cjbprime 2y ago
        There isn't that much training data on reverse engineering Python bytecode, but in my experiments ChatGPT can reconstruct a (unique) Python function's source code from its bytecode with high accuracy. I think it's simulating the language in the way you're describing.
  • jumping_frog 2y ago
    I can suggest one reason why LLM might prefer writing in higher level language like Ruby vs assembly. The reason is the same as why physicists and mathematicians like to work with complex numbers using "i" instead of explicit calculation over 4 real numbers. Using "i" allows us to abstract out and forget the trivial details. "i" allows us to compress ideas better. Compression allows for better prediction.
• WD-42 2y ago
  I don’t buy this. My child communicates with me using emotion and other cues because she can’t speak yet. I don’t know much about early humans or other sapiens but I imagine they communicated long before complex language evolved. These other means of communication are not second order, they are first order.
  • elif 2y ago
    Yep agree with the core of what you are saying. Children are exceptional at being immediate, being present in the moment. It's through learning language that we forget about reality and replace it with concepts.
  • elif 2y ago
    Also remember the "emotions" and "cues" you are recognizing are linguistic concepts you've adopted, and not an inherent aspect of reality.
    • Jarwain 2y ago
      Not exactly. Emotions exist. You feel them. I feel them. Most people feel them unless they've suppressed them sooo far into their subconscious that they don't have a conscious recognition of it. We can know how someone else is feeling by reading their body language and tying that to our personal experience of how we express those feelings. No linguistics necessary. Language is just an easier, more clear way of communicating these fundamental facets of human existence
      • elif 2y ago
        You feel them, but do antisocial animals feel them? Or are emotions derived from mental concepts developed socially through evolution?