3 ms·

It's kind of true story actually. I found one of my license texts for a hobby project[0] in a Microsoft product[1]. I have no idea how they use it. Copilot is

It's kind of true story actually. I found one of my license texts for a hobby project[0] in a Microsoft product[1]. I have no idea how they use it. Copilot is very poor at understanding how the DSL arrangement of the project works or how to assemble validator trees. Before Copilot many other "enterprise" stuff like JetBrains IDEs had problems auto-completing it. But it was always good enough for me, for my own things. It obviously have value. Maybe they're lazy and using it for something unrelated like the TLD list. I really don't know. What I know is that something of my amateur dish ended up being served in what I consider to be a fast food chain. No, copilot is not endlessly variable. It works best with typed languages and an enterprise mindset. That's exactly the point against "vibe coding" they push. MS seems to despise anything that doesn't follow that. Look, I love typescript and all that stuff. As I said, McDonalds meals are delicious. But I don't want that approach for everything I code. This project is not the only one with that issue. I noticed the same in a portable shell compatibility layer, it can only help if I make it to look like other script languages. "You're asking too much, how can copilot help with a language it was not trained on?" I am not asking it to be good at that. Just don't make asshole posts saying anything other than MS pasteurization is "vibes". If they paid me, then that would be another story. But they didn't, and used my "vibe" project anyways. I am not complaining about software I released for free, before anyone tries to go for that. I am making an observation about kinds of software that copilot can't help with. My kind of software development, in which I might take months to decide a name or redo everything several times just to try a new kind of sauce. [0]: https://github.com/Respect/Validation https://github.com/Respect/Validation [1]: https://support.microsoft.com/en-us/office/microsoft-teams-third-party-notice-for-windows-phone-5fd21fb8-601e-418c-adb1-d9f0f8dcceae https://support.microsoft.com/en-us/office/microsoft-teams-t...

motorest 2y ago
> No, copilot is not endlessly variable. It works best with typed languages and an enterprise mindset. I don't think your assumption is true. The key factor is the corpus used to train it, and the context you fed it. I already experienced Copilot fumbling references to methods of a simple Java class, whereas it pulled off thinks like ARM templates flawlessly. You need to understand that internally LLMs do probability-based text completion. If you feed them enough context to maximize the probability they output things you expect, they do so.
- alganet 2y ago
  Please, think about what you just said. How can I feed to it context about a DSL that does not exist yet?
  - motorest 2y ago
    > How can I feed to it context about a DSL that does not exist yet? My example referred to ARM templates. They do exist, don't they?
    - alganet 2y ago
      I guess they do. But if that's the case then your comment brings nothing new to the discussion.
- int_19h 2y ago
  Right, but that's exactly what static typing does in most cases - it provides model with more explicit context for what it's doing. For the same reason, they tend to handle XML better than JSON - sure, both are trees, but XML is has redundancy, and that redundancy helps keep the model on the rails, so to speak. I actually wonder if the perfect LLM language would be something a lot more like COBOL - not in a sense of being similarly high level, but rather verbosity. And perhaps also being closer to natural English, which is, after all, still a lot of its training set, especially for reasoning stuff. For query languages they seem to like SQL the most of all the things I've tried, and I strongly suspect that it's the same underlying cause. Of course, a new language designed like that would have the fundamental problem that there's no existing corpus in such a language. Then again, if it is also designed such that one can reliably convert e.g. from Java to that language, then perhaps we can still pull that off.
  - dragonwriter 2y ago
    > For query languages, at least, they seem to like SQL the most of all the things I've tried, and I strongly suspect that it's the same underlying cause. The reason they like SQL more than other query languages is primarily that their training data has orders of magnitude more of it than any other query language, and that advantage is so huge that any other possible advantage would probably have comparatively negligible effect.

motorest 2y ago

> No, copilot is not endlessly variable. It works best with typed languages and an enterprise mindset. I don't think your assumption is true. The key factor is the corpus used to train it, and the context you fed it. I already experienced Copilot fumbling references to methods of a simple Java class, whereas it pulled off thinks like ARM templates flawlessly. You need to understand that internally LLMs do probability-based text completion. If you feed them enough context to maximize the probability they output things you expect, they do so.

alganet 2y ago
Please, think about what you just said. How can I feed to it context about a DSL that does not exist yet?
- motorest 2y ago
  > How can I feed to it context about a DSL that does not exist yet? My example referred to ARM templates. They do exist, don't they?
  - alganet 2y ago
    I guess they do. But if that's the case then your comment brings nothing new to the discussion.
int_19h 2y ago
Right, but that's exactly what static typing does in most cases - it provides model with more explicit context for what it's doing. For the same reason, they tend to handle XML better than JSON - sure, both are trees, but XML is has redundancy, and that redundancy helps keep the model on the rails, so to speak. I actually wonder if the perfect LLM language would be something a lot more like COBOL - not in a sense of being similarly high level, but rather verbosity. And perhaps also being closer to natural English, which is, after all, still a lot of its training set, especially for reasoning stuff. For query languages they seem to like SQL the most of all the things I've tried, and I strongly suspect that it's the same underlying cause. Of course, a new language designed like that would have the fundamental problem that there's no existing corpus in such a language. Then again, if it is also designed such that one can reliably convert e.g. from Java to that language, then perhaps we can still pull that off.
- dragonwriter 2y ago
  > For query languages, at least, they seem to like SQL the most of all the things I've tried, and I strongly suspect that it's the same underlying cause. The reason they like SQL more than other query languages is primarily that their training data has orders of magnitude more of it than any other query language, and that advantage is so huge that any other possible advantage would probably have comparatively negligible effect.