3 ms·

I try to go to the lowest level I could. During my recent research into PowerPC 32-bit assembly language I have found 1) Not many material online, and what avai

I try to go to the lowest level I could. During my recent research into PowerPC 32-bit assembly language I have found 1) Not many material online, and what available are usually PDF with pictures which could be difficult for LLMs to pick up, and 2) Indeed ChatGPT didn't give good answer even for a Hello, World example. I think hardware manufacturers, including ones that produce chips, are way less encouraged to put things online and thus has a wide moat. "Classic" ones such as 6502 or 8086 definitely have way more material. "Modern" popular ones such as x86/64 too have a lot of material online. But "obscure" ones don't. On software side, I believe LLMs or other AI can easily replace juniors who only knows how to "fill-in" the code designed by someone else, in a popular language (Python, Java, Javascript, etc.), in under 10 years. In fact it has greatly supported my data engineering work in Python and Scala -- does it always produce the most efficient solution? No. Does it greatly reduces the time I need to get to a solution? Yes, definitely!

tetha 2y ago
I've been noticing similar patterns as well. One instructive example was when I was implementing a terraform provider for an in-house application. This thing can template the boilerplate for a terraform resource implementation in about 3-4 auto completes and only gets confused a bit by the plugin-sdk vs the older implementation way. But once it deals with our in-house application, it can guess some things, but it's not good. Here it's ok. In my private gaming projects on Godot... I tried using CoPilot and it's just terrible to the point of turning it off. There is Godot code out there how an entity handles a collision with another entitiy, and there are hundreds of variations out there, and it wildly hallucinates between all of them. It's just so distracting and bad. ChatGPT is OK at navigating the documentation, but that's about it. If I'm thinking about my last job, which -- don't ask why -- was writing Java Code with low-level concurrency primitives like thread pools, raw synchronized statements and atomic primitives... if I think about my experience with CoPilot about code like this, I'm honestly feeling strength leaving my body because that would be so horrible. I've spend literal months chasing a once-in-a-billion concurrency bug in that code once. IMO, the most simple framework-fill-in code segment will suffer from LLMs. But a well-coached junior can move past that stage quite quickly.
- markus_zhang 2y ago
  Yeah I basically treat LLM as a better Google search. It is indeed a lot better than Google if I want to find some public information, but I need to be careful and double-check. Other than that it completely depends on luck I guess. I'm pretty sure if companies feed in-house information to it that will make it much more useful, but those agents would be privately owned and maintained.

tetha 2y ago

I've been noticing similar patterns as well. One instructive example was when I was implementing a terraform provider for an in-house application. This thing can template the boilerplate for a terraform resource implementation in about 3-4 auto completes and only gets confused a bit by the plugin-sdk vs the older implementation way. But once it deals with our in-house application, it can guess some things, but it's not good. Here it's ok. In my private gaming projects on Godot... I tried using CoPilot and it's just terrible to the point of turning it off. There is Godot code out there how an entity handles a collision with another entitiy, and there are hundreds of variations out there, and it wildly hallucinates between all of them. It's just so distracting and bad. ChatGPT is OK at navigating the documentation, but that's about it. If I'm thinking about my last job, which -- don't ask why -- was writing Java Code with low-level concurrency primitives like thread pools, raw synchronized statements and atomic primitives... if I think about my experience with CoPilot about code like this, I'm honestly feeling strength leaving my body because that would be so horrible. I've spend literal months chasing a once-in-a-billion concurrency bug in that code once. IMO, the most simple framework-fill-in code segment will suffer from LLMs. But a well-coached junior can move past that stage quite quickly.

markus_zhang 2y ago
Yeah I basically treat LLM as a better Google search. It is indeed a lot better than Google if I want to find some public information, but I need to be careful and double-check. Other than that it completely depends on luck I guess. I'm pretty sure if companies feed in-house information to it that will make it much more useful, but those agents would be privately owned and maintained.