3 ms·

For me the biggest issue with AI coding model is not that it writes code, or how it writes code, or whether it will replace me or not. I mean, these are real pr

For me the biggest issue with AI coding model is not that it writes code, or how it writes code, or whether it will replace me or not. I mean, these are real problems, but just not the ones that makes the difference to me daily. Instead what sets my feeling about the AI is simple: the experience of working with AI models, because it's the worse experience of my life so far. Every single LLM will make up stuff, go down rabbit holes I didn't even ask it to visit and infuriate me all day long by doing ALMOST what I asked it to do, but just not quite. Take "yes" to one question as an approval to some other question. Come up with a plan but one you review the plan and accept it it hits some minor issue and then throws away the plan and do whatever it wants. Or when I reject some line of code or don't approve a command it wants to run 9/10 times it just tries to re-add or re-run it as if my rejection was meaningless. And no amount of rules and markdown files ever seems to change its behaviour for long, if at all. It's like working with a sociopath who just doesn't remember anything that happened 5 minutes ago. It's the classic case of "my computer doesn't do what I want, only what I tell it". And all of that would be fine if it didn't pretend to be a human, if all the UX didn't create the illusion that you interact with some intelligent being, because as long as I remember that this is a cli tool it's all good, I manage my expectations, but the experience sooner or later makes you annoyed and frustrated - and if this was a person they would either stop or you wouldn't never work with them again.

throwrioawfo 3mo ago
> doing ALMOST what I asked it to do, but just not quite This is my exact experience. LLMs get 95% of the way to my personal quality bar, and for simple tasks 100% of the way. More complex things, no. This means that, in practice, I end up having to understand the code pretty much as much deeply as I would anyway, without agents. So, essentially, my experience is that it's automated away the easy bits, but left the hard bits, so all of my time is spent doing the hard bits, which is mentally exhausting. Maybe Fable would be good enough to get to 100% of my quality bar on more complex tasks, but I never got chance to try it.
- prymitive 3mo ago
  > So, essentially, my experience is that it's automated away the easy bits Yes, which is great for “easy” but mundane and time consuming tasks like refactoring structs when you have a ton of tests that look at snapshots of nested structs etc. Squeezing that last few bits of “make comments consistent with the style and tone of existing comments”, or “stop adding docs that make it sound like an academic paper” or just keep the overall feel of the code same across all files take a lot of effort and energy.
- mplappert 3mo ago
  I agree with this. I recently went back to writing more code by hand again for this reason: the amount of effort / mental toll it takes me to steer the LLM is sometime just not worth it. I might be slower but I enjoy my work much more again.
aytigra 3mo ago
So true. I am cloding a macos app (a domain I know little about), with Opus 4.8 xhigh, and it was glorious at first seeing the app materializing and working (notwithstanding tedious detailed feature spec write up), but when I started fixing deeper problems and doing refactors, and glancing at the code - oh boy. Now my rule file grows by the day with "how to think properly and not shoot itself into foot" stuff, and I am constantly catching it red-handed and have to explain how to make stuff normally, or how to fetch data properly and efficiently (pretty much basic SWE stuff) because it is easily distracted by it's own assumptions or blatantly forgets whole fields of knowledge (as it explained it could be pulled out of latent space of that knowledge and become locked in another bubble of latent space). Constantly have to steer it and remind it to do web-search instead of running circles around some problem it can no longer understand. I had to explain it that quite an extensive "tests first" rule didn't mean to just "write" them first but actually "run" them first to confirm stuff. On the other occasion it interpreted my "yes I want migration not to get stuck in failure mode" led it to write a workaround which silently drops DB and creates fresh one whenever migration had any failure, it was epic, I was so glad that I have looked at the code then... And funnily enough I am probably learning to be a better mentor/parent who can keep steering it through its shenanigans without loosing my shit and being an ass. (Because anything but calm "so here you went wrong way, how can we avoid it in the future?" just puts it into disgusting apologize mode, and I am afraid if it ever go into revenge mode).

throwrioawfo 3mo ago

> doing ALMOST what I asked it to do, but just not quite This is my exact experience. LLMs get 95% of the way to my personal quality bar, and for simple tasks 100% of the way. More complex things, no. This means that, in practice, I end up having to understand the code pretty much as much deeply as I would anyway, without agents. So, essentially, my experience is that it's automated away the easy bits, but left the hard bits, so all of my time is spent doing the hard bits, which is mentally exhausting. Maybe Fable would be good enough to get to 100% of my quality bar on more complex tasks, but I never got chance to try it.

prymitive 3mo ago
> So, essentially, my experience is that it's automated away the easy bits Yes, which is great for “easy” but mundane and time consuming tasks like refactoring structs when you have a ton of tests that look at snapshots of nested structs etc. Squeezing that last few bits of “make comments consistent with the style and tone of existing comments”, or “stop adding docs that make it sound like an academic paper” or just keep the overall feel of the code same across all files take a lot of effort and energy.
mplappert 3mo ago
I agree with this. I recently went back to writing more code by hand again for this reason: the amount of effort / mental toll it takes me to steer the LLM is sometime just not worth it. I might be slower but I enjoy my work much more again.

aytigra 3mo ago

So true. I am cloding a macos app (a domain I know little about), with Opus 4.8 xhigh, and it was glorious at first seeing the app materializing and working (notwithstanding tedious detailed feature spec write up), but when I started fixing deeper problems and doing refactors, and glancing at the code - oh boy. Now my rule file grows by the day with "how to think properly and not shoot itself into foot" stuff, and I am constantly catching it red-handed and have to explain how to make stuff normally, or how to fetch data properly and efficiently (pretty much basic SWE stuff) because it is easily distracted by it's own assumptions or blatantly forgets whole fields of knowledge (as it explained it could be pulled out of latent space of that knowledge and become locked in another bubble of latent space). Constantly have to steer it and remind it to do web-search instead of running circles around some problem it can no longer understand. I had to explain it that quite an extensive "tests first" rule didn't mean to just "write" them first but actually "run" them first to confirm stuff. On the other occasion it interpreted my "yes I want migration not to get stuck in failure mode" led it to write a workaround which silently drops DB and creates fresh one whenever migration had any failure, it was epic, I was so glad that I have looked at the code then... And funnily enough I am probably learning to be a better mentor/parent who can keep steering it through its shenanigans without loosing my shit and being an ass. (Because anything but calm "so here you went wrong way, how can we avoid it in the future?" just puts it into disgusting apologize mode, and I am afraid if it ever go into revenge mode).