7 ms·

I am using Claude Code full-time for about 6 weeks* with the $20/month subscription. I am trying out building different products from ideas I have already had.

I am using Claude Code full-time for about 6 weeks* with the $20/month subscription. I am trying out building different products from ideas I have already had. It frees me a lot of time to talk about my founder journey. I have not needed multiple agents or using CC over an SSH terminal to run overnight. The main reason is that LLMs are not correct many times. So I still need time to test. Like run the whole app, or check what broke in CI (GitHub Actions), etc. I do not go through code line by line anymore and I organize work with tickets (sometimes they are created with CC too). Both https://github.com/pixlie/Pixlie https://github.com/pixlie/Pixlie and https://github.com/pixlie/SmartCrawler https://github.com/pixlie/SmartCrawler are vibe coded (barely any code that I wrote). With LLMs you can generated code 10x than writing manually. It means you can also get 10x the errors. So the manual checks take some time. Our existing engineering practices are very helpful when generating code through LLMs and I do not have mental bandwidth to review a mountain of code. I am not sure if we scale out LLMs, it will help in building production quality software. I already see that sometimes CC makes really poor guesses. Imagine many such guesses in parallel, daily. edit: typo - months/weeks

eisbaw 1y ago
you just rick rolled me!
- brainless 1y ago
  I am so sorry, it was not intentional at all. I had asked Claude Code to keep a placeholder video. I did not even know what "rick rolling" means. Just searched and understood what I have done!
  - eulgro 1y ago
    Apparently this is typical to LLMs: https://chatgpt.com/s/t_687a285d9f708191a884d2ad39ddcb53 https://chatgpt.com/s/t_687a285d9f708191a884d2ad39ddcb53 I mean, it's probably the most linked YouTube video by a factor of 100x so it makes sense for it to be hardcoded in the model.
wg0 1y ago
My trick is - ask for a plan. Revise the plan. Then ask only to work on a single step of the plan making progress incremental then ask for tests for that step and keep hitch hiking in incremental steps.
- mnky9800n 1y ago
  Yes and this also helps to highlight which problems I should be solving and which problems Claude code can solve. Like tbh building efficient data structures is not Claude’s thing he seems happy to just hack together some nonsense that leads to spaghetti being shot into all corners of your repo. But by iteratively building up plans and todo lists i find Claude is able to resist the temptation to hack everything all at once to solve the immediate problem in front of his face.
- bamboozled 1y ago
  After getting lazy and relying on it too much and getting burned. I now try use it as a replacement for typing I have a similar workflow to you. I just go through the plan together, and review what’s happened every few steps and the correct. It’s actually made me a better PR reviewer, I’ve noticed :)
Quarrel 1y ago
If there is barely any code in those repos that you wrote, how can you license them under the GPL? You don't hold the copyright for it. This genuinely isn't an attack, I just don't think you can? The AI isn't granted copyright over what it produces.
- Daviey 1y ago
  I can only talk about the law in England & Wales, but: For code generated by an LLM the human user would likely be considered the author if you provided sufficient creative input, direction, or modification. The level of human involvement matters, simply prompting "write me a function" might not be enough, but providing detailed specifications, reviewing, and modifying the output would strengthen the claim. the Copyright, Designs and Patents Act 1988 (CDPA), Section 9(3) staes, "In the case of a literary, dramatic, musical or artistic work which is computer-generated, the author shall be taken to be the person by whom the arrangements necessary for the creation of the work are undertaken". This was written before LLM's existed, but recent academic literature has supported this position, https://academic.oup.com/jiplp/article/19/1/43/7485196?login=false https://academic.oup.com/jiplp/article/19/1/43/7485196?login... However, a comparable situation was tested with Thaler v Comptroller-General, where courts emphasised that legal rights require meaningful human involvement, not just ownership of the AI system. - https://www.culawreview.org/journal/unlocking-the-canvas-a-legal-analysis-of-ai-patents-in-the-us-and-uk https://www.culawreview.org/journal/unlocking-the-canvas-a-l... and https://www.whitecase.com/insight-our-thinking/uk-supreme-court-rules-against-ai-inventorship-patents https://www.whitecase.com/insight-our-thinking/uk-supreme-co... I do acknowledge there is uncertainty, and this is highlighted here in "The Curious Case of Computer-Generated Works under the Copyright, Designs and Patents Act 1988.", with "section 9(3): the section is either unnecessary or unjustifiably extends legal protection to a class of works which belong in the public domain" - https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4072004 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4072004 Today, I think it's doubtful that a functional application can be entirely vibe coded without decent direction and modification, but I don't think that will always be the case.
  - ffsm8 1y ago
    At least for art there is already precedent in US courts with someone trying to copyright an image generated by midjourney and it getting revoked in 22, because ai generated art cannot be copyrighted. for code it hasn't been challenged yet, but I find it doubtful they'd decide differently there
cess11 1y ago
Looking at ChatInterface.tsx/handleSendMessage in https://github.com/pixlie/Pixlie/commit/3c0bd23ff16c0fcdac803796f6f60a1b1cdd4d31 https://github.com/pixlie/Pixlie/commit/3c0bd23ff16c0fcdac80..., I'd have rejected this if it came up in a PR and would not consider this production quality software. Rewriting it to something sane would be harder and more time consuming than just writing a decent implementation upfront.
- baq 1y ago
  You’re missing the point. This is not production quality software. This is mass produced software. You aren’t supposed to look at all the internals, just the really critical parts.
  - cess11 1y ago
    So, according to you, it's like Wordpress all over again and a lot of work incoming for people that can actually build and maintain software. Edit: And more critically, a lot of work incoming for people that can teach software development.
    - baq 1y ago
      Yes. Someone will need to manage the 10-100x mess of LLM spaghetti codebases after they find product-market fit.
  - brainless 1y ago
    This is a very critical point. I think code generated by LLMs will lower code generation cost, that means mass produced software. But: if we reduce cost, then can we not add deterministic guardrails in software that are also maintained at LLM speed and cost? This is pretty much what I am trying to understand. Choice of Rust/TypeScript in my projects in very intentional and you may see why.
- brainless 1y ago
  This is an excellent point and one that I am chasing. I do not want software (at least ones I produce) to be inferior than what I would hand code. LLMs give me huge velocity but I am still learning where to put guardrails so it keeps quality to what I would do myself. Now the critical point: what if my own quality is inferior to many others. I think we will have this issue a lot since LLMs can generated code at 10x speeds. The quality will be what the human operator sets. Or, maybe, more and more tools will adapt best practices as guardrails and do not give humans much option to steer away from them.
dearilos 1y ago
I'm actually building something to fix this. The biggest bottleneck for background agents is code review. I'm building a tool that can give the first pass so the result of the background agent isn't garbage most of the time.

eisbaw 1y ago

you just rick rolled me!

brainless 1y ago
I am so sorry, it was not intentional at all. I had asked Claude Code to keep a placeholder video. I did not even know what "rick rolling" means. Just searched and understood what I have done!
- eulgro 1y ago
  Apparently this is typical to LLMs: https://chatgpt.com/s/t_687a285d9f708191a884d2ad39ddcb53 https://chatgpt.com/s/t_687a285d9f708191a884d2ad39ddcb53 I mean, it's probably the most linked YouTube video by a factor of 100x so it makes sense for it to be hardcoded in the model.

wg0 1y ago

My trick is - ask for a plan. Revise the plan. Then ask only to work on a single step of the plan making progress incremental then ask for tests for that step and keep hitch hiking in incremental steps.

mnky9800n 1y ago
Yes and this also helps to highlight which problems I should be solving and which problems Claude code can solve. Like tbh building efficient data structures is not Claude’s thing he seems happy to just hack together some nonsense that leads to spaghetti being shot into all corners of your repo. But by iteratively building up plans and todo lists i find Claude is able to resist the temptation to hack everything all at once to solve the immediate problem in front of his face.
bamboozled 1y ago
After getting lazy and relying on it too much and getting burned. I now try use it as a replacement for typing I have a similar workflow to you. I just go through the plan together, and review what’s happened every few steps and the correct. It’s actually made me a better PR reviewer, I’ve noticed :)

Quarrel 1y ago

If there is barely any code in those repos that you wrote, how can you license them under the GPL? You don't hold the copyright for it. This genuinely isn't an attack, I just don't think you can? The AI isn't granted copyright over what it produces.

Daviey 1y ago
I can only talk about the law in England & Wales, but: For code generated by an LLM the human user would likely be considered the author if you provided sufficient creative input, direction, or modification. The level of human involvement matters, simply prompting "write me a function" might not be enough, but providing detailed specifications, reviewing, and modifying the output would strengthen the claim. the Copyright, Designs and Patents Act 1988 (CDPA), Section 9(3) staes, "In the case of a literary, dramatic, musical or artistic work which is computer-generated, the author shall be taken to be the person by whom the arrangements necessary for the creation of the work are undertaken". This was written before LLM's existed, but recent academic literature has supported this position, https://academic.oup.com/jiplp/article/19/1/43/7485196?login=false https://academic.oup.com/jiplp/article/19/1/43/7485196?login... However, a comparable situation was tested with Thaler v Comptroller-General, where courts emphasised that legal rights require meaningful human involvement, not just ownership of the AI system. - https://www.culawreview.org/journal/unlocking-the-canvas-a-legal-analysis-of-ai-patents-in-the-us-and-uk https://www.culawreview.org/journal/unlocking-the-canvas-a-l... and https://www.whitecase.com/insight-our-thinking/uk-supreme-court-rules-against-ai-inventorship-patents https://www.whitecase.com/insight-our-thinking/uk-supreme-co... I do acknowledge there is uncertainty, and this is highlighted here in "The Curious Case of Computer-Generated Works under the Copyright, Designs and Patents Act 1988.", with "section 9(3): the section is either unnecessary or unjustifiably extends legal protection to a class of works which belong in the public domain" - https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4072004 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4072004 Today, I think it's doubtful that a functional application can be entirely vibe coded without decent direction and modification, but I don't think that will always be the case.
- ffsm8 1y ago
  At least for art there is already precedent in US courts with someone trying to copyright an image generated by midjourney and it getting revoked in 22, because ai generated art cannot be copyrighted. for code it hasn't been challenged yet, but I find it doubtful they'd decide differently there

cess11 1y ago

Looking at ChatInterface.tsx/handleSendMessage in https://github.com/pixlie/Pixlie/commit/3c0bd23ff16c0fcdac803796f6f60a1b1cdd4d31 https://github.com/pixlie/Pixlie/commit/3c0bd23ff16c0fcdac80..., I'd have rejected this if it came up in a PR and would not consider this production quality software. Rewriting it to something sane would be harder and more time consuming than just writing a decent implementation upfront.

baq 1y ago
You’re missing the point. This is not production quality software. This is mass produced software. You aren’t supposed to look at all the internals, just the really critical parts.
- cess11 1y ago
  So, according to you, it's like Wordpress all over again and a lot of work incoming for people that can actually build and maintain software. Edit: And more critically, a lot of work incoming for people that can teach software development.
  - baq 1y ago
    Yes. Someone will need to manage the 10-100x mess of LLM spaghetti codebases after they find product-market fit.
- brainless 1y ago
  This is a very critical point. I think code generated by LLMs will lower code generation cost, that means mass produced software. But: if we reduce cost, then can we not add deterministic guardrails in software that are also maintained at LLM speed and cost? This is pretty much what I am trying to understand. Choice of Rust/TypeScript in my projects in very intentional and you may see why.
brainless 1y ago
This is an excellent point and one that I am chasing. I do not want software (at least ones I produce) to be inferior than what I would hand code. LLMs give me huge velocity but I am still learning where to put guardrails so it keeps quality to what I would do myself. Now the critical point: what if my own quality is inferior to many others. I think we will have this issue a lot since LLMs can generated code at 10x speeds. The quality will be what the human operator sets. Or, maybe, more and more tools will adapt best practices as guardrails and do not give humans much option to steer away from them.

dearilos 1y ago

I'm actually building something to fix this. The biggest bottleneck for background agents is code review. I'm building a tool that can give the first pass so the result of the background agent isn't garbage most of the time.