6 ms·

I did a test a few months ago. A friend of mine wanted to develop what i understood to be a simple single page web app. But since she didn’t have any software e

by krisoft 2mo ago

I did a test a few months ago. A friend of mine wanted to develop what i understood to be a simple single page web app. But since she didn’t have any software engineering experience she asked me to help. Around that time everyone was talking about how literally anyone can develop software with LLMs i asked her if she could give it a try first, and if I could watch the attempt. I was fully expecting that writing the code will pose no problem for the AI. But i was curious if the AI will realise that my friend is a novice and needs extra help with things like: copy pasting the code into a text file and saving it with an html extension, helping her host the file online so she can share it with others, buying a domain for it, etc. I assumed they will get there eventually, but i also assumed that it will take a lot of stumbling around and misunderstandings. But i was completely wrong. They didn’t even get to that point. Because my friend didn’t have the vocabulary to ask the AI to write code. They were just going around in circles where the AI was brainstorming with her about possible features and getting thints more and more complicated. We terminated the experiment after one and a half hours and many many messages exchanged between her and the LLM. Whereas to me who knows the terminology would have probably just taken a single exchange of messages to get the result she described to me. I would have prompted with something like “Please write to me an html page which does X, Y, Z.” But since she didn’t know the right terminology she got into a vortex of feature discusion, and she didn’t find a way to tip the AI into “just do it, write it now” mode. In other words in that case the LLM would have rewarded even just a little bit of expertise, but without it there was a confusion about goals between the human and the machine.

Melatonic 2mo ago
Reminds me of watching someone who has no idea how to use a search engine try to use a search engine
matthew-wegner 2mo ago
Are you describing a "chat window" experience here? This is apples to oranges.
- Synthetic7346 2mo ago
  Yeah I wonder if they had given their friend Claude code or Codex, would it have been more likely to create what she wanted?
  - davely 2mo ago
    Perhaps! But I do think the vocabulary issue is real and I think LLMs are still sycophantic enough that they won’t really challenge someone or offer alternative ideas on how to implement something unless they explicitly ask. Interestingly at my work, Claude Code was available before Claude Desktop, so a number of non-technical PMs tried to use it in order to build… anything, with very mixed success. The “hey guys, check out the website I built with Claude: http://localhost:3000/” joke is real! In my experience, the whole “the terminal is a scary place” aspect is very real and some non-technical people can feel intimidated by. I think Claude Code in the desktop app helps alleviate that a bit (perhaps Codex, too, but man what a mess the ‘ol ChatGPT app has become). But I’m sure there are entire repos of web dev skills that someone could use to put together things with a bit of effort.
    - ElectricalUnion 2mo ago
      > the terminal is a scary place Isn't the the powerful, unlimited, unopinionated blank LLM text input waiting for your instructions eerily similar to a scary terminal? WIMP and GUI paradigms are the exact the opposite: intentional dis-empowering, by design restrictions, enumeration of your few possible options. Those feel more constrained therefore safer.
      - andsoitis 2mo ago
        > WIMP and GUI paradigms are the exact the opposite: intentional dis-empowering Are you kidding? WIMP and GUI democratized computing!
        kazinator 2mo ago
AnotherGoodName 2mo ago
OTOH my wife's friends got drunk and made "tinder for horse purchases". They prompted to read typical horse advertisements (we're all horse people) and create an app with mock tinder like entries to swipe right and left to buy horses. A web app was produced with lots of mock "Hi i'm Dominique and i love running through fields and having a bucking good time" type entries complete with silly horse photos. A huge amount of drunken fun even if it boiled a towns water supply and blew through half a subscription to create. I was looking at the results as a dev with 30 years experience and thinking fuck me. The little apps i made here and there before AI are being outdone by a bunch of drunk people on a whim!
- selcuka 2mo ago
  > The little apps i made here and there before AI are being outdone by a bunch of drunk people on a whim! Yes, but the premise of the article is you should be able to outdo a bunch of drunk people with your 30 years experience, if you use AI too.
  - vasco 2mo ago
    The article doesn't hold the universal truth.
RileyJames 2mo ago
I’d love to see this experiment executed with Claude design. Particularly with something static, I don’t think they’d fail to get a result. But without domain knowledge I think they’d misunderstand prototype with finished product. Without knowing what it’s doing, it’s hard to know what it’s not doing.
- andsoitis 2mo ago
  > Particularly with something static Apps ain’t static.
  - folkrav 2mo ago
    They probably mean static site, in the sense of static front end, no backend.
    - andsoitis 2mo ago
      Yes. That's not an app.
      - shukantpal 2mo ago
        Not really, is a offline chess page not an app? It seems like it would be closer to an app.
        andsoitis 2mo ago
        What does this hypothetical chess page do? And how’s does it do it if it is static?
        Tepix 2mo ago
        It uses javascript. Still a static file. Lets you play chess
        2mo ago
yieldcrv 2mo ago
copying and pasting code? a few months ago? 2025 called and wants its test back
- redrove 2mo ago
  Exactly, sounds to me like OP is the one with the LLM skill issue.
ynab6 2mo ago
[dead]
coder-3 2mo ago
We've seen plenty of examples of apps successfully vibe coded by non technical people, including apps making real revenue. Your friend could start with telling the LLM that they are a non technical person who wants to make an app and it will explain all the successive steps.
- roncesvalles 2mo ago
  >We've seen plenty of examples of apps successfully vibe coded by non technical people, including apps making real revenue. Have we? Or is this just something that people say now, without citation?
  - colonelspace 2mo ago
    .
    - jojobas 2mo ago
      That was kinda harnessed and prompted by a team of security experts, so there's that.
  - threatofrain 2mo ago
    I’m afraid this isn’t surprising anymore.
  - fragmede 2mo ago
    Plinq. Plinq was made on Lovable, https://www.aieatingtheworld.com/articles/non-technical-founder-lovable-app-builder-success-story https://www.aieatingtheworld.com/articles/non-technical-foun... Couple more on https://buildthedamnthing.com/resources/articles/case-studies-vibe-coding-builder https://buildthedamnthing.com/resources/articles/case-studie...
    - shepherdjerred 2mo ago
      Wow, I am not sure that I want my safety app to be videcoded by someone without experience
petra 2mo ago
So maybe your friend need a role reversal: A prompt guiding the llm to act as a consultant, guiding people in implementing a software project, asking questions, creating a shared understanding, limiting scope or creating milestones. That seems like something that could be done using an llm, not that complicated probably. And maybe, in other fields as well.
friendzis 2mo ago
> They didn’t even get to that point. Because my friend didn’t have the vocabulary to ask the AI to write code. What harness did you use? In e.g. claude, there are two modes: 1. Spit out code 2. Draft a plan, ask questions, GOTO 1 You literally have to go out of your way to get it NOT to write code. I keep mine on a tight-ish leash because it modify code way too happily even when there's no intention or instruction to do so
- sam_lowry_ 2mo ago
  Harness? I am an avid HN reader but even I don't yet fully understand how this word is used in the AI context. Which is exactly the point of the OP. It all boils down to naming things and cache invalidation, /s
  - tyzoid 2mo ago
    Harness refers to the tooling that allows you to interact with an LLM. Web chat interface is a harness, CLI coding tools (claude code, codex, opencode, pi, etc.) are harnesses, agent systems (openclaw, hermes, etc.) are harnesses. They present different capabilities to the underlying LLM. Codex, for example, is more likely to write code if you say "I want to build an app" than ChatGPT on web will.
    - darkwater 2mo ago
      Which is an abstraction layer that the non-technical should need to know, if we follow GP's line of reasoning.
  - sevenzero 2mo ago
    Personally I even refuse to learn what it is in the AI context. Feels like a waste of energy given new ~~~best ways to use YOUR tokens~~~ are discovered every month and things you learn now will be useless next month. I am doing perfectly fine with the web UI version of these tools... They seem to also not make tokens dissappear as fast as using claude cli tool to automate implementations. Makes my work day more tolerable as well as I actually have something to do over waiting until some implementation can be read through...
raincole 2mo ago
Do your friend at least know what Claude Code (or any harness) is? Of course she wouldn't be able to make a website if she doesn't even know the right tools to use. But I don't think it prove anything. Knowing and installing Claude Code might not be a common sense, but nor is it "expertise" or "skill." I've seen in first hand that people struggle installing Steam. Yes, "people" in the plural. But just because some people struggle with it, it doesn't mean that installing Steam isn't an objectively easy task. Your friend's experience doesn't change the fact that building a website is something that an average person can do in hours if not minutes.
- harvey9 2mo ago
  Knowing what Claude Code is and why you might want it actually is domain knowledge which the op's friend does not have. Your idea of the average person might be biased if you work and socialise with people who have this kind of expertise.
  - vector_spaces 2mo ago
    I'm genuinely a bit floored reading the comments here, but I guess my idea of the average HN commenter's ability to talk to non-technical folks about technical topics is biased because I work and socialize with lots of people who don't have technical expertise (thankfully along with other technical folks who also have lots of experience talking to the former group). Many of them don't even know what Claude is, let alone Claude Code.
    - Maxion 2mo ago
      The average person does not even know what ChatGPT is and has not interacted with an LLM ever.
      - walthamstow 2mo ago
        I don't think this is true. Certainly not in Britain. I went to 3 weddings last summer and each one had a joke in a speech about using ChatGPT to write it and everyone laughed. 68yo father of the bride is a retired plumber and even he's cracking jokes about AI.
sshine 2mo ago
I have the same experience giving my brother an OpenClaw as his personal assistant. His words were: It feels as if I need to know how to program it. I was expecting he could say something like "Oh, it seems like you don't remember the people I'm referring to, perhaps you need some kind of CRM system. Can you investigate if there are any easily available CRM systems you can interface with, so we don't need to make one for you?" Whereas my OpenClaw moment was trying to make it manage its own NixOS installation, so that if I ask it to do something, it doesn't yolo `apt install` commands, but rather improves on the same overview of its own installation. A lot of people had success making their OpenClaw do things without being Linux experts. But you need a tinkerer's mindset, is what I came to conclude.
- walthamstow 2mo ago
  Curiosity and a tinkerer's / engineer's mindset is basically the only thing we can hire for right now
  - sshine 2mo ago
    That and a proven ability to apply it in a given domain. I am really not good with electronics. I’m sure it’s just a mental block, but I see people debug electronics as relentlessly as I debug code.
petesergeant 2mo ago
I’ve had the absolute opposite experience with a friend of mine. I started by setting her up with a terminal emulator on the web linked to Claude Code, and written a CLAUDE.md that told it how to deploy. These days (with no further intervention by me) she’s running Claude Code natively on her laptop, and she tagged me yesterday on Facebook in some update about how much more she enjoyed using Claude Code than plain Claude.
MattGaiser 2mo ago
What terminology is required to make Claude Code write code? My experience with friends has been the opposite. A PM friend made a custom tool. A friend who has never written a line of computer code has an app. If you tell Claude Code "I want a website that does X, Y, and Z" it will write code.
- knollimar 2mo ago
  Especially if the path is well trodden. "Make me a kanban board for construction tasks. I want you to walk me through the process of hosting it" worked for me.
lynx97 2mo ago
Question is if the outcome was LLM related or just a result of a personality trait. After all, talking in circles without really approaching an end-goal is something certain humans do all day long.
downloadram 2mo ago
>she didn’t find a way to tip the AI into “just do it, write it now” mode worse, the longer an LLM conversation goes on, but especially with constricted/free models (yes the simple chat interface they are likely using) the harder it is to get an LLM into this mode even *IF* you know the right words to say at that point the best way forward is to terminate the exchange entirely, and to start off with the right initial message, instantly getting into coding mode. a non technical person will not know this and be stuck in feature theory crafting mode in perpetuity, or worse in an endless "excuses' mode as the LLM diverts ant attempt at coding into reasons why its not going to: "i wont output incomplete/broken code! that would require too many lines of code sorry i wont do it! i wont be able to get it perfect so i wont attempt it! but heres more features and theory crafting" will a non technical person know to end the conversation and start fresh? not likely unless they have a lot of experience already with LLMs
- roncesvalles 2mo ago
  Also, the correct way to LLM is to constantly trial-and-error in new/branched contexts. Remember the LLM is not a human employee. You don't have to say "yes and" to whatever crap they produced so as to not hurt their feelings or infringe upon their creative autonomy, nor do you have to defend the correctness of your original instructions so that they don't think less of you for asking them to chase the wrong goose. I probably generate 20-50 lines of code for every 1 line that I keep. This is also why I think harnesses and things like Claude Code and OpenCode are false efficiency. The only way I can maintain my pace of branched trial-and-error is by using claude.ai/chat and manually extricating code fragments to and from my codebase. The human is still the best harness for production-level code.
  - ChrisMarshallNY 2mo ago
    > The only way I can maintain my pace of branched trial-and-error is by using claude.ai/chat and manually extricating code fragments to and from my codebase. The human is still the best harness for production-level code. That’s been the way I do it. I suppose that it will be considered “quaint,” soon enough, but I have found it to be effective.
ben_w 2mo ago
Interesting, thanks for sharing. > Whereas to me who knows the terminology would have probably just taken a single exchange of messages to get the result she described to me. I would have prompted with something like “Please write to me an html page which does X, Y, Z.” But since she didn’t know the right terminology she got into a vortex of feature discusion, and she didn’t find a way to tip the AI into “just do it, write it now” mode. Even as a developer, when I've been using the web chat interface for things which I know the AI can do easily, I've had this happen to me a few times. I was very surprised the first time I saw ChatGPT respond ~"this would be a few thousand tokens, I can't do that". Even more surprising: ChatGPT was accurate when responding that way this time, despite this being trivial for Claude and well within what ChatGPT could do using the web chat interface 6 months earlier. The ChatGPT output was extremely meh.
didntknowyou 2mo ago
same thing as people just entering a question prompt and copying and pasting the response as gospel. e.g. politicians using it to write speeches, or lawyers for testimonials. the output is programmed to look correct so unless you have some sort of background you won't actually know what errors to look for.
- b112 2mo ago
  And look correct is accurate. Not only does it look correct, it looks correct with an extremely Subject Matter Expert degree of authority. Often I'll work with an LLM, and it simply just misses so many things. I've worked in all sorts of different domains, software, chemistry, material design, everything from power generation through to physics, and in each and every case I see it missing incredibly important things. Any true subject matter expert would immediately bring up and prompt concerns, but not the LLM. This makes sense, of course, because these are language models. They were trained on language. Their first and foremost capability is language. An LLM's true expertise, true subject matter expertness is language. And so anyone working with LLMs who isn't already highly skilled in the field they're asking questions about, will invariably be led astray and miss extremely important parts of a puzzle that need to be solved.
  - lazyasciiart 2mo ago
    It's just like working with engineers.
lazyasciiart 2mo ago
did she try saying "I want to make a webpage"? Even ChatGPT will just build, deploy and host a webpage for you with that request. I don't really understand what system she must have been using.
deleted 2mo ago
[deleted]
bjackman 2mo ago
I have been pondering this and I think it's likely a gap that will get filled sooner or later. Right now there's just so much value in building LLM tools for experts that everyone is focusing on that. But surely at some point we'll have bespoke harnesses that exist exactly to solve this kind of thing. I think this can start with constrained problem spaces like "you are a WordPress developer, you solve problems for people with enough expertise to know they are looking for a WordPress developer" and incrementally expand from there. Maybe I'm naive but I think you can probably get pretty far with this today just by writing loads of skills and picking the right technical preferences to encode in them.
- Cthulhu_ 2mo ago
  Yeah there's lots of companies out there that promise that people can build a website quickly, often using WYSIWYG / drag / drop interfaces, I bet they already have AI integration to speed things up. ("I bet" because I don't actually use those services.) I think people are better helped by using those services than going a level lower and using LLMs directly.
- redhale 2mo ago
  We already have these, and have for a while. Replit and Bolt exist for exactly this case. Not sure why GP didn't direct their friend to these tools (or if these tools are somehow the ones that failed in this example). In my experience, Replit and Bolt (and other similar tools) are quite good at this kind of 0->1 kind of thing.
toasty228 2mo ago
You can see the same thing with restaurants generating their own menus/pictures. Some of them look absolutely terrible, visually ugly, way too information dense, the classic piss filter, etc. idk how they do it, even the most basic prompt I can come up with makes something 10x better, and when I put in my amateur photography knowledge/keywords in it gets pretty close to what I'd consider a good pre LLM quality menu. Literally just adding "make it look nicer/cleaner" seems to get rid of most problems, but people just don't care apparently.
- DrewADesign 2mo ago
  Complete amateurs without these tools were basically limited to making big text in word processors and maybe pasting an image in there. Being able to create a basically coherent, polished looking image is what you’d get on Fiverr for a few bucks. Mostly hustlers filling in templates, or people that know the tools but never learned design fundamentals. Actually being a competent professional: Knowing how to visually communicate showing information hierarchy, what purely visual aspects of an image say, how different things read differently among people who might see it— e.g. does an image of an apple communicate fancy computer? teachers/school? Nutrition? Food? Produce?, etc etc etc (Good kerning and type usage, composition, gestalt, etc all come with that for free. Many think that is the point — those are tools someone can wield to do good design, they aren’t themselves good design.) These tools let amateurs do what the fiverr crowd used to do. Unfortunately, the fiverr crowd is now being pushed into doing what entry-level new graduate professionals used to do, and the job market is kind of fucked.
- chrisjj 2mo ago
  > people just don't care apparently. This is the target user of these chatbots.
nvrmndmnm 2mo ago
I ran a similar test and got completely different results. My girlfriend (hair stylist/artist) with zero coding background mentioned a Telegram bot idea. I asked "Why not build it yourself?" I gave her a Windows laptop, but she said she wanted what I have instead. So I handed her a USB stick and told her she was on her own now. Fast forward: she now runs Arch Linux with Hyprland (I use Xorg/i3 though), fully riced with cats. I only interfered on partitioning to preserve my data. She even installed Steam and got Portal running (that was her "watch this" flex after I told her not to even try). She pulled all of this off using a free-tier Gemini chat. For the bot, a friend gave her a Kimi 2.7 key. She set up their harness and built a working bot in a matter of days. She even got a free Oracle VPS for deployment, though I stopped her there to check security first (still haven't had time, unfortunately). She uses that laptop daily now and says she enjoys it over Windows by a mile.
- NichoPaolucci 2mo ago
  There are big differences in what people call "tinkerers". Some are INTERESTED in building things, give it a little shot and find it to be daunting, and push it no further. True tinkerers have no problem with this, because they enjoy learning how things work. Installing Linux, Steam, Portal are all relatively straightforward tasks for someone who uses computers on the regular - but to some people this is just something they've never done, are scared to do, or just don't want to learn. (Which is fine, but they'll never pick these agents up and run free.) Barrier to entry used to be blog posts, documentation, watching poor quality Youtube videos of a thing that SEEMS related to what you're trying to do. Now we're getting that spoon fed to our particular case, so the friction is essentially just "follow the AI directives". (However, the depth of understanding probably struggles.)
  - cgannett 2mo ago
    You can get pretty dang far asking the AI to explain things and show you where it got that info, especially with IT/DEV stuff. Thing that has been very useful to me is specifically asking AI for that vocabulary. AI is suprisingly useful for giving a vague description of something you want and asking for possible words that match. But you have to KNOW to try that. A tinkerer might have started that convo with the chatbot by asking "What are web pages built from" "Can we use those building blocks to make our own simple app" and gotten there from a place of low expertise in web development but juicing out the knowlege from a place of low expertise is itself a skill.
bartread 2mo ago
Interesting, but... wrong tool, wrong job. And by tool, I mean web based chat interface, not LLMs in general. (Maybe wrong delivery mechanism, if you like.) Your friend needed an agent, not a chatbot. I use Claude within VS Code (as per many others) but I certainly wouldn't recommend that for a beginner. They needed a tool that's specifically aimed at people who want to build software but don't know the first thing about how to do it. I think there are a bunch of these now but the one I'm most aware of is Lovable, and I'm pretty surprised you didn't recommend one of these. An HR person I know was searching for a way get something build, found Lovable, and managed to build a somewhat functional application with it on their first attempt within an hour or two. It was full of holes and far from perfect but they got something working - at least the outline of a potential solution. As I say, you should have recommended your friend to try building with a tool like that: a tool that they're a member of the target market for. They would have got a lot further. I'm not vouching for the quality of the result, but they would have got something. Even for experiened engineers, chatbots have always been a pretty grim experience for software development: from the mind-numbing drudgery of endlessly copying and pasting code, commands, and prompts around, to the fact that they just can't see enough of what you're doing to generate the best quality output or advice. You can do software development with a ChatBot but it seriously sucks, and better tools are (a) probably being shoved at you day in, day out via ads, and (b) only a Google search or a ChatGPT recommendation away. (Obviously, nobody's going to search for "Lovable" without knowing about Lovable, but they might search, or ask ChatGPT or whatever, something like, "How would I build a website without knowing anything about building websites?", which might get them an advert or recommendation.)
- massysett 2mo ago
  The whole point is that the friend wouldn’t know what an agent is. The friend would have heard that AI is changing the world and she would have went into ChatGPT or maybe Claude or Gemini. Presuming that the ideal is that anyone should be able to use these tools for anything (which I do not agree with) the chatbot needs to say “you need to use my agent mode.” This is not a wrong tool, wrong job issue. The issue is that people think any moron can use an LLM and get professional results.
  - 2mo ago
ghm2199 2mo ago
I think this is about teaching problem solving at a young age and it is an abomination that our education system does so poorly at. The key is to know what kind of questions to ask and knowing when to go deeper and what to pay attention to. But that is not how our education system aligns us. One typical example of problem solving kids, and I too, learn in school is how to apply a concept in physics to a free-body-diagram(Indian and Chinese education cram schools are famously good at teaching kids how to do this, the usefulness of which I debate). But it all stops at the exam room. No architect or jr structural engineer position for you kiddo. Another kind of problem solving skill might be how to invest money and understand your own risk appetite to construct portfolios to manage your money. All that is taught in school is a dry compound interest formulae, time discounted cash flows and a black scholes model. Only to find out later I don't need most of it to manage my money.
- ghm2199 2mo ago
  It's also my opinion that Kids should get to do a lot of real work and experimentation at an early age(13 yo IMO) with real responsibilities and prospects of making income. Knowing the failure mode of most real world problems you can solve is the skill you always want but dont nearly get to do enough of until much later in life(sadly true for a large population of kids in the world).
giancarlostoro 2mo ago
Every now and then I'll write a single prompt to Claude to see how much of a web app I can build in one shot. I usually pick Django (though I assume any other web framework could suffice) because of the battery included nature, I still sometimes describe that I want users to login / register, have access to x, y, z I also noticed a friend of mine had way more success by having Claude write code by doing TDD and giving Claude scenarios for things the code should be able to handle, if you do this correctly, and cover edge cases, Claude will work with these in mind.
throwatdem12311 2mo ago
Weird because I had a friend that wanted a web app, also no experience, and he just told Claude (on the iOS app) to make it for him and Claude just made an artifact and put everything there. Buddy sent me a share link and he was just like “dude this is crazy I just told it what I wanted and it just spit it out in a few minutes”. No “experiment” needed he just did it because he knew Claude could do it and it worked exactly like you’d expect. Hell, he did it on the Free tier.
yashpatelcode 2mo ago
I've seen the same thing happen to my brother when he tried making an app with zero experience. Only difference was that he got a front end that didn't work.
enum 2mo ago
This sort of thing has been studied in academic experiments. Although the models studied are now “old”, and I expect the floor is higher, the lack of vocabulary and basic concept familiarity sets the ceiling. https://www.feldmanmolly.com/chiwork2024-author-version.pdf https://www.feldmanmolly.com/chiwork2024-author-version.pdf
RajT88 2mo ago
I am reminded of this one: https://thedailywtf.com/articles/Could-You-Explain-Programming-Please https://thedailywtf.com/articles/Could-You-Explain-Programmi...
- lukan 2mo ago
  The comments feel so outdated: 'him: Keywords? Variables? ... me: (Explaining what programming actually is) him: Oh, I thought I would write something like: "Create football stadium and football players. Start the game when user presses spacebar. Make players have red shirts and white socks."'
  - RajT88 2mo ago
    They are from 2008 is why. lol
    - lukan 2mo ago
      Yes, but usually programming knowledge does not outdate so quickly. Now we actually have prompts, what some people always believed that it always worked like this.
janpeuker 2mo ago
My experience is similar, people succeed with Loveable or similar platforms because they don't need to know _deployment_ and _runtime_ experience. It's like using Adobe vs Canva, most people are now used to the latter and don't even have a mental model of "runs on a server" or "database and server are two different things". This is the key difference for me compared to earlier Low Code solutions such as OutSystems that still required a SDLC mental model. However, at some point this breaks, like in your example of feature discussions, because people struggle to test out new ideas _without_ immediately showing them. So it becomes Canva vs Figma, a system-based model. And we don't even have good terminology for that ourselves yet.
alpha_trion 2mo ago
I've seen some designers and product managers get pretty far with LLMs but mostly because they already know how to build apps just from a non-technical perspective.
sensanaty 2mo ago
I just gave this a quick try with Sonnet 5 Medium [0] on a free Anthropic account. It's a bit of a contrived example I guess, but it probably isn't too far off what a completely non-technical person seeing this for the first time would do. The output is EXTREMELY misleading, as all the data here lives purely locally, yet the AI says that you can "just share the link" and other people will see the schedule you set on the generated artifact. Also, what link? To the Claude chat? It doesn't explain what to do with that `Booking` artifact other than "link to it". I can so easily see someone tapping out a few steps down the line of this once they realize it doesn't work and they have no clue what to do or say to make it work. What do you even ask as a non-technical person at this point? I guess they could explain "The other person doesn't see it", but would the AI actually clarify that it's because it's not fucking hosted anywhere and has no mechanism of persisting the data outside of the current machine, or would it - as I'm almost 100% sure would be the case - not actually point out this error? [0] https://claude.ai/share/0cbfe698-3886-4d4d-86e4-7c697b61dc00 https://claude.ai/share/0cbfe698-3886-4d4d-86e4-7c697b61dc00
honr 2mo ago
...didn't have the vocabulary... That is exactly the key or the sign there. Even with a couple of decades of engineering expertise, when I try to do / research something that I don't know enough about, I find myself in the exact position of not having the vocabulary. To the point that I sometimes have to ask the AI "nicely" to cut the pleasantries and be ruthless against nonsense, whether from its/their side or from mine.
jasondigitized 2mo ago
Having a similar experience. Watching a muggle try to build a website with basic functionality, e.g. auth, database, etc. is an enlightening experience. They simply do not have the vocabulary to guide the LLM. My good friend calls me every night frustrated with his results, and when I watch what he is doing it's amazing what we take for granted being in the software industry. Don't get me started on the UX, that's even more mind boggling.
ryeguy_24 2mo ago
I have counter evidence of this. My wife with absolutely zero skills in computers was at the terminal doing things that Claude was telling her to do and generated some impressive tools. One was a tool to help her organize her day. It involved scripts, PDF generation, printing. She did it all without even asking me. Maybe it fails in some cases but I'm not sure the anecdote above is the average experience.
pvorb 2mo ago
I'm wondering: What did she tell you that made you build the website? Did she ask the same thing from the AI? Did you bridge any gaps the AI didn't do for you? I don't know the answer, but I suspect that she treated the AI differently and would've gotten better results if she had asked what she told you. At least if it's an agentic coding assistant like Claude Code or GitHub Copilot. Of course a simple chat will leave manual tasks for you.
gwbas1c 2mo ago
I find there's a lot of variation between LLMs / models... But there's also some psychology in play too; that we (engineers) see a lot: Some people just let their imagination run away and forget to "do"; without someone in the conversation pushing for results and action, the conversation will just stay within imagination and everyone will be happy in the moment but nothing will get done.
chrisjj 2mo ago
> Because my friend didn’t have the vocabulary to ask the AI to write code. Why did she even need that? > “Please write to me an html page which does X, Y, Z.” But since she didn’t know the right terminology No code there!
plutokras 2mo ago
> My friend didn’t have the vocabulary to ask the AI to write code I had the exact same realization using Claude Design. It’s great that "anyone can design" now, but I don't have the domain knowledge to describe what a good design consists of. I know it when I see it, but that is far from enough.

by krisoft 2mo ago

Melatonic 2mo ago

Reminds me of watching someone who has no idea how to use a search engine try to use a search engine

matthew-wegner 2mo ago

Are you describing a "chat window" experience here? This is apples to oranges.

Synthetic7346 2mo ago
Yeah I wonder if they had given their friend Claude code or Codex, would it have been more likely to create what she wanted?
- davely 2mo ago
  Perhaps! But I do think the vocabulary issue is real and I think LLMs are still sycophantic enough that they won’t really challenge someone or offer alternative ideas on how to implement something unless they explicitly ask. Interestingly at my work, Claude Code was available before Claude Desktop, so a number of non-technical PMs tried to use it in order to build… anything, with very mixed success. The “hey guys, check out the website I built with Claude: http://localhost:3000/” joke is real! In my experience, the whole “the terminal is a scary place” aspect is very real and some non-technical people can feel intimidated by. I think Claude Code in the desktop app helps alleviate that a bit (perhaps Codex, too, but man what a mess the ‘ol ChatGPT app has become). But I’m sure there are entire repos of web dev skills that someone could use to put together things with a bit of effort.
  - ElectricalUnion 2mo ago
    > the terminal is a scary place Isn't the the powerful, unlimited, unopinionated blank LLM text input waiting for your instructions eerily similar to a scary terminal? WIMP and GUI paradigms are the exact the opposite: intentional dis-empowering, by design restrictions, enumeration of your few possible options. Those feel more constrained therefore safer.
    - andsoitis 2mo ago
      > WIMP and GUI paradigms are the exact the opposite: intentional dis-empowering Are you kidding? WIMP and GUI democratized computing!
      - kazinator 2mo ago

AnotherGoodName 2mo ago

OTOH my wife's friends got drunk and made "tinder for horse purchases". They prompted to read typical horse advertisements (we're all horse people) and create an app with mock tinder like entries to swipe right and left to buy horses. A web app was produced with lots of mock "Hi i'm Dominique and i love running through fields and having a bucking good time" type entries complete with silly horse photos. A huge amount of drunken fun even if it boiled a towns water supply and blew through half a subscription to create. I was looking at the results as a dev with 30 years experience and thinking fuck me. The little apps i made here and there before AI are being outdone by a bunch of drunk people on a whim!

selcuka 2mo ago
> The little apps i made here and there before AI are being outdone by a bunch of drunk people on a whim! Yes, but the premise of the article is you should be able to outdo a bunch of drunk people with your 30 years experience, if you use AI too.
- vasco 2mo ago
  The article doesn't hold the universal truth.

RileyJames 2mo ago

I’d love to see this experiment executed with Claude design. Particularly with something static, I don’t think they’d fail to get a result. But without domain knowledge I think they’d misunderstand prototype with finished product. Without knowing what it’s doing, it’s hard to know what it’s not doing.

andsoitis 2mo ago
> Particularly with something static Apps ain’t static.
- folkrav 2mo ago
  They probably mean static site, in the sense of static front end, no backend.
  - andsoitis 2mo ago
    Yes. That's not an app.
    - shukantpal 2mo ago
      Not really, is a offline chess page not an app? It seems like it would be closer to an app.
      - andsoitis 2mo ago
        What does this hypothetical chess page do? And how’s does it do it if it is static?
        Tepix 2mo ago
        It uses javascript. Still a static file. Lets you play chess
        2mo ago

yieldcrv 2mo ago

copying and pasting code? a few months ago? 2025 called and wants its test back

redrove 2mo ago
Exactly, sounds to me like OP is the one with the LLM skill issue.

ynab6 2mo ago

[dead]

coder-3 2mo ago

We've seen plenty of examples of apps successfully vibe coded by non technical people, including apps making real revenue. Your friend could start with telling the LLM that they are a non technical person who wants to make an app and it will explain all the successive steps.

roncesvalles 2mo ago
>We've seen plenty of examples of apps successfully vibe coded by non technical people, including apps making real revenue. Have we? Or is this just something that people say now, without citation?
- colonelspace 2mo ago
  .
  - jojobas 2mo ago
    That was kinda harnessed and prompted by a team of security experts, so there's that.
- threatofrain 2mo ago
  I’m afraid this isn’t surprising anymore.
- fragmede 2mo ago
  Plinq. Plinq was made on Lovable, https://www.aieatingtheworld.com/articles/non-technical-founder-lovable-app-builder-success-story https://www.aieatingtheworld.com/articles/non-technical-foun... Couple more on https://buildthedamnthing.com/resources/articles/case-studies-vibe-coding-builder https://buildthedamnthing.com/resources/articles/case-studie...
  - shepherdjerred 2mo ago
    Wow, I am not sure that I want my safety app to be videcoded by someone without experience

petra 2mo ago

So maybe your friend need a role reversal: A prompt guiding the llm to act as a consultant, guiding people in implementing a software project, asking questions, creating a shared understanding, limiting scope or creating milestones. That seems like something that could be done using an llm, not that complicated probably. And maybe, in other fields as well.

friendzis 2mo ago

> They didn’t even get to that point. Because my friend didn’t have the vocabulary to ask the AI to write code. What harness did you use? In e.g. claude, there are two modes: 1. Spit out code 2. Draft a plan, ask questions, GOTO 1 You literally have to go out of your way to get it NOT to write code. I keep mine on a tight-ish leash because it modify code way too happily even when there's no intention or instruction to do so

sam_lowry_ 2mo ago
Harness? I am an avid HN reader but even I don't yet fully understand how this word is used in the AI context. Which is exactly the point of the OP. It all boils down to naming things and cache invalidation, /s
- tyzoid 2mo ago
  Harness refers to the tooling that allows you to interact with an LLM. Web chat interface is a harness, CLI coding tools (claude code, codex, opencode, pi, etc.) are harnesses, agent systems (openclaw, hermes, etc.) are harnesses. They present different capabilities to the underlying LLM. Codex, for example, is more likely to write code if you say "I want to build an app" than ChatGPT on web will.
  - darkwater 2mo ago
    Which is an abstraction layer that the non-technical should need to know, if we follow GP's line of reasoning.
- sevenzero 2mo ago
  Personally I even refuse to learn what it is in the AI context. Feels like a waste of energy given new ~~~best ways to use YOUR tokens~~~ are discovered every month and things you learn now will be useless next month. I am doing perfectly fine with the web UI version of these tools... They seem to also not make tokens dissappear as fast as using claude cli tool to automate implementations. Makes my work day more tolerable as well as I actually have something to do over waiting until some implementation can be read through...

raincole 2mo ago

Do your friend at least know what Claude Code (or any harness) is? Of course she wouldn't be able to make a website if she doesn't even know the right tools to use. But I don't think it prove anything. Knowing and installing Claude Code might not be a common sense, but nor is it "expertise" or "skill." I've seen in first hand that people struggle installing Steam. Yes, "people" in the plural. But just because some people struggle with it, it doesn't mean that installing Steam isn't an objectively easy task. Your friend's experience doesn't change the fact that building a website is something that an average person can do in hours if not minutes.

harvey9 2mo ago
Knowing what Claude Code is and why you might want it actually is domain knowledge which the op's friend does not have. Your idea of the average person might be biased if you work and socialise with people who have this kind of expertise.
- vector_spaces 2mo ago
  I'm genuinely a bit floored reading the comments here, but I guess my idea of the average HN commenter's ability to talk to non-technical folks about technical topics is biased because I work and socialize with lots of people who don't have technical expertise (thankfully along with other technical folks who also have lots of experience talking to the former group). Many of them don't even know what Claude is, let alone Claude Code.
  - Maxion 2mo ago
    The average person does not even know what ChatGPT is and has not interacted with an LLM ever.
    - walthamstow 2mo ago
      I don't think this is true. Certainly not in Britain. I went to 3 weddings last summer and each one had a joke in a speech about using ChatGPT to write it and everyone laughed. 68yo father of the bride is a retired plumber and even he's cracking jokes about AI.

sshine 2mo ago

I have the same experience giving my brother an OpenClaw as his personal assistant. His words were: It feels as if I need to know how to program it. I was expecting he could say something like "Oh, it seems like you don't remember the people I'm referring to, perhaps you need some kind of CRM system. Can you investigate if there are any easily available CRM systems you can interface with, so we don't need to make one for you?" Whereas my OpenClaw moment was trying to make it manage its own NixOS installation, so that if I ask it to do something, it doesn't yolo `apt install` commands, but rather improves on the same overview of its own installation. A lot of people had success making their OpenClaw do things without being Linux experts. But you need a tinkerer's mindset, is what I came to conclude.

walthamstow 2mo ago
Curiosity and a tinkerer's / engineer's mindset is basically the only thing we can hire for right now
- sshine 2mo ago
  That and a proven ability to apply it in a given domain. I am really not good with electronics. I’m sure it’s just a mental block, but I see people debug electronics as relentlessly as I debug code.

petesergeant 2mo ago

I’ve had the absolute opposite experience with a friend of mine. I started by setting her up with a terminal emulator on the web linked to Claude Code, and written a CLAUDE.md that told it how to deploy. These days (with no further intervention by me) she’s running Claude Code natively on her laptop, and she tagged me yesterday on Facebook in some update about how much more she enjoyed using Claude Code than plain Claude.

MattGaiser 2mo ago

What terminology is required to make Claude Code write code? My experience with friends has been the opposite. A PM friend made a custom tool. A friend who has never written a line of computer code has an app. If you tell Claude Code "I want a website that does X, Y, and Z" it will write code.

knollimar 2mo ago
Especially if the path is well trodden. "Make me a kanban board for construction tasks. I want you to walk me through the process of hosting it" worked for me.

lynx97 2mo ago

Question is if the outcome was LLM related or just a result of a personality trait. After all, talking in circles without really approaching an end-goal is something certain humans do all day long.

downloadram 2mo ago

>she didn’t find a way to tip the AI into “just do it, write it now” mode worse, the longer an LLM conversation goes on, but especially with constricted/free models (yes the simple chat interface they are likely using) the harder it is to get an LLM into this mode even *IF* you know the right words to say at that point the best way forward is to terminate the exchange entirely, and to start off with the right initial message, instantly getting into coding mode. a non technical person will not know this and be stuck in feature theory crafting mode in perpetuity, or worse in an endless "excuses' mode as the LLM diverts ant attempt at coding into reasons why its not going to: "i wont output incomplete/broken code! that would require too many lines of code sorry i wont do it! i wont be able to get it perfect so i wont attempt it! but heres more features and theory crafting" will a non technical person know to end the conversation and start fresh? not likely unless they have a lot of experience already with LLMs

roncesvalles 2mo ago
Also, the correct way to LLM is to constantly trial-and-error in new/branched contexts. Remember the LLM is not a human employee. You don't have to say "yes and" to whatever crap they produced so as to not hurt their feelings or infringe upon their creative autonomy, nor do you have to defend the correctness of your original instructions so that they don't think less of you for asking them to chase the wrong goose. I probably generate 20-50 lines of code for every 1 line that I keep. This is also why I think harnesses and things like Claude Code and OpenCode are false efficiency. The only way I can maintain my pace of branched trial-and-error is by using claude.ai/chat and manually extricating code fragments to and from my codebase. The human is still the best harness for production-level code.
- ChrisMarshallNY 2mo ago
  > The only way I can maintain my pace of branched trial-and-error is by using claude.ai/chat and manually extricating code fragments to and from my codebase. The human is still the best harness for production-level code. That’s been the way I do it. I suppose that it will be considered “quaint,” soon enough, but I have found it to be effective.

ben_w 2mo ago

Interesting, thanks for sharing. > Whereas to me who knows the terminology would have probably just taken a single exchange of messages to get the result she described to me. I would have prompted with something like “Please write to me an html page which does X, Y, Z.” But since she didn’t know the right terminology she got into a vortex of feature discusion, and she didn’t find a way to tip the AI into “just do it, write it now” mode. Even as a developer, when I've been using the web chat interface for things which I know the AI can do easily, I've had this happen to me a few times. I was very surprised the first time I saw ChatGPT respond ~"this would be a few thousand tokens, I can't do that". Even more surprising: ChatGPT was accurate when responding that way this time, despite this being trivial for Claude and well within what ChatGPT could do using the web chat interface 6 months earlier. The ChatGPT output was extremely meh.

didntknowyou 2mo ago

same thing as people just entering a question prompt and copying and pasting the response as gospel. e.g. politicians using it to write speeches, or lawyers for testimonials. the output is programmed to look correct so unless you have some sort of background you won't actually know what errors to look for.

b112 2mo ago
And look correct is accurate. Not only does it look correct, it looks correct with an extremely Subject Matter Expert degree of authority. Often I'll work with an LLM, and it simply just misses so many things. I've worked in all sorts of different domains, software, chemistry, material design, everything from power generation through to physics, and in each and every case I see it missing incredibly important things. Any true subject matter expert would immediately bring up and prompt concerns, but not the LLM. This makes sense, of course, because these are language models. They were trained on language. Their first and foremost capability is language. An LLM's true expertise, true subject matter expertness is language. And so anyone working with LLMs who isn't already highly skilled in the field they're asking questions about, will invariably be led astray and miss extremely important parts of a puzzle that need to be solved.
- lazyasciiart 2mo ago
  It's just like working with engineers.

lazyasciiart 2mo ago

did she try saying "I want to make a webpage"? Even ChatGPT will just build, deploy and host a webpage for you with that request. I don't really understand what system she must have been using.

deleted 2mo ago

[deleted]

bjackman 2mo ago

I have been pondering this and I think it's likely a gap that will get filled sooner or later. Right now there's just so much value in building LLM tools for experts that everyone is focusing on that. But surely at some point we'll have bespoke harnesses that exist exactly to solve this kind of thing. I think this can start with constrained problem spaces like "you are a WordPress developer, you solve problems for people with enough expertise to know they are looking for a WordPress developer" and incrementally expand from there. Maybe I'm naive but I think you can probably get pretty far with this today just by writing loads of skills and picking the right technical preferences to encode in them.

Cthulhu_ 2mo ago
Yeah there's lots of companies out there that promise that people can build a website quickly, often using WYSIWYG / drag / drop interfaces, I bet they already have AI integration to speed things up. ("I bet" because I don't actually use those services.) I think people are better helped by using those services than going a level lower and using LLMs directly.
redhale 2mo ago
We already have these, and have for a while. Replit and Bolt exist for exactly this case. Not sure why GP didn't direct their friend to these tools (or if these tools are somehow the ones that failed in this example). In my experience, Replit and Bolt (and other similar tools) are quite good at this kind of 0->1 kind of thing.

toasty228 2mo ago

You can see the same thing with restaurants generating their own menus/pictures. Some of them look absolutely terrible, visually ugly, way too information dense, the classic piss filter, etc. idk how they do it, even the most basic prompt I can come up with makes something 10x better, and when I put in my amateur photography knowledge/keywords in it gets pretty close to what I'd consider a good pre LLM quality menu. Literally just adding "make it look nicer/cleaner" seems to get rid of most problems, but people just don't care apparently.

DrewADesign 2mo ago
Complete amateurs without these tools were basically limited to making big text in word processors and maybe pasting an image in there. Being able to create a basically coherent, polished looking image is what you’d get on Fiverr for a few bucks. Mostly hustlers filling in templates, or people that know the tools but never learned design fundamentals. Actually being a competent professional: Knowing how to visually communicate showing information hierarchy, what purely visual aspects of an image say, how different things read differently among people who might see it— e.g. does an image of an apple communicate fancy computer? teachers/school? Nutrition? Food? Produce?, etc etc etc (Good kerning and type usage, composition, gestalt, etc all come with that for free. Many think that is the point — those are tools someone can wield to do good design, they aren’t themselves good design.) These tools let amateurs do what the fiverr crowd used to do. Unfortunately, the fiverr crowd is now being pushed into doing what entry-level new graduate professionals used to do, and the job market is kind of fucked.
chrisjj 2mo ago
> people just don't care apparently. This is the target user of these chatbots.

nvrmndmnm 2mo ago

I ran a similar test and got completely different results. My girlfriend (hair stylist/artist) with zero coding background mentioned a Telegram bot idea. I asked "Why not build it yourself?" I gave her a Windows laptop, but she said she wanted what I have instead. So I handed her a USB stick and told her she was on her own now. Fast forward: she now runs Arch Linux with Hyprland (I use Xorg/i3 though), fully riced with cats. I only interfered on partitioning to preserve my data. She even installed Steam and got Portal running (that was her "watch this" flex after I told her not to even try). She pulled all of this off using a free-tier Gemini chat. For the bot, a friend gave her a Kimi 2.7 key. She set up their harness and built a working bot in a matter of days. She even got a free Oracle VPS for deployment, though I stopped her there to check security first (still haven't had time, unfortunately). She uses that laptop daily now and says she enjoys it over Windows by a mile.

NichoPaolucci 2mo ago
There are big differences in what people call "tinkerers". Some are INTERESTED in building things, give it a little shot and find it to be daunting, and push it no further. True tinkerers have no problem with this, because they enjoy learning how things work. Installing Linux, Steam, Portal are all relatively straightforward tasks for someone who uses computers on the regular - but to some people this is just something they've never done, are scared to do, or just don't want to learn. (Which is fine, but they'll never pick these agents up and run free.) Barrier to entry used to be blog posts, documentation, watching poor quality Youtube videos of a thing that SEEMS related to what you're trying to do. Now we're getting that spoon fed to our particular case, so the friction is essentially just "follow the AI directives". (However, the depth of understanding probably struggles.)
- cgannett 2mo ago
  You can get pretty dang far asking the AI to explain things and show you where it got that info, especially with IT/DEV stuff. Thing that has been very useful to me is specifically asking AI for that vocabulary. AI is suprisingly useful for giving a vague description of something you want and asking for possible words that match. But you have to KNOW to try that. A tinkerer might have started that convo with the chatbot by asking "What are web pages built from" "Can we use those building blocks to make our own simple app" and gotten there from a place of low expertise in web development but juicing out the knowlege from a place of low expertise is itself a skill.

bartread 2mo ago

Interesting, but... wrong tool, wrong job. And by tool, I mean web based chat interface, not LLMs in general. (Maybe wrong delivery mechanism, if you like.) Your friend needed an agent, not a chatbot. I use Claude within VS Code (as per many others) but I certainly wouldn't recommend that for a beginner. They needed a tool that's specifically aimed at people who want to build software but don't know the first thing about how to do it. I think there are a bunch of these now but the one I'm most aware of is Lovable, and I'm pretty surprised you didn't recommend one of these. An HR person I know was searching for a way get something build, found Lovable, and managed to build a somewhat functional application with it on their first attempt within an hour or two. It was full of holes and far from perfect but they got something working - at least the outline of a potential solution. As I say, you should have recommended your friend to try building with a tool like that: a tool that they're a member of the target market for. They would have got a lot further. I'm not vouching for the quality of the result, but they would have got something. Even for experiened engineers, chatbots have always been a pretty grim experience for software development: from the mind-numbing drudgery of endlessly copying and pasting code, commands, and prompts around, to the fact that they just can't see enough of what you're doing to generate the best quality output or advice. You can do software development with a ChatBot but it seriously sucks, and better tools are (a) probably being shoved at you day in, day out via ads, and (b) only a Google search or a ChatGPT recommendation away. (Obviously, nobody's going to search for "Lovable" without knowing about Lovable, but they might search, or ask ChatGPT or whatever, something like, "How would I build a website without knowing anything about building websites?", which might get them an advert or recommendation.)

massysett 2mo ago
The whole point is that the friend wouldn’t know what an agent is. The friend would have heard that AI is changing the world and she would have went into ChatGPT or maybe Claude or Gemini. Presuming that the ideal is that anyone should be able to use these tools for anything (which I do not agree with) the chatbot needs to say “you need to use my agent mode.” This is not a wrong tool, wrong job issue. The issue is that people think any moron can use an LLM and get professional results.
- 2mo ago

ghm2199 2mo ago

I think this is about teaching problem solving at a young age and it is an abomination that our education system does so poorly at. The key is to know what kind of questions to ask and knowing when to go deeper and what to pay attention to. But that is not how our education system aligns us. One typical example of problem solving kids, and I too, learn in school is how to apply a concept in physics to a free-body-diagram(Indian and Chinese education cram schools are famously good at teaching kids how to do this, the usefulness of which I debate). But it all stops at the exam room. No architect or jr structural engineer position for you kiddo. Another kind of problem solving skill might be how to invest money and understand your own risk appetite to construct portfolios to manage your money. All that is taught in school is a dry compound interest formulae, time discounted cash flows and a black scholes model. Only to find out later I don't need most of it to manage my money.

ghm2199 2mo ago
It's also my opinion that Kids should get to do a lot of real work and experimentation at an early age(13 yo IMO) with real responsibilities and prospects of making income. Knowing the failure mode of most real world problems you can solve is the skill you always want but dont nearly get to do enough of until much later in life(sadly true for a large population of kids in the world).

giancarlostoro 2mo ago

Every now and then I'll write a single prompt to Claude to see how much of a web app I can build in one shot. I usually pick Django (though I assume any other web framework could suffice) because of the battery included nature, I still sometimes describe that I want users to login / register, have access to x, y, z I also noticed a friend of mine had way more success by having Claude write code by doing TDD and giving Claude scenarios for things the code should be able to handle, if you do this correctly, and cover edge cases, Claude will work with these in mind.

throwatdem12311 2mo ago

Weird because I had a friend that wanted a web app, also no experience, and he just told Claude (on the iOS app) to make it for him and Claude just made an artifact and put everything there. Buddy sent me a share link and he was just like “dude this is crazy I just told it what I wanted and it just spit it out in a few minutes”. No “experiment” needed he just did it because he knew Claude could do it and it worked exactly like you’d expect. Hell, he did it on the Free tier.

yashpatelcode 2mo ago

I've seen the same thing happen to my brother when he tried making an app with zero experience. Only difference was that he got a front end that didn't work.

enum 2mo ago

This sort of thing has been studied in academic experiments. Although the models studied are now “old”, and I expect the floor is higher, the lack of vocabulary and basic concept familiarity sets the ceiling. https://www.feldmanmolly.com/chiwork2024-author-version.pdf https://www.feldmanmolly.com/chiwork2024-author-version.pdf

RajT88 2mo ago

I am reminded of this one: https://thedailywtf.com/articles/Could-You-Explain-Programming-Please https://thedailywtf.com/articles/Could-You-Explain-Programmi...

lukan 2mo ago
The comments feel so outdated: 'him: Keywords? Variables? ... me: (Explaining what programming actually is) him: Oh, I thought I would write something like: "Create football stadium and football players. Start the game when user presses spacebar. Make players have red shirts and white socks."'
- RajT88 2mo ago
  They are from 2008 is why. lol
  - lukan 2mo ago
    Yes, but usually programming knowledge does not outdate so quickly. Now we actually have prompts, what some people always believed that it always worked like this.

janpeuker 2mo ago

My experience is similar, people succeed with Loveable or similar platforms because they don't need to know _deployment_ and _runtime_ experience. It's like using Adobe vs Canva, most people are now used to the latter and don't even have a mental model of "runs on a server" or "database and server are two different things". This is the key difference for me compared to earlier Low Code solutions such as OutSystems that still required a SDLC mental model. However, at some point this breaks, like in your example of feature discussions, because people struggle to test out new ideas _without_ immediately showing them. So it becomes Canva vs Figma, a system-based model. And we don't even have good terminology for that ourselves yet.

alpha_trion 2mo ago

I've seen some designers and product managers get pretty far with LLMs but mostly because they already know how to build apps just from a non-technical perspective.

sensanaty 2mo ago

I just gave this a quick try with Sonnet 5 Medium [0] on a free Anthropic account. It's a bit of a contrived example I guess, but it probably isn't too far off what a completely non-technical person seeing this for the first time would do. The output is EXTREMELY misleading, as all the data here lives purely locally, yet the AI says that you can "just share the link" and other people will see the schedule you set on the generated artifact. Also, what link? To the Claude chat? It doesn't explain what to do with that `Booking` artifact other than "link to it". I can so easily see someone tapping out a few steps down the line of this once they realize it doesn't work and they have no clue what to do or say to make it work. What do you even ask as a non-technical person at this point? I guess they could explain "The other person doesn't see it", but would the AI actually clarify that it's because it's not fucking hosted anywhere and has no mechanism of persisting the data outside of the current machine, or would it - as I'm almost 100% sure would be the case - not actually point out this error? [0] https://claude.ai/share/0cbfe698-3886-4d4d-86e4-7c697b61dc00 https://claude.ai/share/0cbfe698-3886-4d4d-86e4-7c697b61dc00

honr 2mo ago

...didn't have the vocabulary... That is exactly the key or the sign there. Even with a couple of decades of engineering expertise, when I try to do / research something that I don't know enough about, I find myself in the exact position of not having the vocabulary. To the point that I sometimes have to ask the AI "nicely" to cut the pleasantries and be ruthless against nonsense, whether from its/their side or from mine.

jasondigitized 2mo ago

Having a similar experience. Watching a muggle try to build a website with basic functionality, e.g. auth, database, etc. is an enlightening experience. They simply do not have the vocabulary to guide the LLM. My good friend calls me every night frustrated with his results, and when I watch what he is doing it's amazing what we take for granted being in the software industry. Don't get me started on the UX, that's even more mind boggling.

ryeguy_24 2mo ago

I have counter evidence of this. My wife with absolutely zero skills in computers was at the terminal doing things that Claude was telling her to do and generated some impressive tools. One was a tool to help her organize her day. It involved scripts, PDF generation, printing. She did it all without even asking me. Maybe it fails in some cases but I'm not sure the anecdote above is the average experience.

pvorb 2mo ago

I'm wondering: What did she tell you that made you build the website? Did she ask the same thing from the AI? Did you bridge any gaps the AI didn't do for you? I don't know the answer, but I suspect that she treated the AI differently and would've gotten better results if she had asked what she told you. At least if it's an agentic coding assistant like Claude Code or GitHub Copilot. Of course a simple chat will leave manual tasks for you.

gwbas1c 2mo ago

I find there's a lot of variation between LLMs / models... But there's also some psychology in play too; that we (engineers) see a lot: Some people just let their imagination run away and forget to "do"; without someone in the conversation pushing for results and action, the conversation will just stay within imagination and everyone will be happy in the moment but nothing will get done.

chrisjj 2mo ago

> Because my friend didn’t have the vocabulary to ask the AI to write code. Why did she even need that? > “Please write to me an html page which does X, Y, Z.” But since she didn’t know the right terminology No code there!

plutokras 2mo ago

> My friend didn’t have the vocabulary to ask the AI to write code I had the exact same realization using Claude Design. It’s great that "anyone can design" now, but I don't have the domain knowledge to describe what a good design consists of. I know it when I see it, but that is far from enough.