13 ms·

Sometimes I wonder if I'm using the same ChatGPT as everyone else. I pay for GPT4 access in ChatGPT, and it is a really terrible writer. I also pay for Copilot,

by rcme 3y ago

Sometimes I wonder if I'm using the same ChatGPT as everyone else. I pay for GPT4 access in ChatGPT, and it is a really terrible writer. I also pay for Copilot, and I think a can count the number of times it has been helpful with one hand. I'm trying to get on this hype train, but I'm left with an empty feeling every time I try to seriously spend some time with these new tools.

imtringued 3y ago
Everyone knows that using ChatGPT is about signalling your non Luddite status so that the police won't go after you.
- dbtc 3y ago
  I noticed that you haven't been talking to your AI very much lately, what's wrong?
  - marginalia_nu 3y ago
    Please drink a verification can of AI marketing kool aid!
meerita 3y ago
Maybe you're not using the right prompts. But yes, ChatGPT needs more creativity input.
weinzierl 3y ago
There is also a huge difference between code generation (be it Copilot or ChatGPT). With Code you still have the compiler that rejects the utter nonsense and then you hopefully have tests to check if what was generated makes sense. With prose it's all your manual work again to check. If your text is mundane it might be trivial but for anything a pg would write I guess an AI is pretty useless in its current state.
- neodypsis 3y ago
  I wonder if someone has tried to chain ChatGPT's output with static analyzers/compilers/linters in a loop to improve the final response.
  - blatant303 3y ago
    https://github.com/newhouseb/clownfish https://github.com/newhouseb/clownfish Structural Alignment: Modifying Transformers (like GPT) to Follow a JSON Schema
lordnacho 3y ago
I'm surprised about copilot. I find every day it saves me time by "nexting" whatever I'm doing. Whenever it's right about what I wanted to write, I save a bit of time. My go-to example is if conditions. When I start with the first case, it almost always figures out the rest of the cases correctly.
- arvinsim 3y ago
  I thought Github Copilot was useless until I found out that I have to wait for a few seconds for Copilot to display the autosuggested code.
- javchz 3y ago
  I think copilot it's amazing saving time in boilerplate code. But I can understand why some workflows may be a liability rather than help. Similar to chatgpt sometimes allusinate code with reserved words that don't exist in the language (but exists in others), or uses old code from a deprecated api / version of your framework. In other words, somtimes the cleaning process can take as much time, as just writing the code yourself.
hammyhavoc 3y ago
How impressed or unimpressed you are by it is a direct reflection of yourself and what you are trying to achieve with it. I find zero value in it. Others are very impressed by it, but I'm not very impressed by what they hold up as an example of its output. Sometimes I can only laugh when it's horribly wrong, but they herald it as amazing. The whole thing reminds me of the blockchain hype train. Still using and loving databases here for the foreseeable—still writing things by hand for the foreseeable, and loving every moment.
- jiggawatts 3y ago
  You're not impressed!? Here on HN, a few days ago, there was a post about Microsoft publishing a GitHub repo that contained a "table recognizer AI". Basically, you feed it PDFs that contain horrible scanned images of finance records, and it spits out Excel spreadsheets. For some reason, Microsoft had just "thrown this over the fence" and released it to the public for free. This, despite man-years of effort developing the thing. It was working, and everything. I made a comment wondering if Chat GPT 4 with the vision extension could solve the same problem. One of the devs that had worked on the aforementioned AI (for years!) mentioned that yes, yes it can. Game over. Those years of effort had just been replaced with a one-sentence English-language prompt that starts with "Please output a table from..." If this doesn't blow your mind, then... I don't know how to help you understand just how much has changed, virtually overnight.
  - hammyhavoc 3y ago
    I mean, sure, I guess that's neat, but Kofax OmniPage Ultimate has been a thing for years at this point and does the whole OCR from a crappy scan thing, including reformatting it into whatever format you want. Can even take data from forms and digitize it into editable text or stick it into a database. Goes back to the '80s. Sure, different methodology, but it's not something we haven't already been doing for yonks. Users have even been able to train OmniPage Ultimate to recognize someone's bizarre and unique handwriting styles in seconds. OmniPage Ultimate is an entire workflow that's turn-key and ready-to-go.
    - 3y ago
vjk800 3y ago
I suppose you have to play with it enough to get a feeling of what kind of stuff you can trust it with. For most tasks it's pretty bad, and this is why, if you just try it for something you just thought about without any kind of filter or feedback, you're going to be disappointed with the results. I agree that there is currently way too much hype. I mean GPT4 is certainly cool and probably going to be useful in some limited use cases, but it's not the new electricity and it's not going to take over the world.
barrkel 3y ago
The tools are at their most useful when they're replacing Stack Overflow searches for common tasks. An example I had last night: how do you split a pdf into separate pages on the command line, and how do you put them back together again (I replaced one of the pages with a scanned signed version). This is a basic task, a beginner's task in the world of pdf munging, but I didn't know the answer off the top of my head. ChatGPT gave me command lines straight away. That's what they're good for right now. Beginner / common tasks in areas you're not familiar with. And a further implication, they're great for one man bands, generalist roles where simple things need to be done in lots of different areas. For specialist deep roles where you have all the tacit knowledge in mind already, not nearly as useful.
- revelio 3y ago
  By the way, on macos the preview tool can do this. You can drag and drop pages between documents using the sidebar and reorder them the same way.
  - marvin 3y ago
    That's interesting and relevant to know if you'd be doing this a few times. And it doesn't invalidate your point at all, but the time savings of having a machine tell you really fast how to do something simple is valuable. I recently had to do a 'git pull' on 120 repositories, and of course we already had an in-house tool to do that. Of course I could google a script to do it. But it saved me 5 minutes to just have ChatGPT write it for me, and I saved the overhead of a more thorough context-switch. Even the guy sitting next to me. "Hey, check this out, ChatGPT wrote this script for me and saved me 5 minutes". "Oh, but we already have a tool for that". "Yeah, I figured. Where is it?" "Ummmmm, I don't know, let me ask someone. Frank maybe?" And 5 minutes spent chasing Frank.
    - zelphirkalt 3y ago
      Hopefully that ad-hoc second script never makes it into persistent storage or a repo, otherwise you now have 2 of them and Frank will think his part to himself about others duplicating things.
deleted 3y ago
[deleted]
RugnirViking 3y ago
I find copilot and chatgpt constantly useful as just a simple way of automating what would otherwise be tedious work. I think the most important thought in using them is that as you use them more you get a sense if what they are going to be able to do and what they aren't, which lets you know when you can use them (and therefore use them with more confidence and more often) Especially in refactoring or working with poor codebases. I do notice with copilot specifically some people seem to get wildly different response times with it, for me it gives suggestions almost instantly so as im typing ive got an ongoing suggestion I can accept at any time as it gets closer to what I intend or ignore if not. A common really useful thing is when I have to set up a series of something (recently in a power plant code I set up parameters for a certain kind of battery) and then I just start writing for each other kind of battery it suggests the whole boilerplate for every case, I can go back and fill in the details and exceptions. As for chatgpt its more useful at interpreting existing code. I often give it a giant function and say "I want you to split off this part and put it into its own thread, get the output with a queue" or something like that. Again, output can often need a little bit of massaging, but I find it reduces cognitive load involved in doing such a task. The trick is to be naturally skeptical of the output but appreciate how much of your life is spent writing nonsense. I've also found that learning to put what I want exactly into words can be a particular skill that has helped my communication. Making sure you know the name and function of every tool you use. I think if I had said that to myself before I started using these things id have been incredulous, "of course I know the name of things" but no, its a different thing to be talking about them often, especially when you have an idiot savant that takes everything you say too literally. You have to learn to be consise and precise
- BiteCode_dev 3y ago
  That's the gist of it. People want to use LLM hammers like screw drivers and are disapointed. Also, since the quality of the context matters a lot for completion, copilot will be way more useful on clean code with beautiful variable names and nice structure flows than on code-golf snippets. Same got chatgpt: write good prose, and it will help you better.
  - laserlight 3y ago
renewiltord 3y ago
I use Copilot with nvim and C-x C-e at my terminal. I can use it to build out pipelines in the shell so easily it's crazy. But in the end, it's just a tool. If it isn't working for you, it isn't working for you. For me, I prefer thinking in what structure to me as bigger building blocks. So I prefer vim bindings because I used up teenage time learning it. And now I don't think in terms of characters but in terms of text-blocks. I prefer writing Java in IntelliJ and Rust in Clion because I don't think in char terms since I can use the built-in refactor tools. E.g. I write functions inline and then extract them rather than writing them as functions and calling. Likewise when I'm at the shell I think of pipelines as less than pieces. Being an experienced shell user I can usually write a parallel pipeline with conditionals and loops correct first time verbally with no computer usually. But my copilot flow is improved even better. I talk to the computer in components: # get all private ip addresses in Alibaba region cn-zhangjiakou ips=...prompted text... # ssh to each and check last long line ...Prompted text... It's faster than me and this is great for r-i-search with fzf. Overall, quite happy. But just a tool.
TaylorAlexander 3y ago
For writing I find it makes an excellent rubber duck. I absolutely do not expect it to write long form prose for me, but I can explain what I am trying to say and talk to it about how I might present that information.
sberens 3y ago
5 minutes ago I copy pasted a 200 line react component and asked ChatGPT to explain why I was getting a certain bug. Although it didn't quite get it right, it allowed me to quickly identify the cause. It only needs to save 10 minutes of my time a month for it to be worth the $20 - I would estimate it saves hours.
dbtc 3y ago
I think that you are at least using the same one that I am. "Hype" can also mean marketing.
hoseja 3y ago
It seems really great at vacuous american essaywriting.
emmelaich 3y ago
What effort have you put into learning useful prompts? I only use the free 3.5+ ChatGPT and it's written some nice mapping code and some fun poems for me. Poem subjects ranged from rheology to texas hold'em.
shapefrog 3y ago
My experience is very similar. I am neither a great writer, not a great coder, not a grammar nazi - but still have to make major corrections to the output of such tools. I came to the sad conclusion that perhaps what I write and what I do is not as repetitive or as generic as others. Over the years when I have had to do more repetition, writing once well and ctrl-c / ctrl-v.
redox99 3y ago
Tell it the columns of a certain table. Ask it a GraphQL schema for a CRUD for this table. Ask it for a resolver using X libraries. Ask it for unit tests using Y libraries. Couple hours work done in a few minutes.
bombolo 3y ago
> it is a really terrible writer Most people are terrible writers, and never read anything at all. So they have no way to judge if the wordy informationless crap gpt wrote is good or not. It's long, thus is good.
- carlmr 3y ago
  >It's long, thus is good. It's also somewhat structured, which puts it at above average for writing tasks.
Doxin 3y ago
I find copilot most useful when writing comments tbh. Whenever I'm stuck contemplating how to phrase the rest of a comment it comes with a suggestion. It might be a terrible suggestion but that's usually enough to get me unstuck. And every now and then it's exactly what you need and it saves a bunch of typing :P
animuchan 3y ago
I have similar experience. The differentiator is, I guess, expectations: I fully expect my own code to strike a certain balance of readability / optimization / correctness. In this context all but the most trivial code suggestions, sloppy as they usually come, instantly fall out of place. On the other hand, there is a group of people (e.g. those spamming GitHub pull requests...) who expect the output to be kind of visually similar to some code. When they realize that the result also sometimes works and gives sane-looking results, they are absolutely blown away.
nicbou 3y ago
It fills the role of an intern. It can do stuff but the quality is inconsistent and the work needs to be verified. Nonetheless it's better than Google at many things. I find it really good at giving paragraph-long introductions to things, instead of pages-long blogspam. It's also good at suggesting improvements when writing in a second language. But I don't see what those AI startups see when they want to replace all sorts of skilled workers with this bullshit generator.
SergeAx 3y ago
I am trying to use ChatGPT for writing code and that code is mediocre to bad. I really hope that in couple of years LLMs will advance enought to support/refactor what they are generating now.
zelphirkalt 3y ago
It is probably because you have standard / quality expectations and those are above the level of what GPT will give you. Keep your standards and quality expectations, I say.
jiggawatts 3y ago
I paid for it too, but I'm slowly figuring out its utility. The trick is to use it for what it's good at. For example, I had this vague, fuzzy memory of a talk I saw online about a decade ago. It was about a database modelling paradigm. All I remembered was that the country it was developed in was in northern Europe, the methodology was based on splitting out columns as individual objects, and that it had funny little icons such as diamonds on the edges connecting things together in a graph. Good luck finding that on Google. I had tried and failed. It was bugging me, because I needed something like it for my work, but for the life of me I couldn't dig up the reference. Chat GPT 4 found it just three prompts: https://www.anchormodeling.com/ https://www.anchormodeling.com/ That's insane.
futureshock 3y ago
I feel like this is the bias of a very smart, highly educated, highly capable person. If you are already an excellent writer and coder, can take vast fields of knowledge and blossoming ideas and spit them out onto paper in skilled form with minimal anxiety, GPT probably does very little for you. Maybe you could see if you were missing something very obvious and average. I read for probably 8-12 hours a day. I write every day. I think every day. I’ve been doing this for decades. GPT has been like a talking dog. It’s an insane achievement, but he’s not the worlds best conversationalist. But to be 17 again and struggling all night to write a three page paper filled with unoriginal ideas, it would have been a game changer back then.
- hsjqllzlfkf 3y ago
  May I ask - what job do you do that you read 8-12 hours a day?
  - dougmwne 3y ago
    I’m an enterprise architect. I’ve got a pretty wide set of projects I need to keep up to date on, a lot of industry developments and just following general tech trends. Lots of my role focuses on documenting our designs and projects.
- blagie 3y ago
  I would add: And doing the same thing over and over. I find GPT very useful in that I can quickly become mediocre in any domain. Need a legal filing? I can get a good-enough legal filing. Need some code in a framework I've never used? I can get it instantly. Need something written in the language of some esoteric field of science? I can get that instantly from a bullet point list. It doesn't do the hard parts well: What's being written, and how it's structured. That's where I fit back in.
- kenjackson 3y ago
  I think the thing to remember is that there are far more people like me than you. People who are poor writers or middling thinkers outside of specific domains. I agree that GPT isn’t much of a conversationalist. But it is an exceptionally good tool some of us. It has already proved invaluable for me in helping diagnose a medical issue with a family member — one that escaped several ICU doctors and nurses. And a couple of weeks ago helped me fix an issue with my car. A good analogy might be a calculator. There are some people who can do calculations in their heads near instantly. And some of these people could do arithmetic of numbers too large to be represented by calculators even. But for most of us, calculators were immensely useful.
keithnz 3y ago
they are tools, you have to learn how to use them. I use chatgpt everyday and am getting better and better at asking questions to get the answers I want. I used copilot when it was free, and it was best at boilerplate, and sometimes not bad at auto completing the next few lines, I didn't think it was worth paying for at the time. I use Codeium now, that works in a similar way, probably not quite as good as copilot in that it doesn't understand as many languages, but still does a reasonable job with boilerplate type code.
MagicMoonlight 3y ago
The point of it isn’t to write something like a book from scratch. Imagine if I told you “write a book about unicorn”, it would be trash. The genius of it is that you could give it the notes of a meeting and get it to rewrite them in a different form. Nothing else can handle language like that.
sschueller 3y ago
Totally agree, I also pay for api access and I am still trying to figure out how I can benefit from using it when it gives me blatantly incorrect commands or wrong answers. I can see it guide me in the right direction if I have a more generic question but even then, if I am not specific enough I fear I might go down a rabbit hole that ends no where. It may be better than google in finding something but when I manage to find something on google the information is usually correct and not totally wrong. Especially technical problem that someone wrote somewhere on a forum regarding a specific device. Finding the things I am looking for on google allows me to judge the correctness of the data. When I find answers in what is clearly blog spam I know that the info is probably wrong. Which ChatGPT I have no idea how it arrived at the conclusion it gives me. A good example is if I want to know the max current rating for a chip. If I search google and find the manufacturers datasheet, I know the information is correct. If however the same information is on stack overflow or some forum I may not take it as a fact but more as a guidance. ChatGPT would be 100 times more useful is it cited a source. Like "The max rating of chip ... is ... according to the manufactures datasheet dated ... which you can find here ...". I guess Bing and Google are trying to do something like that.
- gwd 3y ago
  > Totally agree, I also pay for api access and I am still trying to figure out how I can benefit from using it when it gives me blatantly incorrect commands or wrong answers. So here's one that saved me a bunch of time: https://gitlab.com/-/snippets/2535443 https://gitlab.com/-/snippets/2535443 Basically, I knew the package could do what I wanted; I knew it was almost certainly in GPT-4's data set; I could do it myself, but it would involve searching through all the documentation and figuring out the right bits. It just did it for me. Now there were a few minor bugs: It duplicated a buggy error message, and at some point later it called a "Subtree" method rather than a "Tree" method. But those were a lot easier to fix than writing the code from scratch. Once I had a list of 27 book names I wanted put into "canonical order" and in the form of a golang array. I could have done it myself but it would have been tedious; I just pasted the list into GPT-4, asked it what I wanted, and out popped the result. Here's another place it was helpful recently; I prompted: "We're hiring a new community manager for the $PROJECT. I'd like you to help me develop some interview questions. What information to you need about the role to help you develop more useful questions?" The questions it asked me about the role were really good; just answering the questions was quite a useful exercise, and I'm sure the resulting document will be a good intro to whomever we hire. I wouldn't say the resulting interview questions were brilliant, but they were solid, and I used a couple of them. There are times when "the form" is there for a reason; if you want something re-written in a specific form, GPT-4 can do a good job. I wrote an email recommending something to somebody's managers in a different company in a different country; then I pasted it into GPT-4 and asked if it had any suggestions. It did a fair amount of rewording, of which I decided to take about half. In this case, the "polite form" is there to avoid offense, and it's exactly what I wanted. I've also asked it to write some Tweets highlighting some specific aspects of an upcoming conference I'm planning. It did a good job coming up with the sort of punchy, tweet-length messages which seem to do really well. Connecting it to the context of the article: My day job is basically arguing with people on the internet. :-). I do read and write all day every day; but I don't write messages where diplomacy is critical, nor do I write tweets. Perhaps I could get better at those, but I don't think it's worth the effort. Am I the worse off for that? Probably not in the way the author thinks; I don't think being diplomatically polished would change my thinking that much -- much less being able to write punchy tweets. If I started relying on it for the core writing, however, I'd certainly be selling myself short.
micromacrofoot 3y ago
ask it to write like someone else (some established author you know) and it gets much better, you need to play with it a bit
dr_pardee 3y ago
You are likely not prompting correctly. https://www.deeplearning.ai/short-courses/chatgpt-prompt-engineering-for-developers/ https://www.deeplearning.ai/short-courses/chatgpt-prompt-eng... GPT-4 Passed the Bar Exam, so I think there's something to it https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4389233 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4389233
School-Cotton 3y ago
Just to make sure, are you selecting the GPT4 model at the top of the conversation in ChatGPT?
efields 3y ago
I needed to read this. Seeing posts about having gpt “write apps from scratch” vs me trying to get it to spit out what I want has been tedious. I’m assuming what’s not written about is the hours of just how tedious this process currently is, and how much “from scratch” is your own human brain doing the prompt edits bc you have a very good idea of what coding step comes next. It’s a lot of work, and not faster than me just concentrating hard and doing it myself.
vagab0nd 3y ago
No offense but you are almost certainly using it wrong. Consider the following: 1. Use it when you know how to do something but don't want to. 2. Use it when you don't know something, and don't know how to find the answer. 3. Never trust. Verify. Clearly, this means the examples that impress others will often not impress you.
- d23 3y ago
  It’s very difficult for me to wrap my head around people that aren’t blown away by both this and generative image AIs, and I tend to be in the skeptical side when it comes to fancy tech adoption.
  - jonahx 3y ago
    It is astonishing, even more so that some in this camp are smart people. I chalk it up to "hype aversion" -- an internal heuristic that anything with so much hype around it must be bullshit. For some, there might also be good old resistance to change, to the unknown consequences of all these developments.
    - laratied 3y ago
      [dead]
  - CamperBob2 3y ago
    The people who are least impressed by GPT seem to be the ones who've never tried to build anything similar, even a text adventure game. Industry outsiders also tend to take ML for granted, even when the 'experts' can see it's completely inappropriate. (My GF, a few years ago: "Why do we still need judges and juries? Can't computers judge cases and author legal opinions?" Me, at the time: "LOL, no, that would be ridiculous, and here's why..." Me, today: "Uh, about that argument we were having...")
  - rcme 3y ago
    I find generative AI very impressive, but impressive in the sense of "wow, it's impressive that an algorithm can do that." If I ignore the fact that something was algorithmically generated, then I'm not impressed by the generated output on its own.
conductr 3y ago
It makes me feel like my grandmother who can’t use search. She can use the browser fine she just lacks any Google Fu and uses the most ridiculous search phrases.
ericmcer 3y ago
Try coding in a language/style you are not familiar with. It is not revolutionary for areas you specialize in, but if I try using it for something like setting up a web scraper with python (something I understand conceptually but have never done) it feels pretty amazing. My one concern is that if I use it for JS I can immediately spot issues with it, but I am too ignorant of python to spot subtle problems.
- d23 3y ago
  It’s definitely awesome for that, but for me, on side projects even in languages I’m familiar with, getting ChatGPT to write the boring annoying stuff, or things that would require a bit of referencing (like a library I’ve not completely memorized), I’ve still found it to be pretty useful.
natch 3y ago
I don’t know how you’re using it but it seems to step up its game if you challenge it more. You can also ask it to refrain from dumbing things down. Copilot wasn’t useful for me because I want a back and forth conversation to straighten out concepts. With Copilot you don’t get that and it just sprays out little messes of code. I use GPT4 like a rubber duck in the sense of “rubber duck debugging” and it can be surprisingly supple and creative in its solutions… many bad ones, but again, push back on those and you get to good ones.
mbrochh 3y ago
No idea what you are doing there, I've been using ChatGPT Pro and Copilot since four weeks and it literally blows my mind every single day. Give it a Django model and ask it to create a data visualisation. It'l perform complex ORM queries with filters, annotations and aggregations, then load it into a pandas dataframe, then create more columns in the dataframe, does more aggregations, then spits out matplotlib code to draw the chart. The time typing the prompt is time I would have needed even in the pre-GPT-era, since I would have to think about what I want to achieve anyways, now I'm typing it into ChatGPT, which makes the thinking process more structured as well! I have built more tools in the last three weeks (for work and personal) than I have in the last three years combined. All thanks to OpenAI's brain constantly pushing me into the right direction.
maytc 3y ago
Is it terrible at the content or the delivery of the content? I find it to be bad at the prior but rather useful for the latter. If you asked chatgpt to generate a document, its generates one that reads well but has terrible content. Eg. Super broad or just plain contradictory. But given some not so well written piece of writing, it can clean that up fairly well.
aprilnya 3y ago
half the time i tell copilot to do something the suggestion is just “return true” and there are no other suggestions
PaulHoule 3y ago
All the time I see blog posts by people who give ChatGPT way too much credit for partially correct answers than it deserves.
Capricorn2481 3y ago
I think ChatGPT is impressive, but it's revealed just how insular and vapid the thoughts of some HNers are. Hearing people's "endgame" for ChatGPT has been very depressing
asdasd213123 3y ago
You got filtered btw
dilippkumar 3y ago
I wonder if it’s time to start seriously looking at why some people have completely embraced copilot/chat-gpt and others can not get it to work at all. I’d love to see which of the following might have any correlations with reported usefulness of chatgpt/copilot: 1. How articulate a person is. 2. How much of an expert they are in what they are trying to do with the tool. 3. How successful they are at giving instructions to a human to perform the same task. 4. How much experience they have managing/coaching junior devs/interne/newbies. 5. How much experience someone has at decomposing a problem into smaller parts and identifying the simple parts and complicated parts There are huge consequences to either answer to the question: “Is using an AI tool effectively a coachable skill?”. I’m sure someone has already looked into this or is looking into this - if it turns out to be a coachable skill, and we can identify what those are, there’s a lot of money to be made in bringing a coursework to market.
noisycarlos 3y ago
Same here! I can feel like the Emperor's clothes sometimes.
Aeolun 3y ago
Well, you made only one mistake in these 4 sentences you wrote, which is like a 400% improvement over the average messages I see outside HN, and an infinite improvement on some other people.

by rcme 3y ago

imtringued 3y ago

Everyone knows that using ChatGPT is about signalling your non Luddite status so that the police won't go after you.

dbtc 3y ago
I noticed that you haven't been talking to your AI very much lately, what's wrong?
- marginalia_nu 3y ago
  Please drink a verification can of AI marketing kool aid!

meerita 3y ago

Maybe you're not using the right prompts. But yes, ChatGPT needs more creativity input.

weinzierl 3y ago

There is also a huge difference between code generation (be it Copilot or ChatGPT). With Code you still have the compiler that rejects the utter nonsense and then you hopefully have tests to check if what was generated makes sense. With prose it's all your manual work again to check. If your text is mundane it might be trivial but for anything a pg would write I guess an AI is pretty useless in its current state.

neodypsis 3y ago
I wonder if someone has tried to chain ChatGPT's output with static analyzers/compilers/linters in a loop to improve the final response.
- blatant303 3y ago
  https://github.com/newhouseb/clownfish https://github.com/newhouseb/clownfish Structural Alignment: Modifying Transformers (like GPT) to Follow a JSON Schema

lordnacho 3y ago

I'm surprised about copilot. I find every day it saves me time by "nexting" whatever I'm doing. Whenever it's right about what I wanted to write, I save a bit of time. My go-to example is if conditions. When I start with the first case, it almost always figures out the rest of the cases correctly.

arvinsim 3y ago
I thought Github Copilot was useless until I found out that I have to wait for a few seconds for Copilot to display the autosuggested code.
javchz 3y ago
I think copilot it's amazing saving time in boilerplate code. But I can understand why some workflows may be a liability rather than help. Similar to chatgpt sometimes allusinate code with reserved words that don't exist in the language (but exists in others), or uses old code from a deprecated api / version of your framework. In other words, somtimes the cleaning process can take as much time, as just writing the code yourself.

hammyhavoc 3y ago

How impressed or unimpressed you are by it is a direct reflection of yourself and what you are trying to achieve with it. I find zero value in it. Others are very impressed by it, but I'm not very impressed by what they hold up as an example of its output. Sometimes I can only laugh when it's horribly wrong, but they herald it as amazing. The whole thing reminds me of the blockchain hype train. Still using and loving databases here for the foreseeable—still writing things by hand for the foreseeable, and loving every moment.

jiggawatts 3y ago
You're not impressed!? Here on HN, a few days ago, there was a post about Microsoft publishing a GitHub repo that contained a "table recognizer AI". Basically, you feed it PDFs that contain horrible scanned images of finance records, and it spits out Excel spreadsheets. For some reason, Microsoft had just "thrown this over the fence" and released it to the public for free. This, despite man-years of effort developing the thing. It was working, and everything. I made a comment wondering if Chat GPT 4 with the vision extension could solve the same problem. One of the devs that had worked on the aforementioned AI (for years!) mentioned that yes, yes it can. Game over. Those years of effort had just been replaced with a one-sentence English-language prompt that starts with "Please output a table from..." If this doesn't blow your mind, then... I don't know how to help you understand just how much has changed, virtually overnight.
- hammyhavoc 3y ago
  I mean, sure, I guess that's neat, but Kofax OmniPage Ultimate has been a thing for years at this point and does the whole OCR from a crappy scan thing, including reformatting it into whatever format you want. Can even take data from forms and digitize it into editable text or stick it into a database. Goes back to the '80s. Sure, different methodology, but it's not something we haven't already been doing for yonks. Users have even been able to train OmniPage Ultimate to recognize someone's bizarre and unique handwriting styles in seconds. OmniPage Ultimate is an entire workflow that's turn-key and ready-to-go.
  - 3y ago

vjk800 3y ago

I suppose you have to play with it enough to get a feeling of what kind of stuff you can trust it with. For most tasks it's pretty bad, and this is why, if you just try it for something you just thought about without any kind of filter or feedback, you're going to be disappointed with the results. I agree that there is currently way too much hype. I mean GPT4 is certainly cool and probably going to be useful in some limited use cases, but it's not the new electricity and it's not going to take over the world.

barrkel 3y ago

The tools are at their most useful when they're replacing Stack Overflow searches for common tasks. An example I had last night: how do you split a pdf into separate pages on the command line, and how do you put them back together again (I replaced one of the pages with a scanned signed version). This is a basic task, a beginner's task in the world of pdf munging, but I didn't know the answer off the top of my head. ChatGPT gave me command lines straight away. That's what they're good for right now. Beginner / common tasks in areas you're not familiar with. And a further implication, they're great for one man bands, generalist roles where simple things need to be done in lots of different areas. For specialist deep roles where you have all the tacit knowledge in mind already, not nearly as useful.

revelio 3y ago
By the way, on macos the preview tool can do this. You can drag and drop pages between documents using the sidebar and reorder them the same way.
- marvin 3y ago
  That's interesting and relevant to know if you'd be doing this a few times. And it doesn't invalidate your point at all, but the time savings of having a machine tell you really fast how to do something simple is valuable. I recently had to do a 'git pull' on 120 repositories, and of course we already had an in-house tool to do that. Of course I could google a script to do it. But it saved me 5 minutes to just have ChatGPT write it for me, and I saved the overhead of a more thorough context-switch. Even the guy sitting next to me. "Hey, check this out, ChatGPT wrote this script for me and saved me 5 minutes". "Oh, but we already have a tool for that". "Yeah, I figured. Where is it?" "Ummmmm, I don't know, let me ask someone. Frank maybe?" And 5 minutes spent chasing Frank.
  - zelphirkalt 3y ago
    Hopefully that ad-hoc second script never makes it into persistent storage or a repo, otherwise you now have 2 of them and Frank will think his part to himself about others duplicating things.

deleted 3y ago

[deleted]

RugnirViking 3y ago

I find copilot and chatgpt constantly useful as just a simple way of automating what would otherwise be tedious work. I think the most important thought in using them is that as you use them more you get a sense if what they are going to be able to do and what they aren't, which lets you know when you can use them (and therefore use them with more confidence and more often) Especially in refactoring or working with poor codebases. I do notice with copilot specifically some people seem to get wildly different response times with it, for me it gives suggestions almost instantly so as im typing ive got an ongoing suggestion I can accept at any time as it gets closer to what I intend or ignore if not. A common really useful thing is when I have to set up a series of something (recently in a power plant code I set up parameters for a certain kind of battery) and then I just start writing for each other kind of battery it suggests the whole boilerplate for every case, I can go back and fill in the details and exceptions. As for chatgpt its more useful at interpreting existing code. I often give it a giant function and say "I want you to split off this part and put it into its own thread, get the output with a queue" or something like that. Again, output can often need a little bit of massaging, but I find it reduces cognitive load involved in doing such a task. The trick is to be naturally skeptical of the output but appreciate how much of your life is spent writing nonsense. I've also found that learning to put what I want exactly into words can be a particular skill that has helped my communication. Making sure you know the name and function of every tool you use. I think if I had said that to myself before I started using these things id have been incredulous, "of course I know the name of things" but no, its a different thing to be talking about them often, especially when you have an idiot savant that takes everything you say too literally. You have to learn to be consise and precise

BiteCode_dev 3y ago
That's the gist of it. People want to use LLM hammers like screw drivers and are disapointed. Also, since the quality of the context matters a lot for completion, copilot will be way more useful on clean code with beautiful variable names and nice structure flows than on code-golf snippets. Same got chatgpt: write good prose, and it will help you better.
- laserlight 3y ago

renewiltord 3y ago

I use Copilot with nvim and C-x C-e at my terminal. I can use it to build out pipelines in the shell so easily it's crazy. But in the end, it's just a tool. If it isn't working for you, it isn't working for you. For me, I prefer thinking in what structure to me as bigger building blocks. So I prefer vim bindings because I used up teenage time learning it. And now I don't think in terms of characters but in terms of text-blocks. I prefer writing Java in IntelliJ and Rust in Clion because I don't think in char terms since I can use the built-in refactor tools. E.g. I write functions inline and then extract them rather than writing them as functions and calling. Likewise when I'm at the shell I think of pipelines as less than pieces. Being an experienced shell user I can usually write a parallel pipeline with conditionals and loops correct first time verbally with no computer usually. But my copilot flow is improved even better. I talk to the computer in components: # get all private ip addresses in Alibaba region cn-zhangjiakou ips=...prompted text... # ssh to each and check last long line ...Prompted text... It's faster than me and this is great for r-i-search with fzf. Overall, quite happy. But just a tool.

TaylorAlexander 3y ago

For writing I find it makes an excellent rubber duck. I absolutely do not expect it to write long form prose for me, but I can explain what I am trying to say and talk to it about how I might present that information.

sberens 3y ago

5 minutes ago I copy pasted a 200 line react component and asked ChatGPT to explain why I was getting a certain bug. Although it didn't quite get it right, it allowed me to quickly identify the cause. It only needs to save 10 minutes of my time a month for it to be worth the $20 - I would estimate it saves hours.

dbtc 3y ago

I think that you are at least using the same one that I am. "Hype" can also mean marketing.

hoseja 3y ago

It seems really great at vacuous american essaywriting.

emmelaich 3y ago

What effort have you put into learning useful prompts? I only use the free 3.5+ ChatGPT and it's written some nice mapping code and some fun poems for me. Poem subjects ranged from rheology to texas hold'em.

shapefrog 3y ago

My experience is very similar. I am neither a great writer, not a great coder, not a grammar nazi - but still have to make major corrections to the output of such tools. I came to the sad conclusion that perhaps what I write and what I do is not as repetitive or as generic as others. Over the years when I have had to do more repetition, writing once well and ctrl-c / ctrl-v.

redox99 3y ago

Tell it the columns of a certain table. Ask it a GraphQL schema for a CRUD for this table. Ask it for a resolver using X libraries. Ask it for unit tests using Y libraries. Couple hours work done in a few minutes.

bombolo 3y ago

> it is a really terrible writer Most people are terrible writers, and never read anything at all. So they have no way to judge if the wordy informationless crap gpt wrote is good or not. It's long, thus is good.

carlmr 3y ago
>It's long, thus is good. It's also somewhat structured, which puts it at above average for writing tasks.

Doxin 3y ago

I find copilot most useful when writing comments tbh. Whenever I'm stuck contemplating how to phrase the rest of a comment it comes with a suggestion. It might be a terrible suggestion but that's usually enough to get me unstuck. And every now and then it's exactly what you need and it saves a bunch of typing :P

animuchan 3y ago

I have similar experience. The differentiator is, I guess, expectations: I fully expect my own code to strike a certain balance of readability / optimization / correctness. In this context all but the most trivial code suggestions, sloppy as they usually come, instantly fall out of place. On the other hand, there is a group of people (e.g. those spamming GitHub pull requests...) who expect the output to be kind of visually similar to some code. When they realize that the result also sometimes works and gives sane-looking results, they are absolutely blown away.

nicbou 3y ago

It fills the role of an intern. It can do stuff but the quality is inconsistent and the work needs to be verified. Nonetheless it's better than Google at many things. I find it really good at giving paragraph-long introductions to things, instead of pages-long blogspam. It's also good at suggesting improvements when writing in a second language. But I don't see what those AI startups see when they want to replace all sorts of skilled workers with this bullshit generator.

SergeAx 3y ago

I am trying to use ChatGPT for writing code and that code is mediocre to bad. I really hope that in couple of years LLMs will advance enought to support/refactor what they are generating now.

zelphirkalt 3y ago

It is probably because you have standard / quality expectations and those are above the level of what GPT will give you. Keep your standards and quality expectations, I say.

jiggawatts 3y ago

I paid for it too, but I'm slowly figuring out its utility. The trick is to use it for what it's good at. For example, I had this vague, fuzzy memory of a talk I saw online about a decade ago. It was about a database modelling paradigm. All I remembered was that the country it was developed in was in northern Europe, the methodology was based on splitting out columns as individual objects, and that it had funny little icons such as diamonds on the edges connecting things together in a graph. Good luck finding that on Google. I had tried and failed. It was bugging me, because I needed something like it for my work, but for the life of me I couldn't dig up the reference. Chat GPT 4 found it just three prompts: https://www.anchormodeling.com/ https://www.anchormodeling.com/ That's insane.

futureshock 3y ago

I feel like this is the bias of a very smart, highly educated, highly capable person. If you are already an excellent writer and coder, can take vast fields of knowledge and blossoming ideas and spit them out onto paper in skilled form with minimal anxiety, GPT probably does very little for you. Maybe you could see if you were missing something very obvious and average. I read for probably 8-12 hours a day. I write every day. I think every day. I’ve been doing this for decades. GPT has been like a talking dog. It’s an insane achievement, but he’s not the worlds best conversationalist. But to be 17 again and struggling all night to write a three page paper filled with unoriginal ideas, it would have been a game changer back then.

hsjqllzlfkf 3y ago
May I ask - what job do you do that you read 8-12 hours a day?
- dougmwne 3y ago
  I’m an enterprise architect. I’ve got a pretty wide set of projects I need to keep up to date on, a lot of industry developments and just following general tech trends. Lots of my role focuses on documenting our designs and projects.
blagie 3y ago
I would add: And doing the same thing over and over. I find GPT very useful in that I can quickly become mediocre in any domain. Need a legal filing? I can get a good-enough legal filing. Need some code in a framework I've never used? I can get it instantly. Need something written in the language of some esoteric field of science? I can get that instantly from a bullet point list. It doesn't do the hard parts well: What's being written, and how it's structured. That's where I fit back in.
kenjackson 3y ago
I think the thing to remember is that there are far more people like me than you. People who are poor writers or middling thinkers outside of specific domains. I agree that GPT isn’t much of a conversationalist. But it is an exceptionally good tool some of us. It has already proved invaluable for me in helping diagnose a medical issue with a family member — one that escaped several ICU doctors and nurses. And a couple of weeks ago helped me fix an issue with my car. A good analogy might be a calculator. There are some people who can do calculations in their heads near instantly. And some of these people could do arithmetic of numbers too large to be represented by calculators even. But for most of us, calculators were immensely useful.

keithnz 3y ago

they are tools, you have to learn how to use them. I use chatgpt everyday and am getting better and better at asking questions to get the answers I want. I used copilot when it was free, and it was best at boilerplate, and sometimes not bad at auto completing the next few lines, I didn't think it was worth paying for at the time. I use Codeium now, that works in a similar way, probably not quite as good as copilot in that it doesn't understand as many languages, but still does a reasonable job with boilerplate type code.

MagicMoonlight 3y ago

The point of it isn’t to write something like a book from scratch. Imagine if I told you “write a book about unicorn”, it would be trash. The genius of it is that you could give it the notes of a meeting and get it to rewrite them in a different form. Nothing else can handle language like that.

sschueller 3y ago

Totally agree, I also pay for api access and I am still trying to figure out how I can benefit from using it when it gives me blatantly incorrect commands or wrong answers. I can see it guide me in the right direction if I have a more generic question but even then, if I am not specific enough I fear I might go down a rabbit hole that ends no where. It may be better than google in finding something but when I manage to find something on google the information is usually correct and not totally wrong. Especially technical problem that someone wrote somewhere on a forum regarding a specific device. Finding the things I am looking for on google allows me to judge the correctness of the data. When I find answers in what is clearly blog spam I know that the info is probably wrong. Which ChatGPT I have no idea how it arrived at the conclusion it gives me. A good example is if I want to know the max current rating for a chip. If I search google and find the manufacturers datasheet, I know the information is correct. If however the same information is on stack overflow or some forum I may not take it as a fact but more as a guidance. ChatGPT would be 100 times more useful is it cited a source. Like "The max rating of chip ... is ... according to the manufactures datasheet dated ... which you can find here ...". I guess Bing and Google are trying to do something like that.

gwd 3y ago
> Totally agree, I also pay for api access and I am still trying to figure out how I can benefit from using it when it gives me blatantly incorrect commands or wrong answers. So here's one that saved me a bunch of time: https://gitlab.com/-/snippets/2535443 https://gitlab.com/-/snippets/2535443 Basically, I knew the package could do what I wanted; I knew it was almost certainly in GPT-4's data set; I could do it myself, but it would involve searching through all the documentation and figuring out the right bits. It just did it for me. Now there were a few minor bugs: It duplicated a buggy error message, and at some point later it called a "Subtree" method rather than a "Tree" method. But those were a lot easier to fix than writing the code from scratch. Once I had a list of 27 book names I wanted put into "canonical order" and in the form of a golang array. I could have done it myself but it would have been tedious; I just pasted the list into GPT-4, asked it what I wanted, and out popped the result. Here's another place it was helpful recently; I prompted: "We're hiring a new community manager for the $PROJECT. I'd like you to help me develop some interview questions. What information to you need about the role to help you develop more useful questions?" The questions it asked me about the role were really good; just answering the questions was quite a useful exercise, and I'm sure the resulting document will be a good intro to whomever we hire. I wouldn't say the resulting interview questions were brilliant, but they were solid, and I used a couple of them. There are times when "the form" is there for a reason; if you want something re-written in a specific form, GPT-4 can do a good job. I wrote an email recommending something to somebody's managers in a different company in a different country; then I pasted it into GPT-4 and asked if it had any suggestions. It did a fair amount of rewording, of which I decided to take about half. In this case, the "polite form" is there to avoid offense, and it's exactly what I wanted. I've also asked it to write some Tweets highlighting some specific aspects of an upcoming conference I'm planning. It did a good job coming up with the sort of punchy, tweet-length messages which seem to do really well. Connecting it to the context of the article: My day job is basically arguing with people on the internet. :-). I do read and write all day every day; but I don't write messages where diplomacy is critical, nor do I write tweets. Perhaps I could get better at those, but I don't think it's worth the effort. Am I the worse off for that? Probably not in the way the author thinks; I don't think being diplomatically polished would change my thinking that much -- much less being able to write punchy tweets. If I started relying on it for the core writing, however, I'd certainly be selling myself short.

micromacrofoot 3y ago

ask it to write like someone else (some established author you know) and it gets much better, you need to play with it a bit

dr_pardee 3y ago

You are likely not prompting correctly. https://www.deeplearning.ai/short-courses/chatgpt-prompt-engineering-for-developers/ https://www.deeplearning.ai/short-courses/chatgpt-prompt-eng... GPT-4 Passed the Bar Exam, so I think there's something to it https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4389233 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4389233

School-Cotton 3y ago

Just to make sure, are you selecting the GPT4 model at the top of the conversation in ChatGPT?

efields 3y ago

I needed to read this. Seeing posts about having gpt “write apps from scratch” vs me trying to get it to spit out what I want has been tedious. I’m assuming what’s not written about is the hours of just how tedious this process currently is, and how much “from scratch” is your own human brain doing the prompt edits bc you have a very good idea of what coding step comes next. It’s a lot of work, and not faster than me just concentrating hard and doing it myself.

vagab0nd 3y ago

No offense but you are almost certainly using it wrong. Consider the following: 1. Use it when you know how to do something but don't want to. 2. Use it when you don't know something, and don't know how to find the answer. 3. Never trust. Verify. Clearly, this means the examples that impress others will often not impress you.

d23 3y ago
It’s very difficult for me to wrap my head around people that aren’t blown away by both this and generative image AIs, and I tend to be in the skeptical side when it comes to fancy tech adoption.
- jonahx 3y ago
  It is astonishing, even more so that some in this camp are smart people. I chalk it up to "hype aversion" -- an internal heuristic that anything with so much hype around it must be bullshit. For some, there might also be good old resistance to change, to the unknown consequences of all these developments.
  - laratied 3y ago
    [dead]
- CamperBob2 3y ago
  The people who are least impressed by GPT seem to be the ones who've never tried to build anything similar, even a text adventure game. Industry outsiders also tend to take ML for granted, even when the 'experts' can see it's completely inappropriate. (My GF, a few years ago: "Why do we still need judges and juries? Can't computers judge cases and author legal opinions?" Me, at the time: "LOL, no, that would be ridiculous, and here's why..." Me, today: "Uh, about that argument we were having...")
- rcme 3y ago
  I find generative AI very impressive, but impressive in the sense of "wow, it's impressive that an algorithm can do that." If I ignore the fact that something was algorithmically generated, then I'm not impressed by the generated output on its own.

conductr 3y ago

It makes me feel like my grandmother who can’t use search. She can use the browser fine she just lacks any Google Fu and uses the most ridiculous search phrases.

ericmcer 3y ago

Try coding in a language/style you are not familiar with. It is not revolutionary for areas you specialize in, but if I try using it for something like setting up a web scraper with python (something I understand conceptually but have never done) it feels pretty amazing. My one concern is that if I use it for JS I can immediately spot issues with it, but I am too ignorant of python to spot subtle problems.

d23 3y ago
It’s definitely awesome for that, but for me, on side projects even in languages I’m familiar with, getting ChatGPT to write the boring annoying stuff, or things that would require a bit of referencing (like a library I’ve not completely memorized), I’ve still found it to be pretty useful.

natch 3y ago

I don’t know how you’re using it but it seems to step up its game if you challenge it more. You can also ask it to refrain from dumbing things down. Copilot wasn’t useful for me because I want a back and forth conversation to straighten out concepts. With Copilot you don’t get that and it just sprays out little messes of code. I use GPT4 like a rubber duck in the sense of “rubber duck debugging” and it can be surprisingly supple and creative in its solutions… many bad ones, but again, push back on those and you get to good ones.

mbrochh 3y ago

No idea what you are doing there, I've been using ChatGPT Pro and Copilot since four weeks and it literally blows my mind every single day. Give it a Django model and ask it to create a data visualisation. It'l perform complex ORM queries with filters, annotations and aggregations, then load it into a pandas dataframe, then create more columns in the dataframe, does more aggregations, then spits out matplotlib code to draw the chart. The time typing the prompt is time I would have needed even in the pre-GPT-era, since I would have to think about what I want to achieve anyways, now I'm typing it into ChatGPT, which makes the thinking process more structured as well! I have built more tools in the last three weeks (for work and personal) than I have in the last three years combined. All thanks to OpenAI's brain constantly pushing me into the right direction.

maytc 3y ago

Is it terrible at the content or the delivery of the content? I find it to be bad at the prior but rather useful for the latter. If you asked chatgpt to generate a document, its generates one that reads well but has terrible content. Eg. Super broad or just plain contradictory. But given some not so well written piece of writing, it can clean that up fairly well.

aprilnya 3y ago

half the time i tell copilot to do something the suggestion is just “return true” and there are no other suggestions

PaulHoule 3y ago

All the time I see blog posts by people who give ChatGPT way too much credit for partially correct answers than it deserves.

Capricorn2481 3y ago

I think ChatGPT is impressive, but it's revealed just how insular and vapid the thoughts of some HNers are. Hearing people's "endgame" for ChatGPT has been very depressing

asdasd213123 3y ago

You got filtered btw

dilippkumar 3y ago

I wonder if it’s time to start seriously looking at why some people have completely embraced copilot/chat-gpt and others can not get it to work at all. I’d love to see which of the following might have any correlations with reported usefulness of chatgpt/copilot: 1. How articulate a person is. 2. How much of an expert they are in what they are trying to do with the tool. 3. How successful they are at giving instructions to a human to perform the same task. 4. How much experience they have managing/coaching junior devs/interne/newbies. 5. How much experience someone has at decomposing a problem into smaller parts and identifying the simple parts and complicated parts There are huge consequences to either answer to the question: “Is using an AI tool effectively a coachable skill?”. I’m sure someone has already looked into this or is looking into this - if it turns out to be a coachable skill, and we can identify what those are, there’s a lot of money to be made in bringing a coursework to market.

noisycarlos 3y ago

Same here! I can feel like the Emperor's clothes sometimes.

Aeolun 3y ago

Well, you made only one mistake in these 4 sentences you wrote, which is like a 400% improvement over the average messages I see outside HN, and an infinite improvement on some other people.