10 ms·
Learnings from employing ChatGPT as a ML Engineer for a day
- stuckinhell 4y agoWhat a frightening conclusion. >Can ChatGPT be used to improve an AI system? Yes. >Would we hire it as our next standalone ML engineer? No. > Let’s wait until GPT4. The arms race for AI that can erase the human from cognitive work is really here.
- reidjs 4y agoWho wants to do cognitive work anyways? I’d rather get paid to dig ditches, at least then I get a free workout in.
- Shish2k 4y agoA large part of ditch digging is cognitive work (a human operating heavy machinery), so I’m not sure that backup career is going to last very long either
- codetrotter 4y agoI think they meant with a shovel, not an excavator
- sebzim4500 4y agoHow many people are being hired to dig a ditch with a shovel?
- tmtvl 4y agoI think that's the point, there are tasks that some would like to do for a decent wage but that aren't available as such. ...well, maybe if someone live streamed the ditch digging it could earn a few dollars.
- awesomeMilou 4y agoWith quality conclusions like these, the author might singlehandedly silence all the disbelievers who claim ChatGPT is not capable of human like intelligence.
- stevenhuang 4y agoEspecially when they use the stochastic parrot argument to further their point (or say something about how LLMs are mere next word predictors as if that ends all discussion on the matter). I've always found arguments of these sort to be weak sauce--our brains are stochastic parrots. Emergent behaviour + the bitter lesson (scaling laws) hits hard.
- hnfong 4y agoThe stochastic parrot is literally what would pass a Turing Test. Think about it. If the parrot imitates humans well enough it passes. And while there may be objections to defining intelligence with respect to the Turing Test, I haven't heard of any definitions that are obviously and significantly better. Our brains are stochastic parrots at least when it comes to communicating with text. People may digress into things like physical abilities, emotions, feelings, subjective experiences etc., but if we're talking about text, I'm not aware of a higher bar. In fact we probably don't know how to tell if an entity is "more intelligent" than the most intelligent humans because we won't be able to assess their intelligence.
- pfisherman 4y agoThat is not going to happen. Calculators allowed us to outsource arithmetic to machines. They did not eliminate the need for mathematicians. Spreadsheets revolutionized bookkeeping. They did not eliminate the need for accountants. Robots automated much of the manual labor in manufacturing. They did not eliminate the need for process engineers, etc. Same story for autocad, protools, avid, etc.
- redox99 4y agoAGIs should basically by definition be able to replace humans[1], so yes, when we get good enough AGIs that will happen. GPT4 won't be an AGI of course, but it still could have a massive impact on the job market. A single engineer with GPT4 could potentially do the work of 10 engineers right now. Edit: To be clear, I used GPT4 as a placeholder for "some near future LLM". I'm not making any bets on GPT4 in particular. [1] See "The employment test" https://en.wikipedia.org/wiki/Artificial_general_intelligence#Tests_for_confirming_human-level_AGI https://en.wikipedia.org/wiki/Artificial_general_intelligenc...
- ASalazarMX 4y ago> A single engineer with GPT4 could potentially do the work of 10 engineers right now And the potential mistakes of several engineers too, so you would need more than one engineer to double-check the output. Remember that, even if they are truly amazing, LLMs are basically incredibly powerful predictive keyboards.
- jackblemming 4y agoIf we're going to pull random numbers from where the sun doesn't shine, why don't we say the work of a hundred engineers? How about a billion? I've actually used ChatGPT and copilot and I can tell you it's not even 10% more productive, so I'm curious how you ended up with your estimate.
- travisjungroth 4y agoThey said “could potentially”. They’re just making the argument (true IMO) that technology doesn’t have to replace a class of employee to have a huge effect. We still have farmers, but 1% of the population instead of >50%. Or, it could be like how assemblers didn’t get rid of software engineers but helped fuel the fire. Who knows.
- xt00 4y agoEvery war has collaborators on the enemy side.. so this will be the same.. tons of people will happily load the gun for the AI overlords if it came to that — especially if they make a buck along the way..
- ldehaan 4y agoI use it every day at the f500 I work at. I have gotten my team to adopt it, and it has increased productivity from those who were already productive. The people who weren't already good can't figure out how to use it properly. I also see a bunch of people online, most?, using it incorrectly, writing prompts that would give bad info. Maybe they're doing it on purpose? To those who have everything, more will be given.
- tomlue 4y agoyeah +1 on this, seeing a lot of weird posts where people claim chatgpt screwed something up, and the prompt is all wrong or it is just a special case that will be solved eventually. The citation thing is a good example of this, just because chatgpt makes up citations today doesn't mean that it will always make up citations, accurate citations is just the wrong use case today. Already there are extensions that make accurate citations work. Still, people point at inaccurate citations as some kind of proof that chatgpt won't work. Probably people want it to fail or don't want to believe how transformative it is.
- barking_biscuit 4y ago>Probably people want it to fail or don't want to believe how transformative it is. Ego defense is a very real, and very powerful driving force in human behavior and it is coming out in full force.
- senthil_rajasek 4y agoWhat type of tasks do you get help from ChatGPT? Curious to know some solid use cases of productivity improvements in real world?
- matwood 4y agoI've used it pretty successfully to grammar check emails. I write a rough version and ask chatGPT to check it. It usually comes back with a slightly tweaked version that I use parts from. Also, naming things.
- 4y ago
- visarga 4y agoWe need models that can read long texts like a bunch of scientific papers, a book, or the whole git repo of an app. And we need models that can do iterative changes - diff models, they can be trained of course on git commits.
- kgarten 4y agothe models we have today will never be able to read .. they are just able to produce something that we cannot distinguish from human output. we should be more careful on how we use them. in my opinion, ChatGPT and similar will be horrible for search on the web, as we are flooded with text that looks like a human wrote it but it does not add any knowledge or new insight.
- hybridity 4y agoWhen I give ChatGPT a piece of literature or code that it has never seen before, it is able to analyze it and respond back at least semi-intelligibly. This would make me conclude that the models we have today can in fact read, and typically rather well.
- humanistbot 4y agoThat is because it has seen literature or code that is syntactically similar enough to what you give it, so that it can autocomplete its way to a semi-intelligent response.
- babyshake 4y agoThis line of thought always brings to mind the "mission accomplished" XKCD: https://xkcd.com/810/ https://xkcd.com/810/
- gitgud 4y ago> the models we have today will never be able to read .. Never say never
- 4y ago
- rustastra 4y agoCurious to learn more about prompt engineering takeaways here. Was feeding more context (or chapters of textbooks, bits of papers, documentation) helpful? It does seem like layering information and being very precise helps a lot. Eerily like with interns
- aabajian 4y agoI'm a physician with a software background, another physician at my work used ChatGPT to write a compelling appeal for a patient when insurance denied coverage. I've started to use it to automatically dictate procedural reports. I provide an example report and tell it to modify the report as needed (e.g. "right iliac bone biopsy report with 11G needle"). Beats tabbing (or F2ing) through autotext fields. These examples make me think, "ChatGPT for form completion" is the killer app as it applies to law, real estate, medicine, etc.
- VectorLock 4y agoHow were you getting the report from ChatGPT into the form? Sounds like you'd have to tab/F2 anyways to input what ChatGPT says.
- CabSauce 4y agoYou're sending patient information to a third party without any contractual agreement?
- aabajian 4y agoNo patient info is sent. In my colleague's example, you can try out a similar query, "Write an appeal letter to a medical insurance company for a patient who needs a biopsy for a bone lesion given prior unclear diagnosis."
- cardosof 4y agoProbably patients consent to using software to process their data and chatgpt is considered just another software in the stack. Edit: parent poster said no info is sent.
- bfeynman 4y agothat's not how it works, PHI and HIPAA usually supersede a lot of these. That's why you hospitals don't just have everyone opt-out data protections if they want services.
- 4y ago
- matmatmatmat 4y agoQuestion on the ML side of this post: How are these "parameterizations" used? Is this really just feature engineering with a new name? Are they including this information when training the model? In the article, they mention using the new labels to build a "more balanced" dataset -- is this a realistic possibility in practice when most teams still have a dearth of data?
- elandau25 4y agoHello! I wrote the article so happy to answer this. It is partially feature engineering but partially not. It’s essentially using feature engineering to curate/correct a dataset, but a neural network as the actual end model without explicit input of these features(we call them quality metrics). I abbreviated a good amount of the process in the article so that it wouldn’t run forever, but essentially we allowed ChatGPT to select and write its own features and then used the strategies it came up with to apply these features to improve the dataset. In terms of if it’s realistic in practice, the answer is yes. Some teams have a dearth of data, but many AI companies we work with have more data than they can use, and it’s more a question of how to sample, curate, and correct the data and labels they have to improve their models rather than collect new data. Great questions!
- VincentEvans 4y agoWhats the deal lately with turning verbs into nouns? Learnings, asks - come to mind, but I periodically hear others. It’s not drippy anymore to simply say lessons and questions, bros don’t find them lit?
- pprotas 4y agoI don't think this is a new concept, Wikitionary seems to confirm this [0]. Btw we don't say "drip" and "lit" anymore, get with the times!!!! --- [0] https://en.wiktionary.org/wiki/learnings https://en.wiktionary.org/wiki/learnings
- WithinReason 4y agoAccording to Wiktionary we do: https://en.wiktionary.org/wiki/drippy https://en.wiktionary.org/wiki/drippy
- jcims 4y agoIt's too bad they didn't get a chance to try it with Bing's chatbot. Based on what I've seen, the ability for it to reach into an index (or to the internet) to grab specific information creates a qualitative change in its ability.