7 ms·
That's wrong. It's poetic, but incorrect. These tools are about as good as a non specialist with time to undertake some bullshitting.
by jvanderbot 4y ago
That's wrong. It's poetic, but incorrect. These tools are about as good as a non specialist with time to undertake some bullshitting.
- paulgb 4y ago> These tools are about as good as a non specialist with time to undertake some bullshitting. You’re not wrong, but AI being passably good at bullshitting (while mostly keeping a rhyme/meter) is still mindblowing to me.
- jvanderbot 4y agoIt's an amazing accomplishment technically, and that deserves to be emphasized, but it is still not producing reliable outputs. Subjectively incredible; objectively flawed.
- rpdillon 4y agoWhat does produce reliable outputs?
- xeromal 4y agoHumans?
- rpdillon 4y agoHumans, in my experience, have even more reasons to be flawed than the AI. Yes, the AI can be incorrect, but humans can be misled in what they believe the same way, and have the added problems of ego and misaligned incentives. Heck, in this very thread we have one human calling out another for being incorrect about whether or not an AI was correct. Locating sources of reliable truth is an unsolved problem AFAIK.
- anigbrowl 4y agolol no. LLMs get things wrong, but they don't lose their shit when you point it out. I find it pleasant rather than frustrating to work on problems with one; the shortcoming for me as a sort of sort of conditioned passivity that arises out of its orientation to search its own knowledge for correct answers rather than asking questions.
- pxue 4y ago"confidently wrong" That's chatgpt right now. To get it to confidently right will take years and years more
- Rastonbury 4y agoI saw dallE and GPT-2 demos and though 'oh pretty cool' about a year later we've got stable diffusion that can run on phones and ChatGPT which can remember context. I thought these things would be 3-5 years out or more. The speed of development is so quick, I think confidently right LLM is around the corner In less than a decade, I believe front line support chat support jobs and even graphic artist jobs will be made obselete
- causi 4y agoIn less than a decade, I believe front line support chat support jobs and even graphic artist jobs will be made obselete We can finally get the eighty hour long book-accurate Lord of the Rings adaptation we deserve. But seriously, I can't wait to feed a novel into one of these things and get a comic book version in return.
- ilaksh 4y agoThe latest models, with the right setup, can already substitute for many (not all) tasks in knowledge jobs such as graphic design, support chat, even programming. I feel like we may be looking at less than 3 or 4 years for a lot of jobs to be mostly obsolete. Or at least, the traditional version of those jobs. It's already at the point where you start expecting any knowledge worker to be significantly more productive by leveraging these tools. It's hard to imagine that it will be more than five years before AI tools are available that can handle almost all tasks in these types of jobs. For example, on my website aidev.codes I just added preliminary knowledgebase support. It can reference the knowledgebase this to write code. I would say that with the code-davinci-002 model at least it seems about at the level of a junior software engineer already since it's pretty effective with close supervision by a senior peogrammer, except for the fact that it cannot interpret visual information. Knowledgebases/embedding search can also be used right now with these models for answering support questions. The only thing holding it back from very very wide scale adoption is the problem of making up information. There are already solutions in progress for this. It's unlikely that will take more than a few years to roll out and replace the current generation of models. Google and Microsoft will probably roll out their internet-scale chat search interfaces this year even if they can't fully mitigate the hallucination problem immediately. I would guess more like 2-3 years for many knowledge-based jobs. If you want employment/contracts you will need to be very good at leveraging AI, or people will just use the AIs instead.
- kevinwang 4y agoI agree that the output is often wrong despite looking right, but in this instance I don't see anything wrong in this example.
- jvanderbot 4y agoDijkstra's cannot be used as generally as described. A-star can potentially be more general, but that's highly dependent on heuristic. Dinkstras is a-star with a "min weight on shortest path first" heuristic. Because a star doesn't expand shortest paths first necessarily, it cannot necessarily find the shortest path to any node except the prescribed endpoint. Dijkstra's will find all shortest paths with cost less than the path to the endpoint, and that's helpful sometimes.
- kevinwang 4y agoI actually think the robot's poetic take matches my thinking more closely than this description Edit: I guess in this description dijkstras is more specific, in that dijkstras is the specific instance of A* with a zero heuristic. But I think what HAL over here was saying is that you can use dijkstras in a superset of scenarios in which you can use (non-trivial) A*, so in that sense dijkstras is more general than A*, so it's not wrong.
- lqet 4y agoWhat is your definition of "general" here? The classic shortest path problem (which the psalm addresses) is described as finding the shortest path between two nodes, and given non-negative edge weights, Dijkstra's algorithm is perfectly general here. It will find such a shortest path in any graph, no further restrictions. For A*, everything depends on the heuristic, as you said. If h(v) = 0, A* is equivalent to Dijkstra's algorithm, so potentially, it can be as general. But the wrong heuristic (inadmissible, inconsistent) will lead to wrong results, and so calling it a "specialized" tool is correct. The heuristic gives you a specialized version of Dijkstra's algorithm which is faster on specific graphs. > Dinkstras is a-star with a "min weight on shortest path first" heuristic. I am not quite sure what you mean here. Dijkstra's algorithm is A* with no heuristic. I am not sure how a "shortest path first" heuristic would look like - do you mean that Dijkstra's algorithm chooses the node to expand next based on its shortest-path cost to the target t? Even if you would construct such a heuristic h(v) = c(v, t) by explicitly calculating c(v, t) each time, this is not what Dijkstra does, as it would basically be a perfect heuristic - you would then only visit nodes on the shortest path. If you meant that Dijkstra's algorithm is A* with a "neighboring node with shortest path from start node first" heuristic, that's also not strictly true, because expanding the nearest node first is already built-in to A*. A* does not chose the next node v with the smallest h(v), but with the smallest g(v) + h(v), where g(v) is the shortest path cost from the source node to v.
- mensetmanusman 4y agoIt is passing medical exams and will be used to help diagnose issues with similar reliability as humans.
- nradov 4y agoThat can only be possible in a limited set of cases where the inputs necessary to make a diagnosis are all present in the patient chart. Human clinicians rely on other inputs to make most diagnoses including visually looking at the patient from multiple angles, asking them questions, manipulating and feeling the patient's body, listening (with or without stethoscope), and even smell. We are a long way from the robotics and sensor integration technology necessary to automate those steps in the diagnostic process.
- tootie 4y agoWe've been down the AI diagnosis path before and even a purpose-built system failed to live up. I think that GPT, much like Tesla autopilot, is going to spend a very long time ironing out it's "last mile" before anyone will trust it. The tendency for AI to be spectacularly wrong when it fails to grasp context is still pretty prevalent.
- markus_zhang 4y agoJust curious is it a political issue (risky and who to sue) or a technical issue (ai has far worse judgement)?
- tootie 4y agoMostly that it faired poorly against human doctors. Real-world medical data is too messy to feed into a machine. https://www.theregister.com/2022/01/31/machine_learning_the_hard_way/ https://www.theregister.com/2022/01/31/machine_learning_the_...
- markus_zhang 4y agoThanks! I wonder what is the most advanced automatic technique right now. I also think it's difficult to replace nurses -- even though basic nurse work is even easier comparing to a family doctor who needs to diagnose. I figure the reason is that to replace nurse we need a lot of automatic hardware, which needs a lot of investment, unlike ChatGPT which can really replace a service agent given enough training because it doesn't involve hardware.
- krona 4y agoI think it captured the essential difference between the two algorithms, i.e. the heuristic, so I don't understand how this is 'bullshit'.
- deleted 4y ago[deleted]
- joaogui1 4y agoA* is more general than Dijkstra, as Dijkstra ia A* with the heuristic set to 0. Also it's not in the style of a psalm
- vijayr02 4y agoSo perfectly suited for a management role? :)
- mjhay 4y agoI did read somewhere that in some tests, ChatGPT-generated MBA papers got better grades than average MBA students.
- treis 4y agoThey blocked people from generating performance reviews so at least some were trying.
- dennis_jeeves1 4y agoWell suited for more most walks of life actually where BS reigns supreme :)
- lqet 4y ago> These tools are about as good as a non specialist with time to undertake some bullshitting. Even if you are right, you still acknowledge that ChatGPT is on the level of a non-specialist bullshitting human you can have a meaningful conversation with. This alone would be enough to freak me out. If you would've told me that this is possible 10 years ago I would've called you crazy.
- jvanderbot 4y agoSubjectively incredible; objectively questionable.
- throwaway675309 4y agoI know a lot of people are vividly impressed by this but to me "writing in the style of" (Victorian sonnets, poems, hip hop lyrics) is the equivalent of transfer learning and feels more like a cheap parlor trick. While I wont deny that it has impressive summarization abilities in terms of making excellent Q&A if you're willing to vet the information, I wouldn't exactly say that chatGPT is capable of meaningful conversation. It has great powers of recollection but its capacitive powers to produce new interesting information feels highly formulaic.
- hammyhavoc 4y agoI laughed when I read someone call its output "meaningful".
- HideousKojima 4y agoAlso psalms don't rhyme, even in the original Hebrew. Also even if it is supposed to be a poem with rhyme and meter, ChatGPT's poem doesn't have correct meter.
- danielvf 4y agoAgreeing with you. Psalms typically employ parallelism not rhyming. Two examples: The sun will not harm you by day, nor the moon by night. and My life is consumed by anguish and my years by groaning; my strength fails because of my affliction,[b] and my bones grow weak.
- magarnicle 4y agoYou have to tell it explicitly the correct number of syllables in a haiku, otherwise it gets it wrong.
- Der_Einzige 4y agoI wrote a whole paper on how to fix this in LLMs! https://paperswithcode.com/paper/most-language-models-can-be-poets-too-an-ai https://paperswithcode.com/paper/most-language-models-can-be... Gwern cited it too!
- dennis_jeeves1 4y ago'poetic': you have described it accurately. I fail to see what the hullabaloo is all about. Call me jaded.
- hammyhavoc 4y agoI'll call you refreshingly grounded.
- soperj 4y agoThis is my exact problem with ChatGTP, it looks great when you don't know what it's talking about, but as soon as you do, it looks foolishly over-confident in it's answers which are very clearly wrong.
- Keyframe 4y agowhich are very clearly wrong but not always; And that's what makes it scary. You know people will use it and trusting it.
- krsw 4y agoNot much worse than a number of people I've talked with, to be honest.
- twic 4y agoHa ha only serious - for me, the most profound thing about ChatGPT and friends is that they show how much of human behaviour is not actually intelligent in some deep sense.
- deleted 4y ago[deleted]
- CamperBob2 4y agoExactly. It doesn't just share our abilities, it shares our flaws, which is actually more interesting and more disturbing. It's not a crystal ball, it's a mirror.
- iliane5 4y agoThe fact that a "dumb", generative model that is simply predicting the next token when given a prompt can talk so well, perform complex tasks and interact with humans in such a convincing manner is pretty fascinating. There is obviously more to human intelligence than text based conversation but it is pretty humbling that such an aspect of ourselves can be replicated and be so convincing at such an early stage and perform better than some humans even when it makes stuff up: toddlers can't talk, kids are smarter but don't have the technical knowledge, most adults only have a few specific areas of expertise, etc.
- CamperBob2 4y agoWhich if you haven't noticed is good enough to land a seat in the United States Congress.
- kragen 4y agowhat's incorrect about it as far as i can tell it seems perfectly correct but admittedly my understanding of a* search is limited to messing with amit's visualizations of it
- PebblesRox 4y agoIt's also not Psalm-like! The Psalms don't rhyme (even in the original Hebrew). What they do have is a parallel structure where each idea is repeated in slightly different words. This is just generic poetry with a sprinkling of "ye's."
- hammyhavoc 4y agoIndeed, I had to laugh.
- Der_Einzige 4y agoNot even always poetic either Notice that it sometimes struggles to rhyme?
- TuringNYC 4y ago>> That's wrong. It's poetic, but incorrect. These tools are about as good as a non specialist with time to undertake some bullshitting. Does it really matter when it comes to real life? Kids take medical advice from TikTok influencers. Half the country believes absurd news they see on Facebook. Half the country believes anything their president says.