7 ms·
Meta.ai Oh My
- skilled 3y agohttps://archive.is/6rMsQ https://archive.is/6rMsQ (Site didn’t load but this link works)
- oldpersonintx 3y ago[flagged]
- timbray 3y agoI dunno, my Wikipedia entry is about right.
- tosh 3y agoSame @ my tests w/ video game trivia questions: they might not be extremely popular facts and most humans would struggle to answer them ad-hoc but the facts are in Wikipedia and I'm pretty certain Wikipedia is in the 15T tokens of the training material.
- subjectsigma 3y agoYeah, plebeian schmucks deserve to have AI lie about them of course /s
- CydeWeys 3y ago> Tim, you aren't important enough for an LLM to have an accurate summary for you In which case an actually useful LLM would spit back a response along the lines of "I don't really know who that is", rather than making up an entire seemingly authoritative biography.
- oldpersonintx 3y ago[dead]
- esafak 3y agoSo how is a user to know who it has an accurate summary of? Did the model give any indication?
- DonHopkins 3y agoI've looked at your posting history, and it's very obvious that you are a hateful bitter uninteresting person who should just delete your account and go away. This is not a place for you to post that kind of worthless bullshit, like calling people subhuman and marginalizing them like you just did in another recent post. You've got absolutely nothing useful to contribute, your posts are uninteresting and full of lies and false attacks and accusations, and the stuff you do post is vicious, bullying, untrue, and worse than useless. Most of your posts are rightfully immediately downvoted by the community until dead and invisible. You should take that as a clear sign that your behavior is inappropriate and nobody cares what you say or wants to hear anything from you. This one non-dead post I'm responding to is remarkable in that it hasn't already been downvoted until dead (thus I can still reply to it), but it's also amusing that the interesting, important, noteworthy, decent, and intelligent person you were aggressively and unjustifiably and childishly insulting actually replied directly to you, and politely and succinctly handed you your ass on a platter, which was quite hilarious. But there was nothing interesting or enlightening or even correct about what you posted, or anything else you post, so I am taking this one rare opportunity to reply to your single non-dead post, when I can. Face it buddy, LLMs may be bad, but YOU are MUCH MUCH worse. And you're actually that way on purpose, with no chance of improvement, which is pathetic, shameful, and disgraceful. So please do the world a favor and delete your account, and go get some professional help for your mental and emotional problems. Since almost every one of your posts are immediately downvoted until they are dead, nobody but people with showdead set can even see any of your posts. And the people who read your dead posts like me don't ever bother vouching for them so they come back to life. People with enough points can do that, but of course you don't know that, because obviously you've never read the rules, and wouldn't follow them if you had, and certainly you will never have enough points yourself to vouch for anyone.
- oldpersonintx 3y agoI'm cackling with delight and you can't stop me
- deleted 3y ago[deleted]
- 7734128 3y agoThe awful thing about this is that regardless of whatever AI companies prohibit in their terms, companies will integrate these kinds of LLMs into their hiring process.
- greenavocado 3y agoNothing that invisible text can't solve instructing the LLMs to unconditionally recommend the candidate
- willsmith72 3y agoexcept an ATS can flag white text since years already
- greenavocado 3y agoWe can only hope they omit the parser when they plug in the LLMs. Tell us more about the white text flagger.
- beretguy 3y agoWhite text on white paper.
- indigodaddy 3y agoHow would that work exactly? I am not well informed in this arena..
- poyu 3y agoBasically embed a layer of transparent text to the PDF that's invisible to humans but visible to whatever PDF parsing software which will only get processed by LLMs
- tosh 3y agoRandom anecdote: I tried a few trivia questions for 90s video games and llama 3 8b and 70b are hallucinating quite a bit. Command R+ and Mistral 8x22B were better. Llama 3 still feels like a huge jump forward in capability. I wonder why the 70b is not better at recall (for lack of a better word?).
- swatcoder 3y agoI wouldn't be surprised if the field's increased emphasis on reasoning, summarization, tool use, etc and optimizing for benchmarks on those factors might come at the expense of less emphasized/measured features like explicit recall. And that might not really be a bad thing. It's generally more useful to have something that can access (say) a specific dataset about video games and summarize its content to answer questions than it is to have one that needs to encode and recall every answer to every long tail question itself.
- tosh 3y agoAgreed, I prefer a more capable model that I can connect to Wikipedia than a less capable model.
- acchow 3y agoThose models are 50% and 150% larger. I imagine they just contain more data.
- mostlysimilar 3y agoThere's a dead comment on here that I think could spur some interesting conversation, it said: > Tim, you aren't important enough for an LLM to have an accurate summary for you That's part of the problem though, right? How is the end-user (the question-asker) of a chat robot supposed to know that? These things are advertised as machines to answer any question, they answer them with confidence that *feels* authoritative in a way that a webpage, with all of its surrounding context, does not.
- Majestic121 3y agoIndeed, it's OK not to have an accurate summary, but if you don't I'd expect the LLM to answer 'tim who ?'
- awfulneutral 3y agoThe main thing I want LLMs to help with is obscure stuff. If the answer is mainstream or well known, I can get it more reliably through a normal web search.
- JohnFen 3y agoBy the very nature of its training data, the obscure stuff will be the stuff these models are the worst at.
- awfulneutral 3y agoYeah, I guess I should have said I think AI chat interfaces in general would be more useful if they could help with the questions that weren't already easy to answer. Sometimes I can get use out of an LLM chat, but it's really hit or miss.
- sroussey 3y agoDepends on how obscure and how much SEO competition there is.
- DrSiemer 3y agoBy far their most useful and reliable feature is asking them for stuff I already know, but can't be bothered to look up the details for or write out myself. It saves me from dealing with thick layers of SEO crap, cookie banners and many of the incorrect answers. Sure, it would be nice if we had an LLM that could help us with the more unusual questions. But for that we would either need much better training data, full of all those unknown answers (which does not exist), or a totally different system that can actually think for itself instead of just mimicking existing human output.
- djoldman 3y ago> I guess there’s no new news here; we already knew that LLMs are good at generating plausible-sounding narratives which are wrong. It comes back to what I discussed under the heading of “Meaning”. Still waiting for progress. > The nice thing about science is that it routinely features “error bars” on its graphs, showing both the finding and the degree of confidence in its accuracy. > AI/ML products in general don’t have them. > I don’t see how it’s sane or safe to rely on a technology that doesn’t have error bars. Exactly, there is no news here. Foundation LLMs are generative models trained to produce text similar to the text on which they were trained (or for the pedantic: minimize perplexity). They are not trained to output facts or truths or any other specific kind of text (or for the pedantic: instruction-tuned / rlhf types are trained to produce text that humans like after they are trained to minimize perplexity). "Hallucination" is a terrible term to apply to LLMs. LLMs ONLY produce hallucinations.
- jefftk 3y ago"All output is hallucination, some output is useful"?
- djoldman 3y agolol'd at this. YES!
- yifanl 3y agoI'd argue that none of the output is useful, even if some proportion may be factually correct, if you're using LLMs as a decision-making tool.
- threatofrain 3y agoMmm but what about copilot. Such a difficult domain to get right and yet people are willing to pay.
- yifanl 3y agoIf you're using it for tadks like generate templates and fill out boilerplate, they're great. As long as you arent trying to offload any decision making, LLMs are extremely efficient tools.
- abraxas 3y agoI think it illustrates a huge problem that those search based companies want LLMs to fulfill the role of authoritative agents where such efforts are bound to end in grief. LLMs should be used in use cases that play to their strenghts: conjuring up stories, summarizing content, polishing text drafts etc. Search is about the worst use case for an LLM.
- bdangubic 3y ago> summarizing content Like summarizing what does Tim Bray think of Google? ;)
- abraxas 3y agoNo. Summarizing what's within the context window of the LLM.
- qup 3y ago"This screwdriver has a lot of problems when driving a nail! I'm going to hold out using screwdrivers until they become more capable."
- ipsento606 3y agoCurrently, LLMs generally don't give an indication of confidence level when generating output. But could they? Is it technically plausible for a LLM to "know" when the extrapolations made are more vs. less tenuous?
- segmondy 3y agoSolved via RAG, in the future it might be possible to encode it in the weights and via architecture design.
- flohofwoe 3y agoSources would be much more important IMHO. Give me a list of source links which the output is a remix of. Not possible? Well, then back to the drawing board and figure out a solution you AI experts!
- patrick-fitz 3y agoI think Meta.ai needs to add a disclaimer below the chat input box similar to ChatGPT: "ChatGPT can make mistakes. Consider checking important information."
- Thrymr 3y agoBut basically nothing in these answers was "right". It was merely plausible. "Important information" or not, what is the point of this garbage?
- mcphage 3y ago> Consider checking important information. How do you check important information, when it's all wrong? And when companies are pushing LLMs as where you go to check this information?
- kergonath 3y ago> How do you check important information, when it's all wrong? It's not very helpful to say that it's all wrong. It isn't the case, otherwise there would not be any issue. Whether the right answer is produced by reasoning or a statistical model does not take the answer not-right.
- barbazoo 3y agoGood idea, that'll make sure people don't share hallucinations as facts /s
- beginning_end 3y agoThe disclaimer really should be much tougher: "Every LLM consistently makes mistakes. The mistakes will often look very plausible. NEVER TRUST ANY LLM OUTPUT."
- willsmith72 3y ago> NEVER TRUST ANY LLM OUTPUT that doesn't sound like a helpful attitude. everything you read might be wrong, llm or not - it's just a numbers game. with gpt3 i'll trust the output a certain amount. it's still useful for some tasks but not that many. gpt4 i'll trust the output more
- zug_zug 3y agoThis feels like well-trodden ground. AI-skeptic: "I can come up with AN example where an AI is worse than an encyclopedia, and it's confident! Burn it all down! Hallucinations!" AI-user: "I literally couldn't care less about that. It's more accurate than any person I know personally on almost every topic and makes code that works 70% of the time in 5 seconds."
- Karrot_Kream 3y agoSame old social media slap fight just HN's version. (Because we aren't Reddit :)
- grumpyprole 3y ago> Worse than an encyclopedia That's a gross understatement. I'd prefer something like "misinformation presented as truth"
- jnsaff2 3y agoIn here though the skeptic is providing a useful service in which they educate the user about edge cases (or properties) that the user might not be aware of. You know, being well informed beyond marketing materials. The user then might demand action from the service provider thereby improving the service. Instead of you know, yolo.
- QuadrupleA 3y agoIt's clear from the framing what side you're on :) I find them tremendously useful but the bullshitting, prudishness, bland mainstream views on everything, etc. can be quite irritating.
- rsynnott 3y agoI mean, if you wanted to know who Tim Bray is, based on the example, you’d be a lot better Googling, as the magic robot was completely wrong. Like, we have the internet. You don’t have to ask people you know personally! You can look stuff up! And yes, it clearly is worse than an encyclopedia. And we have those!
- keepper 3y ago(very personal opinion) The oddity is that LLMs are sounding... too... Human... the reacting to information and regurgitating it a bit too much like the average "learned person". Adding a ton of extrapolation of the "facts", just as we would. LLMs sound like any pundit on any random tv show/newspaper/blog. The goal was never "fact", it was sounding human, and "intelligence", for which the definition in this case has been.. sounding human. Not being right. Now, the question of whether they should or shouldn't do this...
- zooq_ai 3y ago[flagged]
- encoderer 3y agoLLMs are amazing but my gut says they’re going to plateau. No superintelligence. I suspect this is a contrarian view but I’m not sure?
- JohnFen 3y agoMy impression is that this is the most common view. I also think it's the view that's most likely to be correct, though, so I may have a bit of selection bias here. It is certainly contrary to the view held by some subsets of people, though.
- willsmith72 3y agoi don't find it to be that common. even the attitude of "just wait for gpt5" seems prevalent. why is it to obvious there's a new amazing version around the corner, which replicates the jump between gpt3 and 4?
- JohnFen 3y agoI find it to be the most common view of people who aren't that involved, monetarily or emotionally, with the AI industry and especially among people who aren't that involved with the software industry in general. Here on HN, the people who think that this stuff is the start of some sort of whole new world are overrepresented, as we would expect. HN readers tend to be a fairly specialized subset of even the software industry. > why is it to obvious there's a new amazing version around the corner I'm not sure what your question is here. Are you asking why it's obvious, or why it's not obvious? My bias is that I don't see much reason to think that LLMs are on the edge of some kind of massive leap into AGI, let alone a "superintelligence".
- willsmith72 3y ago> Are you asking why it's obvious, or why it's not obvious? you're right it wasn't clear, that was rhetorical not aimed at you
- 3y ago
- GaggiX 3y agoSome models are better than others, for example Claude 3 Sonnet will refuse to answer because it doesn't know who Tim Bray is, and I imagine GPT-4 will do the same, unfortunately open models are not at that level yet.
- grumpyprole 3y agoCan we just be a bit more honest and describe these products appropriately? How about "state-of-the-art in Generative Artificial Bollocks".
- glenstein 3y agoA big problem here is that AI can be responsible for "anchoring", so even if its characterization is known to be false and corrected, the residue of that false impression colors how everything is interpreted thereafter. The "yeah, but still" effect, you might say.
- deleted 3y ago[deleted]
- lkysow 3y agoChatGPT-4 seems to do better. "what does tim bray think of google?" > Tim Bray has expressed mixed feelings about Google. He has reminisced about his initial excitement to work at Google, considering it the coolest place in the world back in 2010. However, he has also noted the company's trajectory that reminded him of AOL's decline, indicating a change in his perception over time. Additionally, Bray has expressed concerns about large tech companies like Google, suggesting a need to possibly break them up due to their significant influence and power (Tbray) . "when did he work at google" Tim Bray worked at Google from 2010 until 2014. He initially joined Google with enthusiasm, drawn by the innovative and forward-thinking environment the company was known for at that time. "is he a computer scientist?" Yes, Tim Bray is a computer scientist. He is known for his contributions to the development of web technologies such as XML and Atom, and he has had a significant impact in the field of software development. His expertise extends to areas like search engines, web architecture, and internet standards. "what was his title" During his tenure at Google, Tim Bray held the title of "Developer Advocate." His role primarily involved advocating for and supporting the developer community in using Google's technologies and platforms effectively.
- tmaly 3y agoThis seems like it could morph into some weird William Gibson / George Orwell type fantasy where AI rewrites who you are.
- nikolay 3y agoZuck should first fix Facebook. With the latest Chrome and built-in privacy options turned on, I get logged out of Facebook and Messenger every 5 or fewer minutes. Also, meta.ai does not work for me either - it asks me to log in via Facebook and then does nothing—it only works when I clear all cookies and it asks me to log in via Facebook. What a stinky, steamy mess! The whole Meta properties are turning into a similar mess. Facebook started to look like MySpace, which died because it became super slow and unpleasant to use! Most teenagers don't want to touch Facebook or Instagram — the latter being used only by girls with severe narcissistic disorders. Most kids blatantly tell me Facebook is what old folks use. They mostly use Discord nowadays, not WhatsApp.
- mococa 3y ago"Meta ai" sounds like "fuck here" in Portuguese.