12 ms·

The author gives two reasons why AI won't replace the need for metadata: 1: LLMs "routinely get stuff wrong" 2: "pricy GPU time" 1: I make a lot of tests on

• monero-xmr 2y ago
  LLMs have no soul, so I like content and curation from real people
  • doe_eyes 2y ago
    The main problem is that the incentive for well-intentioned people to add detailed and accurate metadata is much lower than the incentive for SEO dudes to abuse the system if the metadata is used for anything of consequence. There's a reason why search engines that trusted website metadata went extinct. That's the whole benefit of using LLMs for categorization: they work for you, not for the SEO guy... well, prompt injection tricks aside.
    • monero-xmr 2y ago
      There is value-add if you can prove whatever content you are producing is from an authentic human, because I dislike LLM produced garbage
      • usrusr 2y ago
        The point is that metadata lies. Intentionally, instead of just being coincidentally wrong. For example everybody who wants to spew LLM produced garbage in your face will go out of their way to attach metadata claiming the opposite. The value proposition of LLM categorization would be that the LLM looks at the same content as the eventual human (if, in fact, it does - which is a related but different problem)
  • tsimionescu 2y ago
    All the web metadata I consume is organic and responsively farmed.
  • amarant 2y ago
    Huh, it's not often you hear a religious argument in a technical discussion. Interesting viewpoint!
    • MrVandemar 2y ago
      I don't see it as anything religious. I see the comment about something having an intrinsic, instinctive quality, which we can categorise as having "soul".
      • amarant 2y ago
        That's even more interesting! The only non-religious meaning of soul I've ever heard is a music genre, but then English is my second language. I tried googling it and found this meaning I wasn't aware of: emotional or intellectual energy or intensity, especially as revealed in a work of art or an artistic performance. "their interpretation lacked soul" Is this the definition used? I'm not sure how a JSON document is supposed to convey emotional or intellectual energy, especially since it's basically a collection of tags. Maybe I also lack soul? Or is there yet another definition I didn't find?
        • pessimizer 2y ago
          It's early 20th century (and later) black American dialect to say things "have soul" or "don't have soul." In the West, Black Americans are associated with a mystical connection to the Earth, deeper understandings, and suffering. So LLMs are not gritty and down and dirty, and don't get down. They're not the real stuff.
          • amarant 2y ago
            Mystical connection? Now you're back to religion. If you wanna be down you gotta keep it real, and mysticism is categorically not that.
      • Eisenstein 2y ago
        > intrinsic, instinctive quality, What are a few examples of things with an 'intrinsic, instinctive quality'?
• rapsey 2y ago
  GPU compute price is dropping fast and will continue to do so.
  • philjohn 2y ago
    But is it dropping faster than the needs of the next model that needs to be trained?
    • tossandthrow 2y ago
      Short answer is yes. Also, GPU pricing is hardly relevant. From now on we will see dedicated co-processors on the GPU to handle these things. They will keep on keeping up with the demand until we meet actual physical limits.
  • dspillett 2y ago
    The cost of GPU time isn't just the cost that you see (buying them initially, paying for service if they are not yours, paying for electricity if they are) but the cost to the environment. Data centre power draws are increasing significantly and the recent explosion in LLM model creation is part of that. Yes, things are getting better per unit (GPUs get more efficient, better yet AI-optimised chipsets are an order more efficient than using GPUs, etc.) but are they getting better per unit of compute faster than the number of compute units being used is increasing ATM?
• menzoic 2y ago
  How does Product Chart use LLMs?
  • mg 2y ago
    We research all product data manually and then have AI cross-check the data and see how well it can replicate what the human has researched and whether it can find errors. Actually, building the AI agent for data research takes up most of my time these days.
    • viraptor 2y ago
      Have you seen https://superagent.sh/ https://superagent.sh/ ? It's an interesting one and not terrible in the test cases I tried. (Requires pretty specific descriptions for the fields though)
• throwme_123 2y ago
  For my part, I stopped reading at the free bashing of blockchain•. Reminded me of the angst and negativity of these original "Web3" people, already bashing everything that was not in their mood back then. • The crypto ecosystem is shady, I know, but the tech is great
  • ashkankiani 2y ago
    As someone who stopped getting involved in blockchain "tech" 12 years ago because of the prevalence of scams and bad actors and lack of interesting tech beyond the merkle tree, what's great about it? FWIW I am genuinely asking. I don't know anything about the current tech. There's something about "zero knowledge proofs" but I don't understand how much of that is used in practice for real blockchain things vs just being research. As far as I know, the throughput of blockchain transactions at scale is miserably slow and expensive and their usual solution is some kind of side channel that skips the full validation. Distributed computation on the blockchain isn't really used for anything other than converting between currencies and minting new ones mostly AFAIK as well. What is the great tech that we got from the blockchain revolution?
    • throwme_123 2y ago
      Scams and bad actors haven't changed sadly. But zk-based really decentralized consensus now does 400 tps and it's extraordinary when you think about it and all the safety and security properties it brings. And that's with proof-of-stake of course with decentralized sequencers for L2. But I get that people here prefer centralized databases, managed by admins and censorship-empowering platforms. Your bank stack looks like it's designed for fraud too. Manual operations and months-long audits with errors, but that is by design. Thanks everyone for all the downvotes.
      • dspillett 2y ago
        > But I get that people here prefer For many of us it isn't that we think the status quo is the RightWay™ - we just aren't convinced that crypto as it currently is presents a better answer. It fixes some problems, but adds a number of its own that many of us don't think are currently worth the compromise for our needs. As you said yourself: > The crypto ecosystem is shady, I know, but the tech is great That but is not enough for me to want to take part. Yes the tech is useful, heck I use it for other things (blockchains existed as auditing mechanisms long before crypto-currencies), but I'm not going to encourage others to take part in an ecosystem that is as shady as crypto is. > Thanks everyone for all the downvotes. I don't think you are getting downvoted for supporting crypto, more likely because you basically said “you know that article you are all discussing?, well I think you'll want to know that I didn't bother to read it”, then without a hint of irony made assertions of “angst and negativity”. And if I might make a mental health suggestion: caring about online downvotes is seldom part of a path to happiness :)
• zaik 2y ago
  > Reddit takes hundreds of http requests, parses megabytes of text, image and JavaScript code [...] to show some links to articles Yes, and I hate it. I closed Reddit many times because the wait time wasn't worth it.
  • rfl890 2y ago
    https://old.reddit.com https://old.reddit.com ?
    • jeltz 2y ago
      Gets buggier for every year.
    • dspillett 2y ago
      That definitely seems to be getting less reliable these days. A number of times I've found it refusing to work, or redirecting me to the primary UI arbitrarily, a few months ago there was a time when you couldn't login via that UI (though logging in on main and going back worked for me). These instances seem to be temporary bugs, but they show that it isn't getting any love (why would it? they only maintain it at all under sufferance) so at some point it'll no doubt be cut off as a cost cutting exercise during a time when ad revenue is low.
• atoav 2y ago
  Let's hope you never write articles about court cases then: https://www.heise.de/en/news/Copilot-turns-a-court-reporter-into-a-child-molester-9840612.html https://www.heise.de/en/news/Copilot-turns-a-court-reporter-... The alleged low error rate of 1% can ruin your day/life/company, if it hits the wrong person, regards the wrong problem, etc. And that risk is not adequately addressed by hand-waving and pointing people to low error rates. In fact, if anything such claims would make me less confident in your product. 1% error is still a lot if they are the wrong kind of error in the wrong kind of situation. Especially if in that 1% of cases the system is not just slightly wrong, but catastrophically mind-bogglingly wrong.
  • kqr 2y ago
    This is the thing with errors and automation. A 1 % error rate in a human process is basically fine. A 1 % error rate in an automated process is hundreds of thousands of errors per day. (See also why automated face recognition in public surveillance cameras might be a bad idea.)
    • atoav 2y ago
      Exactly. If your system monitors a place like a halfway decent railway station half a million people per day is a number you could expect. Even with an amazingly low error rate of 1% that would result in 5000 wrong signals a day. If we make the assumption that the people are uniformly spread out througout a 24 hour cycle that means a false alarm every 20 seconds. In reality most of the people are there during the day (false alarm every 10 seconds) and the error percentages are nowhere near 1%. If you do the math to figure out the staff needed to react to those false alarms in any meaningful way you have to come to the conclusion that just putting people there instead of cameras would be a safer way to reach the goal.
    • Terr_ 2y ago
      Another part is that artificial systems can screw up in fundamentally different ways and modes compared to a human baseline, even if the raw count of errors is lower. A human might fail to recognize another person in a photo, but at least they won't insist the person is definitely a cartoon character, or blindly follow "I am John Doe" written on someone's cheek in pen.
• intended 2y ago
  Only slightly tongue in cheek, but if your measure of success is Reddit, perhaps a better example may serve your argument?
  • ramon156 2y ago
    The argument for "LLMs get it right 99% of the time" is also very generalized and doesn't take into account smaller websites
    • klabb3 2y ago
      It’s baffling how defeatist and ignorant engineering culture has become when someone else’s non-deterministic, proprietary and non-debuggable code, running on someone else’s machine, that uses an enormous amount of currently VC-subsidized resources, is touted as a general solution to a data annotation problem. Back in my day people used to bash on JavaScript. Today one can only dream of a world where JS is the worst of our engineering problems.
• 8organicbits 2y ago
  Oh nice, Product Chart looks like a great fit for what LLMs can actually do. I'm generally pretty skeptical about LLMs getting used, but looking at the smart phone tool: this is the sort of product search missing from online stores. Critically, if the LLM gets something wrong, a user can notice and flag it, then someone can manually fix it. That's 100x less work than manually curating the product info (assuming 1% error rate).
• esjeon 2y ago
  > I make a lot of tests on how well LLMs get categorization and data extraction right or wrong for my Product Chart (https://www.productchart.com https://www.productchart.com) project. In fact, what you're doing there is building a local semantic database by automatically mining metadata using LLM. The searching part is entirely based on the metadata you gathered, so the GP's point 1 is still perfectly valid. > In the near future, nobody will see passing a text through an LLM as a noteworthy amount of compute anymore. Even with all that technological power, LLMs won't replace most simple-searching-over-index, as they are bad at adapting to ever changing datasets. They only can make it easier.