lukev

37 ms·

601.
by lukev 3y ago
Can anyone who's worked with fine-tuning models this large weigh in on how much fine-tuning data is typically required to be effective? Can 100k tokens (as mentioned in the docs) really influence the behavior of the base model that muc
602.
by lukev 3y ago
RAG is fundamentally different and there will always be a place for it. A model's weights are inherently lossy and opaque. If a model asserts some fact, it is impossible to tell whether that fact was true or hallucinated just from the
603.
by lukev 3y ago
I'm sure it makes a lot of people feel better, but almost certainly because of the cessation of whatever dietary behavior was causing them to feel bad in the first place, as opposed to eliminating some undefined "toxin."
604.
by lukev 3y ago
Maybe ChatGPT is politically biased, maybe it isn’t. Maybe it’s “aligned”, maybe it isn’t. The point that everyone seems to miss is that it’s a damn useful tool… which is also entirely inappropriate to use for political, philosophical or et
605.
by lukev 3y ago
Congress can pass whatever they like, but there's a bunch of reasons a law like that would be unconstitutional and likely quickly struck down. Not only is it a violation of the First Amendment, it is almost a textbook case of a Bill of
606.
by lukev 3y ago
Is there even any mechanism for this in the US legal system? Companies and products can be regulated but the requirements need to be specific (and I'd welcome rules around personal data collection etc.) Or you can regulate an entire cl
607.
by lukev 3y ago
Sigh. Sarcasm is always lost on the internet. I was replying to the actually insane comment who said the Tractatus was well below Milton on his reading list (presumably to express how little any of them are worth his time…?) I’ve read all
608.
by lukev 3y ago
Yeah completely agreed... who needs real philosophy or literature when we have Harry Potter fanfic that does both?
609.
by lukev 3y ago
It's true. Validity isn't a statistical notion, and that's why I agree with you that LLMs can't "reason" if we're being precise with language. Just like they can't "know" or "believe&qu
610.
by lukev 3y ago
To paraphrase what you're saying: an AI can't reason, because it is built to stochastically predicts tokens, which is not reasoning and which is different from the activity of reasoning. Which I agree with. But also, by observatio
611.
by lukev 3y ago
This is not how large language models work. The only way this could happen is if they deliberately include the prompt and the correct answers (e.g, this paper) in the training data for the next version of the model. Each version of the mode
612.
by lukev 3y ago
I (and everyone else) have known that text in VR is really hard and technically demanding since I received my first Oculus from the first kickstarter. Not sure we can judge how well Apple has solved this extremely hard problem until we actu
613.
by lukev 3y ago
Different vector indexing algorithms under the hood, with different tradeoffs. In this case "hnsw" is the name of the algorithm. If you want to know more, just search "hnsw" and "ivfflat" to understand the diff
614.
by lukev 3y ago
Indeed. Or, more generally, don't over-define your data model. Keep it as simple as possible (but no simpler.)
615.
by lukev 3y ago
I like this a lot and it’s well worth reading over. For a lot of programming tasks, it’s best to consider these kind of lists not as falsehoods but as edge cases or assumptions that should be made explicit. Handling every “falsehood” gracef
616.
by lukev 3y ago
This is pretty ahistorical. Most of the world's great civilizations have been from warmer climates. Northern Europe is an exception, not the norm.
617.
by lukev 3y ago
First of all, we're assuming that agents are (at least initially) designed intentionally, not as harmful viruses. Second, running a LLM inference (nevermind training!) involves lots and lots of system resources. Much more difficult to
618.
by lukev 3y ago
As someone who has worked/is working on building LLM driven agents, this allegory basically reads to me like: 1. I ask a LLM for paperclips. 2. ~Magic happens~. 3. Humanity is doomed. Even if a LLM was good enough to drive a system lik
619.
by lukev 3y ago
Perhaps, I don't discount the possibility that an LLM could be used as a primitive as part of a larger more complex system that could rightfully be called intelligent. But a script running a LLM in an OODA loop is a lot easier to monit
620.
by lukev 3y ago
This is a good discussion of concepts. However, a lot of the folks worried about superintelligence miss an important division within the concept of "intelligence." Specifically with respect to the current generation of LLMs. Namel
621.
by lukev 3y ago
Can we please not do this with every new chatbot out there? LLMs have no visibility into their own workings. The results of asking a LLM about itself are unreliable and honestly, uninteresting.
622.
by lukev 3y ago
Huh. Thanks for the heads up. Used to be a useful service but apparently the domain expired and was replaced by spam.
623.
by lukev 3y ago
I'm a polyglot developer who likes to think they're comfortable in a wide range of programming paradigms and languages across both frontend and backend. This is pretty much exactly my reaction any time I see a package.json longer
624.
by lukev 3y ago
I guess it depends what you're looking for. I travel mostly for work, and I want something above all easy and reliable. I'm expensing all my meals so restaurants aren't a problem. I hate the uncertainty of AirBNBs. They have
625.
by lukev 3y ago
I disagree. Perhaps there’s a case for Uber… having a decent app-based way to summon a ride is a net positive. But the experience is even better in European cities where Uber has successfully integrated with the local taxi system rather tha
626.
by lukev 3y ago
This just makes obvious sense. If someone wants a GPT-generated answer, they can use ChatGPT themselves. If I'm bothering to post my question to a forum , the social contract is that I'd like a response from a human.
627.
by lukev 3y ago
Technically untrue, approximately 7.31 percent of all humans who have ever lived are currently alive (i.e. have avoided death (so far)). Also your argument is just disingenuous. Nonexistence may not be painful, but the concept of nonexisten
628.
by lukev 3y ago
Sometimes things work out that way. But a new official version of a W3C spec is not just "tacking it on." If data or software claims to be compliant with some version of the relevant W3C spec, they almost always actually are. And
629.
by lukev 3y ago
Yes, it's an early draft. The "quoted triples" technique is otherwise known as "RDF-star" Here's an article that explains the motivations and alternatives pretty comprehensively: https://www.ontotext
630.
by lukev 3y ago
RDF and related technologies are heavily used in healthcare and scientific fields, as well as industry. The "semantic web" or "open linked data" concepts never really took off the way people had hoped, but there's t

More ›