4 ms·

I'm Not Consulting an LLM

53 points by birdculture 8mo ago

mark_l_watson 8mo ago
I understand the author’s sentiment but I would like to give a counter example: I like to read philosophy and after I read a passage and think about it, I find it useful to copy the passage into a decent model and ask for its interpretation, or if it is something old ask about word choice or meaning. I realize that I may not be getting perfect information, but LLM output gives me ideas that are a combination of live web searching and whatever innate knowledge the LLM holds in its weights. Another counter example: I have never found runtime error traces from languages like Haskell and Common Lisp to be that clear. If the error is not clear to me, sometimes using a model gets me past an error quickly. All that said, I think the author is right-on correct that using LLMs should not be an excuse to not think for oneself.
- XenophileJKO 8mo ago
  I mean it can also depend on scale. I use hundreds of sub-agent instances to do analysis that I just would not be able to do in a reasonable timeframe. That is a TON of thinking done for me.
  - 4ndrewl 8mo ago
    Is "not having to think" a good metric now?
    - mr_mitm 8mo ago
      I think it could be. It doesn't have to be one or the other. In my opinion it's entirely comparable to anything else that augments human capability. Is it a good thing that I can drive somewhere instead of walking? It can be. If driving 50 miles means I get there in an hour instead of two days, it can be a good thing, even though it could also be a bad thing if I replace all walking with driving. It just expands my horizon in the sense that I can now reach places I otherwise couldn't reasonably reach. Why can't it be the same with thinking? If the machine does some thinking for me, it can be helpful. That doesn't mean I should delegate all thinking to the machine.
    - rgoulter 8mo ago
      While "I don't have to think, I just get the LLM to do the task" is a bit careless (or a "hype" way of putting it)... I'd reckon it's always been true that you want to think about the stuff that matters and the other stuff to be done for minimal effort. e.g. By using a cryptography library / algorithm someone else has written, I don't need to think about it (although someone has done the thinking, I hope!). Or by using a high level language, I don't need to think about how to write assembly / machine code. Or with a tool like a spell-checker: since it checks the document, you don't have to worry about spelling mistakes. What upsets is the imbalance between "tasks which previously required some thought/effort can now be done effortlessly". -- Stuff like "write out a document" used to signal effort had been put into it.
    - XenophileJKO 8mo ago
      I guess, how often do you pay someone to fix your car? Repair something in your house? Give you financial advice? Those are all things many people outsource their thinking to other people with.
      - v3xro 8mo ago
        No, you outsource it because it's not your core competency. I think humans should be able to do anything and not narrowly specialise as narrow specialisation leads to tunnel vision. Sometimes you need to outsource to someone because of legal reasons (and rightly so, mostly because the complexities involved do require someone who is a professional in that area). Can some things be simplified? Of course they can, and there are many barriers that prevent such simplification. But it's absolutely insane to say - nah, we don't need to think at all, and something else can do all the work.
        mr_mitm 8mo ago
        Nobody said "we don't need to think at all" though. The statement was "not having to think", or rephrased: "being able to choose how much to think or what to think about".
  - hectormalot 8mo ago
    There’s both a quality and quantity angle. For some work, similar to the philosophy example of GP, LLMs can help with depth/quality. Is additive to your own thinking. -> quality approach For other things. I take a quantity approach. Having 8 subagents research, implement, review, improve, review (etc) a feature in a non critical part of our code, or investigate a bug together with some traces. It’s displacing my own thinking, but that’s ok, it makes up for it with the speed and amount of work it can do. —> quantity approach. It’s become mostly a matter of picking the right approach depending on the problem I’m trying to solve.
- delusional 8mo ago
  > I realize that I may not be getting perfect information, but LLM output gives me ideas that are a combination of live web searching and whatever innate knowledge the LLM holds in its weights. I don't mean to be judgemental. It's possible this is a personal observation, but I do wonder if it's not universal. I find that if I give an inch to these models thinking, I instantly become lazy. It doesn't really matter if they produce interesting output, but rather that I stop trying to produce interesting thoughts because I can't help wonder if the LLM wouldn't have output the same thing. I become TOO output focused. I mistake reading an interpretation for actually integrating knowledge into my own thinking, I disregard following along with the author. I love reading philosophy as well. Dialectic of Enlightenment profoundy shaped how I view the world, but there was not a single part of that book that I could have given you a coherent interpretation of as a read it. The interpretations all come now, years after I read it. I can't help but wonder if those interpretations would have been different, had my subcouncious been satiated by cheap explanations from the lie robot.
  - kolinko 8mo ago
    For me it's the opposite - sure, for many outputs I don't need to think, but then I end up thinking on a higher level, and doing even more work. An analogy would be - if GPS allows you to not worry about which turn to take, you can finally focus on where you want to get.
  - demorro 8mo ago
    Seconding this. Revelation happens subtly, often far removed from what you might later unpick as its "primary source". Immediate interpretation tend to be plastic and shallow.
    - sph 8mo ago
      Also it might be hard to grasp for most of us, used to constant stimulation and lack of space for contemplation and incorporation of information (I recommend the works of philosopher Byung-Chul Han on the matter) with yet unknown effects on our psyche and creative output. It takes days or weeks for one to sit and digest novel viewpoints; asking a machine to skip all that work for us is just another example of seeking instant gratification. I have no time to think, do it for me, so I can scroll to the next post already.
- carlosjobim 8mo ago
  Why would you even read philosophy if you're then consulting a third party for interpretation? That is the definition of meaningless. That is like listening to music and asking somebody if you liked a song.
  - tylervigen 8mo ago
    I would say it's more like enjoying a song so much that you choose to listen to a cover of that song.
  - therealdrag0 8mo ago
    Philosophy is often considered an activity of engagement with others in thought and discussion. I don’t see why an LLM can’t play a role there.
zaphirplane 8mo ago
I was hoping for a deeper article
NitpickLawyer 8mo ago
Yawn. Just a post++ about white-horse attitudes regarding "muh expertise". And yet the top of the top experts in their fields (Terrence Tao, Karpathy, hell even Linus) are finding ways to make them useful for them. That's the crux imo. If you can't find a way to make these tools useful for you, you are the problem. Not the LLMs. THere's something there, even if currently not much, but there's something there for everyone at this point.
- xyzal 8mo ago
  "If you can't find a way to make today's tech du jour useful for you, you are the problem". Hmmmmm.
  - throwaway0665 8mo ago
    This tech isn't going away anytime soon. It might become prohibitively expensive for individuals but it's here to stay. It's worth trying to find a use for it while it's cheap.
    - coatmatter 8mo ago
      "Motor vehicles aren't going away any time soon. It might become prohibitively expensive for some individuals but they're here to stay. It's worth trying to get into car ownership early despite your bicycle being much cheaper, healthier, and better for your local community."
  - lostmsu 8mo ago
    You omitted the part where he said why it is for AI. Tao doesn't use React AFAIK
- mr_mitm 8mo ago
  Even Knuth is starting to be impressed: https://www-cs-faculty.stanford.edu/~knuth/papers/claude-cycles.pdf https://www-cs-faculty.stanford.edu/~knuth/papers/claude-cyc...
simianwords 8mo ago
If people used LLMs more we would have fewer instances of misinformation. Lots of comments in social media could easily be dispelled by a single LLM search.
- chrysoprace 8mo ago
  LLMs can be injected with biases as well. Just look at Grok's responses any time it's tagged in anything mildly political.
  - simianwords 8mo ago
    That’s like saying journalism is not useful because journalists are biased. Bias is useful and inevitable
    - chrysoprace 8mo ago
      Bad journalists are biased. Good journalists will present a story as factually as possible and as void of bias as possible (of course it's impossible to not have any biases). Opinion pieces can have as much bias as they like as long as they're strictly marked as opinions.
      - delusional 8mo ago
        That's not true. Any journalist would tell you that picking the stories you chose to cover is just as much a bias as how you chose to cover them. Even then, the specific words you pick, how you ask the interviewees, how you place the story on the page, what you pick as the "related stories". All of that is Editorial and reflects an opinion. Good journalists are open about their angle. Bad journalists tell you they are "unbiased" and "just bringing you the facts".
        chrysoprace 8mo ago
        In the case of publications we luckily have such fantastic resources as mediabiasfactcheck[0] to keep their bias in check and to keep them factual. LLMs are much harder to fact-check because they can make anything up based on their training data and weights without sources. [0] https://mediabiasfactcheck.com https://mediabiasfactcheck.com
tony_codes 8mo ago
speaking of chance discovery, what a great personal website! I love the daily virtues from the diary section.
jstanley 8mo ago
> “I’m Feeling Lucky” intelligence is optimized for arrival, not for becoming. You get the answer but nothing else (keep in mind we are assuming that it's a good answer). You don’t learn how ideas fight, mutate, or die. You don’t develop a sense for epistemic smell or the ability to feel when something is off before you can formally prove it. All you're saying is that you can't imagine working on a task that is longer than 1 Google Search. If "I'm feeling lucky" works by magic, that doesn't mean your life is free of all searching, it just means you get the answer to each Google Search in fewer steps, which means the overall complexity of tasks that you can handle goes up. That's good! It doesn't mean you miss out on the journey of learning and being confused, it just means you're learning and being confused about more complicated things.
test001only 8mo ago
> you had to build a model of the world just to survive the tension? The world the author is describing currently has LLM in it. Irrespective of the author liking it or not, it is here to stay. So to build a model of the world, you would still need to consult an LLM, understand how it can give plausible looking answers, learn how to effectively leverage the tool and make it part of your toolkit. It does not mean you stop reading manuals, books or blogs. It just means you include LLM also in those list of things.
perks_12 8mo ago
I google a lot (or rather, Kagi). I loved to explore the web when I was younger. But over time I lost any interest in trying to gather informational bits from increasingly shittier websites designed to have more ads and hide relevant information for as many ad slots as possible. These days I hit the quick answer button inside Kagi more often and just accept that I might have some false information in there. If it is critical to be right, I usually consult primary sources directly anyway.
- siquick 8mo ago
  > These days I hit the quick answer button inside Kagi more often Just incase you didn’t know you can append ? to any query and get a quick answer straight away
  - perks_12 8mo ago
    Actually, I didn't know. Thanks!
po1nt 8mo ago
I don't think the argument is correct. Reasoning LLM will check itself and search multiple sources. It's essentially doing the same mental process as human would. Also consulting multiple LLMs completely breaks this argument.
- heeton 8mo ago
  Author’s central point is that an LLM answer “is optimized for arrival, not for becoming” (to paraphrase from the Google “Lucky” part). So a reasoning LLM that does the comparisons and checks “like a human” still fails the author’s test. That said, this still feels like a skill issue. If you want to learn, see opposing views gather evidence to form your own opinions about, LLMs can still help massively. You just have to treat them research assistants instead of answer providers.
- bsza 8mo ago
  IME, even when an LLM is right, a few follow-up questions always lead to some baffling cracks in its reasoning that expose it has absolutely no idea what it's talking about. Not just about the subject but basic common sense. I definitely wouldn't call it the "same mental process" a human does. It is an alien intelligence, and exposing a human mind to it won't necessarily lead to the same (or better) outcome as learning from other humans would.
- em-bee 8mo ago
  but the point is that the metal process should be done by yourself. it is the difference between finding the answer myself or asking my classmate to just share his answer with me. in the latter case i am not learning what my classmate learned.
antonvs 8mo ago
To me this reads like “I don’t want to be able to learn faster.” The downside of the internet is that we get to see people agonizing over their inability to adapt to change.
- stavros 8mo ago
  Agreed, all this "but if you don't need certain skills any more, you'll lose them!" is tiring, and even more tiring because it's missing the entire point: yes, because I don't need them any more! It feels like I'm reading an article crying "if you buy a car, you will lose your horse-shoeing skills!" every day lately.
ChrisMarshallNY 8mo ago
For myself, I’m very much a “results” guy. Have been, for all my career. I’ve been shipping (as opposed to “writing”) software for most of my adult life. People seem to like to stuff I make. I’m currently working on my first major project that incorporates heavy LLM contributions. It’s coming along great. I started with Machine Code and individual gate ICs, so my knowledge goes way down past the roots. I don’t miss it, at all. Occasionally, my understanding of stuff has been helped by that depth of experience, but, for the most part, it’s been irrelevant. It’s a first-stage booster, dropping back into the atmosphere. I will say that my original training as a bench tech has been very useful, as I’m good at finding and fixing bugs, but a lot of my experience is in the rear-view mirror. I have been routinely googling even the most basic stuff, for many years. It hasn’t corroded my intellect (yet), and I’m doing the same kind of thing with an LLM. Not being sneered at by some insecure kid is nice.
- jmfldn 8mo ago
  I'm somewhere in between. I'm excited about building more things faster and extending my capabilities. But I also love thinking about the underlying language, runtime, algorithms, the wider system. I want LLMs to enhance this for me, I want my understanding to go up as I write less code. It's also key to my job as a lead that I maintain understanding of the system for debugging, security etc. So if I can do both with these tools, then great. I want to cognitively offload in a way that allows me to focus on the important bits. And I'm writing instructions to the LLM to help me do that eg 'help teach me this bit'. A builder and tutor at once.
  - ChrisMarshallNY 8mo ago
    Great approach. I usually use ChatGPT, as a chat (as opposed to an agent). It explains everything quite well (if sometimes a bit verbose). Today, I am going to do an experiment: I’ll be asking it to rewrite the headerdocs in one of my files (I tend to have about a 50% comment/code ratio), so it generates effective docc documentation. I suspect the result will be good.
- v3xro 8mo ago
  > Not being sneered at by some insecure kid is nice. How very adult of you.
cmiles8 8mo ago
I see the SDE pushback on LLMs, but most of it is unfounded. Like any new tool, if used irresponsibly of course bad things can happen. Most of the backlash from devs seems rooted in: 1. It causes a step change in productivity for those that use it well, and as a result a step change in the expectations on productivity for dev teams. Folks simply expect things to be done faster now and that’s annoying folks that have to do the building. 2. It’s removed much of the mystique of dev. When the CEO is vibe coding legit apps on their own suddenly the SDE team is no longer this mysterious oracle that one can’t challenge or question because nobody else can do what they do. Now everyone can do what they do. Not to the same degree, yes, but it’s completely changed the dynamic and that’s just annoying some devs. SDEs aren’t going away, but we will likely need less moving forward and the expectations on how long things take have changed forever. Like anything in tech we’re not going back to the old way so you either evolve or you get cycled out. I also hear the “but LLMs only produce unrecognizable junk that one can’t maintain” angle, but that implies dev teams have been shipping beautiful artwork. Truth is most dev teams have been shipping undocumented fragile junk for years. While LLMs occasionally do odd things, in my experience LLM code is actually better documented and structured than what most dev teams produce and because of that it’s actually easier to hand off to others vs the typical codebase full of hacky workarounds and half-completed documentation.
- disinz 8mo ago
  > in my experience LLM code is actually better documented Most LLM documentation is extremely poor unless you literally can’t understand code. It’s communicating obvious things that the code makes apparent. Good documentation covers what’s not obvious from reading the code. I’m amazed at how many devs think running Claude init is a good idea. I’ve noticed a constant trend throughout my career that low performing devs generate and are impressed with this kind of documentation. > and structured than what most dev teams produce Left alone, LLM structure is terrible. Speed running the cheapest outsourced slop is not a good thing unless you’re already paying for outsourced slop. Huge win in that case. LLMs are median equalizers. I think a large contingent of devs (especially given the money grab around 2021) are not that good, and LLMs really are a step improvement for them. I’m not denying there are use cases for competent devs, but so far that seems much narrower (albeit insanely impressive) than the benefit low performers (think) they receive. This fact is important to keep in mind when discussing the future of dev.
kolinko 8mo ago
I think the author just doesn't know how to use LLMs well. "Because what would be missing isn’t information but the experience. And experience is where intellect actually gets trained." From my experience, LLMs don't cause this effect. You still get to explore a ton of dead ends and whatnot, just on a much higher level. "You get the answer but nothing else (keep in mind we are assuming that it's a good answer)." On the contrary here - you get to answer a ton of followup questions easily, something you don't get to ask books. "I never so far asked GPT about something that I'm specialized at, and it gave me a sufficient answer that I would expect from someone who is as much as expert as me in that given field." LLMs are at a level of junior-mid of any field (and going higher every year), not senior-master. Is that anything new? Their strength is, among other things, in making connections between fields, and also the availability. If you have an option to talk to a specialist in your field that has time 24/7 to discuss ideas with you, that's great, but also highly unusual. If you don't have such a person, an LLM that is junior-mid is way better than plain books.
kator 8mo ago
The "calculator ruined the world" argument was actually studied to death once the panic subsided. Large meta-analyses of 50 years of data show it was mostly a non-problem. Students using calculators generally developed better attitudes toward math and attempted more complex problems because the mechanical drudgery was gone. The only real "catch" researchers found was timing. If you introduce them before a kid has "automaticity" (around 4th grade), they never develop a baseline number sense, which makes high-level math harder later on. It's a pretty clean parallel for LLMs. The tool isn't the problem, but using it to bypass the "becoming" phase of a skill usually backfires. If you use an LLM before you know how to structure an argument or a block of code, you're just building on sand.
- coatmatter 8mo ago
  You're using the motte-and-bailey fallacy. Pocket and scientific calculators have never caused anywhere near as much grief as current LLMs do as a loss-leader for near-trillion dollar corporations that are primarily responsible for artificially keeping the US economy afloat (at least prior to the latest war/not-war effort). Calculators have never led to anything like AI-induced psychosis: https://theconversation.com/ai-induced-psychosis-the-danger-of-humans-and-machines-hallucinating-together-269850 https://theconversation.com/ai-induced-psychosis-the-danger-... That's merely one example of a long list of very real unresolved problems. It's not a "pretty clean parallel" in the slightest.
rsfern 8mo ago
I like this analogy of always choosing “I’m feeling lucky” on Google, I feel like it clarifies a boundary between information retrieval and evaluation that gets blurred by language model summarizations. I’ve been frustrated with the LLM summary at the top of the Google search results for scientific topics because often the sources linked to don’t actually contain the information the summary is citing them for. Then I have a side quest of finding the right backing literature or deciding the summary was just wrong in the first place
maplethorpe 8mo ago
> A tool can be efficient and still be intellectually corrosive, not because it lies all the time, but because it lies well enough. Its smoothness hides uncertainty, which is important unless you want intellect-rot. I keep seeing sentiments like this, but to me they're still very much stuck in the past. We once needed to develop our intellects in order to solve problems. It was a necessary part of the process. Solving a particular problem would flood our brains with dopamine, and we would feel good for achieving our goal, and thus continue to develop our intellect in the hope of achieving a similar rush in the future. Now that we have a machine that can solve our problems for us, intellect just plain isn't necessary anymore. We can solve our problems immediately and skip the roundabout intellect-building process entirely. That's a liberating thing.
- hexaga 8mo ago
  is this satire
carrychains 8mo ago
Same can be said of search engines, encyclopedias, or wikis compared to seeking out books, journals, and other source material. If you don't sit there for 8 hours in a library to find the same information on your own, you've missed out on the experience. It's a standard Luddite's argument. Tools of any kind that enhance efficiency have always actualized lazy outcomes. It has always been the human responsibility to, not only rely on their best effort, but to figure out what actually encompasses their best possible effort.
- readthenotes1 8mo ago
  I have a relative who claims, because hen is dyslexic, that hen thinks better and deeper than any normal person due to the inability to learn by reading since that is so, to the point where hen claims not to be subject to any of the well-known cognitive biases.
- em-bee 8mo ago
  but that is being addressed. there is a difference between looking at 10 search results vs clicking "i am feeling lucky". asking an LLM always just gives you one answer without an opportunity to research and compare multiple possible answers. a search engine always gives multiple results and it is on me to read and compare. i usually look at the first half a dozen or so results. a well written wikipedia article will represent multiple views and critical opinions if there are any. but if there aren't then you may want to expand your search. same goes for other encyclopedias or articles you are reading. the problem, again, is that an LLM answer does not have the quality of a well written article you may find elsewhere. that may change in the future, but until then i find LLMs are a waste of time for me.
  - pod_krad 8mo ago
    >a search engine always gives multiple results and it is on me to read and compare. i usually look at the first half a dozen or so results Why do you think they're the best? You just trust the search engine, right? >the problem, again, is that an LLM answer does not have the quality of a well written article you may find elsewhere When I ask LLM about something, it usually returns several URLs usually, if there are different opinions.
codexetreme 8mo ago
I personally consult LLMs over technical stuff. But for the more nuanced "human" side of things, I feel it gives you answers from both sides For instance, I wanted to know if it's better to setup a call with a client in a coffee shop over their office. Almost all LLMs will give you whatever answer you want based on how much stress you put on one particular answer. I'm not saying that traditional search engines will show you only one sided results, but you'll see reddit threads that will have a bit of conversation before arriving at some conclusion. You'd read it and go hmm this question had XYZ setup and they picked the coffee shop as a great place. My own situation is perhaps (for sake of argument) not XYZ, so it's better to have the meeting in the client office. This is a community driven decision where I have certain basis if I want to revisit my decision. With an llm I find that it will happily just give you one of the two options and keep switching in the middle if you ask it to. That's what makes me a bit a skeptical. With technical stuff, there are direct solutions that fit your usecase and you can just use it and move on.

53 points by birdculture 8mo ago

I understand the author’s sentiment but I would like to give a counter example: I like to read philosophy and after I read a passage and think about it, I find it useful to copy the passage into a decent model and ask for its interpretation, or if it is something old ask about word choice or meaning. I realize that I may not be getting perfect information, but LLM output gives me ideas that are a combination of live web searching and whatever innate knowledge the LLM holds in its weights. Another counter example: I have never found runtime error traces from languages like Haskell and Common Lisp to be that clear. If the error is not clear to me, sometimes using a model gets me past an error quickly. All that said, I think the author is right-on correct that using LLMs should not be an excuse to not think for oneself.

XenophileJKO 8mo ago
I mean it can also depend on scale. I use hundreds of sub-agent instances to do analysis that I just would not be able to do in a reasonable timeframe. That is a TON of thinking done for me.
- 4ndrewl 8mo ago
  Is "not having to think" a good metric now?
  - mr_mitm 8mo ago
    I think it could be. It doesn't have to be one or the other. In my opinion it's entirely comparable to anything else that augments human capability. Is it a good thing that I can drive somewhere instead of walking? It can be. If driving 50 miles means I get there in an hour instead of two days, it can be a good thing, even though it could also be a bad thing if I replace all walking with driving. It just expands my horizon in the sense that I can now reach places I otherwise couldn't reasonably reach. Why can't it be the same with thinking? If the machine does some thinking for me, it can be helpful. That doesn't mean I should delegate all thinking to the machine.
  - rgoulter 8mo ago
    While "I don't have to think, I just get the LLM to do the task" is a bit careless (or a "hype" way of putting it)... I'd reckon it's always been true that you want to think about the stuff that matters and the other stuff to be done for minimal effort. e.g. By using a cryptography library / algorithm someone else has written, I don't need to think about it (although someone has done the thinking, I hope!). Or by using a high level language, I don't need to think about how to write assembly / machine code. Or with a tool like a spell-checker: since it checks the document, you don't have to worry about spelling mistakes. What upsets is the imbalance between "tasks which previously required some thought/effort can now be done effortlessly". -- Stuff like "write out a document" used to signal effort had been put into it.
  - XenophileJKO 8mo ago
    I guess, how often do you pay someone to fix your car? Repair something in your house? Give you financial advice? Those are all things many people outsource their thinking to other people with.
    - v3xro 8mo ago
      No, you outsource it because it's not your core competency. I think humans should be able to do anything and not narrowly specialise as narrow specialisation leads to tunnel vision. Sometimes you need to outsource to someone because of legal reasons (and rightly so, mostly because the complexities involved do require someone who is a professional in that area). Can some things be simplified? Of course they can, and there are many barriers that prevent such simplification. But it's absolutely insane to say - nah, we don't need to think at all, and something else can do all the work.
      - mr_mitm 8mo ago
        Nobody said "we don't need to think at all" though. The statement was "not having to think", or rephrased: "being able to choose how much to think or what to think about".
- hectormalot 8mo ago
  There’s both a quality and quantity angle. For some work, similar to the philosophy example of GP, LLMs can help with depth/quality. Is additive to your own thinking. -> quality approach For other things. I take a quantity approach. Having 8 subagents research, implement, review, improve, review (etc) a feature in a non critical part of our code, or investigate a bug together with some traces. It’s displacing my own thinking, but that’s ok, it makes up for it with the speed and amount of work it can do. —> quantity approach. It’s become mostly a matter of picking the right approach depending on the problem I’m trying to solve.
delusional 8mo ago
> I realize that I may not be getting perfect information, but LLM output gives me ideas that are a combination of live web searching and whatever innate knowledge the LLM holds in its weights. I don't mean to be judgemental. It's possible this is a personal observation, but I do wonder if it's not universal. I find that if I give an inch to these models thinking, I instantly become lazy. It doesn't really matter if they produce interesting output, but rather that I stop trying to produce interesting thoughts because I can't help wonder if the LLM wouldn't have output the same thing. I become TOO output focused. I mistake reading an interpretation for actually integrating knowledge into my own thinking, I disregard following along with the author. I love reading philosophy as well. Dialectic of Enlightenment profoundy shaped how I view the world, but there was not a single part of that book that I could have given you a coherent interpretation of as a read it. The interpretations all come now, years after I read it. I can't help but wonder if those interpretations would have been different, had my subcouncious been satiated by cheap explanations from the lie robot.
- kolinko 8mo ago
  For me it's the opposite - sure, for many outputs I don't need to think, but then I end up thinking on a higher level, and doing even more work. An analogy would be - if GPS allows you to not worry about which turn to take, you can finally focus on where you want to get.
- demorro 8mo ago
  Seconding this. Revelation happens subtly, often far removed from what you might later unpick as its "primary source". Immediate interpretation tend to be plastic and shallow.
  - sph 8mo ago
    Also it might be hard to grasp for most of us, used to constant stimulation and lack of space for contemplation and incorporation of information (I recommend the works of philosopher Byung-Chul Han on the matter) with yet unknown effects on our psyche and creative output. It takes days or weeks for one to sit and digest novel viewpoints; asking a machine to skip all that work for us is just another example of seeking instant gratification. I have no time to think, do it for me, so I can scroll to the next post already.
carlosjobim 8mo ago
Why would you even read philosophy if you're then consulting a third party for interpretation? That is the definition of meaningless. That is like listening to music and asking somebody if you liked a song.
- tylervigen 8mo ago
  I would say it's more like enjoying a song so much that you choose to listen to a cover of that song.
- therealdrag0 8mo ago
  Philosophy is often considered an activity of engagement with others in thought and discussion. I don’t see why an LLM can’t play a role there.

zaphirplane 8mo ago

I was hoping for a deeper article

NitpickLawyer 8mo ago

Yawn. Just a post++ about white-horse attitudes regarding "muh expertise". And yet the top of the top experts in their fields (Terrence Tao, Karpathy, hell even Linus) are finding ways to make them useful for them. That's the crux imo. If you can't find a way to make these tools useful for you, you are the problem. Not the LLMs. THere's something there, even if currently not much, but there's something there for everyone at this point.

xyzal 8mo ago
"If you can't find a way to make today's tech du jour useful for you, you are the problem". Hmmmmm.
- throwaway0665 8mo ago
  This tech isn't going away anytime soon. It might become prohibitively expensive for individuals but it's here to stay. It's worth trying to find a use for it while it's cheap.
  - coatmatter 8mo ago
    "Motor vehicles aren't going away any time soon. It might become prohibitively expensive for some individuals but they're here to stay. It's worth trying to get into car ownership early despite your bicycle being much cheaper, healthier, and better for your local community."
- lostmsu 8mo ago
  You omitted the part where he said why it is for AI. Tao doesn't use React AFAIK
mr_mitm 8mo ago
Even Knuth is starting to be impressed: https://www-cs-faculty.stanford.edu/~knuth/papers/claude-cycles.pdf https://www-cs-faculty.stanford.edu/~knuth/papers/claude-cyc...

simianwords 8mo ago

If people used LLMs more we would have fewer instances of misinformation. Lots of comments in social media could easily be dispelled by a single LLM search.

chrysoprace 8mo ago
LLMs can be injected with biases as well. Just look at Grok's responses any time it's tagged in anything mildly political.
- simianwords 8mo ago
  That’s like saying journalism is not useful because journalists are biased. Bias is useful and inevitable
  - chrysoprace 8mo ago
    Bad journalists are biased. Good journalists will present a story as factually as possible and as void of bias as possible (of course it's impossible to not have any biases). Opinion pieces can have as much bias as they like as long as they're strictly marked as opinions.
    - delusional 8mo ago
      That's not true. Any journalist would tell you that picking the stories you chose to cover is just as much a bias as how you chose to cover them. Even then, the specific words you pick, how you ask the interviewees, how you place the story on the page, what you pick as the "related stories". All of that is Editorial and reflects an opinion. Good journalists are open about their angle. Bad journalists tell you they are "unbiased" and "just bringing you the facts".
      - chrysoprace 8mo ago
        In the case of publications we luckily have such fantastic resources as mediabiasfactcheck[0] to keep their bias in check and to keep them factual. LLMs are much harder to fact-check because they can make anything up based on their training data and weights without sources. [0] https://mediabiasfactcheck.com https://mediabiasfactcheck.com

tony_codes 8mo ago

speaking of chance discovery, what a great personal website! I love the daily virtues from the diary section.

jstanley 8mo ago

> “I’m Feeling Lucky” intelligence is optimized for arrival, not for becoming. You get the answer but nothing else (keep in mind we are assuming that it's a good answer). You don’t learn how ideas fight, mutate, or die. You don’t develop a sense for epistemic smell or the ability to feel when something is off before you can formally prove it. All you're saying is that you can't imagine working on a task that is longer than 1 Google Search. If "I'm feeling lucky" works by magic, that doesn't mean your life is free of all searching, it just means you get the answer to each Google Search in fewer steps, which means the overall complexity of tasks that you can handle goes up. That's good! It doesn't mean you miss out on the journey of learning and being confused, it just means you're learning and being confused about more complicated things.

test001only 8mo ago

> you had to build a model of the world just to survive the tension? The world the author is describing currently has LLM in it. Irrespective of the author liking it or not, it is here to stay. So to build a model of the world, you would still need to consult an LLM, understand how it can give plausible looking answers, learn how to effectively leverage the tool and make it part of your toolkit. It does not mean you stop reading manuals, books or blogs. It just means you include LLM also in those list of things.

perks_12 8mo ago

I google a lot (or rather, Kagi). I loved to explore the web when I was younger. But over time I lost any interest in trying to gather informational bits from increasingly shittier websites designed to have more ads and hide relevant information for as many ad slots as possible. These days I hit the quick answer button inside Kagi more often and just accept that I might have some false information in there. If it is critical to be right, I usually consult primary sources directly anyway.

siquick 8mo ago
> These days I hit the quick answer button inside Kagi more often Just incase you didn’t know you can append ? to any query and get a quick answer straight away
- perks_12 8mo ago
  Actually, I didn't know. Thanks!

po1nt 8mo ago

I don't think the argument is correct. Reasoning LLM will check itself and search multiple sources. It's essentially doing the same mental process as human would. Also consulting multiple LLMs completely breaks this argument.

heeton 8mo ago
Author’s central point is that an LLM answer “is optimized for arrival, not for becoming” (to paraphrase from the Google “Lucky” part). So a reasoning LLM that does the comparisons and checks “like a human” still fails the author’s test. That said, this still feels like a skill issue. If you want to learn, see opposing views gather evidence to form your own opinions about, LLMs can still help massively. You just have to treat them research assistants instead of answer providers.
bsza 8mo ago
IME, even when an LLM is right, a few follow-up questions always lead to some baffling cracks in its reasoning that expose it has absolutely no idea what it's talking about. Not just about the subject but basic common sense. I definitely wouldn't call it the "same mental process" a human does. It is an alien intelligence, and exposing a human mind to it won't necessarily lead to the same (or better) outcome as learning from other humans would.
em-bee 8mo ago
but the point is that the metal process should be done by yourself. it is the difference between finding the answer myself or asking my classmate to just share his answer with me. in the latter case i am not learning what my classmate learned.

antonvs 8mo ago

To me this reads like “I don’t want to be able to learn faster.” The downside of the internet is that we get to see people agonizing over their inability to adapt to change.

stavros 8mo ago
Agreed, all this "but if you don't need certain skills any more, you'll lose them!" is tiring, and even more tiring because it's missing the entire point: yes, because I don't need them any more! It feels like I'm reading an article crying "if you buy a car, you will lose your horse-shoeing skills!" every day lately.

ChrisMarshallNY 8mo ago

For myself, I’m very much a “results” guy. Have been, for all my career. I’ve been shipping (as opposed to “writing”) software for most of my adult life. People seem to like to stuff I make. I’m currently working on my first major project that incorporates heavy LLM contributions. It’s coming along great. I started with Machine Code and individual gate ICs, so my knowledge goes way down past the roots. I don’t miss it, at all. Occasionally, my understanding of stuff has been helped by that depth of experience, but, for the most part, it’s been irrelevant. It’s a first-stage booster, dropping back into the atmosphere. I will say that my original training as a bench tech has been very useful, as I’m good at finding and fixing bugs, but a lot of my experience is in the rear-view mirror. I have been routinely googling even the most basic stuff, for many years. It hasn’t corroded my intellect (yet), and I’m doing the same kind of thing with an LLM. Not being sneered at by some insecure kid is nice.

jmfldn 8mo ago
I'm somewhere in between. I'm excited about building more things faster and extending my capabilities. But I also love thinking about the underlying language, runtime, algorithms, the wider system. I want LLMs to enhance this for me, I want my understanding to go up as I write less code. It's also key to my job as a lead that I maintain understanding of the system for debugging, security etc. So if I can do both with these tools, then great. I want to cognitively offload in a way that allows me to focus on the important bits. And I'm writing instructions to the LLM to help me do that eg 'help teach me this bit'. A builder and tutor at once.
- ChrisMarshallNY 8mo ago
  Great approach. I usually use ChatGPT, as a chat (as opposed to an agent). It explains everything quite well (if sometimes a bit verbose). Today, I am going to do an experiment: I’ll be asking it to rewrite the headerdocs in one of my files (I tend to have about a 50% comment/code ratio), so it generates effective docc documentation. I suspect the result will be good.
v3xro 8mo ago
> Not being sneered at by some insecure kid is nice. How very adult of you.

cmiles8 8mo ago

I see the SDE pushback on LLMs, but most of it is unfounded. Like any new tool, if used irresponsibly of course bad things can happen. Most of the backlash from devs seems rooted in: 1. It causes a step change in productivity for those that use it well, and as a result a step change in the expectations on productivity for dev teams. Folks simply expect things to be done faster now and that’s annoying folks that have to do the building. 2. It’s removed much of the mystique of dev. When the CEO is vibe coding legit apps on their own suddenly the SDE team is no longer this mysterious oracle that one can’t challenge or question because nobody else can do what they do. Now everyone can do what they do. Not to the same degree, yes, but it’s completely changed the dynamic and that’s just annoying some devs. SDEs aren’t going away, but we will likely need less moving forward and the expectations on how long things take have changed forever. Like anything in tech we’re not going back to the old way so you either evolve or you get cycled out. I also hear the “but LLMs only produce unrecognizable junk that one can’t maintain” angle, but that implies dev teams have been shipping beautiful artwork. Truth is most dev teams have been shipping undocumented fragile junk for years. While LLMs occasionally do odd things, in my experience LLM code is actually better documented and structured than what most dev teams produce and because of that it’s actually easier to hand off to others vs the typical codebase full of hacky workarounds and half-completed documentation.

disinz 8mo ago
> in my experience LLM code is actually better documented Most LLM documentation is extremely poor unless you literally can’t understand code. It’s communicating obvious things that the code makes apparent. Good documentation covers what’s not obvious from reading the code. I’m amazed at how many devs think running Claude init is a good idea. I’ve noticed a constant trend throughout my career that low performing devs generate and are impressed with this kind of documentation. > and structured than what most dev teams produce Left alone, LLM structure is terrible. Speed running the cheapest outsourced slop is not a good thing unless you’re already paying for outsourced slop. Huge win in that case. LLMs are median equalizers. I think a large contingent of devs (especially given the money grab around 2021) are not that good, and LLMs really are a step improvement for them. I’m not denying there are use cases for competent devs, but so far that seems much narrower (albeit insanely impressive) than the benefit low performers (think) they receive. This fact is important to keep in mind when discussing the future of dev.

kolinko 8mo ago

I think the author just doesn't know how to use LLMs well. "Because what would be missing isn’t information but the experience. And experience is where intellect actually gets trained." From my experience, LLMs don't cause this effect. You still get to explore a ton of dead ends and whatnot, just on a much higher level. "You get the answer but nothing else (keep in mind we are assuming that it's a good answer)." On the contrary here - you get to answer a ton of followup questions easily, something you don't get to ask books. "I never so far asked GPT about something that I'm specialized at, and it gave me a sufficient answer that I would expect from someone who is as much as expert as me in that given field." LLMs are at a level of junior-mid of any field (and going higher every year), not senior-master. Is that anything new? Their strength is, among other things, in making connections between fields, and also the availability. If you have an option to talk to a specialist in your field that has time 24/7 to discuss ideas with you, that's great, but also highly unusual. If you don't have such a person, an LLM that is junior-mid is way better than plain books.

kator 8mo ago

The "calculator ruined the world" argument was actually studied to death once the panic subsided. Large meta-analyses of 50 years of data show it was mostly a non-problem. Students using calculators generally developed better attitudes toward math and attempted more complex problems because the mechanical drudgery was gone. The only real "catch" researchers found was timing. If you introduce them before a kid has "automaticity" (around 4th grade), they never develop a baseline number sense, which makes high-level math harder later on. It's a pretty clean parallel for LLMs. The tool isn't the problem, but using it to bypass the "becoming" phase of a skill usually backfires. If you use an LLM before you know how to structure an argument or a block of code, you're just building on sand.

coatmatter 8mo ago
You're using the motte-and-bailey fallacy. Pocket and scientific calculators have never caused anywhere near as much grief as current LLMs do as a loss-leader for near-trillion dollar corporations that are primarily responsible for artificially keeping the US economy afloat (at least prior to the latest war/not-war effort). Calculators have never led to anything like AI-induced psychosis: https://theconversation.com/ai-induced-psychosis-the-danger-of-humans-and-machines-hallucinating-together-269850 https://theconversation.com/ai-induced-psychosis-the-danger-... That's merely one example of a long list of very real unresolved problems. It's not a "pretty clean parallel" in the slightest.

rsfern 8mo ago

I like this analogy of always choosing “I’m feeling lucky” on Google, I feel like it clarifies a boundary between information retrieval and evaluation that gets blurred by language model summarizations. I’ve been frustrated with the LLM summary at the top of the Google search results for scientific topics because often the sources linked to don’t actually contain the information the summary is citing them for. Then I have a side quest of finding the right backing literature or deciding the summary was just wrong in the first place

maplethorpe 8mo ago

> A tool can be efficient and still be intellectually corrosive, not because it lies all the time, but because it lies well enough. Its smoothness hides uncertainty, which is important unless you want intellect-rot. I keep seeing sentiments like this, but to me they're still very much stuck in the past. We once needed to develop our intellects in order to solve problems. It was a necessary part of the process. Solving a particular problem would flood our brains with dopamine, and we would feel good for achieving our goal, and thus continue to develop our intellect in the hope of achieving a similar rush in the future. Now that we have a machine that can solve our problems for us, intellect just plain isn't necessary anymore. We can solve our problems immediately and skip the roundabout intellect-building process entirely. That's a liberating thing.

hexaga 8mo ago
is this satire

carrychains 8mo ago

Same can be said of search engines, encyclopedias, or wikis compared to seeking out books, journals, and other source material. If you don't sit there for 8 hours in a library to find the same information on your own, you've missed out on the experience. It's a standard Luddite's argument. Tools of any kind that enhance efficiency have always actualized lazy outcomes. It has always been the human responsibility to, not only rely on their best effort, but to figure out what actually encompasses their best possible effort.

readthenotes1 8mo ago
I have a relative who claims, because hen is dyslexic, that hen thinks better and deeper than any normal person due to the inability to learn by reading since that is so, to the point where hen claims not to be subject to any of the well-known cognitive biases.
em-bee 8mo ago
but that is being addressed. there is a difference between looking at 10 search results vs clicking "i am feeling lucky". asking an LLM always just gives you one answer without an opportunity to research and compare multiple possible answers. a search engine always gives multiple results and it is on me to read and compare. i usually look at the first half a dozen or so results. a well written wikipedia article will represent multiple views and critical opinions if there are any. but if there aren't then you may want to expand your search. same goes for other encyclopedias or articles you are reading. the problem, again, is that an LLM answer does not have the quality of a well written article you may find elsewhere. that may change in the future, but until then i find LLMs are a waste of time for me.
- pod_krad 8mo ago
  >a search engine always gives multiple results and it is on me to read and compare. i usually look at the first half a dozen or so results Why do you think they're the best? You just trust the search engine, right? >the problem, again, is that an LLM answer does not have the quality of a well written article you may find elsewhere When I ask LLM about something, it usually returns several URLs usually, if there are different opinions.

codexetreme 8mo ago

I personally consult LLMs over technical stuff. But for the more nuanced "human" side of things, I feel it gives you answers from both sides For instance, I wanted to know if it's better to setup a call with a client in a coffee shop over their office. Almost all LLMs will give you whatever answer you want based on how much stress you put on one particular answer. I'm not saying that traditional search engines will show you only one sided results, but you'll see reddit threads that will have a bit of conversation before arriving at some conclusion. You'd read it and go hmm this question had XYZ setup and they picked the coffee shop as a great place. My own situation is perhaps (for sake of argument) not XYZ, so it's better to have the meeting in the client office. This is a community driven decision where I have certain basis if I want to revisit my decision. With an llm I find that it will happily just give you one of the two options and keep switching in the middle if you ask it to. That's what makes me a bit a skeptical. With technical stuff, there are direct solutions that fit your usecase and you can just use it and move on.