7 ms·

As Google has been unable to keep spammy crap out of their search index since at least 2006 when we were doing Blekko I doubt they will have much success fighti

As Google has been unable to keep spammy crap out of their search index since at least 2006 when we were doing Blekko I doubt they will have much success fighting this. But it is another good example that "AI" is just glorified search and there is not reasoning or thinking going on behind the covers.

K0balt 5mo ago
Hmm. I don’t think that novel code generation can be accounted for with glorified search. I can have my agentic system read a few data sheets, then I explain the project requirements and have it design driver specifications, protocols, interfaces, and state machines. Taking those, develop an implementation plan. Working from that, write the skeleton of the application, then fill it in to create a functional system using a novel combination of hardware. Done correctly, I end up with better, more maintainable, smaller code than I used to with a small team, at 1/100 the cost and 1/4 the time. Whatever that is, it more closely resembles reasoning than search. Unless, of course, you’d also call bare metal C development on novel hardware search, in which case I guess all dev is search?
- Raphael_Amiard 5mo ago
  It’s pattern matching. A big part of reasoning for sure, but not reasoning per se
  - K0balt 5mo ago
    That could be, but if that is the case than development apparently doesn’t require reasoning? Or maybe that’s the part that the senior developer supervising the pipeline injects. Thats certainly a plausible position.
    - freejazz 5mo ago
      >but if that is the case than development apparently doesn’t require reasoning? Certainly plenty of it does not.
      - K0balt 5mo ago
        Ctrlc stack exchange lol
- rkozik1989 5mo ago
  How do you even know those numbers are correct? Realistically for what you've described you need more QA time that a traditional application to ensure its actually working properly. Especially with regards to any part of the application that deals with LLM inference. Its not hard to write unique content for niche topics where there are few relevant results and have LLMs take it as fact. For example, I poisoned the well for research on early Arab Americans immigrants by repeatedly posting about how many family passed as different ethnicity to make their lives easier, so now if you ask LLMs about that subject it'll include information I wrote which isn't entirely correct because I hadn't figured everything out before the LLM trained on it. EDIT: Now imagine if I had done this on an obscure programming-related problem, yeah? I could potentially make the LLM reference packages that do not actually exist and put backdoors in applications.
  - K0balt 5mo ago
    Because I have 100 percent test coverage (of the software, some hardware edge cases pop up that aren’t documented in the data sheets), and over 10k hours of field deployment over 130 devices? This rollout has been much more bug free than any we have done in the last six years, and it’s the first that has been almost zero hand coded. (Our system is far from vibe coding however, there is a very strict pipeline) I’m not saying that AI can solve every problem or that it is without problems (we spent hundreds of hours developing a concept to production pipeline just to make sure it doesn’t go off the rails) But the net result is that a good senior dev with an acutely olfactory paranoia can supervise a production pipeline and produce efficient, maintainable code at a much faster rate (and ridiculously lower cost) that he was doing before supervising 3 or 4 devs on a complex hardware project. I can’t speak for other types of development, but our applications devs are also leveraging AI code generation and it -seems- to be working out. Now, where those senior devs are going to come from in the future… that imho is a huge problem. It’s definitely some flavor of eating the goose that lays the golden egg here.
    - ACCount37 5mo ago
      It's blindingly obvious what the big bet is. The senior devs are going to come from the next generations of AI systems.
- freejazz 5mo ago
  >I can have my agentic system read a few data sheets, then I explain the project requirements and have it design driver specifications, protocols, interfaces, and state machines. Taking those, develop an implementation plan. Working from that, write the skeleton of the application, then fill it in to create a functional system using a novel combination of hardware. When you put it that way, isn't it crazy you have to tell it to do that? Like shouldn't it just figure out it needs to do that?
  - Silamoth 5mo ago
    This is exactly it. A human capable of reasoning might not know how to write code. But they can learn and be taught. Eventually, you can give them a vague problem, and they’ll know what clarifying questions to ask and how to write the code. LLMs cannot do that. If you have to do the reasoning and tell the LLM the results of your reasoning before it can generate the code you want, surely that tells you the LLM isn’t reasoning. Agentic workflows hide some of it, but anyone who’s interacted even a little with an LLM can tell they’re not reasoning, no matter how OpenAI and Anthropic label their models.
    - K0balt 5mo ago
      I’m not really sure. I’m constantly presented with a blurry line and it isn’t getting less blurry. If anything it’s slowly dissolving. Or maybe it’s me, falling victim to AI psychosis lol.
  - K0balt 5mo ago
    To be fair, I also have had to explain this same basic workflow to junior devs in the past, so I guess not?
    - freejazz 5mo ago
      > I also have had to explain this same basic workflow to junior devs That would not surprise me.
      - 5mo ago
- ChuckMcM 5mo ago
  You'd have to define 'novel code generation' and why dealing someone a poker hand they have never seen before isn't 'novel poker hand generation.' Not being snarky here, just understanding the way that LLMs work I am well aware that you can come up with things that nobody has seen before, and the 'how' is very much like the 'genetic' programming of times past.
  - K0balt 5mo ago
    Sure, apply this pattern to that set of specifications. The very fact that the language has a fixed set of defined keywords sort of makes it all “pattern matching”, but computabillity theory implies that you can definitely use patterns to create novel solutions. I guess it’s where you draw the line?
    - ChuckMcM 5mo ago
      > ... it more closely resembles reasoning than search. I get that, to you, it feels like reasoning. I'm not arguing about that. I expect we have different ideas of what sort of steps constitute reasoning. I'm also entirely unclear that we have the same understanding of computability theory. For example, a program can start at the beginning of a maze, and "compute" a path through it with a recursive algorithm that splits at every branch. Is is "reasoning" about how to solve the maze? If you believe that it is, then I understand your position and, as you surmised, I have a different definition of 'reasoning' than that one. For me, a classic "reasoning"[1] test is diagramming English sentences. That's because in order to diagram a sentence you need to understand both the rules around nouns, verbs, adverbs, and such, and what the sentence is actually saying. Some of the rules have exceptions and those exceptions are perfectly valid. In computation you might say this problem is not NP complete, and yet people do it all the time. Anyway, I appreciate the additional context you've provided. [1] using quotes here because I am operating under the understanding that substituting your version of what reasoning means in this context might not parse well.
keeda 5mo ago
> But it is another good example that "AI" is just glorified search and there is not reasoning or thinking going on behind the covers. I don't think that follows. This is just LLMs being, for a lack of a better word, "gullible." How is it different from a person believing whatever they read on the Internet? People fall for spam and scams all the time, doesn't mean they are just glorified searches ;-) It does highlight the problem facing any search engine though. AI-generated spam will be much harder to defend against with traditional, statistical mechanisms. And this is before we get to the existential problem of prompt injection. Maybe this is where news organizations can win back their proper place in their relationship with Big Tech: by becoming the sources of verified, vetted information that LLMs can trust blindly. Possibly that's what deals like the OpenAI / Atlantic one are about.
- cess11 5mo ago
  "How is it different from a person believing whatever they read on the Internet?" Because a person is alive while the LLM is a floating point number database with a questionable degree of determinism.
  - bigfishrunning 5mo ago
    It's worse then that. The neural network itself is completely deterministic, but the software around it adds some RNG to make it seem more intelligent. It would be much more useful if you could get repeatability, but that wouldn't support the illusion that it's "thinking".
    - cess11 5mo ago
      Right, this is true. However, when I make this point I try to be as succinct as possible to leave as little cognitive wiggle room for the reader as I can. "It's just a database", up to roughly the sentence above, is short, quick to interpret and requires much more effort to start an attempt at refutation against than a longer sentence which might be more precise and technically satisfying due to the larger semantic and syntactic 'surface'. I think of it more like an advert rather than an argument. People already know intellectually that it's a computer software thingie, it's the feeling I'd like to target. 'Coca-Cola is a watery drink with syrup and acid and a secret but quite long recipe of tasty flavours and aromas' is correct, but "Coke is it!" hits differently.
marginalia_nu 5mo ago
Google has had ample ability to address this problem, it's really not that hard. The reason it remains such a difficult problem for them to solve is that most of the things that would solve the problem would also decimate their ad revenue.
dlenski 5mo ago
> "AI" is just glorified search Even aside from out-and-out spam one of the extremely frustrating things about Google's AI overviews, compared to traditional search, is that the results are presented as coherent verging on authoritative even when they're not. If you do an "old-fashioned" (udm=14) Google search for, let's say "vendor scsi commands appotech USB NAND flash chip": https://www.google.com/search?q=vendor+scsi+commands+appotech+USB+NAND+flash+chip&udm=14 https://www.google.com/search?q=vendor+scsi+commands+appotec... … you'll see that there are only a few links, and a lot of them are people who are trying to reverse-engineer the devices' behavior, and uncertain or confused about what they're doing. You get instant feedback that you're looking a dark corner for something that has little public documentation. If you remove that `&udm=14` and look at the AI overview, Google gives you a confident-looking reply about available tools and techniques, even though some of what it links to are bit-rotted Russian-language forums and file download sites, and other places that likely won't solve your problem in a straightforward way… because that's all that's available for Google to mine.
winddude 5mo ago
unable, nah, more profitable for the ads business. Yea.
ge96 5mo ago
I did notice I had made videos/reddit post about vintage lenses and I was trying to figure out how old it was. The LLM would say an age eg. "made in 1940s" and reference my post which never mentioned the manufacturer date.
- ChuckMcM 5mo ago
  I've seen this happen when the backend image searches a picture, gets a description of what is in the picture, and adds that description to the bag of things it will produce as a summary. The whole 'put some text in the image frame that misleads the AI' lead to some hilarious results (man holding a puppy which has a postit stuck to it saying "Siamese kitten" for example, results saying "this man is holding a Siamese kitten." That led to some changes but it would be interesting to see if you could still poison results that way.
kristianp 5mo ago
> "AI" is just glorified search Google's AI overview seems to be using RAG of their search snippets that is summarised by a very fast LLM. I wouldn't call that glorified search.
jandrewrogers 5mo ago
At the limit AI and indexing are the same problem. It is supposed to be glorified search. The only thing that changes is the depth and complexity of the patterns you can search for using the “index”.
in-silico 5mo ago
> But it is another good example that "AI" is just glorified search and there is not reasoning or thinking going on behind the covers A bold claim given that the current top post on HN is "An OpenAI model has disproved a central conjecture in discrete geometry": https://news.ycombinator.com/item?id=48212493 https://news.ycombinator.com/item?id=48212493

K0balt 5mo ago

Hmm. I don’t think that novel code generation can be accounted for with glorified search. I can have my agentic system read a few data sheets, then I explain the project requirements and have it design driver specifications, protocols, interfaces, and state machines. Taking those, develop an implementation plan. Working from that, write the skeleton of the application, then fill it in to create a functional system using a novel combination of hardware. Done correctly, I end up with better, more maintainable, smaller code than I used to with a small team, at 1/100 the cost and 1/4 the time. Whatever that is, it more closely resembles reasoning than search. Unless, of course, you’d also call bare metal C development on novel hardware search, in which case I guess all dev is search?

Raphael_Amiard 5mo ago
It’s pattern matching. A big part of reasoning for sure, but not reasoning per se
- K0balt 5mo ago
  That could be, but if that is the case than development apparently doesn’t require reasoning? Or maybe that’s the part that the senior developer supervising the pipeline injects. Thats certainly a plausible position.
  - freejazz 5mo ago
    >but if that is the case than development apparently doesn’t require reasoning? Certainly plenty of it does not.
    - K0balt 5mo ago
      Ctrlc stack exchange lol
rkozik1989 5mo ago
How do you even know those numbers are correct? Realistically for what you've described you need more QA time that a traditional application to ensure its actually working properly. Especially with regards to any part of the application that deals with LLM inference. Its not hard to write unique content for niche topics where there are few relevant results and have LLMs take it as fact. For example, I poisoned the well for research on early Arab Americans immigrants by repeatedly posting about how many family passed as different ethnicity to make their lives easier, so now if you ask LLMs about that subject it'll include information I wrote which isn't entirely correct because I hadn't figured everything out before the LLM trained on it. EDIT: Now imagine if I had done this on an obscure programming-related problem, yeah? I could potentially make the LLM reference packages that do not actually exist and put backdoors in applications.
- K0balt 5mo ago
  Because I have 100 percent test coverage (of the software, some hardware edge cases pop up that aren’t documented in the data sheets), and over 10k hours of field deployment over 130 devices? This rollout has been much more bug free than any we have done in the last six years, and it’s the first that has been almost zero hand coded. (Our system is far from vibe coding however, there is a very strict pipeline) I’m not saying that AI can solve every problem or that it is without problems (we spent hundreds of hours developing a concept to production pipeline just to make sure it doesn’t go off the rails) But the net result is that a good senior dev with an acutely olfactory paranoia can supervise a production pipeline and produce efficient, maintainable code at a much faster rate (and ridiculously lower cost) that he was doing before supervising 3 or 4 devs on a complex hardware project. I can’t speak for other types of development, but our applications devs are also leveraging AI code generation and it -seems- to be working out. Now, where those senior devs are going to come from in the future… that imho is a huge problem. It’s definitely some flavor of eating the goose that lays the golden egg here.
  - ACCount37 5mo ago
    It's blindingly obvious what the big bet is. The senior devs are going to come from the next generations of AI systems.
freejazz 5mo ago
>I can have my agentic system read a few data sheets, then I explain the project requirements and have it design driver specifications, protocols, interfaces, and state machines. Taking those, develop an implementation plan. Working from that, write the skeleton of the application, then fill it in to create a functional system using a novel combination of hardware. When you put it that way, isn't it crazy you have to tell it to do that? Like shouldn't it just figure out it needs to do that?
- Silamoth 5mo ago
  This is exactly it. A human capable of reasoning might not know how to write code. But they can learn and be taught. Eventually, you can give them a vague problem, and they’ll know what clarifying questions to ask and how to write the code. LLMs cannot do that. If you have to do the reasoning and tell the LLM the results of your reasoning before it can generate the code you want, surely that tells you the LLM isn’t reasoning. Agentic workflows hide some of it, but anyone who’s interacted even a little with an LLM can tell they’re not reasoning, no matter how OpenAI and Anthropic label their models.
  - K0balt 5mo ago
    I’m not really sure. I’m constantly presented with a blurry line and it isn’t getting less blurry. If anything it’s slowly dissolving. Or maybe it’s me, falling victim to AI psychosis lol.
- K0balt 5mo ago
  To be fair, I also have had to explain this same basic workflow to junior devs in the past, so I guess not?
  - freejazz 5mo ago
    > I also have had to explain this same basic workflow to junior devs That would not surprise me.
    - 5mo ago
ChuckMcM 5mo ago
You'd have to define 'novel code generation' and why dealing someone a poker hand they have never seen before isn't 'novel poker hand generation.' Not being snarky here, just understanding the way that LLMs work I am well aware that you can come up with things that nobody has seen before, and the 'how' is very much like the 'genetic' programming of times past.
- K0balt 5mo ago
  Sure, apply this pattern to that set of specifications. The very fact that the language has a fixed set of defined keywords sort of makes it all “pattern matching”, but computabillity theory implies that you can definitely use patterns to create novel solutions. I guess it’s where you draw the line?
  - ChuckMcM 5mo ago
    > ... it more closely resembles reasoning than search. I get that, to you, it feels like reasoning. I'm not arguing about that. I expect we have different ideas of what sort of steps constitute reasoning. I'm also entirely unclear that we have the same understanding of computability theory. For example, a program can start at the beginning of a maze, and "compute" a path through it with a recursive algorithm that splits at every branch. Is is "reasoning" about how to solve the maze? If you believe that it is, then I understand your position and, as you surmised, I have a different definition of 'reasoning' than that one. For me, a classic "reasoning"[1] test is diagramming English sentences. That's because in order to diagram a sentence you need to understand both the rules around nouns, verbs, adverbs, and such, and what the sentence is actually saying. Some of the rules have exceptions and those exceptions are perfectly valid. In computation you might say this problem is not NP complete, and yet people do it all the time. Anyway, I appreciate the additional context you've provided. [1] using quotes here because I am operating under the understanding that substituting your version of what reasoning means in this context might not parse well.

keeda 5mo ago

> But it is another good example that "AI" is just glorified search and there is not reasoning or thinking going on behind the covers. I don't think that follows. This is just LLMs being, for a lack of a better word, "gullible." How is it different from a person believing whatever they read on the Internet? People fall for spam and scams all the time, doesn't mean they are just glorified searches ;-) It does highlight the problem facing any search engine though. AI-generated spam will be much harder to defend against with traditional, statistical mechanisms. And this is before we get to the existential problem of prompt injection. Maybe this is where news organizations can win back their proper place in their relationship with Big Tech: by becoming the sources of verified, vetted information that LLMs can trust blindly. Possibly that's what deals like the OpenAI / Atlantic one are about.

cess11 5mo ago
"How is it different from a person believing whatever they read on the Internet?" Because a person is alive while the LLM is a floating point number database with a questionable degree of determinism.
- bigfishrunning 5mo ago
  It's worse then that. The neural network itself is completely deterministic, but the software around it adds some RNG to make it seem more intelligent. It would be much more useful if you could get repeatability, but that wouldn't support the illusion that it's "thinking".
  - cess11 5mo ago
    Right, this is true. However, when I make this point I try to be as succinct as possible to leave as little cognitive wiggle room for the reader as I can. "It's just a database", up to roughly the sentence above, is short, quick to interpret and requires much more effort to start an attempt at refutation against than a longer sentence which might be more precise and technically satisfying due to the larger semantic and syntactic 'surface'. I think of it more like an advert rather than an argument. People already know intellectually that it's a computer software thingie, it's the feeling I'd like to target. 'Coca-Cola is a watery drink with syrup and acid and a secret but quite long recipe of tasty flavours and aromas' is correct, but "Coke is it!" hits differently.

marginalia_nu 5mo ago

Google has had ample ability to address this problem, it's really not that hard. The reason it remains such a difficult problem for them to solve is that most of the things that would solve the problem would also decimate their ad revenue.

dlenski 5mo ago

> "AI" is just glorified search Even aside from out-and-out spam one of the extremely frustrating things about Google's AI overviews, compared to traditional search, is that the results are presented as coherent verging on authoritative even when they're not. If you do an "old-fashioned" (udm=14) Google search for, let's say "vendor scsi commands appotech USB NAND flash chip": https://www.google.com/search?q=vendor+scsi+commands+appotech+USB+NAND+flash+chip&udm=14 https://www.google.com/search?q=vendor+scsi+commands+appotec... … you'll see that there are only a few links, and a lot of them are people who are trying to reverse-engineer the devices' behavior, and uncertain or confused about what they're doing. You get instant feedback that you're looking a dark corner for something that has little public documentation. If you remove that `&udm=14` and look at the AI overview, Google gives you a confident-looking reply about available tools and techniques, even though some of what it links to are bit-rotted Russian-language forums and file download sites, and other places that likely won't solve your problem in a straightforward way… because that's all that's available for Google to mine.

winddude 5mo ago

unable, nah, more profitable for the ads business. Yea.

ge96 5mo ago

I did notice I had made videos/reddit post about vintage lenses and I was trying to figure out how old it was. The LLM would say an age eg. "made in 1940s" and reference my post which never mentioned the manufacturer date.

ChuckMcM 5mo ago
I've seen this happen when the backend image searches a picture, gets a description of what is in the picture, and adds that description to the bag of things it will produce as a summary. The whole 'put some text in the image frame that misleads the AI' lead to some hilarious results (man holding a puppy which has a postit stuck to it saying "Siamese kitten" for example, results saying "this man is holding a Siamese kitten." That led to some changes but it would be interesting to see if you could still poison results that way.

kristianp 5mo ago

> "AI" is just glorified search Google's AI overview seems to be using RAG of their search snippets that is summarised by a very fast LLM. I wouldn't call that glorified search.

jandrewrogers 5mo ago

At the limit AI and indexing are the same problem. It is supposed to be glorified search. The only thing that changes is the depth and complexity of the patterns you can search for using the “index”.

in-silico 5mo ago

> But it is another good example that "AI" is just glorified search and there is not reasoning or thinking going on behind the covers A bold claim given that the current top post on HN is "An OpenAI model has disproved a central conjecture in discrete geometry": https://news.ycombinator.com/item?id=48212493 https://news.ycombinator.com/item?id=48212493