6 ms·

Building AI without a neural network

61 points by max_sendfeld 3y ago

troika 3y ago
Early in my career, I wrote sales funnel optimization algorithms for online-shops and travel websites. We've used an evolutionary algorithm approach - basically, a number of algos implementing competing strategies and aspects that could mutate, crossover, reproduce and die. We ran this over two years and saw impressive results - especially for the time. I feel, this very much was a "thinking system" in the same sense. And I concur - there are many architectures and approaches to built this sort of thing.
- wolframhempel 3y ago
  Yeah - I've been toying with a similar approach in my banking days. Basically, we've used "Genetic Programming" (very pompous term) to optimize order routing to minimize slippage (the difference between the expected price of a trade and the actual executed price). Basically, you create a "population" out of possible placement strategies for orders. You then cycle through "generations" that quickly adapt to the changing market conditions. Ultimately, you end up with something that - on average - provides execution closer to the target price than traditional, more static strategies. But the problem for us was that the outliers where really far out. Basically, if it got it wrong, it got it really wrong and you were often stuck with a set of half executed orders that had to be cleaned up manually.
- bumby 3y ago
  It seems the value of this type of approach is not the "thinking" (i.e., contextual understanding) but rather the computational speed of assessing random trials. Compared to evolutionary timescales, it's very, very fast but I wouldn't characterize "thinking" to the randomness of evolution, despite its success.
  - WJW 3y ago
    And yet it is a repeatable process that: - uses the information it already has combined with new inputs to form new information based on both (aka reasoning) - retains learned information over time (aka memory) - contains feedback loops to eventually eradicate "wrong" information (aka learning) - converges to similar conclusions in similar circumstances (aka reasoning or possibly instict) At least several of the most important parts of thought are there, though it is obviously very different than our own mind. That's OK, both submarines and penguins can move underwater but they do it in very different ways. No reason to think that thought wouldn't have multiple ways in which it could be implemented. Evolution is also almost certainly not self-aware but neither is a dog and we consider dogs to be capable of thinking too.
    - bumby 3y ago
      I agree, and I didn't mean to imply it's not useful. But I do think it's different and important to understand the limitations regarding those differences. In one important aspect, contextual understanding allows for better decisions in novel environments. The "random trial and error" approach has much less additional benefit in that regard.
- WJW 3y ago
  I think that in a few decades after we've had a lot more experience with various NN-based AIs, someone will come up with a more general quantification of "intelligence" that can integrate all these approaches into some vaguely orderable classes. After all a genetic algorithm seems clearly smarter (in some ways) than a rock but also it's clearly not as smart (in some way) as a dog. In that sense you could also say a physical lock is smarter than a rock (in that it processes a little bit of information) but not as smart as a human because it cannot learn. Similar to how we have things like NP-hard > NP > (maybe) P, you could have a classification based on how many (types of) information can be processed or something like that. Maybe a similar but separate scale for learning capacity?
- naasking 3y ago
  I think there's a meaningful difference between adaptive systems and thinking systems. I would say that the latter have to build internal models of the system they're analyzing and adapting to in order to qualify as "thinking". Evolution by natural selection is adaptive in the sense you describe, but no one would consider to be "thinking".
andy99 3y ago
It's interesting how when discussing neural networks "inspired by the human brain" always comes up when the brain, as far as we know, is emergent behavior from many little things coordinating, not architected. "Top down" never really works for complex systems, the economy being an obvious example. But we tend to ignore that when thinking about neural networks.
- voidhorse 3y ago
  The concept has a long history going back to the 1940s (Pitts and McCulloch) and was based on their ideas at the time about how neurons in the brain functioned. That's why the association with the brain is still with us, even if our biological and neurological understanding has evolved. I wish the industry would be more transparent about where all these brilliant ideas come from (spoiler alert: academics from the 1930s-70s, not "genius founders").
- nightski 3y ago
  The inspiration isn't top down. It's bottom up.
- somewhereoutth 3y ago
  Bear in mind that for most (all?) advanced economies the state runs about 40-50%, so top down is working fine for about half of the economy.
  - polotics 3y ago
    Going on a tangent sorry, you started it! So... in a private company the employees are very often a lot more beholden to the managers/owners, so top-down command and control can be applied a lot more than in an administration where job security is often much higher. For better or for worse, this can result in a lot more bottom-up decision-making... In view of general wastefulness, lack of longer term (environmental) planning, I would challenge the "working fine" a bit.
- famouswaffles 3y ago
  ANNs aren't top down either though. The architecture isn't as important as people think. https://nonint.com/2023/06/10/the-it-in-ai-models-is-the-dataset/ https://nonint.com/2023/06/10/the-it-in-ai-models-is-the-dat... Even then, the architectures we use are essentially stumbled on. This is alchemy, not modern chemistry.
skilled 3y ago
So what exactly are these guys building? For a product that’s supposedly 1.0 it sure lacks even the most basic of examples. I also think the title should reflect the product name because it’s clickbait as it is right now.
rvz 3y ago
> But that leaves large and important areas that GPTs are entirely unfit for: Real-time problem solving in dynamic environments, understanding and reacting to current events or spatial reasoning and coordination in the physical world. It is important to recognize the limitations of these deep neural networks and especially GPTs which both have been hyped to the point where most see it as the solution to everything; since many deep-learning supporters continue to compare it to the human brain, despite the severely limited explainability in these models to the point where not even researchers know what it is doing. It's one of the reasons why there's very low trust in deep neural networks in general. Humans can be held to account for their actions but with almost all these neural network systems cannot be held to account when something goes wrong and accept the output as the answer whilst 'it is thinking' or 'reasoning'. > The mechanism behind real world complex systems requires both simple rulesets and a communicative fabric that allows for real time feedback loops. For flocks of starlings this can be as easy as keeping eye contact with the next bird. For Termites, it is pheromone trails. For societies, it is language, reputation and status. For capital markets, it is money. And for the internet, its the physical wiring and network architecture that makes it all possible. As long as this solution provides transparency and explainable results rather than giving an answer from a black box system then I'm looking forward to this approach.
- why_only_15 3y ago
  Human explainability/interpretability is terrible -- it takes years to understand a person well enough to model their actions well in novel situations, and there's tons of literature that backs up that people consistently invent bullshit explanations when asked why they did things. Humans can only be held to account by telling them to change, which we can do with neural networks too.
  - rvz 3y ago
    > Humans can only be held to account by telling them to change, which we can do with neural networks too. Even if humans invent bullshit explanations, in serious cases of accountability is done in the courts system which humans investigate about the whole timeline of events of a dispute which there is very little room for perverting the course of justice and making everything up. Hence this scenario, lawyers would liked to have known as to why did an AI system give hallucinated citations when it was used in a legal proceeding? It's even worse that legal experts knowingly trusted it and failed to reason with the results from this AI system; because fundamentally it cannot explain why that issue happened. [0] It even goes beyond basic citations, with autonomous cars without humans behind the wheel [1] with the company (Cruise) being unable to convince the regulators or even explain the crashes and had to pull the vehicles off the road due to this high amount of risk. So yet again, explainability in AI with neural networks is still far worse than humans, even when these systems cannot be trusted in high risk and novel situations. [0] https://www.theguardian.com/technology/2023/jun/23/two-us-lawyers-fined-submitting-fake-court-citations-chatgpt https://www.theguardian.com/technology/2023/jun/23/two-us-la... [1] https://www.theguardian.com/us-news/2023/oct/24/driverless-car-self-driving-california-cruise-gm https://www.theguardian.com/us-news/2023/oct/24/driverless-c...
tyronehed 3y ago
[dead]
deleted 3y ago
[deleted]
Arson9416 3y ago
This is an ad for hivekit.io and contains almost no substance.
- mistermann 3y ago
  "When I change the way I look at things, the things I look at change."
- outofpaper 3y ago
  At least it's a good example of how to write a great clickbate title for hacker news ;)
- mcv 3y ago
  From the text, it sounds to me like they're selling multi-agent systems of some sort. They sounded really promising in the 1990s and then fell silent. Much like neural networks. But neural networks are back, so I guess they think multi-agent systems might be ready for a comeback too. They could be right.
  - PurpleRamen 3y ago
    Multi-agent-setups with LLMs (AutoGPT for example) were hyped some weeks ago. And OpenAI with their specialized Bots(?) or what they introduced recently goes in the same direction.
- m3kw9 3y ago
  They selling a blockchain coin?
machinekob 3y ago
Nice AD/clickbait
mromanuk 3y ago
Is there any replication of a transformer, GPT, LLM using "hivekit"? If not they should aim at that first.
abeppu 3y ago
Building a platform for spatially aware communication and coordination seems pretty valuable. But it seems pretty unrelated to AI. Their only other blogpost is also a vacuous AI buzz piece. I can only guess that someone not involved with the product has decided that putting out AI-related content for non-technical audiences is a good way to build brand awareness right now? https://hivekit.io/blog/ https://hivekit.io/blog/
facu17y 3y ago
Well, if it's so useless why is it on the HN front page? Are there "PR" companies behind promoting items to the HN front page? I'm sure there are because sometimes an article like this comes up at #3 and everyone says it's got no substance, clickbait, etc
- wolframhempel 3y ago
  I'll take the "PR Company" thing as a compliment. But no, afraid not. I'm one of the two founders and I wrote this one - sorry if it came across as click-baity, but I just wanted to outline the basic, long term idea that Hivekit is going for.
stayux 3y ago
Very good clickbait title. I am envious:)
toddmorey 3y ago
I’m going to admit to a bit of confusion here. “But that leaves large and important areas that GPTs are entirely unfit for: Real-time problem solving in dynamic environments.” Isn’t Tesla using GPTs at least for vision? What do you call the AI technologies employed by self driving projects and robotics?
- rvz 3y ago
  > Isn’t Tesla using GPTs at least for vision? What do you call the AI technologies employed by self driving projects and robotics? It is certainly not a generative pre-trained model (GPT) used in FSD or autopilot. Most likely advanced deep learning models for object detection or segmentation with very low latency. Whatever they are using, it still easily can get confused with similar looking objects on the road, which is why Tesla requires drivers to keep their eyes on the road and hands on the wheel. Not even Tesla, trusts its own AI systems.
efitz 3y ago
Swarms in nature are composed of nodes (organisms) that each have a neural network, and which have light and ephemeral connectivity (sensory input) to adjacent nodes. I guess if you wanted to be pedantic, there is no swarm, or in other words the swarm is just a shorthand way of describing the collective decisions of all the nodes. Where exactly does the emergent behavior come from if the nodes are dumb, e.g. non-autonomous?

61 points by max_sendfeld 3y ago

Early in my career, I wrote sales funnel optimization algorithms for online-shops and travel websites. We've used an evolutionary algorithm approach - basically, a number of algos implementing competing strategies and aspects that could mutate, crossover, reproduce and die. We ran this over two years and saw impressive results - especially for the time. I feel, this very much was a "thinking system" in the same sense. And I concur - there are many architectures and approaches to built this sort of thing.

wolframhempel 3y ago
Yeah - I've been toying with a similar approach in my banking days. Basically, we've used "Genetic Programming" (very pompous term) to optimize order routing to minimize slippage (the difference between the expected price of a trade and the actual executed price). Basically, you create a "population" out of possible placement strategies for orders. You then cycle through "generations" that quickly adapt to the changing market conditions. Ultimately, you end up with something that - on average - provides execution closer to the target price than traditional, more static strategies. But the problem for us was that the outliers where really far out. Basically, if it got it wrong, it got it really wrong and you were often stuck with a set of half executed orders that had to be cleaned up manually.
bumby 3y ago
It seems the value of this type of approach is not the "thinking" (i.e., contextual understanding) but rather the computational speed of assessing random trials. Compared to evolutionary timescales, it's very, very fast but I wouldn't characterize "thinking" to the randomness of evolution, despite its success.
- WJW 3y ago
  And yet it is a repeatable process that: - uses the information it already has combined with new inputs to form new information based on both (aka reasoning) - retains learned information over time (aka memory) - contains feedback loops to eventually eradicate "wrong" information (aka learning) - converges to similar conclusions in similar circumstances (aka reasoning or possibly instict) At least several of the most important parts of thought are there, though it is obviously very different than our own mind. That's OK, both submarines and penguins can move underwater but they do it in very different ways. No reason to think that thought wouldn't have multiple ways in which it could be implemented. Evolution is also almost certainly not self-aware but neither is a dog and we consider dogs to be capable of thinking too.
  - bumby 3y ago
    I agree, and I didn't mean to imply it's not useful. But I do think it's different and important to understand the limitations regarding those differences. In one important aspect, contextual understanding allows for better decisions in novel environments. The "random trial and error" approach has much less additional benefit in that regard.
WJW 3y ago
I think that in a few decades after we've had a lot more experience with various NN-based AIs, someone will come up with a more general quantification of "intelligence" that can integrate all these approaches into some vaguely orderable classes. After all a genetic algorithm seems clearly smarter (in some ways) than a rock but also it's clearly not as smart (in some way) as a dog. In that sense you could also say a physical lock is smarter than a rock (in that it processes a little bit of information) but not as smart as a human because it cannot learn. Similar to how we have things like NP-hard > NP > (maybe) P, you could have a classification based on how many (types of) information can be processed or something like that. Maybe a similar but separate scale for learning capacity?
naasking 3y ago
I think there's a meaningful difference between adaptive systems and thinking systems. I would say that the latter have to build internal models of the system they're analyzing and adapting to in order to qualify as "thinking". Evolution by natural selection is adaptive in the sense you describe, but no one would consider to be "thinking".

andy99 3y ago

It's interesting how when discussing neural networks "inspired by the human brain" always comes up when the brain, as far as we know, is emergent behavior from many little things coordinating, not architected. "Top down" never really works for complex systems, the economy being an obvious example. But we tend to ignore that when thinking about neural networks.

voidhorse 3y ago
The concept has a long history going back to the 1940s (Pitts and McCulloch) and was based on their ideas at the time about how neurons in the brain functioned. That's why the association with the brain is still with us, even if our biological and neurological understanding has evolved. I wish the industry would be more transparent about where all these brilliant ideas come from (spoiler alert: academics from the 1930s-70s, not "genius founders").
nightski 3y ago
The inspiration isn't top down. It's bottom up.
somewhereoutth 3y ago
Bear in mind that for most (all?) advanced economies the state runs about 40-50%, so top down is working fine for about half of the economy.
- polotics 3y ago
  Going on a tangent sorry, you started it! So... in a private company the employees are very often a lot more beholden to the managers/owners, so top-down command and control can be applied a lot more than in an administration where job security is often much higher. For better or for worse, this can result in a lot more bottom-up decision-making... In view of general wastefulness, lack of longer term (environmental) planning, I would challenge the "working fine" a bit.
famouswaffles 3y ago
ANNs aren't top down either though. The architecture isn't as important as people think. https://nonint.com/2023/06/10/the-it-in-ai-models-is-the-dataset/ https://nonint.com/2023/06/10/the-it-in-ai-models-is-the-dat... Even then, the architectures we use are essentially stumbled on. This is alchemy, not modern chemistry.

skilled 3y ago

So what exactly are these guys building? For a product that’s supposedly 1.0 it sure lacks even the most basic of examples. I also think the title should reflect the product name because it’s clickbait as it is right now.

rvz 3y ago

> But that leaves large and important areas that GPTs are entirely unfit for: Real-time problem solving in dynamic environments, understanding and reacting to current events or spatial reasoning and coordination in the physical world. It is important to recognize the limitations of these deep neural networks and especially GPTs which both have been hyped to the point where most see it as the solution to everything; since many deep-learning supporters continue to compare it to the human brain, despite the severely limited explainability in these models to the point where not even researchers know what it is doing. It's one of the reasons why there's very low trust in deep neural networks in general. Humans can be held to account for their actions but with almost all these neural network systems cannot be held to account when something goes wrong and accept the output as the answer whilst 'it is thinking' or 'reasoning'. > The mechanism behind real world complex systems requires both simple rulesets and a communicative fabric that allows for real time feedback loops. For flocks of starlings this can be as easy as keeping eye contact with the next bird. For Termites, it is pheromone trails. For societies, it is language, reputation and status. For capital markets, it is money. And for the internet, its the physical wiring and network architecture that makes it all possible. As long as this solution provides transparency and explainable results rather than giving an answer from a black box system then I'm looking forward to this approach.

why_only_15 3y ago
Human explainability/interpretability is terrible -- it takes years to understand a person well enough to model their actions well in novel situations, and there's tons of literature that backs up that people consistently invent bullshit explanations when asked why they did things. Humans can only be held to account by telling them to change, which we can do with neural networks too.
- rvz 3y ago
  > Humans can only be held to account by telling them to change, which we can do with neural networks too. Even if humans invent bullshit explanations, in serious cases of accountability is done in the courts system which humans investigate about the whole timeline of events of a dispute which there is very little room for perverting the course of justice and making everything up. Hence this scenario, lawyers would liked to have known as to why did an AI system give hallucinated citations when it was used in a legal proceeding? It's even worse that legal experts knowingly trusted it and failed to reason with the results from this AI system; because fundamentally it cannot explain why that issue happened. [0] It even goes beyond basic citations, with autonomous cars without humans behind the wheel [1] with the company (Cruise) being unable to convince the regulators or even explain the crashes and had to pull the vehicles off the road due to this high amount of risk. So yet again, explainability in AI with neural networks is still far worse than humans, even when these systems cannot be trusted in high risk and novel situations. [0] https://www.theguardian.com/technology/2023/jun/23/two-us-lawyers-fined-submitting-fake-court-citations-chatgpt https://www.theguardian.com/technology/2023/jun/23/two-us-la... [1] https://www.theguardian.com/us-news/2023/oct/24/driverless-car-self-driving-california-cruise-gm https://www.theguardian.com/us-news/2023/oct/24/driverless-c...

tyronehed 3y ago

[dead]

deleted 3y ago

[deleted]

Arson9416 3y ago

This is an ad for hivekit.io and contains almost no substance.

mistermann 3y ago
"When I change the way I look at things, the things I look at change."
outofpaper 3y ago
At least it's a good example of how to write a great clickbate title for hacker news ;)
mcv 3y ago
From the text, it sounds to me like they're selling multi-agent systems of some sort. They sounded really promising in the 1990s and then fell silent. Much like neural networks. But neural networks are back, so I guess they think multi-agent systems might be ready for a comeback too. They could be right.
- PurpleRamen 3y ago
  Multi-agent-setups with LLMs (AutoGPT for example) were hyped some weeks ago. And OpenAI with their specialized Bots(?) or what they introduced recently goes in the same direction.
m3kw9 3y ago
They selling a blockchain coin?

machinekob 3y ago

Nice AD/clickbait

mromanuk 3y ago

Is there any replication of a transformer, GPT, LLM using "hivekit"? If not they should aim at that first.

abeppu 3y ago

Building a platform for spatially aware communication and coordination seems pretty valuable. But it seems pretty unrelated to AI. Their only other blogpost is also a vacuous AI buzz piece. I can only guess that someone not involved with the product has decided that putting out AI-related content for non-technical audiences is a good way to build brand awareness right now? https://hivekit.io/blog/ https://hivekit.io/blog/

facu17y 3y ago

Well, if it's so useless why is it on the HN front page? Are there "PR" companies behind promoting items to the HN front page? I'm sure there are because sometimes an article like this comes up at #3 and everyone says it's got no substance, clickbait, etc

wolframhempel 3y ago
I'll take the "PR Company" thing as a compliment. But no, afraid not. I'm one of the two founders and I wrote this one - sorry if it came across as click-baity, but I just wanted to outline the basic, long term idea that Hivekit is going for.

stayux 3y ago

Very good clickbait title. I am envious:)

toddmorey 3y ago

I’m going to admit to a bit of confusion here. “But that leaves large and important areas that GPTs are entirely unfit for: Real-time problem solving in dynamic environments.” Isn’t Tesla using GPTs at least for vision? What do you call the AI technologies employed by self driving projects and robotics?

rvz 3y ago
> Isn’t Tesla using GPTs at least for vision? What do you call the AI technologies employed by self driving projects and robotics? It is certainly not a generative pre-trained model (GPT) used in FSD or autopilot. Most likely advanced deep learning models for object detection or segmentation with very low latency. Whatever they are using, it still easily can get confused with similar looking objects on the road, which is why Tesla requires drivers to keep their eyes on the road and hands on the wheel. Not even Tesla, trusts its own AI systems.

efitz 3y ago

Swarms in nature are composed of nodes (organisms) that each have a neural network, and which have light and ephemeral connectivity (sensory input) to adjacent nodes. I guess if you wanted to be pedantic, there is no swarm, or in other words the swarm is just a shorthand way of describing the collective decisions of all the nodes. Where exactly does the emergent behavior come from if the nodes are dumb, e.g. non-autonomous?