3 ms·

“888 KiB Assistant” but the assistant itself is a multi terabyte rental-only model stored in some mysterious data center.

amelius 7mo ago
I'm getting "serverless" flashbacks.
- pgt 7mo ago
  modelless
  - stuaxo 7mo ago
    This thread reminds me how Javas heavy GUI written in Java itself was called "lightweight" when in fact it did not feel lightweight at all on the hardware of the time.
Rebelgecko 7mo ago
It seems to support connecting to your own LLM on the same LAN
- dheera 7mo ago
  I tried connecting OpenClaw to ollama with a V100 running qwen3.5:35b but it was really, really, really slow (despite ollama itself feeling fairly fast). These "claw" agents really multiply the tokens used by an obscenely huge factor for the same request.
  - jcgrillo 7mo ago
    i recently decided to get into this ocean boiling game too, the 32GB V100 seems like a pretty good VRAM/$. if i may ask, do you make any special accommodations for cooling? i've never dealt with a passively cooled card before and i'm curious whether my workstation fans (HP Z840) will be sufficient. i'm going to try 2 cards at first but i think i might be able to squeeze a third in there
    - dheera 7mo ago
      No. I have a Titan V CEO edition, which is basically a 32GB V100 but has full active fan cooling which I'm finding works just fine.
      - jcgrillo 7mo ago
        Oh very cool. Some folks are printing shrouds for dual 40mm fans so I'll probably try that if the stock case fans don't do it
- croes 7mo ago
  The point is the agent is still the LLM. No LLM, no agent.
  - otterley 7mo ago
    LLMs are not agents. LLMs are language models that simply respond to a text prompt with a textual response. Agents are middleware that take input from the user and then use LLMs to drive tools.
    - croes 7mo ago
      They are just a to-do list. The real work is done by the LLMs
      - otterley 7mo ago
        An LLM has no motive power, like a script without an a cast, or a program without a computer to execute it.
kristianpaul 7mo ago
My model is at home... just 16Gb still a lot but just FYI
seertaak 7mo ago
The whole point is that this fits on an ESP32, which has wifi. We're not quite at the point where it makes sense to run the whole thing locally - if you do try it, it will need a fan, and be loud etc. For my part, I installed Nanoclaw on my Arch derived OS (I love Arch!), and it worked fine until the next day some update decided to revert the power management settings, and now my glorious assistant is dead. There's something to be said for a barebones OS. No bullshit, no updates. Also, playing with hardware watchdog timers and GPIOs and DACs can be so much fun.

amelius 7mo ago

I'm getting "serverless" flashbacks.

pgt 7mo ago
modelless
- stuaxo 7mo ago
  This thread reminds me how Javas heavy GUI written in Java itself was called "lightweight" when in fact it did not feel lightweight at all on the hardware of the time.

Rebelgecko 7mo ago

It seems to support connecting to your own LLM on the same LAN

dheera 7mo ago
I tried connecting OpenClaw to ollama with a V100 running qwen3.5:35b but it was really, really, really slow (despite ollama itself feeling fairly fast). These "claw" agents really multiply the tokens used by an obscenely huge factor for the same request.
- jcgrillo 7mo ago
  i recently decided to get into this ocean boiling game too, the 32GB V100 seems like a pretty good VRAM/$. if i may ask, do you make any special accommodations for cooling? i've never dealt with a passively cooled card before and i'm curious whether my workstation fans (HP Z840) will be sufficient. i'm going to try 2 cards at first but i think i might be able to squeeze a third in there
  - dheera 7mo ago
    No. I have a Titan V CEO edition, which is basically a 32GB V100 but has full active fan cooling which I'm finding works just fine.
    - jcgrillo 7mo ago
      Oh very cool. Some folks are printing shrouds for dual 40mm fans so I'll probably try that if the stock case fans don't do it
croes 7mo ago
The point is the agent is still the LLM. No LLM, no agent.
- otterley 7mo ago
  LLMs are not agents. LLMs are language models that simply respond to a text prompt with a textual response. Agents are middleware that take input from the user and then use LLMs to drive tools.
  - croes 7mo ago
    They are just a to-do list. The real work is done by the LLMs
    - otterley 7mo ago
      An LLM has no motive power, like a script without an a cast, or a program without a computer to execute it.

kristianpaul 7mo ago

My model is at home... just 16Gb still a lot but just FYI

seertaak 7mo ago

The whole point is that this fits on an ESP32, which has wifi. We're not quite at the point where it makes sense to run the whole thing locally - if you do try it, it will need a fan, and be loud etc. For my part, I installed Nanoclaw on my Arch derived OS (I love Arch!), and it worked fine until the next day some update decided to revert the power management settings, and now my glorious assistant is dead. There's something to be said for a barebones OS. No bullshit, no updates. Also, playing with hardware watchdog timers and GPIOs and DACs can be so much fun.