3 ms·

The model running well is only one constraint for consumer adoption. The other ones, namely RAM and making the CPU super warm and rapidly draining battery, are

The model running well is only one constraint for consumer adoption. The other ones, namely RAM and making the CPU super warm and rapidly draining battery, are another which is much harder to solve.

falcor84 3y ago
I'm somewhat bullish on the Edge Computing approach to this. For example, how about a co-op neighborhood LLM rack?
- alberth 3y ago
  How is “Edge Computing” any different than what the GP was stating about current cloud hosted (eg OpenAI)?
  - falcor84 3y ago
    There are different possible approaches here, but for me, the main benefit would be of having control over the model(s) used. Other than that, low latency would be crucial for effective voice interaction.
- sho_hn 3y ago
  I currently use a TrueNAS as my home server, and I'm sure whatever I replace it with will add HW for local inference and act as my Home Brain for various automations and assistants.
- runjake 3y ago
  > how about a co-op neighborhood LLM rack? It'll work about as well as those "wire your neighborhood for Internet as a collective" movements of the 1990s. In fact, you'll need those to deal with latency issues, if my experience with consumer ISP quality is any indication.
  - dharmab 3y ago
    Funny enough, I know two separate groups of people doing that. One in an apartment in SF running a shared Wi-Fi network with VLANs so they don't step on each other's frequencies, and another group sharing WISP infrastructure in semi-rural Utah where they can't convince ISPs to lay fiber. (Although I think the second group now uses that as a backup for Starlink.)
    - runjake 3y ago
      Yes, UBNT nanobeam 5ac AC wireless[1] is pretty dirt cheap. Can get a pair for about $199 for a wireless P2P link that can work over several kilometers. I have deployed a number of these myself, along with more high bandwidth airFiber stuff. 1. https://store.ui.com/us/en/pro/category/all-wireless/products/nanobeam-5ac https://store.ui.com/us/en/pro/category/all-wireless/product...
- ozr 3y ago
  Why would you do that, and when else has it worked?
- zztop44 3y ago
  I would much rather send my personal information to Microsoft and the NSA than people in my neighborhood

The model running well is only one constraint for consumer adoption. The other ones, namely RAM and making the CPU super warm and rapidly draining battery, are another which is much harder to solve.

falcor84 3y ago

I'm somewhat bullish on the Edge Computing approach to this. For example, how about a co-op neighborhood LLM rack?

alberth 3y ago
How is “Edge Computing” any different than what the GP was stating about current cloud hosted (eg OpenAI)?
- falcor84 3y ago
  There are different possible approaches here, but for me, the main benefit would be of having control over the model(s) used. Other than that, low latency would be crucial for effective voice interaction.
sho_hn 3y ago
I currently use a TrueNAS as my home server, and I'm sure whatever I replace it with will add HW for local inference and act as my Home Brain for various automations and assistants.
runjake 3y ago
> how about a co-op neighborhood LLM rack? It'll work about as well as those "wire your neighborhood for Internet as a collective" movements of the 1990s. In fact, you'll need those to deal with latency issues, if my experience with consumer ISP quality is any indication.
- dharmab 3y ago
  Funny enough, I know two separate groups of people doing that. One in an apartment in SF running a shared Wi-Fi network with VLANs so they don't step on each other's frequencies, and another group sharing WISP infrastructure in semi-rural Utah where they can't convince ISPs to lay fiber. (Although I think the second group now uses that as a backup for Starlink.)
  - runjake 3y ago
    Yes, UBNT nanobeam 5ac AC wireless[1] is pretty dirt cheap. Can get a pair for about $199 for a wireless P2P link that can work over several kilometers. I have deployed a number of these myself, along with more high bandwidth airFiber stuff. 1. https://store.ui.com/us/en/pro/category/all-wireless/products/nanobeam-5ac https://store.ui.com/us/en/pro/category/all-wireless/product...
ozr 3y ago
Why would you do that, and when else has it worked?
zztop44 3y ago
I would much rather send my personal information to Microsoft and the NSA than people in my neighborhood