3 ms·

I feel this is going into increasingly-unlikely mixes of constraints and needs in order to try to keep a "wouldn't it be cool if" hypothetical-tool dream alive.

by Terr_ 4mo ago

I feel this is going into increasingly-unlikely mixes of constraints and needs in order to try to keep a "wouldn't it be cool if" hypothetical-tool dream alive. [0] But OK, let's assume that: The power is out, but you have a generator with so much fuel you can run a desktop just fine; Your neighborhood will somehow make a mesh network; Your neighbors need some already stored information and the best solution for that is texting a chatbot rather than a survival/emergency handbook or Wikipedia; Your mesh-network will also be good enough to match the time-sensitivity of the questions. Under those assumption, which of these sounds better? 1. Buying an "LLM-in-a-box for emergency supply kits", which you deploy so that your neighbors can ask questions (text over the mesh) of the offline chatbot. 2. Buying a satellite internet transciever for your emergency supply kit, so that your neighbors can ask questions of a much better chatbot and communicate with human experts, their worried relatives, and coordinate with rescue/relief efforts... [0] https://xkcd.com/2128/ https://xkcd.com/2128/

zmgsabst 4mo ago
Option 1 sounds better: I’m only out the cost of the drive, which is like $40 and doesn’t require anybody on the other side cooperate with me. - - - More broadly… You call it unlikely mixes, but we see it all the time: - people already have a computer for gaming or work - people (ie, “preppers” like we’re discussing) buy a generator for emergencies - local emergency response sets up mesh networking during disasters, both official and unofficial Have you ever tried to use a handbook you’re not intimately familiar with during an emergency? It’s rough. For personal preparedness, nothing replaces familiarity and practice — eg, weekend survival trips and reading your manual ahead of time. But for providing information in a random lookup manner to unpracticed people who weren’t prepared? Yes, I think an LLM/chatbot is the practical way to operationalize all that information which you stored (eg, survival guides or machine manuals). Also, it’s unlikely a general purpose chatbot would be superior at survival advice to one specialized for that purpose — and indeed, is likely to refuse your questions as “unsafe” or “criminal”.
- swiftcoder 4mo ago
  > I’m only out the cost of the drive, which is like $40 and doesn’t require anybody on the other side cooperate with me. At current prices you are also out about $4k for a Spark to actually run the inference on, if you want a full LLM in a low-power package. In general, I'm not sure why one would want to pin your survival to an expensive, hallucination-prone data source, when an offline copy of wikipedia with a little vector search attached to a Raspberry Pi can fulfil the same role...
- root_axis 4mo ago
  > Have you ever tried to use a handbook you’re not intimately familiar with during an emergency? It’s rough. Sounds like the absolute worst time to rely on a crappy little model that will inevitably hallucinate.

by Terr_ 4mo ago

zmgsabst 4mo ago

Option 1 sounds better: I’m only out the cost of the drive, which is like $40 and doesn’t require anybody on the other side cooperate with me. - - - More broadly… You call it unlikely mixes, but we see it all the time: - people already have a computer for gaming or work - people (ie, “preppers” like we’re discussing) buy a generator for emergencies - local emergency response sets up mesh networking during disasters, both official and unofficial Have you ever tried to use a handbook you’re not intimately familiar with during an emergency? It’s rough. For personal preparedness, nothing replaces familiarity and practice — eg, weekend survival trips and reading your manual ahead of time. But for providing information in a random lookup manner to unpracticed people who weren’t prepared? Yes, I think an LLM/chatbot is the practical way to operationalize all that information which you stored (eg, survival guides or machine manuals). Also, it’s unlikely a general purpose chatbot would be superior at survival advice to one specialized for that purpose — and indeed, is likely to refuse your questions as “unsafe” or “criminal”.

swiftcoder 4mo ago
> I’m only out the cost of the drive, which is like $40 and doesn’t require anybody on the other side cooperate with me. At current prices you are also out about $4k for a Spark to actually run the inference on, if you want a full LLM in a low-power package. In general, I'm not sure why one would want to pin your survival to an expensive, hallucination-prone data source, when an offline copy of wikipedia with a little vector search attached to a Raspberry Pi can fulfil the same role...
root_axis 4mo ago
> Have you ever tried to use a handbook you’re not intimately familiar with during an emergency? It’s rough. Sounds like the absolute worst time to rely on a crappy little model that will inevitably hallucinate.