5 ms·
I'm convinced now that “personal LLMs” are going to be a huge thing
- mentos 3y agoOne reddit comment in opposition to this idea I agreed with was "If Google turned on an LLM tomorrow that had access to your whole text message history, location history, calendar, google docs & drive, and all of your gmail (sent and received), almost everyone would start using it and keep using it." https://old.reddit.com/r/LocalLLaMA/comments/16au3ga/im_convinced_now_that_personal_llms_are_going_to/jzbwsrr/ https://old.reddit.com/r/LocalLLaMA/comments/16au3ga/im_conv...
- unixhero 3y agoSo then all we need is Charlize Theron's voice in an all voice command world, and were there where Her brought us. The future came faster than I thought
- sublinear 3y agoI think the public opinion of LLMs is so severely damaged right now that if Google made that available tomorrow it would be a media circus of bad incidents, experts, and even politicians chiming in. That the public has a poor understanding of technology cuts both ways. There's no such thing as user error if they're talking to an LLM. High expectations require high amounts of testing, regulation, etc.
- myth_drannon 3y agothat's multiple orders of magnitude more training data, I don't think it makes sense financialy (or possible in the near future).
- totallywrong 3y agoThat's both very true and very sad.
- throw7 3y agoUsers will get one wrong answer at a critical time and then google will shut it down. Actually, they'll probably just shut it down after a couple of years.
- barryrandall 3y agoIf that was true, Google Assistant would have been archived years ago.
- dragonwriter 3y agoYou mean if Google Assistant was backed by an LLM instead of whatever combination of tools is backing it now? Yes, 100% this is correct. And its been fairly overtly the direction Google’s AI efforts have been directed at since long before “LLM” was the an implementation technology they could use for it. If Google was any good at productizing its AI efforts, it would already have happened.
- russellbeattie 3y agoHere's a fundamental truth that HNers can't or won't accept: Consumers don't care about privacy, at all. Maybe conceptually, they might object to Big Tech knowing as much as they do, but it doesn't affect their decisions in any way. They simply don't care that Google Maps is always tracking their location, that an Echo Show has a camera and mic that can be remotely activated, that TikTok is logging their viewing habits down to the millisecond, nor that their Smart TVs are logging their media choices. They just don't care. It doesn't affect their willingness to use or buy technology in the slightest. The comment you quoted and your agreement shows this quite plainly. This actually bodes well for AI services, as they will be more useful the more they know about you. Your age, gender, sexual preferences, religion, work, family, possessions, money, tax records, account logins, schedule, habits, health records, etc. will help AIs to help you. This is the inevitable end game for AI services, so the fact that your average joe has no real conviction about privacy will hasten that future. Side note: If your business or product plan includes "privacy" as one of its core features, expect limited success as you're targeting a very niche part of the market. No one cares.
- Yizahi 3y agoHumans can tolerate increasing amounts of discomfort as long as it's incremented slowly or alternatives are blocked. Privacy loss is simply one of the many discomforts which we tolerate. Which is why pointing a flaw in any long used product of service causes such antagonistic reaction - that flaw was already normalized by us.
- tayo42 3y agoThe only way that would be possible is if an llm were in a mode of always being trained and could count on being accurate. Which i dont think is possible? So trying to use the calculated weights as a massive compressed data source that knows everything is a mistake and is just leaning into the hype. which this is doing imo. The only thing it seems to enable is using natural language as an interface, instead of a programmed api. Text goes in, translates to something the computer understands, text comes back to the use. Instead of like rpc and and error code. And there is code auto complete.
- theK 3y agoYeah sure, some time in the future. For now I think the fuzzyness and mediocre quality of the responses you get at scale are probably big showstoppers for usage at large. Also, I've not followed this domain over the summer, are we still at 50k token caps on the state of the art models?
- nraford 3y agoClaude has been able to do 100k for months and is as good, if not better (IMHO) than GPT4 for some things.
- theK 3y agoOkay, 100k means we are getting to the point where one can safely edit a large piece of text, say a blogpost or a small scientific paper if you are lucky. It still means most of the long running interactions will get clipped/compressed relatively soon. But good to see the industry evolving
- Aaronstotle 3y agoMy first thought after ChatGPTs launch last year was that the eventual goal is a personal AI. I'm very forgetful, having an AI assistant would be immensely helpful. Can go further than that of course, I like to cycle and weight lift, I can see a future where I have an AI let me know where I should push and conserve energy on a bike ride/develop a training plan for me.
- totallywrong 3y agoI don't know why I can't seem to get on the LLM hype train everyone is so ready to jump on. I either lack vision or see it for what it is, a next-gen search engine. And don't get me wrong that's amazing, but not world-shattering as people want to believe.
- syndicatedjelly 3y agoIt's world-shattering for people who have never had a good way to manage their personal knowledge base. Personally, I just let my mind organize my thoughts. If something's important I'll remember it, if it's not then I don't. Obviously I write stuff down where needed but that's more to help commit it to memory, not because I'm ever going to read these notes again.
- dieselgate 3y agoThis resonates with me because personally I don't see a lot of use for my (dev) workflows. Was talking with my SO recently on this topic, they are close to finishing a PhD and do scientific research. For their workflows, consuming a lot of written information and producing a lot of written information, a case could be made it is more useful.
- eichin 3y agoyeah, as a developer I've yet to see any development tasks that an LLM would be useful for, and it's wasting resources that could be going into abstract reasoning about software, or really doing anything correctly/accurately...
- spiderxxxx 3y agoas a developer, I can see a lot of tasks that an LLM would be useful for. In a large organization, new people get onboarded all the time, and generally they have the same exact questions, like 'how do i do a web request out of the proxy?' and 'how can I do multithreading' or 'how do i use library x for our use case y'? whereas an LLM can look at the code used throughout the business and offer suggestions like "hey, I see you're trying to make an https connection to an external site without using a proxy, try this instead". Getting enterprise code up to the enterprise standard, whatever that is, can be very useful. Personally, I'd love to go in depth with an AI on crazy ideas I have for code, and work through a lot of ideas before I implement something. For example, I might want to try different ways to accomplish the same task. I could write my version of it, and instead of rewriting it to try a different way, I might ask an LLM to rewrite it that other way, so I can check which way is better.
- jjcm 3y agoThis is essentially what Windows Copilot seems to be attempting to do currently. There's obviously downsides to this approach from a security concern - it doesn't live on device, it's on the cloud. I actually think Apple is incredibly positioned here. All of their devices have dedicated NPUs, if they can optimize an LLM to run locally on these in a way that's performant/accurate, the power could be incredible for a local AI that helps out. It'd likely be a huge sales driver as well - Pro devices might have faster/more accurate AI assistants. If it's purely local, the access to personal data becomes less of a concern. Super curious to see how this evolves.
- JohnMakin 3y agoApple's first foray into this kind of tech would have been siri, but they made the baffling decision to send siri requests to their servers and back rather than running locally. I know at some point that changed a while back, can't remember when, but siri is still so bad at doing/understanding very basic things that my confidence in apple is not super high to wield this tech.
- tick_tock_tick 3y ago> I actually think Apple is incredibly positioned here. Does Apple even have an AI story? Siri is still the worst of the lot for voice. I don't think they are doing anything in the LLM space so....?
- flangola7 3y agoThere's all the ML features that have that Android and windows are still catching up to. Let me know when I can search for that paper document I took a photo of by ctrl+f for a string I remember in it.
- throwawa14223 3y agoWe're at that part of the Gartner hype cycle.
- readyplayernull 3y agoI recently switched to Claude after using ChatGPT for several months. CG tells you why something is great, while Claude tells you how it's useful. CG is more of a psychoanalysis chatbot, which would be good as a virtual partner.
- choeger 3y agoPersonal LLMs make a lot of sense. Until they don't. Right now, having a OCR combined with an LLM would open the way to a personalized assistant (rather a well working search machine and letter/email generator). No need to manually manage or catalogue documents, photos, videos, contacts, it would all be available by grepping your input stream smartly. But once everyone uses such a tool, why bother generating human-readable text in the first place? There's so much efficiency to be gained: Send plaintext instead of html or pdf. Replace plaintext with its embedding. The sending AI could learn what the receiver understands. Finally, don't send stuff at all. Just point to the relevant vector DB entry. In the end, the LLM itself will be obsolete, except as an appendix to communicate with humans. For all other tasks it will be superseded by a evolvednit engineered, form of machine-to-machine language.
- jackschultz 3y agoI've been thinking about how these personal LLMs fit into Klara and The Sun[0], book by Ishiguro from a few years ago. Written in first person from someone named Klara, who is an AF (artificial friend) "human" who is an assistants for families and children. The scene in the book that stuck out to me, and that I feel fits with personal AI / LLM, was the scene where a person in the AF store was looking to possibly buy an upgraded AF. The one the family had was something like 3.5, and now that 4 was around, they should ditch the outdated AF and get a new one with more learning power. If I was looking to get into the personal LLM space, and I agree with the reddit post linked, I feel like the process will be to be able to quickly spin these up and train them, and be aware that new ones are going to come and don't be overly attached to the old. Weird to type that out. [0]https://en.wikipedia.org/wiki/Klara_and_the_Sun https://en.wikipedia.org/wiki/Klara_and_the_Sun
- beepbooptheory 3y agoSurely if you got a new model you would want to feed it back up with all you had inputted in the last one? What else would there to be attached to? Something like its "personality" is necessarily just fleeting effect of a given prompt, right?
- nologic01 3y agoMaybe the year of the LLLM has finally arrived? LLLM, as in: the Linux Large Language Model [enabled Desktop]. sudo apt install llm, use it with any app, you get the drift... This is not about hype, AGI, singularity or gazillions. Its about enabling individuals to get on top of the information fire-hose with the help of an important data processing tool. In a private and empowering manner. Nothing more, nothing less. Sure there are certain hardware and data issues (where / how to train the models, with what data, how to ensure speedy inference in interactive use etc). But those are tangible targets people can chew on. No obvious show-stopper.
- craighay1 3y agoIs the majority of current LLM research reaching for the mythical grail of AGI? Or is there enough research effort being poured into how to retrain networks efficiently and how to finetune accurately on small amounts of data? My personal knowledge base and personal correspondence is miniscule in comparison to the great swaths of the internet these things are trained on. Vague, inaccurate or non-comittal answers based on my personal data are perfectly well served by my magic 8 ball at the moment.
- KaiserPro 3y agoI am adjacent to personal LLMs at work. They've been trying to make them work for a number of years. LLMs make it simpler to query the data, but its not quite there yet for useful work. For example if you have an assistant and you talk off and on about booking for $band, they will know that you like music, and that genre. They can alter the context of replies to suit. With LLMs, the link between shortterm and long term recall is broken. Now, this might be partially solvable by having an intermediary LLM that search the interaction database for context and feeds it into the main interaction LLM. But I'm not convinced that its a long term or viable solution. the LLM is not building a personal insight into your responses, its just having one shot context injected.
- _fs 3y agoAre there any open source projects that make this possible today? Something you can just throw data at (in any format) and build up a training library?
- tqi 3y agoTo do what though? Even the linked thread has no concrete examples. If I had a human personal assistant, most (all?) the value to me would be doing physical errands. I'm not so busy or important that I'd need someone to manage my calendar or correspondence. Travel planning could be helpful, but I don't know if the "personal" aspect of it has that much incremental value over a centralized model...
- interstice 3y agoThe personal aspect would be connections to your contacts, apps and devices (if you have that kind of thing). If it was smart/reliable enough it could book appointments, order dinner, plan your weekend based on events people pinged you about.. that kind of thing. That is, if you trust it enough to plug into your personal ecosystem and spend your money..
- smugglerFlynn 3y agoI’d assume anyone living such a busy life he needs a personal assistant to make dinner appointment decisions for him, most likely already has one. One of the most basic tricks in product management, when validating a product idea, is to ask your potential customer “how do you currently try solve this problem?”. Because oftentimes these problems are fantasies nobody actually cares enough to solve.
- tqi 3y ago> If it was smart/reliable enough it could book appointments, order dinner, plan your weekend based on events people pinged you about.. that kind of thing. The way I see it, if I had an ever present human assistant following me around 24/7, I would definitely get some value from being able to offload parts of those tasks. However my gut says that the amount is quite small, since I would still need to direct them to do stuff for me, and confirm everything. Most of the value I've seen from EAs is helping to coordinate competing priorities, and I just don't have that many. On top of that, a lot of the little ways that people get value from assistants (calling up hard to reach restaurants, reminding you to send cards and gifts) lose it's value if EVERYONE has access to an assistant.