8 ms·
Opera becomes the first major browser with built-in access to local AI models
- glintik 3y agoHow much RAM browser wants to run LLM local processing?
- skeeter2020 3y agoprobably less than it needs for a few dozen open tabs based on my past profiling experiences...
- kevindamm 3y agoHow long are you willing to wait?
- dchest 3y agoUsually around 5 GB for a 7B 4-bit quantized model.
- constantcrying 3y agoThe question I have is, why should your browser also be the interface to local AI models? Seems totally unrelated to be honest.
- nerdponx 3y agoBecause we consume and interact with so much of our information via browser that it's a natural pairing? Why does my browser have a PDF reader and print manager built in?
- amlib 3y agoThe very things that should ideally be part of an operating system and be called/embedded by the browser as need be?
- vengefulduck 3y agoBrowsers are just mini OSs at this point. It’s probably best just to accept it. Honestly in some respects (security, isolation, resource management) they do a better job than the operating system they run on top of.
- pluc 3y agoIt makes it possible to do, in half an hour, tasks which were completely unnecessary to do before.
- notachatbot1234 3y agoBut why use the web browser as local interface for it? Why not a separate, native application?
- mdrzn 3y agoIf my data is already in the browser (Sheets, Gmail, other websites to scrape) why have another app instead of doing the work in the browser?
- some_furry 3y agoWhile we're at it, why not shove our financial passwords etc. into our HTTP headers for every website we visit? That would be awfully convenient. You'd never lose them or need to deal with another native application. Just open any web page that echoes all HTTP headers and then check your account balance. Or maybe having a separation of concerns for different applications in a general purpose computer is worth the friction.
- foverzar 3y agoI don't get it.
- shutupnerd0000 3y agoYou missed the joke
- notachatbot1234 3y agoI apologize for the mistake in my previous response. It seems that my information about the poster's intent was outdated or inaccurate. Upon further research, I have found that I misread the word "unnecessarily".
- amelius 3y agoMaybe if you consider your browser to be your OS.
- papichulo2023 3y agoTask automation?
- ben_w 3y agoSave on bandwidth by generating content on the fly rather than having to visit websites which used their own GenAI to generate content without regard for accuracy as its sole purpose is to attract the attention of real humans which can then be sold via a real-time auction run by another AI (Google ads etc.) to the real customers, who are likewise generating advertising with GenAI and testing which works better via another (A/B, multi-armed bandit, or genetic algorithm). GenAI on the browser skips the intermediaries and probably loads faster than all the JS needed for 1200 analytics "partners" and reinventing the wheel badly by making it load text on the fly based on your scroll position rather than letting the browser handle it because someone just assumed a few thousand lines must be slow and never stopped to try it. Plus it being local means it might even work when you have no signal.
- RobotToaster 3y agoSo I can have an AI summarise the 5000 word essay the author used an AI to write?
- aurareturn 3y agoNo, but I'd like an LLM to filter out comments like yours and delete it before I read it.
- eimrine 3y ago:s/yours/this_one/
- postalrat 3y agoAll I need is to do is block more ads.
- raul-pilla 3y agoThe browser is the new OS.
- wkat4242 3y agoOnly according to some specific streams of thought within IT. It's not really the mainstream view. That's while apps still exist both on mobile and desktop.
- jabroni_salad 3y agoOpera has been bundling in a mail client, rss reader, calendar app etc for ages now. I also always thought it was weird but Opera people seem to like it, to the point that inspired browsers like Vivaldi also do that.
- nailz1911 3y agoFYI Vivaldi is run by ex-CEO and co-founder of Opera. =)
- jcranmer 3y agoDidn't Netscape bundle in a mail client before Opera did?
- guestbest 3y agoNetscape had a separate email and html composer application suite in the late 1990s. Opera had it built in along with a torrent client that couldn’t be separated from what I remember in a package that was smaller than almost all feature complete stand alone web browsers from 2003 and onwards.
- abdullahkhalids 3y agoSomething like that. Mozilla used the Netscape code to first create Mozilla Application Suite and then SeaMonkey[1]. It had Mail, IRC chat and a composer. [1] https://en.wikipedia.org/wiki/SeaMonkey https://en.wikipedia.org/wiki/SeaMonkey
- asadotzler 3y agoYes. 3 years before Opera. Netscape shipped Communicator in June of '97, the suite that added email and calendaring to browsing, based in part on their Collabra groupware acquisition in '96. Opera released v4 in June of 2000, three years later, with its email client.
- Rinzler89 3y ago>Opera has been bundling in a mail client, rss reader, calendar app etc for ages now. Opera has been bundling stuff like that, even a BitTorrent client, even when they were still a Norwegian owned company.
- PurpleRamen 3y agoOpera is an anything-Internet-suite. It's not just a browser, but aims to deliver interfaces for anything else people are doing with browsers. So anything for which most people would use a website or web-extensions, they deliver out of the box. And AI seems to be something that should be integrated deep into such an environment.
- noufalibrahim 3y agoMozilla (the browser) was like that wasn't it? Email, browser etc. and FF was the stripped down version.
- PurpleRamen 3y agoYes, it was called Mozilla Suite or Mozilla Application Suite, and is now named SeaMonkey.
- asadotzler 3y agoMozilla was like that because Mozilla was a clone of Netscape Communicator that had been like that since 1997 when Netscape acquired Collabra and turned the standalone browser into a groupware suite with email, calendaring, directory services, and more.
- asystole 3y agoExactly. You can claim to be the "first" at something very easily this way. Check out my browser, we're the first with an integrated flight simulator!
- bastawhiz 3y ago> why should your browser also be the interface to local AI models? Opera has historically been a junk drawer of technology. A BitTorrent client. A mail app. A chat app. A web server. There have been innumerable features which were often unnecessary or puzzling. It does not surprise me at all to hear that they're adding local LLM support.
- codercowmoo 3y agoI really wish people developing local applications would allow users to specify an API endpoint. Most applications use an OpenAI compatible API, and if they don't the browser's implementation of local model inference can be used.
- CharlesW 3y agoI've also been looking for an "LLM browsers" for iOS, i.e. apps which can work with LLM endpoints that I host, but I haven't been able to find anything.
- 29athrowaway 3y agoOpera was the reason CSS exists.
- 112233 3y agoYeah. Anyone knows how much of that opera is left over in the current opera?
- GreenVulpine 3y agoNot much. They are Chinese spying apparatus nowadays.
- AJ007 3y agoNothing, don't use it.
- asadotzler 3y agoNone. The board got rid of JSvT in 2011 I think and Haakon bailed in 2016 and today it's just a Blink skin owned by the Chinese.
- robin_reala 3y agoThat’s fairly revisionist. Bert Bos was never involved with Opera to the best of my knowledge, and Håkon Wium Lie only joined Opera later. The initial drafts of the CSS1 spec were done in 1994 well before Opera even existed, and the W3C’s history of CSS lists Opera as the third browser to implement at least part of the spec.[1] [1] https://www.w3.org/Style/CSS20/history.html https://www.w3.org/Style/CSS20/history.html
- 29athrowaway 3y agoViola had stylesheets I think.
- Eriks 3y agoBrowsers are for browsing internet. So many of them are now turning into bloatware..
- marban 3y agoOr sheep in wolf's clothing – like Arc.
- al_borland 3y agoCan you explain what you mean by this?
- rpastuszak 3y agoTheir PR narrative is mostly about being an ethical, user-first company. They’ve received a tonne of investment which is spent on expensive to maintain features like AI support in search. Someone will have to pay the bill and the general attitude when it comes to monetising browsers are adtech partnerships, not subscriptions.
- abhinavk 3y agoI think he meant to write wolf in sheep's clothing. It probably refers to their user-first marketing in contrast with their privacy issues and imminent enshittification to provide returns on VC money.
- mrb 3y agoOpposite viewpoint: browsers are for much more than "browsing". It's been 10-20 years that browsers have become the only app many people ever use, on a computer. So they already almost do everything: video conferencing, instant messaging, editing documents, sending emails, image editors, video games, calendar management, etc So at this point, frankly, "running a local LLM" is a pretty minor feature compared to the entire feature set of an average browser.
- deleted 3y ago[deleted]
- bdw5204 3y agoDoes that mean Opera doesn't consider Brave a "major browser" or that Brave's Leo is not a local model?
- dinkleberg 3y agoBrave's Leo appears to be on their servers, not run locally.
- tiptup300 3y agoThey advertise it to the user as though it's local. They just say "built right into the browser". Certainly misleading.
- eastendguy 3y agoIf browser extensions could access the local LLMs - Now THAT would be a very cool feature. Imagine what (test) automation extensions like Selenium IDE or Ui.vision could do then. Or translation extensions. Or.... so many cool options.
- evrimoztamur 3y agoDo the models get the content of the page fed into the context or not? Otherwise it is just plasterwork.
- stoev 3y agoI actually just submitted something quite relevant - a side project we soft-launched today that uses LLMs to analyse and group browser tabs across various browsers and devices. It turns out that it’s super demanding for today’s models, but with some optimisations and experimentation, we made it work pretty well. Running local LLMs might be a heavy order currently to get the job done, but in the near future I don't see a reason why it couldn't be done and it would definitely be quite helpful to people who have either many tabs open or a lot of bookmarks they want to organise. Ironically, Opera's extension shop is dead and no one reviews new submissions, but for anyone else who is interested in learning more about how we use LLMs to improve the browsing experience or even give the product a shot, here is a link: https://tabcrunch.com/technical_deep_dive.html https://tabcrunch.com/technical_deep_dive.html
- authorfly 3y agoThis is useful. Why the step of involving LLMs though? I note you cluster the tabs and then GPT-4 is involved to name them. But my tab groups don't usually need names - just the icons tells me what I mostly need to know. Could this work locally better using much smaller sentence-transformer models?
- stoev 3y agoThat's a good question. If you have many tabs open from the same few websites (depending on what those websites are), maybe just grouping them based on domain names would be enough to provide context. But LLMs are needed if you want the product to have a deeper understanding of everything that you are reading and really organise it into groups. You might be reading about architecture across three devices, multiple browsers, from a bunch of different websites. This gives you the opportunity to reunite them and really dive into that topic when you need to. LLMs are also used to create summaries of each page. So, if some content takes 30 minutes to read, you can have them extract all the interesting information for you in bullet points and based on that you can decide if its worth spending the 30 minutes or if you would rather just close that tab. So, in short, it's about capabilities. You can have just simple statistical models and regex rules filtering similar websites into predefined categories, or you can have a tool that truly organises your reading and shortens it for you. But for the latter, you need complex models handling a lot of context.
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- nerdjon 3y agoDon't get me wrong, I appreciate that there is at least some effort to bring these models local instead of shoving even more personal data into servers and who knows where it is going. But I am really frustrated by LLM's being installed into nearly everything with no real value. We still have fundemtal issues with the tech like it hallucinating and giving straight up bad information. The idea that it is being shoved into things, being normalized, is going to bite us real hard and I frankly hope soon. Also, like someone else said. Browsers are getting way too complicated.
- beefnugs 3y agoIts the perfect back door to get more data. "It could be useful someday! just keep giving us all the training data until it does something useful" It captures the cypto suckers, it traps smart software people with "unlimited potential!", it traps every single company management because they can fire everyone, it traps government in endless new reasons to authoritarian-smackdown. There is no stopping it, there is only people who stop trusting most technology
- eimrine 3y agoWhat are any possible reasons to have some generative network pre-trained by someone else as a part of some proprietary software?
- deleted 3y ago[deleted]
- lawlessone 3y ago[flagged]
- deleted 3y ago[deleted]
- josefresco 3y agoFYI for those not aware: Vivaldi(1) is the new "true Opera". The original Opera sold in 2016 to "a Chinese consortium"(2) 1: https://en.wikipedia.org/wiki/Vivaldi_(web_browser) https://en.wikipedia.org/wiki/Vivaldi_(web_browser) 2: https://www.nytimes.com/2016/02/11/business/dealbook/china-opera-kunlun-qihoo-golden-brick.html https://www.nytimes.com/2016/02/11/business/dealbook/china-o...
- dumbo-octopus 3y agoThat's a rather xenophobic take to be stating so loudly. Is there any reason to believe that being associated with Chinese intrinsically makes it lesser? Edit: they're all chromium anyways.
- trackflak 3y agohttps://en.wikipedia.org/wiki/Chinese_Communist_Party https://en.wikipedia.org/wiki/Chinese_Communist_Party
- scanny 3y agoFrom my understanding it comes from a law passed that allows unlimited access by state intelligence services in China to any firms customer data. "The most controversial sections of the law include Article 7 which potentially compels businesses registered or operating in the People's Republic of China to hand over information to Chinese intelligence agencies such as the MSS and to conceal the fact that they do so." [1] [1] https://en.wikipedia.org/wiki/National_Intelligence_Law_of_the_People%27s_Republic_of_China#Provisions https://en.wikipedia.org/wiki/National_Intelligence_Law_of_t...
- dumbo-octopus 3y agoThe same laws apply to American companies for American data requested by American agencies, is that better? And likely Norwegian too, but I don't know the specifics there.
- 3y ago
- ivanjermakov 3y agoI want my user agent to have nothing to do with LLMs. Last time I checked, the browser's purpose was to serve web applications.
- 6510 3y agoThe step from pages to applications was bigger.
- asadotzler 3y agoIncorrect. It's the web server's purpose to serve web applications. The browser's purpose is to act as the client, the user's agent, in rendering those web applications. If you're using Firefox, you've probably already got LLMs (well, NMT models which are built using large language corpora, and among other things, a transformer architecture, so close enough) and they're a good thing because they allow you to translate websites without sending that site and its address and content to a third party for remote translation and surveillance.
- ivanjermakov 3y agoGood point. I guess it boils down to providing value and solving real problems. Opposite to "we added AI to your browser because it's trendy".
- FredPret 3y agoPlaying fast and loose with "major"
- katsura 3y agoBased on some stats they are at more or less the same level as Firefox[0]. Obviously, you can't trust these 100%. [0] https://gs.statcounter.com/browser-market-share https://gs.statcounter.com/browser-market-share
- notatoad 3y agoif you limit it to desktop browsers, opera is 3% to firefox's 6%. i suspect the mobile stats include opera mini, which is not really the same thing.
- bootlooped 3y agoVery unfortunate, but true. Opera was the first browser I remember using besides Internet Explorer, and the amount of features it had compared to IE was a total game changer. I want to say Firefox was just called "Mozilla" back then and used a dinosaur logo. Eventually I switched to Firefox for quite a long time, but I'll always remember Opera fondly.
- asadotzler 3y agoI was an Opera user in the second half of the '90s when Netscape and IE were battling it out. When Netscape launched Mozilla in '98 I moved to that because it was the first open source browser and I had to be a part of that. I was for 25 years and during that time Opera went from good to bad to awful. In the 90s, Opera was a leader in CSS and had MDI. In the 2000s as we at Mozilla realized the groupware suite was a bad fit for most folks and leaned down from Seamonkey to Firefox, Opera went in the opposite direction, adding more and more features until it was almost unusable. Then they eventually kicked out their founder and his top lieutenant (CSS inventor) and finally sold to the Chinese. Tetzchner now leads Vivaldi and Håkon now does woodworking and occasionally runs for office as the founding member of the Pirate Party in Oslo, Norway.
- timetraveller26 3y agothe Opera guys seem to be trowing any idea they have to attract people, from a gaming browser, later a chess browser!? and who knows how many integrations. This seems like a good idea though, too bad they are untrustable at this point.
- deleted 3y ago[deleted]
- ddmma 3y agoIt can be instructed to browse on user behalf, this might be interesting feature presumably
- superkuh 3y agoThere is no Opera much in the way there is no Burma, Corsica, Bengal, or Ottoman Empire. People still know the name but the thing the name is associated with is not the thing. "Opera" is just some random chrome skin run for profitable spying by Zhou Yahui.
- reustle 3y agoFriendly reminder that Opera is now Chinese owned and you probably don’t want you be giving them your browsing history or account session data. https://en.m.wikipedia.org/wiki/Opera_(web_browser) https://en.m.wikipedia.org/wiki/Opera_(web_browser)
- jtietema 3y agoHasn't Firefox been doing this for a while already with their local translate feature? https://www.mozilla.org/en-US/firefox/features/translate/ https://www.mozilla.org/en-US/firefox/features/translate/ Maybe they mean LLM, instead of AI (not sure if Firefox translate models are LLM ir not)
- SushiHippie 3y agoThey use neural machine translation [0] models, they are artificial neural networks, but I don't think that you can really call them "AI". https://en.wikipedia.org/wiki/Neural_machine_translation https://en.wikipedia.org/wiki/Neural_machine_translation See also: https://browser.mt/ https://browser.mt/ https://blog.mozilla.org/en/mozilla/local-translation-add-on-project-bergamot/ https://blog.mozilla.org/en/mozilla/local-translation-add-on...
- asadotzler 3y agoFirefox's translation feature was built on Mozilla's Bergamot collab with Europe. It's an NMT as you note but it's also a set of language models trained on large language corpora using, among other techniques, the transformer architecture, so people calling it LLM translation don't bother me at all, and I'm the guy that brought it to Firefox (with my architect partner Andre Natal). The best article is probably https://hacks.mozilla.org/2022/06/training-efficient-neural-network-models-for-firefox-translations/ https://hacks.mozilla.org/2022/06/training-efficient-neural-... though that was before we started integrating it into Firefox proper.
- SushiHippie 3y agoThanks for the work you do, and your reply, very interesting!
- asadotzler 3y agoYes. I was responsible for kicking off that project. At Mozilla we'd been working with Europe on the Bergamot LLM translation tech and in 2022 we started work integrating it into Firefox. It began shipping last year. New language models are added regularly, all pivoting off English so French to German would actually be French -> English -> German and so you'd need two models for that. Each model was a few megabytes when I was starting on this but I've been away for a bit now so maybe they've improved on that but it did mean that we had to pull models down on demand because doubling the binary size for an initial Firefox download wasn't an option. (BTW, I was the guy that shipped Firefox v1 and my target for its download size back then was 3.5 MB or "about the size of an mp3". We made it down to about 4.5 MB which was close enough. It's over 10 times that size today.)
- saurik 3y agoSo while I am usually the person who would much rather the browser do almost nothing that isn't a hardware interface, requiring all software (including rendering) to be distributed as code by the website via the end-to-end principal--making the browser easy to implement and easy to secure / sandbox, as it is simply too important of an attack surface to have a billion file format parsing algorithms embedded within it--I actually would love (and I realize this isn't what Opera is doing, at least yet) to have the browser provide a way to get access to a user-selected LLM: the API surface for them--opaque text streaming in both directions--is sufficiently universal that I don't feel bad about the semantic lock-in and I just don't see any reasonable way to do this via the end-to-end principal that preserves user control over tradeoffs in privacy, functionality, and cost... if I go to a website that uses an LLM I should be the one choosing which LLM it is using, NOT the website!!, and if I want it to use some local model or the world's most powerful cloud model, I 1) should be in control of that selection and 2) pretty much have to be for local models to be feasible at all as I can't sit around downloading and caching gigabytes of data, separately, from every service that might make use of an LLM. (edit: Ok, in thinking about it a lot more maybe it makes more sense for this to be a separate daemon run next to the web browser--even if it comes with the web browser--which merely provides a localhost HTTP interface to the LLM, so it can also be shared by native apps... though, I am then unsure how web applications would be able to access them securely due to all of the security restrictions on cross-origin insecure port access.)