12 ms·

I'm reading a lot of criticism here, some I understand, but honestly, this sounds pretty good for a free LLM service: "Orbit doesn't require account creation or

I'm reading a lot of criticism here, some I understand, but honestly, this sounds pretty good for a free LLM service: "Orbit doesn't require account creation or save your session data." Maybe we shouldn't be too critical of Mozilla for providing a privacy-first LLM service - for free. The other big privacy-first LLM provider is Apple, which requires users to have their devices/subscriptions to use, and definitely uses advanced telemetry by default. Not to speak of all the other providers who are either paid or free-but-mine-your-data.

makingstuffs 2y ago
There’s also Leo from Brave: https://brave.com/leo/ https://brave.com/leo/
FireInsight 2y ago
A very similar service is https://duck.ai/ https://duck.ai/ from DDG.
- mentalgear 2y ago
  Definitely like DuckDuckGo - I have recently moved from Google to DDG as I noticed it serves the same, and even better plus privacy-first, results. Yet what you provide is not quite the same: An AI Chat Website vs a Browser Extension (Orbit).
  - pixxel 2y ago
    When the Bing API went offline a short while back, the DDG website was a blank page. No fall back, nothing. A Bing wrapper with ‘trust us’ privacy marketing.
    - oneeyedpigeon 2y ago
      How often does the Bing API go offline? Meanwhile, every single google search I carry out includes, right at the top, 'AI' generated nonsense which is sometimes wildly inaccurate. DDG never does that.
      - pixxel 2y ago
        > How often does the Bing API go offline? That’s not the point. Use any search engine in a terminal to strip the shit. Welcome to Hacker News.
kome 2y ago
[flagged]
- mentalgear 2y ago
  Agreed, I also often get that feeling of a double standard.
- vasco 2y ago
  Mozilla's problem for me is they wasted their market dominance by hiring grifters as their leaders and allowed bad actors to come in. I have nothing to base it off other than the results, but someone with a tinfoil hat would wonder if Google chose or influenced the choice of Mozillas leadership to allow Chrome to grow by destroying it from the inside. They were already basically bankrolling the whole company so it's not a big stretch.
- Y_Y 2y ago
  I didn't have the impression that HN was fine with all those things. Supposing it was, wouldn't it be ok to continue criticizing Mozilla, while also criticizing the bad behaviour of the others?
- rpastuszak 2y ago
  > Mozilla only problem is being transparent about telemetry. It's far from that: https://untested.sonnet.io/notes/defaults-matter-dont-assume-consent/#h-dollar510000000 https://untested.sonnet.io/notes/defaults-matter-dont-assume... (I still think FF is one of the best choices privacy wise, it's just that we don't have that many choices left) > HN is fine with tracking, smartphones, and every surveillance capitalist trick in the book (meta comment, thinking aloud here, feel free to skip) You might be conflating two different groups of users, each vocal about different subjects. That said, there's a big group of people on HN, who just enjoy being annoyingly contrarian. Then some people derive pleasure from pointing out some "moral fallacy" on whatever they perceive as the opposite part of their political/ideological spectrum ("you think my flashlight app collecting fingerprinting data from your phone is evil? well, your browser doesn't block all cookies by default you hypocrite!"). Life's too short for psychoanalysing the orange site, so I'll stop here.
- 2y ago
rtpg 2y ago
> Maybe we shouldn't be too critical of Mozilla for providing a privacy-first LLM service - for free. The other big privacy-first LLM provider is Apple, which requires users to have their devices/subscriptions to use, and definitely uses advanced telemetry by default. There is a very fundamental critique here: a service being offered for free like this is being subsidized, messing with the general market dynamics that really should be making all of these tools cost way more money to begin with. Of course Apple is also doing similar things, but for Mozilla to be doing it is quite frustrating.
- mentalgear 2y ago
  I do agree with your sentiment on cost, especially regarding exploding co2 emissions for LLMs, but it sounds like you're putting the blame mostly on Mozilla for what is market dynamics in capitalism, with most of the contenders being heavily VC-funded, and where Mozilla maybe just tries to signal that "AI" can be done responsibly also. If they don't try, people will blame them for "being late to the game/out of touch/ irrelevant" - so better to do it according to your principles and in-time I think.
  - caoilte 2y ago
    people will blame anyone for anything. mozilla is a non-profit so doesn't need to respond to market signals. it would do a lot more and a lot more sustainably if it focused on the core mission of its browser. but these costly "big swings" justify outrageous compensation packages for its executives and so it lets its browser wither.
    - rpastuszak 2y ago
      > it would do a lot more and a lot more sustainably if it focused on the core mission of its browser. Some numbers to put things in perspective: https://untested.sonnet.io/notes/defaults-matter-dont-assume-consent/ https://untested.sonnet.io/notes/defaults-matter-dont-assume... FF is still a solid choice, privacy-wise (with some manual tweaking), but just in the past few years 80-90% of their revenue came from adtech partnerships, so expect a series of rug pulls like the recent ones.
cdata 2y ago
It is overly generous to describe this as "privacy first." This looks like it's one ToS change away from being a privacy violating service. In Apple's case, they are putting some amount of work into making their privacy claims verifiable. Good will is no longer good enough. Verifiability should be the bar for trust in 3P privacy claims.
- mentalgear 2y ago
  This might be true for any run-of-the-mill service, but I do give Mozilla upfront credit as an entity and Firefox's privacy-leading track record. I haven't read the fine print, but I would be very surprised if there wasn't a robust layer of privacy/anonymisation involved. (Side note: I think the future is in-browser LLM (a la Gemini Nano), so I suspect they will eventually move there.) Also consider that Apple has the big pockets to build their own server hardware, to claim multiple layers of privacy - but also remember that when they first introduced "differential privacy" and claimed it would be totally anonymous, privacy researchers soon found out that Apple set the epsilon so low that even after a few requests to their service, the user could be de-anonymized. source: "Apple has boasted of its use of a cutting-edge data science known as "differential privacy." Researchers say they're doing it wrong." https://www.wired.com/story/apple-differential-privacy-shortcomings/ https://www.wired.com/story/apple-differential-privacy-short...
  - nulbyte 2y ago
    Mozilla's privacy-leading track record includes making Google the default search engine, running opt-in-by-default privacy-violating experiments, such as the Mr Robot fiasco[1], and opt-in-by-default collaboration with advertisers[2]. I still use Firefox, but I try to stay aware of changes, precisely because of Mozilla's privacy-leading gaffe record. 1. https://itsfoss.com/firefox-looking-glass-controversy/ https://itsfoss.com/firefox-looking-glass-controversy/ 2. https://www.pcmag.com/news/firefox-mozilla-data-collection-feature-sparks-privacy-concerns https://www.pcmag.com/news/firefox-mozilla-data-collection-f...
    - legacynl 2y ago
politelemon 2y ago
> The other big privacy-first LLM provider is Apple, which requires users to have their devices/subscriptions to use, and definitely uses advanced telemetry by default. By definition then, Apple's is most certainly not privacy first by any stretch of mental gymnastics.
Kuinox 2y ago
This is a local device LLM, outdated of a whole year, which is massive for LLMs, running on servers. Privacy side, there is little reason to run this on servers, a Pixel 6 could run 7B models at 5 token/s a year ago. It's bad by incompetence, 7B models of a year ago were terribly bad. It's not privacy-first enough, as it's possible to run the AI directly in the browser, but for some reason they didnt do it.
- mentalgear 2y ago
  It's unlikely that they are doing only a plain old LLM query. It's probably a more advanced setup (RAG, Page/LLM Summary Result Querying, maybe even some sort of Summarization validation checking). Also consider accessibility for low-end devices and development countries.
  - Kuinox 2y ago
    I don't see any indication that they use any RAG. It still doesn't explain why they use an old model. More recent models can run on any phone and have better quality than the model they are currently using.
- zozbot234 2y ago
  > it's possible to run the AI directly in the browser, but for some reason they didnt do it. Perhaps this is a proof of concept and they will have optional Firefox integration at a later time. Firefox uses local AI for webpage translation already.
  - freehorse 2y ago
    A locally run 7B model would consume a lot of resources for most people, but it would be nice as an option.
    - Kuinox 2y ago
      Todat 1B models are better that the 7B models they use currently.
throawayonthe 2y ago
[dead]
ksec 2y ago
>Maybe we shouldn't be too critical of Mozilla for providing a privacy-first LLM service It is just Mozilla have a tendency of chasing hype rather than focusing on what they are doing. During early smartphone era they spend most of the resources trying to write an OS with Javascript ( Firefox OS ) that works on a $35 Smartphone. Now they are doing it again with AI. Although this time around Firefox is in fairly good shape I guess this isn't too bad. But they need to figure out a way to generate revenue rather than relying on Google. And LLM service isn't it.
- facialwipe 2y ago
  During early smartphone era they spend most of the resources trying to write an OS Most of Mozilla’s resources have always been spent on Firefox. There was never a cycle where most of Mozilla’s resources were spent on FirefoxOS. But they need to figure out a way to generate revenue rather than relying on Google. The percentage of revenue from Google has fallen every year since 2016. Mozilla Corporation had a 33% profit margin in 2022 (The latest data on Wikipedia). It’s a fantastic business model.
  - idoubtit 2y ago
    > The percentage of revenue from Google has fallen every year since 2016. "Fallen" is too strong a word: Google still provided 81% of Mozilla's revenue in 2022. This share probably decreased in 2023, but that's mostly because the revenue increased by $40M thanks to financial operations (see "Interest and dividend income" in Mozilla's annual report for 2023).
    - sleepdeep 2y ago
      Who will pay the bills when Google is forced to end that search contracts?
      - Widdershin 2y ago
        One of the other bidders, like when Yahoo won the bidding in the early 2010s.
Sephr 2y ago
Privacy-first generally entails designing the service on a technical level to be unable to leak personal data. A promise not to store your data is not sufficient.
67n76n67n6 2y ago
That is always how this stuff works. Capture people with nice features and a friendly approach and then rug pull. There is zero chance I ever use this BS. I don't trust Apple either. I do not want LLM integration period because no corporation can ever under any circumstance be trusted with privacy in regards to it.
aforty 2y ago
All this costs money though and the eventual question is if you aren’t paying for it then who is?

makingstuffs 2y ago

There’s also Leo from Brave: https://brave.com/leo/ https://brave.com/leo/

FireInsight 2y ago

A very similar service is https://duck.ai/ https://duck.ai/ from DDG.

mentalgear 2y ago
Definitely like DuckDuckGo - I have recently moved from Google to DDG as I noticed it serves the same, and even better plus privacy-first, results. Yet what you provide is not quite the same: An AI Chat Website vs a Browser Extension (Orbit).
- pixxel 2y ago
  When the Bing API went offline a short while back, the DDG website was a blank page. No fall back, nothing. A Bing wrapper with ‘trust us’ privacy marketing.
  - oneeyedpigeon 2y ago
    How often does the Bing API go offline? Meanwhile, every single google search I carry out includes, right at the top, 'AI' generated nonsense which is sometimes wildly inaccurate. DDG never does that.
    - pixxel 2y ago
      > How often does the Bing API go offline? That’s not the point. Use any search engine in a terminal to strip the shit. Welcome to Hacker News.

kome 2y ago

[flagged]

mentalgear 2y ago
Agreed, I also often get that feeling of a double standard.
vasco 2y ago
Mozilla's problem for me is they wasted their market dominance by hiring grifters as their leaders and allowed bad actors to come in. I have nothing to base it off other than the results, but someone with a tinfoil hat would wonder if Google chose or influenced the choice of Mozillas leadership to allow Chrome to grow by destroying it from the inside. They were already basically bankrolling the whole company so it's not a big stretch.
Y_Y 2y ago
I didn't have the impression that HN was fine with all those things. Supposing it was, wouldn't it be ok to continue criticizing Mozilla, while also criticizing the bad behaviour of the others?
rpastuszak 2y ago
> Mozilla only problem is being transparent about telemetry. It's far from that: https://untested.sonnet.io/notes/defaults-matter-dont-assume-consent/#h-dollar510000000 https://untested.sonnet.io/notes/defaults-matter-dont-assume... (I still think FF is one of the best choices privacy wise, it's just that we don't have that many choices left) > HN is fine with tracking, smartphones, and every surveillance capitalist trick in the book (meta comment, thinking aloud here, feel free to skip) You might be conflating two different groups of users, each vocal about different subjects. That said, there's a big group of people on HN, who just enjoy being annoyingly contrarian. Then some people derive pleasure from pointing out some "moral fallacy" on whatever they perceive as the opposite part of their political/ideological spectrum ("you think my flashlight app collecting fingerprinting data from your phone is evil? well, your browser doesn't block all cookies by default you hypocrite!"). Life's too short for psychoanalysing the orange site, so I'll stop here.
2y ago

rtpg 2y ago

> Maybe we shouldn't be too critical of Mozilla for providing a privacy-first LLM service - for free. The other big privacy-first LLM provider is Apple, which requires users to have their devices/subscriptions to use, and definitely uses advanced telemetry by default. There is a very fundamental critique here: a service being offered for free like this is being subsidized, messing with the general market dynamics that really should be making all of these tools cost way more money to begin with. Of course Apple is also doing similar things, but for Mozilla to be doing it is quite frustrating.

mentalgear 2y ago
I do agree with your sentiment on cost, especially regarding exploding co2 emissions for LLMs, but it sounds like you're putting the blame mostly on Mozilla for what is market dynamics in capitalism, with most of the contenders being heavily VC-funded, and where Mozilla maybe just tries to signal that "AI" can be done responsibly also. If they don't try, people will blame them for "being late to the game/out of touch/ irrelevant" - so better to do it according to your principles and in-time I think.
- caoilte 2y ago
  people will blame anyone for anything. mozilla is a non-profit so doesn't need to respond to market signals. it would do a lot more and a lot more sustainably if it focused on the core mission of its browser. but these costly "big swings" justify outrageous compensation packages for its executives and so it lets its browser wither.
  - rpastuszak 2y ago
    > it would do a lot more and a lot more sustainably if it focused on the core mission of its browser. Some numbers to put things in perspective: https://untested.sonnet.io/notes/defaults-matter-dont-assume-consent/ https://untested.sonnet.io/notes/defaults-matter-dont-assume... FF is still a solid choice, privacy-wise (with some manual tweaking), but just in the past few years 80-90% of their revenue came from adtech partnerships, so expect a series of rug pulls like the recent ones.

cdata 2y ago

It is overly generous to describe this as "privacy first." This looks like it's one ToS change away from being a privacy violating service. In Apple's case, they are putting some amount of work into making their privacy claims verifiable. Good will is no longer good enough. Verifiability should be the bar for trust in 3P privacy claims.

mentalgear 2y ago
This might be true for any run-of-the-mill service, but I do give Mozilla upfront credit as an entity and Firefox's privacy-leading track record. I haven't read the fine print, but I would be very surprised if there wasn't a robust layer of privacy/anonymisation involved. (Side note: I think the future is in-browser LLM (a la Gemini Nano), so I suspect they will eventually move there.) Also consider that Apple has the big pockets to build their own server hardware, to claim multiple layers of privacy - but also remember that when they first introduced "differential privacy" and claimed it would be totally anonymous, privacy researchers soon found out that Apple set the epsilon so low that even after a few requests to their service, the user could be de-anonymized. source: "Apple has boasted of its use of a cutting-edge data science known as "differential privacy." Researchers say they're doing it wrong." https://www.wired.com/story/apple-differential-privacy-shortcomings/ https://www.wired.com/story/apple-differential-privacy-short...
- nulbyte 2y ago
  Mozilla's privacy-leading track record includes making Google the default search engine, running opt-in-by-default privacy-violating experiments, such as the Mr Robot fiasco[1], and opt-in-by-default collaboration with advertisers[2]. I still use Firefox, but I try to stay aware of changes, precisely because of Mozilla's privacy-leading gaffe record. 1. https://itsfoss.com/firefox-looking-glass-controversy/ https://itsfoss.com/firefox-looking-glass-controversy/ 2. https://www.pcmag.com/news/firefox-mozilla-data-collection-feature-sparks-privacy-concerns https://www.pcmag.com/news/firefox-mozilla-data-collection-f...
  - legacynl 2y ago

politelemon 2y ago

> The other big privacy-first LLM provider is Apple, which requires users to have their devices/subscriptions to use, and definitely uses advanced telemetry by default. By definition then, Apple's is most certainly not privacy first by any stretch of mental gymnastics.

Kuinox 2y ago

This is a local device LLM, outdated of a whole year, which is massive for LLMs, running on servers. Privacy side, there is little reason to run this on servers, a Pixel 6 could run 7B models at 5 token/s a year ago. It's bad by incompetence, 7B models of a year ago were terribly bad. It's not privacy-first enough, as it's possible to run the AI directly in the browser, but for some reason they didnt do it.

mentalgear 2y ago
It's unlikely that they are doing only a plain old LLM query. It's probably a more advanced setup (RAG, Page/LLM Summary Result Querying, maybe even some sort of Summarization validation checking). Also consider accessibility for low-end devices and development countries.
- Kuinox 2y ago
  I don't see any indication that they use any RAG. It still doesn't explain why they use an old model. More recent models can run on any phone and have better quality than the model they are currently using.
zozbot234 2y ago
> it's possible to run the AI directly in the browser, but for some reason they didnt do it. Perhaps this is a proof of concept and they will have optional Firefox integration at a later time. Firefox uses local AI for webpage translation already.
- freehorse 2y ago
  A locally run 7B model would consume a lot of resources for most people, but it would be nice as an option.
  - Kuinox 2y ago
    Todat 1B models are better that the 7B models they use currently.

throawayonthe 2y ago

[dead]

ksec 2y ago

>Maybe we shouldn't be too critical of Mozilla for providing a privacy-first LLM service It is just Mozilla have a tendency of chasing hype rather than focusing on what they are doing. During early smartphone era they spend most of the resources trying to write an OS with Javascript ( Firefox OS ) that works on a $35 Smartphone. Now they are doing it again with AI. Although this time around Firefox is in fairly good shape I guess this isn't too bad. But they need to figure out a way to generate revenue rather than relying on Google. And LLM service isn't it.

facialwipe 2y ago
During early smartphone era they spend most of the resources trying to write an OS Most of Mozilla’s resources have always been spent on Firefox. There was never a cycle where most of Mozilla’s resources were spent on FirefoxOS. But they need to figure out a way to generate revenue rather than relying on Google. The percentage of revenue from Google has fallen every year since 2016. Mozilla Corporation had a 33% profit margin in 2022 (The latest data on Wikipedia). It’s a fantastic business model.
- idoubtit 2y ago
  > The percentage of revenue from Google has fallen every year since 2016. "Fallen" is too strong a word: Google still provided 81% of Mozilla's revenue in 2022. This share probably decreased in 2023, but that's mostly because the revenue increased by $40M thanks to financial operations (see "Interest and dividend income" in Mozilla's annual report for 2023).
  - sleepdeep 2y ago
    Who will pay the bills when Google is forced to end that search contracts?
    - Widdershin 2y ago
      One of the other bidders, like when Yahoo won the bidding in the early 2010s.

Sephr 2y ago

Privacy-first generally entails designing the service on a technical level to be unable to leak personal data. A promise not to store your data is not sufficient.

67n76n67n6 2y ago

That is always how this stuff works. Capture people with nice features and a friendly approach and then rug pull. There is zero chance I ever use this BS. I don't trust Apple either. I do not want LLM integration period because no corporation can ever under any circumstance be trusted with privacy in regards to it.

aforty 2y ago

All this costs money though and the eventual question is if you aren’t paying for it then who is?