4 ms·

This is a lot of money going into compute. I will say this again. EU is sleeping on the opportunity to throw money in an opensource initiative, in a field were

by v4dok 3y ago

This is a lot of money going into compute. I will say this again. EU is sleeping on the opportunity to throw money in an opensource initiative, in a field were money matter and the field is still (kind of) level.

bko 3y ago
> [2008] France has been given the green light by the European Commission for a $152 million government grant for a consortium building a European rival to U.S. internet search giant Google. I don't think government funding to compete with private businesses works well https://www.hollywoodreporter.com/business/business-news/ec-funds-france-build-google-106934/ https://www.hollywoodreporter.com/business/business-news/ec-...
- slashdev 3y ago
  Governments that think they can innovate through consortiums. That's either ignorance or pork barrel politics. Either way it's a sad waste of tax payer money.
- hbbio 3y ago
  They also funded 2 competitors to AWS, just to make sure we own cloud computing. Two is always better than one, right, by ensuring they'll compete with each other. Another bright idea was to let both projects be managed by large reputable French corporations that everybody trusts. With no software DNA. How come did both fail? Edit: One of the largest European provider today, OVH, who existed at the time and was already the leader in France was explicitly left out of both projects... Because the founder is not a guy we can trust you know, he didn't attend the best schools.
  - tough 3y ago
    C'est la vie Govt=Legal grift It's the same all over europe mostly, sadly. We were pioneers in the medieval times, we can follow up the leaders barely now
  - emptysongglass 3y ago
    Scaleway is actually fantastic for what it's worth. You can get extremely cheap K8s clusters and most things a startup would need.
    - hbbio 3y ago
      Another private company. The two heavily subsidized projects were: - https://en.wikipedia.org/wiki/Cloudwatt https://en.wikipedia.org/wiki/Cloudwatt - https://login.numergy.com/login?service=https%3A%2F%2Fwww.numergy.com%2Fespace-client https://login.numergy.com/login?service=https%3A%2F%2Fwww.nu... For the one still "live", details include French URL names :) Edit: A Google Translate of the home page. Close your eyes, imagine a homepage highlighting the essence of cloud computing: Your Numergy space Access the administration of your virtual machines. Secure connection Username Password Forgot your password ? Administration of your VMs Administer your virtual machines in real time, monitor their activity, your bandwidth consumption and the use of your storage spaces. Changing your personal information Access the customer area and modify your personal information in just a few clicks: surname, first name, address. Securing your data Remember to change your password regularly to maintain an optimal level of security.
- v4dok 3y ago
  No consortiums, just subsidize cloud or infra costs only.
seydor 3y ago
Sure, let's make an EU commercial LLM. Let's start by scraping all the Francophone internet. Then let's remove all the PII data and potentially PII-data. Easy-peasy. Then let's train our network so as not to spew out or make up PII data - easy peasy Then let's make it able to delete PII data that it has inadvertedly collected on request. Simultaneously it should be recording all the conversations for safety reasons. that must be possible somehow And let's make sure it never impersonates or makes up defamatory content - that must be super easy. And let's make it explain itself. But explain truthfully, by giving an oath, not like ChatGPT that likes making things up. Looks very doable to me
- omneity 3y ago
  BLOOM has been trained on a 3M€ grant from French research agencies CNRS and GENCI. Doesn’t have any of the constraints you’re talking about.
  - lhl 3y ago
    BLOOM's training corpus ROOTS did make some efforts at removing PII https://arxiv.org/pdf/2303.03915.pdf https://arxiv.org/pdf/2303.03915.pdf btw, but AFAICT that was not at the behest of the French government.
- raverbashing 3y ago
  The secret is not scraping PII in the first place (which is not really difficult, though it requires some planning)
  - seydor 3y ago
    not sure if it can ever be possible. i can ask chatGPT to do stylography analysis on our comments, find our other accounts and go from there. I'm pretty sure most pieces of human-generated data is identifiable at this point
    - raverbashing 3y ago
truetraveller 3y ago
[flagged]
in3d 3y ago
France paid for Bloom: https://www.technologyreview.com/2022/07/12/1055817/inside-a-radical-new-project-to-democratize-ai/ https://www.technologyreview.com/2022/07/12/1055817/inside-a... It hasn’t been very impressive (undertrained I believe).
0xDEF 3y ago
Devil's advocate: Why should EU tax payers fund open source initiatives and not proprietary European initiatives that will help Europe compete against American and Chinese tech giants?
- crop_rotation 3y ago
  They should fund something. A proprietary European initiative is definitely better. The open source alternative should be EU's last resort. But as it stands the EU is nowhere to be found. I am not sure how impactful LLMs will be on a scale of autocomplete to industrial revolution, but the EU needs to notice it and plan for something.
fancyfredbot 3y ago
"Let someone else pay for the open source LLM weights" said everyone.

by v4dok 3y ago

bko 3y ago

> [2008] France has been given the green light by the European Commission for a $152 million government grant for a consortium building a European rival to U.S. internet search giant Google. I don't think government funding to compete with private businesses works well https://www.hollywoodreporter.com/business/business-news/ec-funds-france-build-google-106934/ https://www.hollywoodreporter.com/business/business-news/ec-...

slashdev 3y ago
Governments that think they can innovate through consortiums. That's either ignorance or pork barrel politics. Either way it's a sad waste of tax payer money.
hbbio 3y ago
They also funded 2 competitors to AWS, just to make sure we own cloud computing. Two is always better than one, right, by ensuring they'll compete with each other. Another bright idea was to let both projects be managed by large reputable French corporations that everybody trusts. With no software DNA. How come did both fail? Edit: One of the largest European provider today, OVH, who existed at the time and was already the leader in France was explicitly left out of both projects... Because the founder is not a guy we can trust you know, he didn't attend the best schools.
- tough 3y ago
  C'est la vie Govt=Legal grift It's the same all over europe mostly, sadly. We were pioneers in the medieval times, we can follow up the leaders barely now
- emptysongglass 3y ago
  Scaleway is actually fantastic for what it's worth. You can get extremely cheap K8s clusters and most things a startup would need.
  - hbbio 3y ago
    Another private company. The two heavily subsidized projects were: - https://en.wikipedia.org/wiki/Cloudwatt https://en.wikipedia.org/wiki/Cloudwatt - https://login.numergy.com/login?service=https%3A%2F%2Fwww.numergy.com%2Fespace-client https://login.numergy.com/login?service=https%3A%2F%2Fwww.nu... For the one still "live", details include French URL names :) Edit: A Google Translate of the home page. Close your eyes, imagine a homepage highlighting the essence of cloud computing: Your Numergy space Access the administration of your virtual machines. Secure connection Username Password Forgot your password ? Administration of your VMs Administer your virtual machines in real time, monitor their activity, your bandwidth consumption and the use of your storage spaces. Changing your personal information Access the customer area and modify your personal information in just a few clicks: surname, first name, address. Securing your data Remember to change your password regularly to maintain an optimal level of security.
v4dok 3y ago
No consortiums, just subsidize cloud or infra costs only.

seydor 3y ago

Sure, let's make an EU commercial LLM. Let's start by scraping all the Francophone internet. Then let's remove all the PII data and potentially PII-data. Easy-peasy. Then let's train our network so as not to spew out or make up PII data - easy peasy Then let's make it able to delete PII data that it has inadvertedly collected on request. Simultaneously it should be recording all the conversations for safety reasons. that must be possible somehow And let's make sure it never impersonates or makes up defamatory content - that must be super easy. And let's make it explain itself. But explain truthfully, by giving an oath, not like ChatGPT that likes making things up. Looks very doable to me

omneity 3y ago
BLOOM has been trained on a 3M€ grant from French research agencies CNRS and GENCI. Doesn’t have any of the constraints you’re talking about.
- lhl 3y ago
  BLOOM's training corpus ROOTS did make some efforts at removing PII https://arxiv.org/pdf/2303.03915.pdf https://arxiv.org/pdf/2303.03915.pdf btw, but AFAICT that was not at the behest of the French government.
raverbashing 3y ago
The secret is not scraping PII in the first place (which is not really difficult, though it requires some planning)
- seydor 3y ago
  not sure if it can ever be possible. i can ask chatGPT to do stylography analysis on our comments, find our other accounts and go from there. I'm pretty sure most pieces of human-generated data is identifiable at this point
  - raverbashing 3y ago

truetraveller 3y ago

[flagged]

in3d 3y ago

France paid for Bloom: https://www.technologyreview.com/2022/07/12/1055817/inside-a-radical-new-project-to-democratize-ai/ https://www.technologyreview.com/2022/07/12/1055817/inside-a... It hasn’t been very impressive (undertrained I believe).

0xDEF 3y ago

Devil's advocate: Why should EU tax payers fund open source initiatives and not proprietary European initiatives that will help Europe compete against American and Chinese tech giants?

crop_rotation 3y ago
They should fund something. A proprietary European initiative is definitely better. The open source alternative should be EU's last resort. But as it stands the EU is nowhere to be found. I am not sure how impactful LLMs will be on a scale of autocomplete to industrial revolution, but the EU needs to notice it and plan for something.

fancyfredbot 3y ago

"Let someone else pay for the open source LLM weights" said everyone.