5 ms·

>2018-06-28 - Application filed by Google LLC 2019-10-22 - Application granted Should probably put a (2019) in the title. Furthermore considering how fast the

>2018-06-28 - Application filed by Google LLC 2019-10-22 - Application granted Should probably put a (2019) in the title. Furthermore considering how fast the ML space moves, the fact that google hasn't used this to create a model significantly better than competitors seems to show that the patented architecture did not perform better than others.

dragonwriter 3y ago
Or, it shows that Google can’t deliver products in this space to (literally) save its life, no matter how good its technology is (which they’ve also shown lots of other ways.)
- onlyrealcuzzo 3y ago
  Yes, the company with 8 billion user products can't do anything.
  - brookst 3y ago
    Do you think Google has 8 billion products in the AI space? Because it seems pretty clear that the criticism is that Google is an advertising company that uses search as leadgen, and is struggling to ship AI products.
    - cma 3y ago
      Google translate is widely used. Search also uses transformers for generating snippit summary boxes.
      - brookst 3y ago
        So, would you say Google has been as successful with AI products as anyone else?
        onlyrealcuzzo 3y ago
        Who is making a ton of money on AI? What AI product has more users than Google Translate?
gerash 3y ago
The patent was granted in 2019. The patented architecture is the Transformer architecture that everyone and their mom is now using for LLMs. Patents in your portfolio help defend against trolls and other hostile entities.
- tough 3y ago
  Let's hope Google doesn't become the Troll here
  - lettergram 3y ago
    Is it really trolling if you’re utilizing the tech and someone is infringing?
    - mabbo 3y ago
      Consider Amazon One-click purchase. That patent was one of very few (out of tens of thousands) that they defended heavily. Every other company on the planet had to have at least two steps to purchase an item or Amazon would come knocking with lawyers. They did this until the day that patent finally expired. One has to ask: was that to the benefit of society? Google may make good use of this tech. But it would be better for all of us if everyone did and didn't have to pay them a fee for the privilege.
      - amelius 3y ago
        > Google may make good use of this tech. But it would be better for all of us if everyone did and didn't have to pay them a fee for the privilege. This is reverse logic because then Google would not have published the tech and would have kept it a trade secret.
      - lettergram 3y ago
        100% it’s probably worth it in the end. You quite literally may not have had Amazon without it. In this case, google might not have funded the development of transformers which could dramatically reduce costs of everything for humanity. Same goes for drug development. That said, I think there’s a question around software patents and how long they should last. Perhaps it should just be to recoup costs, plus some multiple. I’m not sure.
theGnuMe 3y ago
I think this patent is invalid based on the disclosure date. The attention is all you need paper was on Arxiv on 12 Jun 2017 and you have to patent within one year. In this case June 28th 2018 is too late..
- avsteele 3y ago
  I doubt it. They probably filed a provisional earlier.
  - theGnuMe 3y ago
    Ah yes, looks like they did.
- kurthr 3y ago
  Apparently, they thought of that, Line 3 of the patent application US10452978B2: Provisional Application No. 62/510,256, filed on May 23, 2017 There are however other patents and applications: Neural machine translation with latent tree attention US20180300317A1 James BRADBURY I'm not an expert so I can't read the spec and claims as to relevance with authority. However, it still wouldn't count directly as prior art as it was published in Oct 2018. Patents are now first to file and not first to invent (to match the rest of the world). edit: They do also site non-patent prior are regarding attention. Whether this covers self attention I'm not clear and of course their claims have been reviewed by the examiner in light of the art so they're presumed valid until re-exam. Luong et al. "Effective approaches to attention based neural machine translation," arXiv 1508.04025v2, Sep. 20, 2015, 11 pages.Luong et al. "Effective approaches to attention based neural machine translation," arXiv 1508.04025v2, Sep. 20, 2015, 11 pages.
  - cma 3y ago
    > However, it still wouldn't count directly as prior art as it was published in Oct 2018. Patents are now first to file and not first to invent (to match the rest of the world). There is an investor's grace period, as long as your own public prior art disclosure is the earliest, you get 1 year to file even under first to file. First to file refers to people filing for undisclosed inventions, disclosure acts as prior art against anyone else filing, along with the grace period for delaying your filing. Often "wow the company detailing internals at this conference is so generous!" is actually them getting an extra year on the of patent expiration date by taking advantage of the grace period.
CaptainNegative 3y ago
Silly question, how did they get their patent granted in 16 months when I have some patents pending for more than 3x as long?
- netdur 3y ago
  they have better lawyer

dragonwriter 3y ago

Or, it shows that Google can’t deliver products in this space to (literally) save its life, no matter how good its technology is (which they’ve also shown lots of other ways.)

onlyrealcuzzo 3y ago
Yes, the company with 8 billion user products can't do anything.
- brookst 3y ago
  Do you think Google has 8 billion products in the AI space? Because it seems pretty clear that the criticism is that Google is an advertising company that uses search as leadgen, and is struggling to ship AI products.
  - cma 3y ago
    Google translate is widely used. Search also uses transformers for generating snippit summary boxes.
    - brookst 3y ago
      So, would you say Google has been as successful with AI products as anyone else?
      - onlyrealcuzzo 3y ago
        Who is making a ton of money on AI? What AI product has more users than Google Translate?

gerash 3y ago

The patent was granted in 2019. The patented architecture is the Transformer architecture that everyone and their mom is now using for LLMs. Patents in your portfolio help defend against trolls and other hostile entities.

tough 3y ago
Let's hope Google doesn't become the Troll here
- lettergram 3y ago
  Is it really trolling if you’re utilizing the tech and someone is infringing?
  - mabbo 3y ago
    Consider Amazon One-click purchase. That patent was one of very few (out of tens of thousands) that they defended heavily. Every other company on the planet had to have at least two steps to purchase an item or Amazon would come knocking with lawyers. They did this until the day that patent finally expired. One has to ask: was that to the benefit of society? Google may make good use of this tech. But it would be better for all of us if everyone did and didn't have to pay them a fee for the privilege.
    - amelius 3y ago
      > Google may make good use of this tech. But it would be better for all of us if everyone did and didn't have to pay them a fee for the privilege. This is reverse logic because then Google would not have published the tech and would have kept it a trade secret.
    - lettergram 3y ago
      100% it’s probably worth it in the end. You quite literally may not have had Amazon without it. In this case, google might not have funded the development of transformers which could dramatically reduce costs of everything for humanity. Same goes for drug development. That said, I think there’s a question around software patents and how long they should last. Perhaps it should just be to recoup costs, plus some multiple. I’m not sure.

theGnuMe 3y ago

I think this patent is invalid based on the disclosure date. The attention is all you need paper was on Arxiv on 12 Jun 2017 and you have to patent within one year. In this case June 28th 2018 is too late..

avsteele 3y ago
I doubt it. They probably filed a provisional earlier.
- theGnuMe 3y ago
  Ah yes, looks like they did.
kurthr 3y ago
Apparently, they thought of that, Line 3 of the patent application US10452978B2: Provisional Application No. 62/510,256, filed on May 23, 2017 There are however other patents and applications: Neural machine translation with latent tree attention US20180300317A1 James BRADBURY I'm not an expert so I can't read the spec and claims as to relevance with authority. However, it still wouldn't count directly as prior art as it was published in Oct 2018. Patents are now first to file and not first to invent (to match the rest of the world). edit: They do also site non-patent prior are regarding attention. Whether this covers self attention I'm not clear and of course their claims have been reviewed by the examiner in light of the art so they're presumed valid until re-exam. Luong et al. "Effective approaches to attention based neural machine translation," arXiv 1508.04025v2, Sep. 20, 2015, 11 pages.Luong et al. "Effective approaches to attention based neural machine translation," arXiv 1508.04025v2, Sep. 20, 2015, 11 pages.
- cma 3y ago
  > However, it still wouldn't count directly as prior art as it was published in Oct 2018. Patents are now first to file and not first to invent (to match the rest of the world). There is an investor's grace period, as long as your own public prior art disclosure is the earliest, you get 1 year to file even under first to file. First to file refers to people filing for undisclosed inventions, disclosure acts as prior art against anyone else filing, along with the grace period for delaying your filing. Often "wow the company detailing internals at this conference is so generous!" is actually them getting an extra year on the of patent expiration date by taking advantage of the grace period.

CaptainNegative 3y ago

Silly question, how did they get their patent granted in 16 months when I have some patents pending for more than 3x as long?

netdur 3y ago
they have better lawyer