9 ms·

I think this patent is invalid based on the disclosure date. The attention is all you need paper was on Arxiv on 12 Jun 2017 and you have to patent within one

I think this patent is invalid based on the disclosure date. The attention is all you need paper was on Arxiv on 12 Jun 2017 and you have to patent within one year. In this case June 28th 2018 is too late..

avsteele 3y ago
I doubt it. They probably filed a provisional earlier.
- theGnuMe 3y ago
  Ah yes, looks like they did.
kurthr 3y ago
Apparently, they thought of that, Line 3 of the patent application US10452978B2: Provisional Application No. 62/510,256, filed on May 23, 2017 There are however other patents and applications: Neural machine translation with latent tree attention US20180300317A1 James BRADBURY I'm not an expert so I can't read the spec and claims as to relevance with authority. However, it still wouldn't count directly as prior art as it was published in Oct 2018. Patents are now first to file and not first to invent (to match the rest of the world). edit: They do also site non-patent prior are regarding attention. Whether this covers self attention I'm not clear and of course their claims have been reviewed by the examiner in light of the art so they're presumed valid until re-exam. Luong et al. "Effective approaches to attention based neural machine translation," arXiv 1508.04025v2, Sep. 20, 2015, 11 pages.Luong et al. "Effective approaches to attention based neural machine translation," arXiv 1508.04025v2, Sep. 20, 2015, 11 pages.
- cma 3y ago
  > However, it still wouldn't count directly as prior art as it was published in Oct 2018. Patents are now first to file and not first to invent (to match the rest of the world). There is an investor's grace period, as long as your own public prior art disclosure is the earliest, you get 1 year to file even under first to file. First to file refers to people filing for undisclosed inventions, disclosure acts as prior art against anyone else filing, along with the grace period for delaying your filing. Often "wow the company detailing internals at this conference is so generous!" is actually them getting an extra year on the of patent expiration date by taking advantage of the grace period.
  - cma 3y ago
    *inventor's grace period

avsteele 3y ago

I doubt it. They probably filed a provisional earlier.

theGnuMe 3y ago
Ah yes, looks like they did.

kurthr 3y ago

Apparently, they thought of that, Line 3 of the patent application US10452978B2: Provisional Application No. 62/510,256, filed on May 23, 2017 There are however other patents and applications: Neural machine translation with latent tree attention US20180300317A1 James BRADBURY I'm not an expert so I can't read the spec and claims as to relevance with authority. However, it still wouldn't count directly as prior art as it was published in Oct 2018. Patents are now first to file and not first to invent (to match the rest of the world). edit: They do also site non-patent prior are regarding attention. Whether this covers self attention I'm not clear and of course their claims have been reviewed by the examiner in light of the art so they're presumed valid until re-exam. Luong et al. "Effective approaches to attention based neural machine translation," arXiv 1508.04025v2, Sep. 20, 2015, 11 pages.Luong et al. "Effective approaches to attention based neural machine translation," arXiv 1508.04025v2, Sep. 20, 2015, 11 pages.

cma 3y ago
> However, it still wouldn't count directly as prior art as it was published in Oct 2018. Patents are now first to file and not first to invent (to match the rest of the world). There is an investor's grace period, as long as your own public prior art disclosure is the earliest, you get 1 year to file even under first to file. First to file refers to people filing for undisclosed inventions, disclosure acts as prior art against anyone else filing, along with the grace period for delaying your filing. Often "wow the company detailing internals at this conference is so generous!" is actually them getting an extra year on the of patent expiration date by taking advantage of the grace period.
- cma 3y ago
  *inventor's grace period