6 ms·

Is there an update to this article? Since the article was written: - AMD has gen4 available - Genoa - there's NVMe drives 15tb capacity each

rz2k 3y ago
And, it looks like with PCIe 5.0 x4 performance exceeds 2,500,000 IOPS reads, and 14 GBps sequential read rate.
tyingq 3y ago
I would guess they try to get 3-5 years out of a new server, and this was done in 2021...so still a bit early for a refresh. The servers prior to this one had E5-2650 processors, so they would have been bought sometime between 2013-2016. Meaning their last refresh cycle was at least 5 years.
wmf 3y ago
Those 2021 servers will probably last them 4-5 years so they haven't upgraded yet.
mcpherrinm 3y ago
The same hardware is still in use today, and I expect will continue to be used for some time. The biggest thing that’s happened in the meantime is reducing load on the databases by pushing some of the OCSP load out to redis caches: https://letsencrypt.org/2022/12/15/ocspcaching.html https://letsencrypt.org/2022/12/15/ocspcaching.html By not storing OCSP in the MariaDBs, we reduced write volume and storage size significantly. The next big thing is going to be sharding the database so multiple servers can share the write loads, and some queries moved out to redis caches. (I work for Let’s Encrypt)
- efxhoy 3y ago
  Why add caching and sharding? Is the current architecture not going to keep up with demand?
  - mcpherrinm 3y ago
    Growth, plus reliability by reducing the SPOF. We are beginning to hit the limits of what we can do with the current single-writer database in this hardware. As well, the size of the data is larger than I’d like for a single MariaDB cluster. That makes operations like setting up new replicas take a long time. Being able to operate on smaller shards will help a lot at parallelizing some of that work.

6 ms·

Is there an update to this article? Since the article was written: - AMD has gen4 available - Genoa - there's NVMe drives 15tb capacity each

by PaywallBuster 3y ago

rz2k 3y ago
And, it looks like with PCIe 5.0 x4 performance exceeds 2,500,000 IOPS reads, and 14 GBps sequential read rate.
tyingq 3y ago
I would guess they try to get 3-5 years out of a new server, and this was done in 2021...so still a bit early for a refresh. The servers prior to this one had E5-2650 processors, so they would have been bought sometime between 2013-2016. Meaning their last refresh cycle was at least 5 years.
wmf 3y ago
Those 2021 servers will probably last them 4-5 years so they haven't upgraded yet.
mcpherrinm 3y ago
The same hardware is still in use today, and I expect will continue to be used for some time. The biggest thing that’s happened in the meantime is reducing load on the databases by pushing some of the OCSP load out to redis caches: https://letsencrypt.org/2022/12/15/ocspcaching.html https://letsencrypt.org/2022/12/15/ocspcaching.html By not storing OCSP in the MariaDBs, we reduced write volume and storage size significantly. The next big thing is going to be sharding the database so multiple servers can share the write loads, and some queries moved out to redis caches. (I work for Let’s Encrypt)
- efxhoy 3y ago
  Why add caching and sharding? Is the current architecture not going to keep up with demand?
  - mcpherrinm 3y ago
    Growth, plus reliability by reducing the SPOF. We are beginning to hit the limits of what we can do with the current single-writer database in this hardware. As well, the size of the data is larger than I’d like for a single MariaDB cluster. That makes operations like setting up new replicas take a long time. Being able to operate on smaller shards will help a lot at parallelizing some of that work.