3 ms·
110MB/s is gigabit. It’s limited to gigabit networking and only has 1Gbps out from the cluster board. So there’s no way to do an aggregate speed of more than 1G
by lathiat 4y ago
110MB/s is gigabit. It’s limited to gigabit networking and only has 1Gbps out from the cluster board. So there’s no way to do an aggregate speed of more than 1Gbps/110MBs on this particular cluster board.
If each pis Ethernet was broken out individually and you used a 10G uplink switch or multiple 1G client ports then you could do better.
The write speed being lower than the read speed will be because writes have to be replicated to two other nodes in the ceph cluster (everything has 3 replicas) which are also sharing bandwidth on those same 1G links. Reads don’t need to replicate so can consume the full bandwidth.
So basically it’s all network limited for this use case. Needs a 2.5G uplink, LACP link aggregation or individual Ethernet ports to do better.
- sitkack 4y agoWhich ICs would you use for this? Do you have something in mind?
- Retr0id 4y agoI'm not sure about specific ICs, but if you took a look inside a Netgear GS110MX you'd have a good candidate IC.
- magicalhippo 4y agoJust a random search on Mouser, but something like the BCM53134O[1] as four 1GbE ports, and one 2.5GbE port. A bit pricier you have the BCM53156XU[2] with eight 1GbE ports and a 10G port for fiber. Not my field so could be other, better parts. [1]: https://eu.mouser.com/ProductDetail/Broadcom-Avago/BCM53134OKFBG?qs=gev7jUp%252BQ%252BiefC9t0Lfu%252Bg%3D%3D https://eu.mouser.com/ProductDetail/Broadcom-Avago/BCM53134O... [2]: https://eu.mouser.com/ProductDetail/Broadcom-Avago/BCM53156XUB1KFBG?qs=PqoDHHvF648C%252BWq0llwsOg%3D%3D https://eu.mouser.com/ProductDetail/Broadcom-Avago/BCM53156X...