3 ms·
How much bandwidth does Twitter use for images and videos? Less than 1.4Tb/s globally? If so, we could probably fit that onto a second machine. We can curr
by drewg123 4y ago
How much bandwidth does Twitter use for images and videos? Less than 1.4Tb/s globally? If so, we could probably fit that onto a second machine. We can currently serve over 700Gb/s from a dual-socket Milan based server[1]. I'm still waiting for hardware, but assuming there are no new bottlenecks, that should directly scale up to 1.4Tb/s with Genoa and ConnectX-7, given the IO pathways are all at least twice the bandwidth of the previous generation.
There are storage size issues (like how big is their long tail; quite large I'd imagine), but its a fun thing to think about.
[1] https://people.freebsd.org/~gallatin/talks/euro2022.pdf https://people.freebsd.org/~gallatin/talks/euro2022.pdf
- cortesoft 4y agoIt is way more than 1.4TBs a second globally.
- xyzzy123 4y agoI wonder how much is api traffic and how much is assets & images.
- koolba 4y agoI wonder how much of that is crypto spam bots replying to each other.
- skywhopper 4y agoCan you detect and block those spam bots with less effort than it would take to process them?
- koolba 4y agoThat’s a not so simple calculation as comparing the raw request process time isn’t the complete picture. Spam bot content must be persistently stored, indexed, archived, etc so the long term cost is much more than the one off POST to create the entity. You’d also have to quantify the improved user experience from seeing less spam v.s. inflated ad revenue for garbage views / content.
- deleted 4y ago[deleted]
- _zoltan_ 4y agoin this specific discussion it's very important to use Tb and TB correctly.
- Gigachad 4y agoThis is never going to happen. If I want to indicate it's correct I'd write Tbit/s or Tbyte/s. Otherwise its a coin flip if TB and Tb has been used correctly.
- jrockway 4y agoI like /s for bytes and ps for bits. 100Mbps = 100 million bits per second. 100MB/s = 100 million bytes per second. (The capitalization is important of course. I tried writing some examples where it's not preserved and it's too weird. I tend to not use capital letters on things like Slack, but for bytes, you just have to. The difference between milli and Mega can also be important, but since nobody talks about the negative powers with bandwidth, you are probably OK if your shift key breaks.)
- JosephRedfern 4y agoI suppose that in practice you'd need to consider burst bandwidth and not just 95/99 percentiles.
- quickthrower2 4y ago… HN thread that reinvents CDN …
- thorncorona 4y agoTwitter is big on real time though. Each user gets served their own feed in real time.
- icedchai 4y agoThis doesn't seem like an issue. You can store images / other assets on the CDN, and serve the real time stuff from the origin without caching.