12 ms·
The end of Moore's Law forced YouTube to make its own video chip
- cosmiccatnap 4y ago...or they needed a chip that supported av1 and despite all the major tech companies being a part of the working group for years none of them had implemented it because Nvidia has a stranglehold on the industry because it makes money from h26x codecs
- fomine3 4y agoIt's not obvious from this article why they need their own original chip. Video transcoding is a quite common task done by dedicated chip (or circuit, like QSV). Perhaps because they push new codecs VP9 and AV1, these are also general format but Google is a developer and early adapter. Also perhaps their use case (batch processing, transmitted many times) isn't very suitable for existing transcoding chips that might be optimized for realtime transcoding.
- r3013 4y agoThe actual papers are very detailed. https://ieeexplore.ieee.org/document/9567040 https://ieeexplore.ieee.org/document/9567040 https://dl.acm.org/doi/abs/10.1145/3445814.3446723 https://dl.acm.org/doi/abs/10.1145/3445814.3446723 They basically have computing per energy as an efficiency spec and then they show how that spec increased by 20-33x compared to an "Intel Skylake" machine. Google also added some features, like a custom speed vs quality tuning thing, and single and multi-output transcoding. What was interesting to me is the video core was made with "Catapult, a C++ HLS flow from Siemens".
- j45 4y agoCompute per watt is a great way to look at it. A very trivial example of this is the concept in homelab circles to compare a cpus geekbench score to its watts to its price. Cost per point.
- IshKebab 4y agoThat is interesting. I guess it makes the most sense if you already have a full C++ implementation, which they do.
- lerp 4y agoThere's also a recording of the conference presentation at https://www.youtube.com/watch?v=vdXCVuY2CJs https://www.youtube.com/watch?v=vdXCVuY2CJs.
- ahartmetz 4y agoSiemens bought several EDA companies over the last ten years or so, in case anyone wonders why that old German company is offering advanced software.
- sp332 4y agoI think it's the batch thing. They can put in one input file and get out multiple formats and resolutions. Also, they have very specific requirements like only encoding 5 seconds at a time for DASH streaming, so they can build the chips with a specific amount of RAM and leave out features like B-frames or whatever they're not using.
- brigade 4y agoTypically one chip has the hardware for a total encode throughput of like 4k60 pixels per second, optimized for the possibility of one real-time 4k60 encode. The chip also includes an entire GPU and/or a bunch of CPU cores. Putting more fixed encode HW instead of GPU or CPU cores is more useful and efficient for YouTube, and at the same time they don’t need individual encodes to be real-time, so it becomes possible to design the hardware for higher quality if you don’t need to guarantee 16ms frame encode times (maybe? at least, "realtime" is the reason I got when asked why hardware can't use recon for intra mode decision...) Intel went partly this way by slapping three 4-core Xeons on a PCIe card (VCA2), abandoned it, then announced variants of Xe to replace it. But ultimately they’re repurposing their existing silicon and it’s not as important for Intel as it is for Youtube.
- yaantc 4y agoThe article [1] is a bit old (2010) but still relevant to answer your question. It compares the efficiency of a HD.264 implemented first on a general purpose CPU, then on a SIMD+VLIW DSP, then on the same DSP with additional optimized instruction, then on this platform with macro memory based accelerators, and lastly a pure ASIC implementation. There are significant efficiency gains at each steps, the big gap is between optimized ops and macro accelerators. Flexibility do have a price. With efficiency gains due to new nodes getting lower (and pricer!), it's normal to see more dedicated hardware accelerators. [1] https://courses.cs.washington.edu/courses/cse591n/10au/Papers/Hameed2010_SourcesOfInefficiency.pdf https://courses.cs.washington.edu/courses/cse591n/10au/Paper...
- walrus01 4y agonot every hardware-encoding AVC (H264) or HEVC (H265) chip is in any way comparable once you start digging down into the various tweaks of encoder settings that are possible. it's very likely that youtube wants capabilities that they may not be able to guarantee exist in commercially available H264/H265 and similar encoding ASICs.
- diydsp 4y agoMoore's law is not over. It's not about density. It's about quantity of transistors on a chip.
- r3013 4y agoIt depends how literally one wants to take the law. The "law" is doubling density every two years. So strictly speaking it is done because it has taken longer than two years. But if you choose the looser interpretation and simple read it as, transistor density increases with time, then it's not dead. Probably the bigger deal is cost per transistor is actually going up with the new nodes. https://www.fabricatedknowledge.com/p/the-rising-tide-of-semiconductor?triedSigningIn=true https://www.fabricatedknowledge.com/p/the-rising-tide-of-sem... and https://www.eetimes.com/moores-law-dead-by-2022-expert-says/ https://www.eetimes.com/moores-law-dead-by-2022-expert-says/
- onlyrealcuzzo 4y agoKoomey’s Law is where it's at - performance per watt growth is slowing - but it's still doubling every ~2.6 years.
- diydsp 4y agoPeople often misinterpret this because they think Moore was only talking about technology. However the number of transistors is more interesting than the technology alone because it represents what people actually do with chips. e.g. GPUs added to CPUs, and later AI added to the M1. These additions may not have been much more dense technologically, but the needs and desires of users increase as well as the technology. The tech doesn't have to improve 2x to reach 2x, because peoples' needs for AI, games, etc. is also increasing. And that's more useful in anticipating and predicting markets than the mere tech process. So if you still think you're right, kindly post your citations. Here are mine: "Moore's law is a term used to refer to the observation made by Gordon Moore in 1965 that the number of transistors in a dense integrated circuit (IC) doubles about every two years. "[0] "Moore's law is the observation that the number of transistors in a dense integrated circuit (IC) doubles about every two years"[1] "Moore's Law refers to Gordon Moore's perception that the number of transistors on a microchip doubles every two years..."[2] [0] https://www.synopsys.com/glossary/what-is-moores-law.html https://www.synopsys.com/glossary/what-is-moores-law.html [1] https://en.wikipedia.org/wiki/Moore%27s_law https://en.wikipedia.org/wiki/Moore%27s_law [2] https://www.investopedia.com/terms/m/mooreslaw.asp https://www.investopedia.com/terms/m/mooreslaw.asp
- pstoll 4y agoWe talk power, not servers, at work because it’s the limiting resource at any scale. “How many MWatts do you need next year”. Its not that you can’t do these workloads on CPUs or GPUs - you can. But you can do it at 10x-30x less power on custom silicon. That’s why Amazon did Graviton3; Google did TPUs and VPUs. Ditto on personal computer with Apple & M1/M2.
- fezfight 4y agoYou skipped over the entire smart phone era.
- pstoll 4y agoSure - add the Apple A chips; Qualcomm chips for communications, etc…
- sthreeaway 4y agoAn important caveat is that this is practically and theoretically true for "embarrassingly parallel" (in the term-of-art, not perjorative sense) compute problems, and only if one sets aside supply-chain issues. It's theoretically (but rarely practically) true for network-bound problems of all sorts. It's not theoretically (and usually not practically) true for storage-bound problems. (Though Wh/bytes-at-rest is a useful bounding metric to understand if you're trying to build a storage business. It's never zero if the integrity of a byte at rest is at all important. Integrity is complicated.) But in any case, supply chain issues tend to dominate as the limiting factor at the largest scale in my experience. This has always been true, even since before the current clusterfuck. Source: no comment.
- deepsquirrelnet 4y agoOn an unrelated note, I find it funny that we call things like “Moore’s law” laws at all. Everything in science is a theory now, and these strictly observational “laws” are borne on much shakier grounds. Meanwhile, everybody expects the law to break at some point (relatively soon, if not already). So what is it then? The observation formerly known as Moore’s law? Seems a bit of a silly way to chronicle contemporary, short lived phenomena.
- scoopertrooper 4y ago> I find it funny that we call things like “Moore’s law” laws at all. I think it had more to do with 'Moore' rhyming with the word 'law'. There hasn't been a radical shift in the use of the word 'law' over time.
- wonnage 4y agowouldn't Moore's Lore rhyme a bit better
- barrysteve 4y agoGot a better name for those waves of temporarily useful theory?
- mdrachuk 4y agoMeme? Moore’s meme.
- nyokodo 4y ago> I find it funny that we call things like “Moore’s law” laws at all Because “Moore’s contemporary short lived phenomena” isn’t as catchy? Words can be used in all sorts of non-literal ways.
- mr_mitm 4y agoMoore conjecture? Moore approximation? Moore prediction? Moore relation?
- pdimitar 4y agoMaybe we'll need co-processors in the future again? Although when you think about it, TPUs / VPUs etc. being pluggable to a PCIe slot is mostly getting us there?
- BooneJS 4y agoWe've already got them! They're integrated into the "System on Chip" or as chiplets on the package. Just considering Apple's M1: https://en.wikipedia.org/wiki/Apple_M1 https://en.wikipedia.org/wiki/Apple_M1 Because pure-software implementations of algorithms are too "expensive" in terms of power and/or latency, the M1 package has logic blocks that implements a GPU, a Neural Network processor, and H.264/H.265/VP9/JPEG/ProRes codecs. When Apple quotes battery life, the fine print says "1080P HD video with brightness set 8 clicks from bottom". Powering down most of the cores and lighting up just one of the codecs is clearly a good way to achieve 18 hours of battery life. AMD and Nvidia sell GPU "co-processors" that plays games and has been used for machine learning in "GPU mode" (think games using "software rendering"), but Nvidia GPUs now have "Tensor Cores" to accelerate and decrease the electrical power required for some of the linear algebra functions common with ML. Laptop chips from Intel and AMD will have embedded video on package to eliminate an external GPU.
- dmw_ng 4y agoYou can get similar functionality very cheaply on AWS from the vt1 instance type. They're a PITA to work with, which is probably why they're still around $160/mo. for enough transcode to power a small city on spot. These cards can transcode a full ladder (5 resolutions) of a 60 minute 1080p video in around 4 minutes, or the full ladder for 8 60 minute videos in parallel in about 30 minutes. Those instances are based on off-the-shelf cards that cost somewhere around $2000 each. Guessing Google went the custom route for wider codec support, but standard hardware for this stuff is readily available in the professional broadcast industry
- hot_gril 4y agoWould be funny if Google did custom hardware entirely for VP8/9, since VT1 on the face of it doesn't seem to support that. But more likely, Google didn't want to rely on a competitor, and in general with datacenters that have a reputation for doing things their own more-optimized way even if it's a hassle.
- alexklarjr 4y agoIt must be a law of common sense that if all you do is encoding every stupid video on planet, you will need an encoding accelerator because gpu manufacturers likes to lock down and charge alot for tiny part of silicon that does it.
- wtallis 4y ago> because gpu manufacturers likes to lock down and charge alot for tiny part of silicon that does it. Those shenanigans have a far smaller effect than the fundamental problem with buying a large GPU when you have no use for 80% of what's on that chip. That's obviously not going to be economical at large scale (especially not during a GPU shortage driven by demand for exactly the portions of a GPU that you never use).
- thunkshift1 4y agoYt milking tf out of ‘that one time they designed an ASIC’
- ksec 4y agoI submitted [1] a similar story when they announced it a year ago. Google VCU Video Coding Unit at Hot Chips 33. It is also sort of strange ( or interesting ) their PR hype on AV1 has toned down a lot. To the point of practically silent. [1] https://news.ycombinator.com/item?id=28302975 https://news.ycombinator.com/item?id=28302975
- shric 4y agoDoes it annoy anyone else that many articles insist on quoting part of the article in a larger font to highlight it? It's a relic left over from magazines where it would draw the reader's attention to a particular segment. It makes no sense whatsoever to do it on a web article that you scroll through to read. This particular one contains the text "After a 10-minute meeting with YouTube chief Susan Wojcicki, the company's first video chip project was approved." then it's repeated immediately after in a larger font.