4 ms·

Looks like playback depends on "ticks" arriving from browser to the JS library, and library selects a frame according to the scroll location. Mobile devices se

Looks like playback depends on "ticks" arriving from browser to the JS library, and library selects a frame according to the scroll location. Mobile devices send this data constantly and periodically, making it smoother; yet desktop browsers do not, appearing choppy. It works as intended on my iPhone X, yet stutters on an infinitely more powerful desktop, and only possible explanation is this.

throw10920 3y ago
Not to undermine your point at all, but as a tangent, you might be surprised by how powerful phone compute is getting these days. The iPhone 15 Pro approaches the multi-threaded GeekBench score of my Ryzen 3700x (which, while not new or high-end, is still better than what 90% of desktop users have) and handily beats its single-threaded score. I agree that the difference in smoothness is due to software design decisions and not computational power, but thought it might be novel to see how the gap between nubile and desktop has shrunk over the years!
- bayindirh 3y ago
  Well, the devices I compare are a seven year old iPhone X, which has 20% of the performance of latest iPhone (in Metal API, at least), a desktop system which's equipped with a Radeon RX550 with 4GB of RAM, and an Apple MacBook Air M1. Both of these systems are infinitely more powerful than my phone, and the latest iPhone possibly smoke that old AMD card, but not the M1, not by a large margin, at least.
  - amadeuspagel 3y ago
    Isn't the most important difference that the phone has a smaller screen?
    - bayindirh 3y ago
      That smaller screen on iPhone X has 2436x1125 resolution, which is only 33% less pixels than my desktop monitor (which is 2560x1440). Considering that the video plays full screen on the phone, and in a much smaller window on my desktop system, the inferior GPU can render that much more smoother than the desktop, which shows that GPU is not the problem here. Also, grabbing the scrollbar on Firefox Linux restores the smoothness, supporting "tick" theory.
- londons_explore 3y ago
  Be wary comparing benchmark scores between platforms. Different compilers for different platforms will make a massive difference to scores achieved, mostly due to the fact benchmarks typically do 'useless' work that compilers aim to detect and eliminate.
  - vlovich123 3y ago
    The compilers for iOS and OSX are largely the same and if you’re consistently using the same compiler family you’re probably ok even if the version isn’t completely identical (major LLVM versions aren’t frequently bringing massive changes to how the optimizer is working). Additionally the UB optimization stuff you’re referencing is all within a tight margin even across families (ie GCC and clang tend to apply very similar optimizations).
- marginalia_nu 3y ago
  > Not to undermine your point at all, but as a tangent, you might be surprised by how powerful phone compute is getting these days. ... some phones, at least. Not everyone's on a state of the art flagship device.
- timthelion 3y ago
  Much more importantly for ARM phones video isn't decoded in the CPU at all, the SOC has video codecs built in. This allows things like all day netflix/hbo without draining the battery. AMD cpus don't have video codec coprocessors as far as I know. Not sure about GPUs though...
  - bayindirh 3y ago
    All of the big three GPU vendors (Intel, AMD, NVIDIA) support hardware video decoding and encoding onboard their GPUs for a very long time. Intel and NVIDIA generally adds dedicated silicon for these tasks, and AMD prefers to use shaders for video tasks. They started doing shader based video processing in Radeon 8500 days, and they're very experienced in video processing, since they had AiO wonder cards which did also capture and even TV in some models. Trying to render analog signals nicely on a PC screen provided them considerable R&D and know how during that journey. Correction: AMD has dedicated video encoder DSP called VCE[0]. [0]: https://en.wikipedia.org/wiki/Video_Coding_Engine https://en.wikipedia.org/wiki/Video_Coding_Engine
basch 3y ago
Small anecdote, I tried scrolling three different ways on a laptop. Two finger scroll, scroll bar on the right, and holding the middle red button and putting pressure on the trackpoint. the third appeared smoother than the other two. (in my mouse drivers, I tried setting the middle button set to scroll and middle click.) It also appears all the desktop people in the thread reporting choppyness are doing so from OSX.
- bayindirh 3y ago
  I both tested from Linux and macOS. Safari on macOS is same with the iPhone X Safari on smoothness. Firefox on macOS is same with Firefox on Linux, too. FF updates when scrolling stops, or updates more frequently when you grab the scrollbar. So, it's how the library gets the scroll signal, not about video decoding performance.
stabbles 3y ago
Desktop is secondary in mobile first development
- j45 3y ago
  Most apps pretty much would benefit from being mobile first and offline first.
danShumway 3y ago
Yeah, this isn't an issue of compute power, the only explanation I can think of off the top of my head is the scroll ticks. There are potentially ways around this, the video could lerp its playback position instead of snapping it? Maybe that would impose too much of a delay though. I also feel like I have seen smooth scrolling video examples on Firefox desktop, so I don't know that it's an insurmountable problem, either other solutions aren't relying on scroll ticks or they're using some method of making them come faster on desktop I would guess? But performance/responsiveness are kind of the least of my concerns when looking at this, because sure it may actually be performant, but it doesn't feel performant if the video is constantly snapping around like this.

throw10920 3y ago

Not to undermine your point at all, but as a tangent, you might be surprised by how powerful phone compute is getting these days. The iPhone 15 Pro approaches the multi-threaded GeekBench score of my Ryzen 3700x (which, while not new or high-end, is still better than what 90% of desktop users have) and handily beats its single-threaded score. I agree that the difference in smoothness is due to software design decisions and not computational power, but thought it might be novel to see how the gap between nubile and desktop has shrunk over the years!

bayindirh 3y ago
Well, the devices I compare are a seven year old iPhone X, which has 20% of the performance of latest iPhone (in Metal API, at least), a desktop system which's equipped with a Radeon RX550 with 4GB of RAM, and an Apple MacBook Air M1. Both of these systems are infinitely more powerful than my phone, and the latest iPhone possibly smoke that old AMD card, but not the M1, not by a large margin, at least.
- amadeuspagel 3y ago
  Isn't the most important difference that the phone has a smaller screen?
  - bayindirh 3y ago
    That smaller screen on iPhone X has 2436x1125 resolution, which is only 33% less pixels than my desktop monitor (which is 2560x1440). Considering that the video plays full screen on the phone, and in a much smaller window on my desktop system, the inferior GPU can render that much more smoother than the desktop, which shows that GPU is not the problem here. Also, grabbing the scrollbar on Firefox Linux restores the smoothness, supporting "tick" theory.
londons_explore 3y ago
Be wary comparing benchmark scores between platforms. Different compilers for different platforms will make a massive difference to scores achieved, mostly due to the fact benchmarks typically do 'useless' work that compilers aim to detect and eliminate.
- vlovich123 3y ago
  The compilers for iOS and OSX are largely the same and if you’re consistently using the same compiler family you’re probably ok even if the version isn’t completely identical (major LLVM versions aren’t frequently bringing massive changes to how the optimizer is working). Additionally the UB optimization stuff you’re referencing is all within a tight margin even across families (ie GCC and clang tend to apply very similar optimizations).
marginalia_nu 3y ago
> Not to undermine your point at all, but as a tangent, you might be surprised by how powerful phone compute is getting these days. ... some phones, at least. Not everyone's on a state of the art flagship device.
timthelion 3y ago
Much more importantly for ARM phones video isn't decoded in the CPU at all, the SOC has video codecs built in. This allows things like all day netflix/hbo without draining the battery. AMD cpus don't have video codec coprocessors as far as I know. Not sure about GPUs though...
- bayindirh 3y ago
  All of the big three GPU vendors (Intel, AMD, NVIDIA) support hardware video decoding and encoding onboard their GPUs for a very long time. Intel and NVIDIA generally adds dedicated silicon for these tasks, and AMD prefers to use shaders for video tasks. They started doing shader based video processing in Radeon 8500 days, and they're very experienced in video processing, since they had AiO wonder cards which did also capture and even TV in some models. Trying to render analog signals nicely on a PC screen provided them considerable R&D and know how during that journey. Correction: AMD has dedicated video encoder DSP called VCE[0]. [0]: https://en.wikipedia.org/wiki/Video_Coding_Engine https://en.wikipedia.org/wiki/Video_Coding_Engine

basch 3y ago

Small anecdote, I tried scrolling three different ways on a laptop. Two finger scroll, scroll bar on the right, and holding the middle red button and putting pressure on the trackpoint. the third appeared smoother than the other two. (in my mouse drivers, I tried setting the middle button set to scroll and middle click.) It also appears all the desktop people in the thread reporting choppyness are doing so from OSX.

bayindirh 3y ago
I both tested from Linux and macOS. Safari on macOS is same with the iPhone X Safari on smoothness. Firefox on macOS is same with Firefox on Linux, too. FF updates when scrolling stops, or updates more frequently when you grab the scrollbar. So, it's how the library gets the scroll signal, not about video decoding performance.

stabbles 3y ago

Desktop is secondary in mobile first development

j45 3y ago
Most apps pretty much would benefit from being mobile first and offline first.

danShumway 3y ago

Yeah, this isn't an issue of compute power, the only explanation I can think of off the top of my head is the scroll ticks. There are potentially ways around this, the video could lerp its playback position instead of snapping it? Maybe that would impose too much of a delay though. I also feel like I have seen smooth scrolling video examples on Firefox desktop, so I don't know that it's an insurmountable problem, either other solutions aren't relying on scroll ticks or they're using some method of making them come faster on desktop I would guess? But performance/responsiveness are kind of the least of my concerns when looking at this, because sure it may actually be performant, but it doesn't feel performant if the video is constantly snapping around like this.