7 ms·

Android should really use a modern kernel. All the forking mess involved in Android updates is a terrible problem predicated by the lack of generic drivers on m

Android should really use a modern kernel. All the forking mess involved in Android updates is a terrible problem predicated by the lack of generic drivers on mobile devices. Copperhead[0] has been working to apply security patches to the kernel for some time and PostMarketOS[1] has an eventual goal of using the mainline upstream kernel. Really pulling for PMOS. [0]: https://copperhead.co/android/ https://copperhead.co/android/ [1]: https://www.postmarketos.org/ https://www.postmarketos.org/

kelnos 9y ago
Yes, this was my first thought as I was reading the article. A better title for it would be "Backporting Modern Linux Kernel Features to Our Really Old Kernels Instead of Doing the Right Thing and Keeping Up To Date". I do -- I really do -- appreciate that version churn is difficult, and the Linux kernel also doesn't make it easy since they don't guarantee any stable internal APIs, but they're also adding a lot of work on themselves by having to maintain their own kernel trees that diverge significantly from mainline. They're also at the mercy (to some extent) of many chipset manufacturers and whatever they've chosen to base their efforts on. A quick look at some kernel release timeframes from the versions they mention in the article: 4.3 - 11/2015 4.4 - 02/2016 4.6 - 05/2016 4.8 - 10/2016 4.10 - 02/2017 The only kernel out of that list I'd unquestioningly accept as it being unrealistic to upgrade to for Oreo is 4.10. 4.8 might be a stretch since I'm guessing they'd already branched internally for Oreo by then, though they likely had a month or so of RCs that they could have used as a base before that. There's certainly risk to basing your work on a newly-released (or soon-to-be-released) kernel, but given the general high quality of kernel releases, I imagine that'd be pretty far down on their list of risks. Regardless, 4.6 or 4.7 would be entirely reasonable to use as a base, and since they own the conformance test criteria, they could also require that all their vendors use that as a minimum version. And yet, they are backporting some features as far back as 3.18, which was originally released in December of 2014, and, while it was designated a LTS kernel, it, at this point in time, has moved into end-of-life status. And we wonder why Android security is a nightmare.
- c2h5oh 9y ago
  Main reason is that there is a lot of hardware that never got binary blobs / drivers updated for newer kernels. We're talking input controllers, wifi, bluetooth, nfc chips, gyroscopes, amps and half a dozen other parts that never attempted to have drivers mainlined in Linux kernel. It's a chicken and egg problem - they won't make updated blobs until Android doesn't include newer kernel. Android won't use newer kernel because that would block 2/3 of the phones from being upgraded because of Broadcom bluetooth chips alone - Samsung might decide they are better off forking Android than waiting for Broadcom. Edit: Sony has a bunch of phones that can run on 4.4 kernel: https://developer.sonymobile.com/open-devices/ https://developer.sonymobile.com/open-devices/
  - endorphone 9y ago
    The Galaxy S8 runs 4.4, and has since release. Oreo the AOSP uses the 4.10 headers. But ultimately any vendor can integrate with a wide variance of kernels depending upon their needs.
  - kelnos 9y ago
    Yes, and I addressed that in my comment. Google, however, can effectively do whatever they want. If they require kernel X, and a manufacturer doesn't support it, they'll either get their shit together, or they'll get left behind. I bet most of them do enough business supplying parts for Android phones that they'd get their shit together. And it' not hard! Writing an initial driver for some hardware might take a lot of effort, but keeping it up-to-date as new kernel releases happen will not, at least in the vast majority of cases.
    - zaphirplane 9y ago
      I expect if google had that strength in position, they would. Perhaps that would lead to more fragmentation and less vendors releasing the latest android, leading to a highly dominant maker squashing the others and then having a stronger position in negotiating with google.
      - 9y ago
tadfisher 9y ago
Agreed, but let's point fingers at the whole stack if we want this to happen. Treble [0] will finally add a HAL to the base system, so updates to the kernel and drivers should in theory be easier on devices that ship with 8.0 (of which there are exactly zero so far). So at least Google's on the right track. But Google doesn't create the drivers and Google doesn't ship the board support packages which vendors build upon. And so we come to Qualcomm, Samsung, Mediatek, and whoever else is shipping proprietary drivers for their SoCs and radios, and who don't provide binaries for new ABIs. The workaround is libhybris, which Ubuntu Touch, Sailfish, and now PMOS are using to support drivers built against Android kernels and its userspace, but that is so fraught with issues that it can't possibly be supported by Google or the vendors. So we have Android 8.0 shipping with a 4.4 minimum to support the lowest common denominator of BSPs, and it kind of has to be that way until some market force changes the landscape. I don't have high hopes for either of the projects you mentioned, sadly. [0] https://source.android.com/devices/architecture/treble https://source.android.com/devices/architecture/treble
- richdougherty 9y ago
  Agreed, it really is a shared problem. It seems like a kind of "collective action problem" [1], where coordination between different parties is required - but they all have different interests and they're not all interested in splitting the costs. Google is in a difficult position. In order for Android to catch up to the iPhone they needed to encourage a wide consortium of vendors to adopt and sell Android hardware. Part of the way they encourage vendors to sell Android phones is by being very permissive about how vendors install, update and use Android. This helps Android by growing market share. Unfortunately, it leads to a lot of variation and fragmentation in devices, which in the long term can hurt the Android ecosystem. I guess there's a balance that Google is trying to strike between being too permissive and too restrictive in how they work with vendors. [1] https://en.wikipedia.org/wiki/Collective_action#Collective_action_problem https://en.wikipedia.org/wiki/Collective_action#Collective_a...
  - pjmlp 9y ago
    It worked for Microsoft to impose hardware designs on PCs, the big difference was that OEMs did not had the source code of MS-DOS and later Windows available to them, to do whatever they felt like it.
exDM69 9y ago
Yes, they should use modern kernels but that has not happened yet. I am appalled by the practice of keeping customers on old software and sometimes even without proper security patches. I hope regulators (EU in particular) will act on this and require manufacturers to provide support for products for a reasonable amount of time. Devices should be clearly labelled with a "best before" date, until which the vendors should provide (at least) security updates. Right now the situation is unbearable, you buy a phone for $200 to $700 and you don't know how long it is good for. A technically oriented or security minded person can deal with the situation but almost everyone has a smartphone. Billions of unsecure devices out there isn't good for anyone. It shouldn't be unreasonable to assume that a smartphone should be good for 5+ years.
kuschku 9y ago
The big issue with this is the – afterwards you always know better – completely stupid decision of Torvalds to never support a stable API for kernel modules, or drivers. A similar issue happened recently between AMD and the kernel maintainers (AMD wanted a stable API for GPU drivers that would allow the same drivers to run on Windows, Mac, desktop Linux, and Android; and AMD had already built this API and was willing to maintain it). In the end, this can only be solved if the Linux kernel gets a stable driver API, and with Oreo, Google added exactly that for their own fork of the kernel.
- majewsky 9y ago
  > The big issue with this is the – afterwards you always know better – completely stupid decision of Torvalds to never support a stable API for kernel modules, or drivers. It's not stupid. APIs need to change eventually. Even famously backwards-compatible Windows has its fair share of API changes. And when public APIs change, you can either drop support for the old one (thereby also dropping support for old hardware), or provide a compatibility mapping from the old to the new API (if that is even possible). Linus knows that he does not have the manpower to do either. All he can realistically do is only support the current API, and require devs who change the API to update all usages inside the kernel source tree when doing so.
  - deleted 9y ago
    [deleted]
  - kuschku 9y ago
    There’s several problems with this. (a) all drivers live in kernel space, even the sketchy drivers you have to download from NVIDIA (b) if a driver wants to be included easily, it needs to become part of the kernel source tree, but that can only happen if Torvalds and his maintainers get full control. (c) the API is broken with every release As a result of all of this, the kernel maintainers refused even in any way to cooperate with AMD on their open driver (AMD needs a stable layer at some point to run the same additional functionality on windows, mac, and linux – the alternative is no linux support), we get ancient drivers on Android, with Google building their own HAL, and more shit. At the same time, the syscalls, where maintaining a stable API is completely irrelevant and could simply be done via a small userspace library that you call to instead of doing actual syscalls, and which also massively would improve security if the translation between old and new syscalls would happen in userspace, is the one thing Torvalds maintains in the kernel. The decisions made by Torvalds are reckless, massively hurt security and usability of open source, and the entire concept of open source ("we’d rather have only a proprietary AMD driver than an open AMD driver that relies on an HAL").
voltagex_ 9y ago
Place the blame squarely at the feet of vendors like Qualcomm - they won't update the BSPs.

kelnos 9y ago

Yes, this was my first thought as I was reading the article. A better title for it would be "Backporting Modern Linux Kernel Features to Our Really Old Kernels Instead of Doing the Right Thing and Keeping Up To Date". I do -- I really do -- appreciate that version churn is difficult, and the Linux kernel also doesn't make it easy since they don't guarantee any stable internal APIs, but they're also adding a lot of work on themselves by having to maintain their own kernel trees that diverge significantly from mainline. They're also at the mercy (to some extent) of many chipset manufacturers and whatever they've chosen to base their efforts on. A quick look at some kernel release timeframes from the versions they mention in the article: 4.3 - 11/2015 4.4 - 02/2016 4.6 - 05/2016 4.8 - 10/2016 4.10 - 02/2017 The only kernel out of that list I'd unquestioningly accept as it being unrealistic to upgrade to for Oreo is 4.10. 4.8 might be a stretch since I'm guessing they'd already branched internally for Oreo by then, though they likely had a month or so of RCs that they could have used as a base before that. There's certainly risk to basing your work on a newly-released (or soon-to-be-released) kernel, but given the general high quality of kernel releases, I imagine that'd be pretty far down on their list of risks. Regardless, 4.6 or 4.7 would be entirely reasonable to use as a base, and since they own the conformance test criteria, they could also require that all their vendors use that as a minimum version. And yet, they are backporting some features as far back as 3.18, which was originally released in December of 2014, and, while it was designated a LTS kernel, it, at this point in time, has moved into end-of-life status. And we wonder why Android security is a nightmare.

c2h5oh 9y ago
Main reason is that there is a lot of hardware that never got binary blobs / drivers updated for newer kernels. We're talking input controllers, wifi, bluetooth, nfc chips, gyroscopes, amps and half a dozen other parts that never attempted to have drivers mainlined in Linux kernel. It's a chicken and egg problem - they won't make updated blobs until Android doesn't include newer kernel. Android won't use newer kernel because that would block 2/3 of the phones from being upgraded because of Broadcom bluetooth chips alone - Samsung might decide they are better off forking Android than waiting for Broadcom. Edit: Sony has a bunch of phones that can run on 4.4 kernel: https://developer.sonymobile.com/open-devices/ https://developer.sonymobile.com/open-devices/
- endorphone 9y ago
  The Galaxy S8 runs 4.4, and has since release. Oreo the AOSP uses the 4.10 headers. But ultimately any vendor can integrate with a wide variance of kernels depending upon their needs.
- kelnos 9y ago
  Yes, and I addressed that in my comment. Google, however, can effectively do whatever they want. If they require kernel X, and a manufacturer doesn't support it, they'll either get their shit together, or they'll get left behind. I bet most of them do enough business supplying parts for Android phones that they'd get their shit together. And it' not hard! Writing an initial driver for some hardware might take a lot of effort, but keeping it up-to-date as new kernel releases happen will not, at least in the vast majority of cases.
  - zaphirplane 9y ago
    I expect if google had that strength in position, they would. Perhaps that would lead to more fragmentation and less vendors releasing the latest android, leading to a highly dominant maker squashing the others and then having a stronger position in negotiating with google.
    - 9y ago

tadfisher 9y ago

Agreed, but let's point fingers at the whole stack if we want this to happen. Treble [0] will finally add a HAL to the base system, so updates to the kernel and drivers should in theory be easier on devices that ship with 8.0 (of which there are exactly zero so far). So at least Google's on the right track. But Google doesn't create the drivers and Google doesn't ship the board support packages which vendors build upon. And so we come to Qualcomm, Samsung, Mediatek, and whoever else is shipping proprietary drivers for their SoCs and radios, and who don't provide binaries for new ABIs. The workaround is libhybris, which Ubuntu Touch, Sailfish, and now PMOS are using to support drivers built against Android kernels and its userspace, but that is so fraught with issues that it can't possibly be supported by Google or the vendors. So we have Android 8.0 shipping with a 4.4 minimum to support the lowest common denominator of BSPs, and it kind of has to be that way until some market force changes the landscape. I don't have high hopes for either of the projects you mentioned, sadly. [0] https://source.android.com/devices/architecture/treble https://source.android.com/devices/architecture/treble

richdougherty 9y ago
Agreed, it really is a shared problem. It seems like a kind of "collective action problem" [1], where coordination between different parties is required - but they all have different interests and they're not all interested in splitting the costs. Google is in a difficult position. In order for Android to catch up to the iPhone they needed to encourage a wide consortium of vendors to adopt and sell Android hardware. Part of the way they encourage vendors to sell Android phones is by being very permissive about how vendors install, update and use Android. This helps Android by growing market share. Unfortunately, it leads to a lot of variation and fragmentation in devices, which in the long term can hurt the Android ecosystem. I guess there's a balance that Google is trying to strike between being too permissive and too restrictive in how they work with vendors. [1] https://en.wikipedia.org/wiki/Collective_action#Collective_action_problem https://en.wikipedia.org/wiki/Collective_action#Collective_a...
- pjmlp 9y ago
  It worked for Microsoft to impose hardware designs on PCs, the big difference was that OEMs did not had the source code of MS-DOS and later Windows available to them, to do whatever they felt like it.

exDM69 9y ago

Yes, they should use modern kernels but that has not happened yet. I am appalled by the practice of keeping customers on old software and sometimes even without proper security patches. I hope regulators (EU in particular) will act on this and require manufacturers to provide support for products for a reasonable amount of time. Devices should be clearly labelled with a "best before" date, until which the vendors should provide (at least) security updates. Right now the situation is unbearable, you buy a phone for $200 to $700 and you don't know how long it is good for. A technically oriented or security minded person can deal with the situation but almost everyone has a smartphone. Billions of unsecure devices out there isn't good for anyone. It shouldn't be unreasonable to assume that a smartphone should be good for 5+ years.

kuschku 9y ago

The big issue with this is the – afterwards you always know better – completely stupid decision of Torvalds to never support a stable API for kernel modules, or drivers. A similar issue happened recently between AMD and the kernel maintainers (AMD wanted a stable API for GPU drivers that would allow the same drivers to run on Windows, Mac, desktop Linux, and Android; and AMD had already built this API and was willing to maintain it). In the end, this can only be solved if the Linux kernel gets a stable driver API, and with Oreo, Google added exactly that for their own fork of the kernel.

majewsky 9y ago
> The big issue with this is the – afterwards you always know better – completely stupid decision of Torvalds to never support a stable API for kernel modules, or drivers. It's not stupid. APIs need to change eventually. Even famously backwards-compatible Windows has its fair share of API changes. And when public APIs change, you can either drop support for the old one (thereby also dropping support for old hardware), or provide a compatibility mapping from the old to the new API (if that is even possible). Linus knows that he does not have the manpower to do either. All he can realistically do is only support the current API, and require devs who change the API to update all usages inside the kernel source tree when doing so.
- deleted 9y ago
  [deleted]
- kuschku 9y ago
  There’s several problems with this. (a) all drivers live in kernel space, even the sketchy drivers you have to download from NVIDIA (b) if a driver wants to be included easily, it needs to become part of the kernel source tree, but that can only happen if Torvalds and his maintainers get full control. (c) the API is broken with every release As a result of all of this, the kernel maintainers refused even in any way to cooperate with AMD on their open driver (AMD needs a stable layer at some point to run the same additional functionality on windows, mac, and linux – the alternative is no linux support), we get ancient drivers on Android, with Google building their own HAL, and more shit. At the same time, the syscalls, where maintaining a stable API is completely irrelevant and could simply be done via a small userspace library that you call to instead of doing actual syscalls, and which also massively would improve security if the translation between old and new syscalls would happen in userspace, is the one thing Torvalds maintains in the kernel. The decisions made by Torvalds are reckless, massively hurt security and usability of open source, and the entire concept of open source ("we’d rather have only a proprietary AMD driver than an open AMD driver that relies on an HAL").

voltagex_ 9y ago

Place the blame squarely at the feet of vendors like Qualcomm - they won't update the BSPs.