3 ms·

> M1 Linux does not have a cool logo or name Wonder if this is a subtle jab at Asahi Linux. Hopefully not, as I was thoroughly unamused by Corellium’s previous

by jchw 5y ago

> M1 Linux does not have a cool logo or name Wonder if this is a subtle jab at Asahi Linux. Hopefully not, as I was thoroughly unamused by Corellium’s previous antics regarding Asahi Linux. It would be a lot nicer to see at least polite relations in the future, if not collaboration…

monocasa 5y ago
Isn't the feeling mutual? I was under the impression that Asahi didn't really want Corellium's contributions because they considered Corellium's looks at proprietary Apple code to be an existential threat to the project's need for clean room RE.
- rowanG077 5y ago
  I believe this is false. The Corellium guys are (much?) further along technically. But they don't care to put their code in a format that upstream will ever accept. I believe this was the main reason collaboration went nowhere. Corelliums forked kernel will always be little more then a marketing toy because of this.
  - marcan_42 5y ago
    FWIW, they had a ~year of head start working on their A14 kernel, but they also haven't done any graphics work and our userspace stack now passes >90% of the GLESv2 tests... so I'm not sure it's fair to say their demo kernel (which they haven't touched since February) is further along technically any more. We're focusing on different things, and now that the DART code is in good shape (which makes PCI and USB work) and ASC/RTKit support is coming along, the only significant driver code they have that we don't is the thunderbolt/USB4 stuff. The rest is trivial stuff (SPI, clocking, CPU frequency scaling) that takes more bureaucracy to figure out how to do properly and upstreamably than actual code. And the drivers which we are writing are higher quality than theirs (e.g. their NVMe support sometimes wedges in a way that can apparently only be fixed with a hard power cycle; sven's patches for Asahi Linux not only don't do that, but actively fix the wedge so you can use the Corellium driver again after booting with our kernel... I don't know why this happens, but I suspect it has to do with their ASC/RTKit driver being buggy).
    - rowanG077 5y ago
      I assumed they had a working graphics stack when I said they are further along :p. But yeah I follow the asahi channels on IRC and am a github sponsor myself! So I'm definitely not knocking on you guys effort or anything. I'm very grateful I will soon have the opportunity to buy a thin & light laptop that doesn't throttle itself into the ground if you so much as even think about using the GPU.
- jchw 5y ago
  In any case, it never had to turn petty, even if they really did feel like RE knowledge from Sandcastle was being used. Even if code wasn’t shared, knowledge could have been. I came out with the feeling that they had bigger concerns about optics above all, and they couldn’t use the same approaches that might’ve worked when they were the David and not the Goliath. I don’t want to be the asshole dredging up drama needlessly, but the line just feels snide after all that happened. Like, if you wanted a slick logo, name and marketing page, you all could just make one; you had absolutely no trouble doing so for Project Sandcastle. I don’t see why this statement needs to be made here and now, as nobody would’ve batted an eye if they hadn’t said anything.
  - monocasa 5y ago
    Totally agreed, the pettiness in this case (if we're reading a one-off sentence fragment in a slide correctly) is unacceptable. I was just commenting that I don't see collaboration occurring for the well meaning base differences of opinion even once you unwrap away the pettiness.
    - deleted 5y ago
      [deleted]
  - coldtea 5y ago
    >Even if code wasn’t shared, knowledge could have been. Legally it would be still an issue to share "knowledge" with someone who has seen the code.
    - jchw 5y ago
      Isn’t that false? I thought the fact that you could is basically the principle behind Chinese wall reverse engineering.
      - not2b 5y ago
        Emphasis on "wall" here. People who are being careful (and when your potential opponent is as big and powerful as Apple it's best to be careful) don't let the engineers who have seen the secrets talk directly to those who are doing the clean-room reimplementation. The first group documents what they found and how they found it, lawyers check the report for anything iffy, and if the implementers need to talk to the first group they don't do so directly, everything goes through legal. But better to get legal advice on this than just listen to some random HN person (like me).
- coldtea 5y ago
  Well, they objectively were.
- deleted 5y ago
  [deleted]
- marcan_42 5y ago
  Sometimes we look at Apple code too, within reason and where strictly necessary. The problem is that there are safe ways of doing this and terrible ways of doing this. When I look at vendor code, my goal is to understand what I need to do to the hardware to get it to behave how I want. I don't care how Apple's code works; I only want to use it to help answer questions about the hardware, and then build my own driver with my own approach. However, I've seen other people (including a Corellium co-founder, in a past project that predates the company) "reverse engineer" code by literally translating it back to compilable C code, with the same logic, functions, and everything. And that is where you run into copyright problems. And the issue is that unless you re-do all the work from scratch, it's not easy to tell from the output whether this has happened or not. I've been bitten by this in the past. And so, as the Asahi Linux project founder, I can't afford to accept contributions from any random person that are not clean-room, without having some level of trust. I have a few long-time friends in the project who I know won't screw this up, so I'm not worried about them taking a peek at Apple's drivers in Ghidra. But if it's someone I don't know, they need to build trust and we need to have a conversation about this first. Corellium have so far refused to have any conversations with us. They never replied to any emails they were CCed on, nor have they joined our project IRC channels. So, without being able to build trust that their approach is kosher, I can't just take their code and use it. Practically speaking, Corellium certainly have internal documentation on the hardware that they could share (much like we're dumping info on our wiki) that would be immensely useful to us, but I doubt they'd ever share that, as it would potentially help someone compete with their product. Asahi Linux is, in this way, fundamentally at odds with Corellium: we're trying to reverse engineer the M1 and share that knowledge, while they did so to build a proprietary product and keeping that reverse engineered knowledge a trade secret is important to them.
marcan_42 5y ago
I'm pretty sure it is. They also apparently still haven't figured out that they have a bad setting in their tunables (that macOS does not do) which disables the SError reports when I/O writes are using the wrong type. They definitely don't get "silently" ignored if you don't turn off the error reports :-) But this presentation finally answers the question of why Corellium did this. It wasn't a fun side project or a way to contribute to the community. Their Linux port is a validation platform for their emulation/VM product. Now it makes perfect sense. Releasing it and claiming they were going to upstream it was a PR stunt; they haven't updated their repo since February and didn't reply to any of the upstreaming mailing list threads they were CCed on. I'm pretty sure they don't have any actual interest in collaborating with anyone or upstreaming anything. To them, this is a platform validation tool for their commercial product, and it only has to work once, not be maintainable or upstreamable. They have no business reason to spend time on that. Apple have done the same, by the way. They have internal Linux ports to their SoCs that they use for silicon validation. I should probably give a talk on m1n1 and the hypervisor I built for M1 reverse engineering...
- xuki 5y ago
  > I should probably give a talk on m1n1 and the hypervisor I built for M1 reverse engineering... Please do. Pretty please.
  - marcan_42 5y ago
    Not in "proper talk" format, but I do have a 3h stream where I go over the hypervisor, why it was made, and how all the different parts of the code work, if you're interested. https://youtu.be/igYgGH6PnOw https://youtu.be/igYgGH6PnOw
asddubs 5y ago
haha, I have to admit I did think that the asahi linux logo has way too many different colors

by jchw 5y ago

monocasa 5y ago

Isn't the feeling mutual? I was under the impression that Asahi didn't really want Corellium's contributions because they considered Corellium's looks at proprietary Apple code to be an existential threat to the project's need for clean room RE.

rowanG077 5y ago
I believe this is false. The Corellium guys are (much?) further along technically. But they don't care to put their code in a format that upstream will ever accept. I believe this was the main reason collaboration went nowhere. Corelliums forked kernel will always be little more then a marketing toy because of this.
- marcan_42 5y ago
  FWIW, they had a ~year of head start working on their A14 kernel, but they also haven't done any graphics work and our userspace stack now passes >90% of the GLESv2 tests... so I'm not sure it's fair to say their demo kernel (which they haven't touched since February) is further along technically any more. We're focusing on different things, and now that the DART code is in good shape (which makes PCI and USB work) and ASC/RTKit support is coming along, the only significant driver code they have that we don't is the thunderbolt/USB4 stuff. The rest is trivial stuff (SPI, clocking, CPU frequency scaling) that takes more bureaucracy to figure out how to do properly and upstreamably than actual code. And the drivers which we are writing are higher quality than theirs (e.g. their NVMe support sometimes wedges in a way that can apparently only be fixed with a hard power cycle; sven's patches for Asahi Linux not only don't do that, but actively fix the wedge so you can use the Corellium driver again after booting with our kernel... I don't know why this happens, but I suspect it has to do with their ASC/RTKit driver being buggy).
  - rowanG077 5y ago
    I assumed they had a working graphics stack when I said they are further along :p. But yeah I follow the asahi channels on IRC and am a github sponsor myself! So I'm definitely not knocking on you guys effort or anything. I'm very grateful I will soon have the opportunity to buy a thin & light laptop that doesn't throttle itself into the ground if you so much as even think about using the GPU.
jchw 5y ago
In any case, it never had to turn petty, even if they really did feel like RE knowledge from Sandcastle was being used. Even if code wasn’t shared, knowledge could have been. I came out with the feeling that they had bigger concerns about optics above all, and they couldn’t use the same approaches that might’ve worked when they were the David and not the Goliath. I don’t want to be the asshole dredging up drama needlessly, but the line just feels snide after all that happened. Like, if you wanted a slick logo, name and marketing page, you all could just make one; you had absolutely no trouble doing so for Project Sandcastle. I don’t see why this statement needs to be made here and now, as nobody would’ve batted an eye if they hadn’t said anything.
- monocasa 5y ago
  Totally agreed, the pettiness in this case (if we're reading a one-off sentence fragment in a slide correctly) is unacceptable. I was just commenting that I don't see collaboration occurring for the well meaning base differences of opinion even once you unwrap away the pettiness.
  - deleted 5y ago
    [deleted]
- coldtea 5y ago
  >Even if code wasn’t shared, knowledge could have been. Legally it would be still an issue to share "knowledge" with someone who has seen the code.
  - jchw 5y ago
    Isn’t that false? I thought the fact that you could is basically the principle behind Chinese wall reverse engineering.
    - not2b 5y ago
      Emphasis on "wall" here. People who are being careful (and when your potential opponent is as big and powerful as Apple it's best to be careful) don't let the engineers who have seen the secrets talk directly to those who are doing the clean-room reimplementation. The first group documents what they found and how they found it, lawyers check the report for anything iffy, and if the implementers need to talk to the first group they don't do so directly, everything goes through legal. But better to get legal advice on this than just listen to some random HN person (like me).
coldtea 5y ago
Well, they objectively were.
deleted 5y ago
[deleted]
marcan_42 5y ago
Sometimes we look at Apple code too, within reason and where strictly necessary. The problem is that there are safe ways of doing this and terrible ways of doing this. When I look at vendor code, my goal is to understand what I need to do to the hardware to get it to behave how I want. I don't care how Apple's code works; I only want to use it to help answer questions about the hardware, and then build my own driver with my own approach. However, I've seen other people (including a Corellium co-founder, in a past project that predates the company) "reverse engineer" code by literally translating it back to compilable C code, with the same logic, functions, and everything. And that is where you run into copyright problems. And the issue is that unless you re-do all the work from scratch, it's not easy to tell from the output whether this has happened or not. I've been bitten by this in the past. And so, as the Asahi Linux project founder, I can't afford to accept contributions from any random person that are not clean-room, without having some level of trust. I have a few long-time friends in the project who I know won't screw this up, so I'm not worried about them taking a peek at Apple's drivers in Ghidra. But if it's someone I don't know, they need to build trust and we need to have a conversation about this first. Corellium have so far refused to have any conversations with us. They never replied to any emails they were CCed on, nor have they joined our project IRC channels. So, without being able to build trust that their approach is kosher, I can't just take their code and use it. Practically speaking, Corellium certainly have internal documentation on the hardware that they could share (much like we're dumping info on our wiki) that would be immensely useful to us, but I doubt they'd ever share that, as it would potentially help someone compete with their product. Asahi Linux is, in this way, fundamentally at odds with Corellium: we're trying to reverse engineer the M1 and share that knowledge, while they did so to build a proprietary product and keeping that reverse engineered knowledge a trade secret is important to them.

marcan_42 5y ago

I'm pretty sure it is. They also apparently still haven't figured out that they have a bad setting in their tunables (that macOS does not do) which disables the SError reports when I/O writes are using the wrong type. They definitely don't get "silently" ignored if you don't turn off the error reports :-) But this presentation finally answers the question of why Corellium did this. It wasn't a fun side project or a way to contribute to the community. Their Linux port is a validation platform for their emulation/VM product. Now it makes perfect sense. Releasing it and claiming they were going to upstream it was a PR stunt; they haven't updated their repo since February and didn't reply to any of the upstreaming mailing list threads they were CCed on. I'm pretty sure they don't have any actual interest in collaborating with anyone or upstreaming anything. To them, this is a platform validation tool for their commercial product, and it only has to work once, not be maintainable or upstreamable. They have no business reason to spend time on that. Apple have done the same, by the way. They have internal Linux ports to their SoCs that they use for silicon validation. I should probably give a talk on m1n1 and the hypervisor I built for M1 reverse engineering...

xuki 5y ago
> I should probably give a talk on m1n1 and the hypervisor I built for M1 reverse engineering... Please do. Pretty please.
- marcan_42 5y ago
  Not in "proper talk" format, but I do have a 3h stream where I go over the hypervisor, why it was made, and how all the different parts of the code work, if you're interested. https://youtu.be/igYgGH6PnOw https://youtu.be/igYgGH6PnOw

asddubs 5y ago

haha, I have to admit I did think that the asahi linux logo has way too many different colors