12 ms·
I lead the Microsoft Open Source Programs Office team. I'm sorry this happened. We have merged a pull request that restored the correct LICENSE file and copyri
by jeffwilcox 5y ago
I lead the Microsoft Open Source Programs Office team. I'm sorry this happened.
We have merged a pull request that restored the correct LICENSE file and copyright, and are in touch with the upstream author Leśny Rumcajs who emailed us this morning. We'll look to revert the entire commit that our bot made, too, since it updated the README with a boilerplate getting started guide.
The bug was caused by a bot that was designed to commit template files in new repositories. It's code that I wrote to try to prevent other problems we have had with releasing projects in the past. It's not supposed to run on forks.
I'm going to make sure that we sit down and audit all of our forked repositories and revert similar changes to any other projects.
We have a lot of process around forking, and have had to put controls in place to make sure that people are aware of that guidance. Starting a few years ago, we even "lock" forks to enforce our process. We prefer that people fork projects into their individual GitHub accounts, instead of our organization, to encourage that they participate with the upstream project. In this situation, a team got approval to fork the repository, but hasn't yet gotten started.
To be as open as I can, I'd like to point to the bug:
- The templates we apply on new repositories live at https://github.com/microsoft/repo-templates https://github.com/microsoft/repo-templates
- The bug seems to be at this line of the new repository workflow: https://github.com/microsoft/opensource-management-portal/blob/main/routes/org/repoWorkflowEngine.ts#L373 https://github.com/microsoft/opensource-management-portal/bl...
- The system we have in place even tries to educate our engineers with this log message (https://github.com/microsoft/opensource-management-portal/blob/main/routes/org/repoWorkflowEngine.ts#L375 https://github.com/microsoft/opensource-management-portal/bl...): "this.log.push({ message: `Repository ${subMessage}, template files will not be committed. Please check the LICENSE and other files to understand existing obligations.` });"
- deleted 5y ago[deleted]
- gautamcgoel 5y agoA lot of commenters are sharpening their pitchforks, but this comment, in my opinion, makes it very likely that it was an honest mistake. Amazing what taking personal responsibility and earnestly apologizing can do to restore trust and credibility!
- tptacek 5y agoIt was obviously a mistake. Still nice to have someone involved give the backstory.
- sharken 5y agoYes, very nice writeup from Microsoft. Had a similar experience with AzureCli v2.30, where the environment variable to ignore certificate errors suddenly did not work anymore. It turned out it was removed but there was no mention of it in the release notes. On the GitHub page quick response was provided by Microsoft.
- beckman466 5y ago
- _zzaw 5y agoYeah, I think this is an example of addressing a mistake that other companies should take note of. I don’t trust Microsoft-sized corporations as a matter of principle, and I don’t typically give them the benefit of the doubt, but when one of their own engineers explains in human-readable terms what specifically happened—on a holiday, no less—I’m impressed enough to believe him. Some PR flack showing up with vague boilerplate about how Microsoft values the open-source community and they’ll look into it would only have encouraged more outrage. I always appreciate communication that acknowledges I’m a person, not a data point or a customer. I wish more companies ditched the greasy PR approach and allowed folks like Jeff to do their talking for them.
- victorvscn 5y ago
- itzprime 5y agoHow has Microsoft reformed? Windows 11 is more intrusive than ever, it changes default programms more frequently to their desired programms and it is so annyoing to switch browser. THey still are the same old.
- ModernMech 5y agoThank you for being transparent about this and dealing with it appropriately, even on Christmas. I think that goes a long way.
- 3np 5y agoReally appreciate the prompt and transparent response, Jeff. Especially at a time when I assume you were not expecting to jump in on duty. I hope that despite all the harsh words, you can have sympathy with that behind them are legitimate concerns and suspicion stemming from past bad behavior from various part of your organization. Due to this, and Microsoft's position of power, you have a much, much lower budget for these kinds of mistakes compared to the most other orgs. Even if there was nothing intentionally malicious at play here, it would not be far-fetched for an outsider to interpret as "implicit maliciousness through neglect". Here's hoping that 2022 will be a year of bridging the divide and sincere alignment.
- tptacek 5y agoYes, it would be far-fetched. The conspiracy theory here is self-evidently implausible. We need to stop pretending that the accusations here were made in good faith, or are anything more than wishcasting. People write about these things because it's a lot more fun for them than to write about what actually might have happened, even if what actually happened is probably a better, more lastingly valuable conversation to have.
- 3np 5y agoSo, I'm of the opinion that a collective (for example a company) can exhibit malicious behavior despite no malicious intention of any particular individual in it. It can be emergent, and moreso the larger the organization. Along similar lines of systemic discrimination, there does not need to exist any conscious conspiracy or malintent. For better or worse, the whole is greater than the sum of its parts. This comment speaks towards that this is the case: https://news.ycombinator.com/item?id=29686347 https://news.ycombinator.com/item?id=29686347 > I know Jeff personally and he's great. This happens all the time at Microsoft though. Teams try to do OSS themselves, haven't a clue how GitHub or licensing works (e.g. they think the CLA transfers copyright), and after a slap aside the head, I send them to Jeff for guidance and all is well. (I mostly agree with your sentiment, though. I do get the impression that leadership is sincere in wanting to do right. It's just that it's not that black-and-white or easy. As another MS employee commented, this is something that has to take time and they need to be held accountable along the way. https://news.ycombinator.com/item?id=29684127 https://news.ycombinator.com/item?id=29684127)
- sarahnovotny 5y agoThis is my favorite part of the culture change which is happening at Microsoft. We are still working on it. And, it will take time. Jeff's team, my team, the java team whose forked repo this unintentionally highlighted are working with dozens of other teams every week. Satya says we're all in on open source. Hold Jeff and Me and all the leaders of Microsoft to that vision. Keep us accountable. And, know this takes time. Let's make technology and humanity healthier and more sustainable in 2022.
- xafnuaetrf8764 5y ago
- dang 5y agoWe've banned this account. Nobody gets to attack others like that here. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- Shared404 5y agoUnlike the sibling to my comment, I appreciate both this and the GP. I was all in on MS as a kid/young teen. Eventually I started looking at the actions of Microsoft, and as stupid as it is, felt betrayed. I still don't trust MS, and still hate the direction Windows is heading, I will probably never daily drive it again. However, comments like these give me hope that MS can turn itself around. Maybe some day MS will be the company I thought it was! Tl;dr: Thanks for making changes!
- deleted 5y ago[deleted]
- phkahler 5y agoI realize that Microsoft is composed of many pieces, but that doesnt prevent me from treating it like a single entity. And that entity has IMHO done horrible things to it's own products and customers, so I'd rather they keep theirs hands off FLOSS as much as possible.
- j4hdufd8 5y agoI'm really sorry you had to deal with this during the festive period. I certainly understand how scandalous this looks to crowds like Hacker News. But this seems a little bit blown out of proportion, as if Microsoft just forked the kernel or something and put their name on the license.
- victorvscn 5y ago>I certainly understand how scandalous this looks to crowds like Hacker News. I don't. I'm with the Microsoft employee who was pissed at how people think it's edgy to diss on Microsoft. What were the chances that Microsoft was openly doing that? Some dude who has now deleted his post said Microsoft was trying to "create a monopoly of web IDEs". These are clearly people who barely have a passing knowledge of how Microsoft works these days. People think critical thinking means complaining endlessly. It doesn't. You can't think critically if you don't think clearly. And you can't think clearly if you're only looking for a reason to lift the pitchforks.
- 3np 5y agoI don't know if you're aware of other not-that-ancient incidents like these? https://keivan.io/the-day-appget-died/ https://keivan.io/the-day-appget-died/ https://web.archive.org/web/20180715225433/https://threadreaderapp.com/thread/1002696910266773505.html https://web.archive.org/web/20180715225433/https://threadrea... It took significant public outrage and press coverage before either of those were even acknowledged, a long time after. > What were the chances that Microsoft was openly doing that? After reading the above, is it really that edgy to be assuming the worst? If it's truly just recurring instances of different rogue employees, doesn't that speak to a systemic and/or cultural issue that needs to be addressed with additional internal safeguards and/or deterrents to prevent it from happening again? To the credit of the relevant team here, today this was promptly addressed as soon as it got their attention. But it will take more than that to set to rest decades of precedence. (I did not partake in the flaming and don't find it constructive or beneficial; just saying I have full understanding of the suspicion and understand that MS are still on probation)
- tptacek 5y agoWhat happened here was obviously a mistake. The thread is full of lurid accusations, because those are fun to write and talk about, but it shouldn't take even a minute's thought to see how dumb a heist this would have been. The thread would have been a lot more fun if we could have spent it talking about what prompted your team to build this thingy, and bounce other people's approaches to the same problem off, and maybe share some war stories about dumb things bots have done on our behalf. Thanks, regardless, for the information you've provided here. It's interesting.
- HugoDaniel 5y ago
- tptacek 5y agoAre you still trying to make this a thing? I'm pretty sure it's over.
- HugoDaniel 5y agoad hominem
- tptacek 5y agohttps://www.youtube.com/watch?v=dTRKCXC0JFg https://www.youtube.com/watch?v=dTRKCXC0JFg
- PragmaticPulp 5y ago> The thread is full of lurid accusations, because those are fun to write and talk about, but it shouldn't take even a minute's thought to see how dumb a heist this would have been. Another good reminder that Hacker News is not above assuming the worst and gathering pitchfork mobs like any other social media. The issue looked like a mistake from the start to anyone paying attention (committed by a bot, changes were consistent with a boilerplate LICENSE file being checked in). If someone at Microsoft wanted to steal some code, forking it on Github and then publicly documenting the history of the code in the most visible way possible would truly be the dumbest way to do it.
- obert 5y agoThanks for clarifying. This also highlights the lack of sufficient testing, and subpar processes, eg automation should be tested against critical paths, and changing a license should require human approval.
- jeffwilcox 5y agoAgree. The lack of tests has been the biggest regret of mine for this project. It started as a hackathon project a long time ago, and as it grew up, it never got the testing investment it deserved. I imagine the code coverage is about 0.001% and there's no end-to-end tests in place.
- mst 5y agoMy local tooling doesn't fail as visibly as this but it certain fails just as catastrophically leaving me feeling excessively silly. So, yeah, hugops, mate.
- giancarlostoro 5y agoThank you for the full transparency, sadly we may hear for years of people saying how Microsoft blatantly ripped off someone else's copyright, but that's probably okay, those people probably wouldn't use your projects for whatever reason anyway, when in fact it was a bot meant to keep you guys from releasing code prematurely without a reasonable license to begin with (at least that's what it sounds like?).
- andrei_says_ 5y agoThank you for taking responsibility for this and for the transparency. It is good to be reminded that care and integrity can exist, even in large corporations.
- withinrafael 5y agoI know Jeff personally and he's great. This happens all the time at Microsoft though. Teams try to do OSS themselves, haven't a clue how GitHub or licensing works (e.g. they think the CLA transfers copyright), and after a slap aside the head, I send them to Jeff for guidance and all is well.
- dapids 5y agoDo you mind addressing the reason why an employee did exactly what you are claiming the bot did in error? https://github.com/microsoft/cups/commit/8100595a3a3a6d5c7d0ced7598807bad5c6d4d6c https://github.com/microsoft/cups/commit/8100595a3a3a6d5c7d0... Again, this is a person, not a robot. How does this play into a "software bug"?
- cdcarter 5y agoIf you read the repo history carefully, you'll see a bot was responsible for rewriting the "LICENSE" file from an Apache license to the Microsoft (c) stamped MIT license. This human commit simply copied that same language to the file named "LICENSE.txt". It's unclear why they did that, but that human was not responsible for introducing the license text into the repo.
- will4274 5y ago? Instead of a software bug, it was a human error. Is it really surprising that with a company of Microsoft's size, some employees fuck up? Likely the employee was trying to say that the contributions in this fork that were not present in upstream are covered by the new license, but failed to do so properly (by leaving the original license intact and identifying precisely which files the new license applied to and which it didn't). Courts will take a far more generous view than you are here. If Microsoft is not profiting from the change, and fixes it promptly when pointed out, the courts will shrug at any case - no harm, no foul. It's not even clear to me that it's illegal to have the wrong license on GitHub, assuming the shipping product does not violate the correct license. As nobody has pointed to any infringing Microsoft product... What are we talking about?
- jeffwilcox 5y agoHonestly, I'm not sure what happened here. My guess is that they were going through a checklist of what to do when releasing open source changes, and didn't understand what they were doing. A lot of why we've had to put some guardrails in our system has been to point people to guidance and training on open source. I've sent the team that works on this repository an e-mail, but I don't expect to get a response on the holiday.
- mkdirp 5y agoI saw your comment on the cups repo. Please don't feel horrible about it. An honest mistake is an honest mistake, as long as the issues are remediated. Merry Christmas
- tiahura 5y agoIt was immediately obvious that it was a script gone awry. Sorry your team had to spend your holiday on something so trivial.
- freediver 5y agoHanlon's razor to the rescue!
- pjmlp 5y agoThanks for jumping in and trying to explain what actually happened.
- smnscu 5y agoFeels like the reverse Streisand effect at play here; a high-profile minor fuckup that helps popularize a positive thing. This was fun to meme on, but I'm glad Microsoft appears to be on an upward slant ethically. It also makes me more comfortable being a Microsoft customer.
- filomeno 5y ago
- yawaramin 5y agoThanks for explaining. Out of curiosity, does Microsoft have any external-facing GPL-licensed projects? Are there any restrictions to using (i.e. open sourcing something developed internally or forking something from outside MS) GPL-licensed projects? Specifically, would teams be able to get approval to fork GPL repos?
- jeffwilcox 5y agoGit for Windows comes to mind. Teams can absolutely get approval for any open source license; however, for a GPL project, we'd have their open source legal team work with them to brief them on the license obligations and requirements, such as publishing code to https://3rdpartysource.microsoft.com/ https://3rdpartysource.microsoft.com/.
- yawaramin 5y agoInteresting, so that's specifically for GPL-licensed projects? Or am I misunderstanding and you would have dev teams work with Legal for any open source licensed project?
- jeffwilcox 5y agoCopyleft has more process, since we absolutely need our engineers to understand the obligations we have, and for some of us, it may be the first time we're being introduced to open source communities and licensing, so we have to do more education in the GPL case. Our process revolves more about _using_ open source than forking specifically. Whenever a build runs at the company, we have a detection task that identifies the open source that is used, storing an inventory. We evaluate the open source licenses for that inventory, and have automation depending on the license that will help inform a team that has taken a new dependency with specific legal obligations - could be to get business and legal approval for something, to take training and learn about copyleft software and licensing, or that they need to post third-party buildable source. We're also able to use that inventory to help with incident response and blast radius analysis. To scale, we need to make sure that our guidance and policies are in front of people, but we know that engineers want to get work done (or will find a way around what we have in place), and so need to be efficient and straightforward. Not all situations will require a business or legal approval. Our motto has been "eliminate, automate, delegate" - eliminate onerous bureaucracy and policies - automate licensing compliance and inventory and approvals - and delegate to business leaders and others when there's a need for humans to be involved. Sorry for the long answer.
- throwawaymanbot 5y ago
- yardie 5y agoAnd here I was ready to sharpen my pitchfork. Thanks for communicating the error and the correction. I know this time of year can be especially challenging.
- brobinson 5y agoFYI: when linking to a line of code, simply press Y on your keyboard to have Github switch from the _branchname/path/to/file.xyz_ URL to the _sha1/path/to/file.xyz_ URL. The former can result in your URL pointing to unrelated code if lines are added or removed in future commits on the referenced branch. https://github.com/microsoft/opensource-management-portal/blob/b47c9675595cdbf81a3888e150faf81a2eb73878/routes/org/repoWorkflowEngine.ts#L373 https://github.com/microsoft/opensource-management-portal/bl...
- jeffwilcox 5y agoTIL. Thanks!
- sillysaurusx 5y agoAmusingly, this works on gitlab too. I was surprised that the shortcut was "so good that competitors had to implement it."
- tentacleuno 5y agoIt's like Super+E: You never know you need it until you try it, and it doesn't work on your DE :-(
- bloqs 5y agoI have enormous respect for your response here.
- wnevets 5y agoI figured something like this was the cause. I must say I'm quite disappointed by all of the negative comments before anyone from MS had a chance to explain what happened.
- phillipcarter 5y agoHugs Jeff, keep up the good work and hopefully the rest of your holidays are a lot more cheerful
- rdl 5y agoThank you for showing up on Christmas to address this. Have a great holiday!
- sydney6 5y ago
- squidgyhead 5y agoSo, you are adding licenses automatically? This seems pretty risky. Why not just prevent commits that don't have a license? Shouldn't there be a human somewhere in that loop?
- snthd 5y agoPlease consider https://github.com/microsoft/azuredatastudio/issues/102#issuecomment-345249558 https://github.com/microsoft/azuredatastudio/issues/102#issu... >SQL Operations Studio was built on the back of many open source projects that all use the MIT License for a reason: it's the right way to keep moving the community forward, empowering your users to do cool stuff and build useful things for the community. >We're just asking SQL Operations Studio to use the same license that Visual Studio Code does.
- danesparza 5y agoDamn dude. You posted this comment on CHRISTMAS no less. Either your phone exploded or you are a VERY PASSIONATE hacker news fan. Either way ... props to you for trying your best to straighten this out immediately.
- mst 5y agoThis feels like the sort of thing where straightening it out immediately, even on christmas day, was well worthwhile simply because it would let him enjoy his christmas dinner without worrying about an inevitable building dramastorm. That is not, however, a complaint - being smart enough and giving a damn enough to realise that straightening it out immediately was a really good idea is impressive and laudable in and of itself.
- junon 5y agoThanks, this is the correct sort of response to this problem. It wasn't clear by the title this was an automated change, hence the pitchforks.
- citygm 5y ago
- mrVentures 5y agoGood job owning the mistake and planning to prevent it in the future.
- explaingarlic 5y agoVery nice to see someone take ownership of a problem. Thanks Jeff :)
- krzyk 5y agoLeśny Rumcajs? :) For those that don't know Polish it means Forest Rumcajs (https://en.wikipedia.org/wiki/Rumcajs https://en.wikipedia.org/wiki/Rumcajs), probably a joke of the author that wants to be anonymous.
- severino 5y ago> It's code that I wrote to try to prevent other problems [...] Don't worry, man, nobody expects people from Microsoft to write code that behaves as intended. Merry Christmas.
- brunoborges 5y agoJeff's reply needs no addendum. But as for some background: I am a PM for Microsoft Build of OpenJDK and from late last year to around May this year I made contributions to the gRPC_bench repo as a Microsoft employee for some experiments we have been working on, to evaluate and improve different ways of implementing gRPC exchange in Java. [1] This fork was intended for newer experiments, one of them being about coding and running these benchmarks on GitHub Codespaces. For that, I needed the repo on an org where we, as employees, have Codespaces enabled. The rest is HN history (back to Jeff's reply above [2]). Merry Christmas all! [1] https://github.com/LesnyRumcajs/grpc_bench/commits?author=brunoborges https://github.com/LesnyRumcajs/grpc_bench/commits?author=br... [2] https://news.ycombinator.com/item?id=29685628 https://news.ycombinator.com/item?id=29685628
- artursapek 5y agoVery well handled, kudos