15 ms·

Matrix.org hacked

324 points by nektro 8y ago

m_b 8y ago
For a bit of context: Matrix.org infrastructure has been hacked a second time in 24h, after restoring everything they went down again, story developing here: https://twitter.com/matrixdotorg/status/1116304867683905537 https://twitter.com/matrixdotorg/status/1116304867683905537
- m_b 8y ago
  The hacker is now doing a post-mortem in the GitHub issues of the project: https://github.com/matrix-org/matrix.org/issues https://github.com/matrix-org/matrix.org/issues
  - nothrabannosir 8y ago
    This is gold... > I noticed in your blog post that you were talking about doing a postmortem and steps you need to take. As someone who is intimately familiar with your entire infrastructure, I thought I could help you out. > There I was, just going about my business, looking for ways I could get higher levels of access and explore your network more, when I stumbled across GPG keys that were used for signing your debian packages. It gave me many nefarious ideas. I would recommend that you don't keep any signing keys on production hosts, and instead do all of your signing in a secure environment.
    - paavoova 8y ago
      Why would they do this? It's pure negligence. I don't even sign anything important and still worry about my keys.
      - henvic 8y ago
        I have been asked twice or more why I insisted on not using a Continuous Integration environment for publishing some software releases that are installed by third-parties. My team was automating the infrastructure to build internal software and naturally they wanted to be able to simplify things. The idea that was proposed to me was the following: once I push a new version tag to GitHub, the deployment CI server is going to build and release it as an unstable version. Some important detail here: I use the same key to sign packages regardless if they are released as unstable or stable. That would mean that if someone, somehow, managed to push a tag that was pushed upstream to GitHub, hypothetically they would be able to eventually gain access to consumers machines (basically, developers) when the consumers update it after getting a notification telling them a new version is available. No way I'd allow this to happen, but I would not be surprised if most people just took this as an acceptable risk.
        simias 8y ago
        Depending on your threat model I think that signing packages directly from your CI is acceptable, assuming that your CI runs is a reasonably isolated environment (e.g. on your company's LAN) and people who are able to trigger a release are correctly vetted. If I understand the parent comment correctly they were somehow shipping the release signing key on their production environment which is a whole other level of bad.
        wcoenen 8y ago
        > That would mean that if someone, somehow, managed to push a tag that was pushed upstream to GitHub You have to define what the signature means. IMHO it is fine for it to mean "this software was built on our build server from a well-defined state of the source code, which is only changable by our employees and contractors, and for which we have the full change log". So I deploy the code signing key to build servers, which is the only place where it is used. I'm interested in what alternative meaning you would give to a signature. I have considered the possibility of tying it to the QA processes, but then a build can only be signed after checking it manually, which is problematic when many signatures are needed at multiple packaging layers (exe/dll, msi, setup.exe).
        rakoo 8y ago
        The problem is when a malicious package is produced, either because a flaw was introduced in the code, or because a dev machine was compromised, or _when the CI machine sad compromised_; the malicious package will be signed as if it were legit. One middle point between automated and manual signing is, as usual, key rotation: have the signing keys expire in a short duration of time (say 2 weeks) and manually push them every week, so that the window of attack is as small as possible.
        yolo1 8y ago
        What does a key rotation solve? Either your build server is compromised or it's not.
        notyourday 8y ago
    - lelf 8y ago
      Another gem: RRREEEEEEEE> I noticed you missed a doctype in your html page. In order for web browsers to know what type of html to render you should include a doctype. Thanks! matrixnotorg> @RRREEEEEEEE Thank you, I will consider that for the next release Edit: it got deleted But see also: https://github.com/matrixnotorg/matrixnotorg.github.io/pull/2 https://github.com/matrixnotorg/matrixnotorg.github.io/pull/...
      - justaj 8y ago
        Wait, did Github delete matrixnotorg's profile or did matrixnotorg? If Github deleted that profile, I don't really see that as being very hacker-friendly.
        aeternus 8y ago
        Although 'hacker' is often used as a positive term on HN, breaking into a company's production server is clearly illegal activity and should not be condoned. If Github deleted the account, they are simply acting in accordance to published TOS & policy.
        justaj 8y ago
        If the attacker placed sensitive information on Github, that would indeed warrant a deletion of the account. However from what I saw from the archives, the attacker merely published details about Matrix.org's infrastructure and its vulnerabilities. Is that something that's against Github's ToS?
  - m00dy 8y ago
    hilarious...
    - meruru 8y ago
      I can only think of the despair I would be in if I was in Aragorn's shoes
      - rabidrat 8y ago
        Arathorn (Aragorn is son of Arathorn).
        meruru 8y ago
        Does being a parent prepare you to handle emergency situations like this better?
  - deleted 8y ago
    [deleted]
  - atsjie 8y ago
    > Escalation could have been avoided if developers only had the access they absolutely required and did not have root access to all of the servers. I would like to take a moment to thank whichever developer forwarded their agent to Flywheel. I'd feel so small if I were this developer right now :-| A couple of his issues appear to have to do with the use of SSH. An Ops-guy whom I worked with had setup a bastion host with ip whitelisting that automatically shut down after 1 hour. He didn't like it as his credo was "if you're using SSH when using a cloud provider you're probably doing something wrong"; meaning to say you should automate and be able to recreate any infra at all times with logs accessible without the need for SSH. I never forgot that.
    - robrtsql 8y ago
      Did you work with Rich Adams? https://wblinks.com/notes/aws-tips-i-wish-id-known-before-i-started/ https://wblinks.com/notes/aws-tips-i-wish-id-known-before-i-...
      - atsjie 8y ago
        no, but I wouldn't be surprised if my colleague got the idea from him. It was around the same time (end of 2014 I think).
  - BartBoch 8y ago
    I have just checked quickly the comments and post-mortem, and I start wondering - it seems that the attack itself would be not really possible if Matrix would not be open source (as this would restrict access to the sensitive data)? Is that right?
    - t3chguy 8y ago
      This is not right at all, the hack was due to an outdated Jenkins instance and could have happened regardless of what other software was running on the infrastructure.
      - notyourday 8y ago
        Absolutely not. There's a zero reason for development infrastructure ( which includes Jenkins ) to have any connection to production outside a well defined "transfer this tarball" to a deploy staging server path during a known deploy window which cannot be controlled via development credentials.
- F30 8y ago
  I am highly skeptical when people taking about "rebuilding [the whole] infrastructure" in a few hours. Even more so when restoring all data from breached systems and before a thorough incident analysis. Show me the org which can just pull that off.
  - pferde 8y ago
    It's quite common in enterprise to have a RTO of a few hours, and RPO of a few minutes, even for infrastructure with terrabytes of data. Of course, many moneys are paid for being able to do that.
  - erikbye 8y ago
    This is doable with proper IaC implementation, and if your org does not have RPO/RTO on lock they're doing it wrong. Events like Matrix experienced now do not lead to panicked frenzy when this is in place.
    - F30 8y ago
      It is certainly doable, but I doubt that most people have IaC which is complete, reproducible and tested enough. And the data migration from the breached host still means some risk.
- proboscis 8y ago
  They closed the threads and deleted all the comments, but luckily the issues page was archived beforehand: https://web.archive.org/web/20190412090123/https://github.com/matrix-org/matrix.org/issues https://web.archive.org/web/20190412090123/https://github.co...
  - gtirloni 8y ago
    Luckily yeah. So much wisdom in those comments. /s
perttir 8y ago
They had root account activated in hebe? Am i reading this right? He got an passlist of 5 million users?
- m_b 8y ago
  yeah
- saagarjha 8y ago
  Yup: https://github.com/matrix-org/matrix.org/issues/358 https://github.com/matrix-org/matrix.org/issues/358
- deleted 8y ago
  [deleted]
- beagle3 8y ago
  But even if they didn't, "sudo -u root /bin/bash" or similar gives it to you unless sudo is extremely locked down (which, from audits I have done, is "rarely if ever"). There are hundreds of ways to get root prompt even with the root account nominally deactivated.
  - deleted 8y ago
    [deleted]
GrumpyNl 8y ago
Everything gets hacked, period. They have to remove this from there slogan, "An open standard for decentralised secure communication."
- tsukurimashou 8y ago
  Nice try NSA
- johnchristopher 8y ago
  Really ? That's a bold claim. Can this string be hacked ? $6$WqQvgIwr$/BA/19FRJZf.z4buUwX5Sbls07ovv/lVZJ3V2En7VgTR4Skdpz98hFNuq3VX4nIaiSDKObO9jKbkPb5tdt7zY1
  - bdibs 8y ago
    Eventually :)
  - chme 8y ago
    I guess the point is that you cannot protect anything from the future if you don't just destroy it completely.
    - meruru 8y ago
      On that note, the matrix.org server currently doesn't delete messages from rooms that have become empty. It probably would be a good idea to do so.
      - acct1771 8y ago
        Does synapse allow that for self hosted?
        meruru 8y ago
        Yes, IIRC it's a script you have to run periodically.
cjslep 8y ago
I believe this is meant to show that it is a targeted attack on the project lead: https://news.ycombinator.com/user?id=Arathorn https://news.ycombinator.com/user?id=Arathorn Unfortunately I don't have any background context for possible reasons why "actual transparency" on the top line is the issue chosen by the attacker, but makes it seem ideologically driven.
- JD557 8y ago
  I don't think that this is a targeted attack. Seems more like a way of showing "I got access to 5493973 passwords and to show that, instead of picking some random users, I'll pick the one responsible for the shoddy security".
  - saagarjha 8y ago
    Or it might be a clear, concise way of showing that he has access to the entire file without disclosing the information of random users, which also happens to be a particularly short command.
- erikbye 8y ago
  As for motivation I don't know, but I would like to state that I have followed the Matrix project for some time and have found it, and the CEO, to be transparent.
zaarn 8y ago
Doesn't surprise me that much, Matrix doesn't seem to be too concerned with security, more with security theatre (considering you can still not easily disable read receipts in your client, a major privacy leak IMO, among other issues).
- meruru 8y ago
  Disabling read receipts is a client feature. Yes, the currently most mature client doesn't have that feature, but nothing in Matrix precludes it.
  - zaarn 8y ago
    There is plenty of other issues with matrix and the reference clients on top of something as simple as mandatory leaking of your presence in a chatroom. I've run a matrix homeserver for almost 3 weeks and it as an utter pain to maintain, despite not a single version upgrade and I was plagued with issues that no chat platform would have if the protocol was remotely sane. edit: That is on top of the numerous security issues this hack uncovered. Apparently the matrix.org devs kept a users.txt file with a dump of users + passwords on the server. Signing keys for debian packages were stored unencrypted on the production server. People used unsafe SSH settings (SSH Agent Forwarding), ran outdated servers with known root-priv RCEs for months and root privileges for all users on a server. Why should I ever trust a matrix developer with their protocol or reference implementations ever again if they can't be trusted with the simple task of updating a service when a critical CVE comes out?
    - pferde 8y ago
      While you bring up valid concerns about the Matrix team's security hygiene, the point of an open standard is that anyone can (try to) spot flaws in it, and anyone can (try to) create their own implementation. I myself am waiting for a healthy ecosystem of servers and clients to spring up before starting to rely on Matrix for anything non-ephemeral - even if it takes years. Perhaps I'll even try my hand at writing a client, if I ever run out of things to do. In the meantime, I will dick around with a throwaway matrix.org account to play with it, and to watch progress happen.
deleted 8y ago
[deleted]
fxbl0i 8y ago
Wow, piping the output of cat to another utility. What a n00b. I hope the hacker gets hacked
SpaceInvader 8y ago
As na linux/unix sysadmin with 15+ years of experience my eyes are bleeding everytime I see cat <file> | grep <pattern> instead of just grep <pattern> <file> ;D
- jyap 8y ago
  Reason why you would do that (and I often do) is that you have further piping options so it becomes standard work flow and muscle memory. What I usually do is cat the file to inspect it, hit Control+C, then up arrow for previous command, then further pipe and head/tail/grep the file. Starting a grep command is fine if you know that's all you're going to be doing.
  - laumars 8y ago
    You can use Bash (and I assumed it works in zsh too) variables to save you there. eg grep something !$ `!$` will be replaced with the last parameter of the previous command. Crude example cat /etc/hosts grep 12.34.56.78 $! # will be executed as grep 12.34.56.78 /etc/hosts I find this extremely useful for when I'm cat'ing a file to get a sense of it's output then wanting to do something more meaningful with it in the next command. That all said, I have absent-mindedly done my fair share of stuff like this too: !! | less # eg will run as `cat /etc/hosts | less` # if `cat /etc/hosts` was the previous command run so I certainly wouldn't look down on people who have "abused" cat due to muscle memory.
    - shpx 8y ago
      You can also hit Alt-. to insert the last argument of the previous command.
      - laumars 8y ago
        Unfortunately not on Macs :( That's one of the features I miss the most when using terminals on a Mac.
fxbl0i 8y ago
I implore the hacker to stop. The 7 users of Matrix are very concerned about the future of the service.
subutux 8y ago
It seems he used github for hosting his content on matrix.org https://github.com/matrixnotorg/matrixnotorg.github.io https://github.com/matrixnotorg/matrixnotorg.github.io
F30 8y ago
The (presumed) attacker opened a bunch of issues in Matrix' GitHub issue tracker, explaining the security issues leading to this compromise: https://github.com/matrix-org/matrix.org/issues/created_by/matrixnotorg https://github.com/matrix-org/matrix.org/issues/created_by/m... TL;DR: A collection of inadvertences and suboptimal practices, some (like having GPG signing keys on production systems) more worrying than others. Something that could probably have happened to most orgs without dedicated security resources.
azhenley 8y ago
The hacker seems nice: “Anyways, that's all for now. I hope this series of issues has given you some good ideas for how to prevent this level of compromise in the future. Security doesn't work retroactively, but I believe in you and I think you'll come back from this even stronger than before. Or at least, I hope so -- My own information is in this user table.” https://github.com/matrix-org/matrix.org/issues/365 https://github.com/matrix-org/matrix.org/issues/365
- meruru 8y ago
  Hopefully, but it could be just posturing.
- chatmasta 8y ago
  I wonder if he searched for his own credentials on the production server, and if so, is there a log of it?
  - legooolas 8y ago
    Just grabbing it to elsewhere and searching there would be trivial, so I'd expect not.
- erikbye 8y ago
  "Or at least, I hope so -- My own information is in this user table... jk, I use EFNet."
  - user9182031 8y ago
    I enjoyed the shout out for EFnet.
- DuskStar 8y ago
  That gives a 404 now. I'm not enormously happy with GitHub's willingness to completely hide bug reports like this...
  - 8y ago
dininski 8y ago
I can see a lot of people trashing on Matrix.org or the "hacker" themselves (the hacker opened a series of issues, detailing how he managed to get in - https://github.com/matrix-org/matrix.org/issues/created_by/matrixnotorg https://github.com/matrix-org/matrix.org/issues/created_by/m...). However everyone seems to be missing the point - matrix seems like a pretty cool and open project. And someone taking over their infrastructure in such an open way is also great for the community. Even though a little dubious on the legal side of things, I believe it's great it was approached with transparency and a dose of humor. Some might argue that this is harmful to matrix as a product and as a brand. But as long as there was no actual harm done and they react appropriately by taking infrastructure security seriously, it could play out well in the end for them. This whole ordeal could end up actually increase trust in the project, if they take swift steps to ensure that something like this does not happen again.
- chronogram 8y ago
  There are some weird people in those issue threads..
  - executesorder66 8y ago
    What did they say? The comments have been deleted.
    - deleted 8y ago
      [deleted]
  - ozzmotik 8y ago
    checking in as an internet weird person here, any time you have a platform that synthesizes anonymity with collaboration/social interaction, weirdos like us are gonna pop out like clockwork because we find a safe haven for our, uh, weird stuff. a place to not be judged or whatever. i think the sjw type terminology for it is a safe space. and of course due to the human element being so easily corruptible, many people do also tend to use such things for illicit purposes. hey anybody else remember the days of T-Philez?
growt 8y ago
I'm probably really out of the loop, but what is matrix.org? Looks like an open source slack clone? Why do they have >5 million user accounts? Is that everybody who uses that chat tool?
- kzcqt 8y ago
  Matrix is what happened when somebody looked at XMPP and yelled "NIH".
  - yjftsjthsd-h 8y ago
    Matrix is what happened when somebody looked at XMPP and yelled "wow, this aged poorly and has some major usability issues".
    - kzcqt 8y ago
      And instead of fixing the issues they just went to do a completely new and incompatible thing. That's the very definition of NIH
      - seba_dos1 8y ago
        Especially that, after some period of stagnation, XMPP is doing pretty fine these days with stepping into the modern world.
        vertex-four 8y ago
        Ehh... not really. I still can't find a good combination of server, desktop client and iOS client that support things like OMEMO, history sharing between clients, and voice/video chat. And the one iOS client (ChatSecure) looks really dodgy and regularly fails while setting up push notifications.
        kzcqt 8y ago
        And instead of adding those features to existing clients, let's create a brand new protocol, server, desktop client and mobile client. Because why not?
beedogs 8y ago
what a mess.
Arathorn 8y ago
Project lead for Matrix.org here - you can see our initial statement on this at http://matrix.org/blog/2019/04/11/security-incident/ http://matrix.org/blog/2019/04/11/security-incident/. It will be updated shortly to reflect the DNS defacement linked here (which was because we failed to rotate a leaked cloudflare API token; we aimed to rotate the master API token but rotated a personal one instead). To our knowledge the rebuilt production infrastructure itself is secure. We've revoked the compromised GPG keys, and are obviously going to do everything we can to improve our production security to avoid a recurrence in future. We can only apologise to everyone caught in the crossfire of this incident.
- notyourday 8y ago
  Have you implemented hole punching?
  - syntonym 8y ago
    What is hole punching and how would it have helped against the attack? I only know the term for UDP firewall transversal.
    - notyourday 8y ago
      The lame version of port knocking that solves 99.9% of issues. 1. Default policy for access to all of the development environment is deny all. 2. A developer triggers a temporary addition of developers current address to the allow list with an idle timer, punching a hole for developer's edge IP to access the infrastructure. 3. When the idle timer expires or when the developer says "i'm done", the allow rule is removed. Obviously, a full blown port knocking with keys and policies would be better for a large organization with hundreds of developers and hundreds of hosts but it is the case where 99.9% of the issues can be solved using a very simple system as in order to get to the vulnerable entry point the attacker would need to do it from an IP address used by a developer at that specific time.
      - peterwwillis 8y ago
odensc 8y ago
TL;DR: Looks like there was a server with an unpatched Jenkins instance running, which allowed RCE. [0] Someone (presumably a developer) was connected to that compromised server via SSH, and had forwarded their SSH agent to it. [1] Apparently that person had root access to the production servers, allowing the attacker to login via the forwarded agent. Yikes. [0]: https://matrix.org/blog/2019/04/11/security-incident/ https://matrix.org/blog/2019/04/11/security-incident/ [1]: https://github.com/matrix-org/matrix.org/issues/358 https://github.com/matrix-org/matrix.org/issues/358
- lucb1e 8y ago
  Thanks for that summary, the twitter thread that I read on it was not quite as enlightening as this small summary!
nukeop 8y ago
The Github issues created by the hacker just got wiped clean, I guess they want to censor any negative opinions users may have about the quality and security of their service.
snvzz 8y ago
4chan is circulating this picture. It shows the defaced website frontpage. https://i.4cdn.org/g/1555048975736.png https://i.4cdn.org/g/1555048975736.png I have a hard time with the idea that they run the webserver and the matrix server on the same computer. (Regarding users.txt) It seems they do urgently need to hire capable infrastructure people.
- josephmx 8y ago
  I can't access that image on my corporate network, any chance of an imgur mirror?
  - couterSpell 8y ago
    It's just a screenshot of the same info shown on the archive.org page linked by the title. https://web.archive.org/web/20190412060115/http://matrix.org/ https://web.archive.org/web/20190412060115/http://matrix.org... If you can't get to archive.org, just respond and I'll imgur it.
- vbrandl 8y ago
  They didn't host production and the website on the same server. The attacker had access to the whole network. After that was detected and cleared, the attacker was still able to change DNS records. The domain was redirected to an attacker controlled site (https://github.com/matrixnotorg/matrixnotorg.github.io https://github.com/matrixnotorg/matrixnotorg.github.io) where some logs of the production servers where posted.
- snvzz 8y ago
  Correction: Not on the same server. They just managed to repoint DNS. See 2019-04-12 update on the incident article[0]. [0]: https://matrix.org/blog/2019/04/11/security-incident/index.html https://matrix.org/blog/2019/04/11/security-incident/index.h...
rglullis 8y ago
The interesting thing for me here is that none of the other homeservers were affected. Despite the weak security on the largest servers, the ecosystem stays alive. Antifragility at its finest.
- kzcqt 8y ago
  If I paid monthly for a server and then spent time configuring it, now I would be able to talk to people who pay monthly for their servers and spent time configuring them.
  - rglullis 8y ago
    Alternatively, if you paid monthly to any hosting provider, you could be talking with customers of any other hosting providers. And if there were any other hosting providers that came doing such shoddy things in their production systems, they would be wiped out of the ecosystem, but the ecosystem would still be alive. Just like email, or phone lines... antifragile.
- maxidorius 8y ago
  ... or is it? https://github.com/matrix-org/matrix-doc/issues/1194 https://github.com/matrix-org/matrix-doc/issues/1194 and https://github.com/matrix-org/matrix-doc/pull/1915 https://github.com/matrix-org/matrix-doc/pull/1915 and https://github.com/matrix-org/synapse/issues/4540 https://github.com/matrix-org/synapse/issues/4540 would tell a different story: Potentially deleting data on remote server just because being matrix.org (or anyone with an access)
  - rglullis 8y ago
    I might be misreading it, but it seems that the issues you are pointing out related to 3PID, which is still somewhat centralized. Sure, work on this needs to be done yet the system is evolving to be more independent.
santamarias 8y ago
in order to regain trust in matrix.org, what other options exist than an external security audit?
- swalladge 8y ago
  Audit the source code, host your own homeserver, and use e2e encryption. The power of open source and decentralisation! :D
- ndnxhs 8y ago
  I'm far less concerned about this because all my messages were end to end encrypted.
nullc 8y ago
I never use SSH agent forwarding but instead use -D socks5 proxying to directly ssh from my host to the ultimate target hosts.
wink 8y ago
I think it boils down to the fact that infrastructure for projects (no matter the size) is usually a second class citizen at best. Either no one is eager to care for it, or the people who are actually focused on developing the software run it because they need to, or worst case - no contributor is trusted enough to handle infrastructure work, with access being given even more sparsely than commit rights to the whole software. Which is fine by itself, but there are so many (big) projects where infra is kind of terrible because 3 out 100 people involved are doing all the work. Or don't.
- proy24 8y ago
  It's just a problem with the industry that's like "let's just get something out there and achieve product market first..will worry about infra and security later" that later is just pushed into a backlog and forgotten.
  - dijit 8y ago
    As a classically trained sysadmin I've seen this trend going for the last decade. Some developers seem to be pushing that ops shouldn't exist any longer or should be outsourced to google (who don't hire ops) or amazon (who do). Managers see this trend and think that hiring only developers is a good way to save costs and do things the "new way". Traditional ops roles are indeed not as required but security/process/reliability focused people should not be the same people who write new features. They're in contradiction of each other often. If you're a developer who thinks ops shouldn't exist any longer consider this: I can write software and design websites as a sysadmin, does that mean I don't need you now, or that I know everything you do? I argue that it doesn't. A focus on automation is one thing but defenestrating the notion of operations/SRE is going to net you a bad time.
- deleted 8y ago
  [deleted]
Oblongoid 8y ago
I've been slightly annoyed with matrix ever since they boasted at FOSDEM with the fact that they backdoored their encryption so that the French government could Virus scan sent files. :/
- Arathorn 8y ago
  we didn't backdoor the encryption. instead, we specced how clients could securely pass attachment keys to an AV server, if they need to. but in practice none of them (other than the french app) do. the whole point was to spell out that we haven't backdoored the encryption, and instead been transparent about how content filtering could be done in the most responsible manner, if it's really needed.
couterSpell 8y ago
I like the fact matrixnotorg decided to alert Matrix to Elasticsearch's existence. But Matrix probably should first figure out how to fix the whole 'all server management ports are open to the internet' problem detailed here: https://github.com/matrix-org/matrix.org/issues/360 https://github.com/matrix-org/matrix.org/issues/360 The last thing we need is another Elasticsearch instance listening on a public IP accessible to the world.
xyproto 8y ago
It's "usless use of cat". He/she should have gone: `grep arathorn users.txt | head -1` Instead of: `cat users.txt | grep arathorn | head -n1` Hackers these days.
- draw_down 8y ago
  That is fine.
- 0FDAA764 8y ago
  It's a "useless use of head". He/she should have gone: `grep -m1 arathorn users` Instead of: `grep arathorn users.txt | head -1` Commenters these days.
  - xyproto 8y ago
    Cool!
  - shanth 8y ago
    It's been a while seeing UUoC awards on a random internet discussion.
- kqr 8y ago
  I still dont get that logic. What if it turns out i want to stick a prerprocessing step before grep? With a "useless" use of cat, thats easy. Without it, i need to do some rearranging. Not convinced.
  - xyproto 8y ago
    It's the YAGNI principle.
standinator 8y ago
Stop cat abuse.
humantiy 8y ago
From reading the headline couldn't help but think to myself was it Neo?
trw999 8y ago
As someone running a Matrix homeserver I take this incident as an example of the benefits of decentralization. Unlike in more centralized services, the security lapses of Matrix.org have had no affect on my homeserver.
IronCoderXYZ 8y ago
Looks like all issues created by the "hacker" have been removed? https://github.com/matrix-org/matrix.org/issues?utf8=%E2%9C%93&q=author%3Amatrixnotorg+ https://github.com/matrix-org/matrix.org/issues?utf8=%E2%9C%...
- gerogerke 8y ago
  Seems like the user itself has been deleted, which might cause Github to remove all content created by that user.
  - deleted 8y ago
    [deleted]
  - Arathorn 8y ago
    we (Matrix.org) haven't deleted the issues; we were deliberately leaving them up for reference.
NotOscarWilde 8y ago
I lost all my messages with my girlfriend (of course). Can anyone clarify: if I use their "server key backup" and set a passphrase, I am now two passwords away from giving the next hacker read access to all my messages, is that right?

324 points by nektro 8y ago

m_b 8y ago

For a bit of context: Matrix.org infrastructure has been hacked a second time in 24h, after restoring everything they went down again, story developing here: https://twitter.com/matrixdotorg/status/1116304867683905537 https://twitter.com/matrixdotorg/status/1116304867683905537

m_b 8y ago
The hacker is now doing a post-mortem in the GitHub issues of the project: https://github.com/matrix-org/matrix.org/issues https://github.com/matrix-org/matrix.org/issues
- nothrabannosir 8y ago
  This is gold... > I noticed in your blog post that you were talking about doing a postmortem and steps you need to take. As someone who is intimately familiar with your entire infrastructure, I thought I could help you out. > There I was, just going about my business, looking for ways I could get higher levels of access and explore your network more, when I stumbled across GPG keys that were used for signing your debian packages. It gave me many nefarious ideas. I would recommend that you don't keep any signing keys on production hosts, and instead do all of your signing in a secure environment.
  - paavoova 8y ago
    Why would they do this? It's pure negligence. I don't even sign anything important and still worry about my keys.
    - henvic 8y ago
      I have been asked twice or more why I insisted on not using a Continuous Integration environment for publishing some software releases that are installed by third-parties. My team was automating the infrastructure to build internal software and naturally they wanted to be able to simplify things. The idea that was proposed to me was the following: once I push a new version tag to GitHub, the deployment CI server is going to build and release it as an unstable version. Some important detail here: I use the same key to sign packages regardless if they are released as unstable or stable. That would mean that if someone, somehow, managed to push a tag that was pushed upstream to GitHub, hypothetically they would be able to eventually gain access to consumers machines (basically, developers) when the consumers update it after getting a notification telling them a new version is available. No way I'd allow this to happen, but I would not be surprised if most people just took this as an acceptable risk.
      - simias 8y ago
        Depending on your threat model I think that signing packages directly from your CI is acceptable, assuming that your CI runs is a reasonably isolated environment (e.g. on your company's LAN) and people who are able to trigger a release are correctly vetted. If I understand the parent comment correctly they were somehow shipping the release signing key on their production environment which is a whole other level of bad.
      - wcoenen 8y ago
        > That would mean that if someone, somehow, managed to push a tag that was pushed upstream to GitHub You have to define what the signature means. IMHO it is fine for it to mean "this software was built on our build server from a well-defined state of the source code, which is only changable by our employees and contractors, and for which we have the full change log". So I deploy the code signing key to build servers, which is the only place where it is used. I'm interested in what alternative meaning you would give to a signature. I have considered the possibility of tying it to the QA processes, but then a build can only be signed after checking it manually, which is problematic when many signatures are needed at multiple packaging layers (exe/dll, msi, setup.exe).
        rakoo 8y ago
        The problem is when a malicious package is produced, either because a flaw was introduced in the code, or because a dev machine was compromised, or _when the CI machine sad compromised_; the malicious package will be signed as if it were legit. One middle point between automated and manual signing is, as usual, key rotation: have the signing keys expire in a short duration of time (say 2 weeks) and manually push them every week, so that the window of attack is as small as possible.
        yolo1 8y ago
        What does a key rotation solve? Either your build server is compromised or it's not.
        notyourday 8y ago
  - lelf 8y ago
    Another gem: RRREEEEEEEE> I noticed you missed a doctype in your html page. In order for web browsers to know what type of html to render you should include a doctype. Thanks! matrixnotorg> @RRREEEEEEEE Thank you, I will consider that for the next release Edit: it got deleted But see also: https://github.com/matrixnotorg/matrixnotorg.github.io/pull/2 https://github.com/matrixnotorg/matrixnotorg.github.io/pull/...
    - justaj 8y ago
      Wait, did Github delete matrixnotorg's profile or did matrixnotorg? If Github deleted that profile, I don't really see that as being very hacker-friendly.
      - aeternus 8y ago
        Although 'hacker' is often used as a positive term on HN, breaking into a company's production server is clearly illegal activity and should not be condoned. If Github deleted the account, they are simply acting in accordance to published TOS & policy.
        justaj 8y ago
        If the attacker placed sensitive information on Github, that would indeed warrant a deletion of the account. However from what I saw from the archives, the attacker merely published details about Matrix.org's infrastructure and its vulnerabilities. Is that something that's against Github's ToS?
- m00dy 8y ago
  hilarious...
  - meruru 8y ago
    I can only think of the despair I would be in if I was in Aragorn's shoes
    - rabidrat 8y ago
      Arathorn (Aragorn is son of Arathorn).
      - meruru 8y ago
        Does being a parent prepare you to handle emergency situations like this better?
- deleted 8y ago
  [deleted]
- atsjie 8y ago
  > Escalation could have been avoided if developers only had the access they absolutely required and did not have root access to all of the servers. I would like to take a moment to thank whichever developer forwarded their agent to Flywheel. I'd feel so small if I were this developer right now :-| A couple of his issues appear to have to do with the use of SSH. An Ops-guy whom I worked with had setup a bastion host with ip whitelisting that automatically shut down after 1 hour. He didn't like it as his credo was "if you're using SSH when using a cloud provider you're probably doing something wrong"; meaning to say you should automate and be able to recreate any infra at all times with logs accessible without the need for SSH. I never forgot that.
  - robrtsql 8y ago
    Did you work with Rich Adams? https://wblinks.com/notes/aws-tips-i-wish-id-known-before-i-started/ https://wblinks.com/notes/aws-tips-i-wish-id-known-before-i-...
    - atsjie 8y ago
      no, but I wouldn't be surprised if my colleague got the idea from him. It was around the same time (end of 2014 I think).
- BartBoch 8y ago
  I have just checked quickly the comments and post-mortem, and I start wondering - it seems that the attack itself would be not really possible if Matrix would not be open source (as this would restrict access to the sensitive data)? Is that right?
  - t3chguy 8y ago
    This is not right at all, the hack was due to an outdated Jenkins instance and could have happened regardless of what other software was running on the infrastructure.
    - notyourday 8y ago
      Absolutely not. There's a zero reason for development infrastructure ( which includes Jenkins ) to have any connection to production outside a well defined "transfer this tarball" to a deploy staging server path during a known deploy window which cannot be controlled via development credentials.
F30 8y ago
I am highly skeptical when people taking about "rebuilding [the whole] infrastructure" in a few hours. Even more so when restoring all data from breached systems and before a thorough incident analysis. Show me the org which can just pull that off.
- pferde 8y ago
  It's quite common in enterprise to have a RTO of a few hours, and RPO of a few minutes, even for infrastructure with terrabytes of data. Of course, many moneys are paid for being able to do that.
- erikbye 8y ago
  This is doable with proper IaC implementation, and if your org does not have RPO/RTO on lock they're doing it wrong. Events like Matrix experienced now do not lead to panicked frenzy when this is in place.
  - F30 8y ago
    It is certainly doable, but I doubt that most people have IaC which is complete, reproducible and tested enough. And the data migration from the breached host still means some risk.
proboscis 8y ago
They closed the threads and deleted all the comments, but luckily the issues page was archived beforehand: https://web.archive.org/web/20190412090123/https://github.com/matrix-org/matrix.org/issues https://web.archive.org/web/20190412090123/https://github.co...
- gtirloni 8y ago
  Luckily yeah. So much wisdom in those comments. /s

perttir 8y ago

They had root account activated in hebe? Am i reading this right? He got an passlist of 5 million users?

m_b 8y ago
yeah
saagarjha 8y ago
Yup: https://github.com/matrix-org/matrix.org/issues/358 https://github.com/matrix-org/matrix.org/issues/358
deleted 8y ago
[deleted]
beagle3 8y ago
But even if they didn't, "sudo -u root /bin/bash" or similar gives it to you unless sudo is extremely locked down (which, from audits I have done, is "rarely if ever"). There are hundreds of ways to get root prompt even with the root account nominally deactivated.
- deleted 8y ago
  [deleted]

GrumpyNl 8y ago

Everything gets hacked, period. They have to remove this from there slogan, "An open standard for decentralised secure communication."

tsukurimashou 8y ago
Nice try NSA
johnchristopher 8y ago
Really ? That's a bold claim. Can this string be hacked ? $6$WqQvgIwr$/BA/19FRJZf.z4buUwX5Sbls07ovv/lVZJ3V2En7VgTR4Skdpz98hFNuq3VX4nIaiSDKObO9jKbkPb5tdt7zY1
- bdibs 8y ago
  Eventually :)
- chme 8y ago
  I guess the point is that you cannot protect anything from the future if you don't just destroy it completely.
  - meruru 8y ago
    On that note, the matrix.org server currently doesn't delete messages from rooms that have become empty. It probably would be a good idea to do so.
    - acct1771 8y ago
      Does synapse allow that for self hosted?
      - meruru 8y ago
        Yes, IIRC it's a script you have to run periodically.

cjslep 8y ago

I believe this is meant to show that it is a targeted attack on the project lead: https://news.ycombinator.com/user?id=Arathorn https://news.ycombinator.com/user?id=Arathorn Unfortunately I don't have any background context for possible reasons why "actual transparency" on the top line is the issue chosen by the attacker, but makes it seem ideologically driven.

JD557 8y ago
I don't think that this is a targeted attack. Seems more like a way of showing "I got access to 5493973 passwords and to show that, instead of picking some random users, I'll pick the one responsible for the shoddy security".
- saagarjha 8y ago
  Or it might be a clear, concise way of showing that he has access to the entire file without disclosing the information of random users, which also happens to be a particularly short command.
erikbye 8y ago
As for motivation I don't know, but I would like to state that I have followed the Matrix project for some time and have found it, and the CEO, to be transparent.

zaarn 8y ago

Doesn't surprise me that much, Matrix doesn't seem to be too concerned with security, more with security theatre (considering you can still not easily disable read receipts in your client, a major privacy leak IMO, among other issues).

meruru 8y ago
Disabling read receipts is a client feature. Yes, the currently most mature client doesn't have that feature, but nothing in Matrix precludes it.
- zaarn 8y ago
  There is plenty of other issues with matrix and the reference clients on top of something as simple as mandatory leaking of your presence in a chatroom. I've run a matrix homeserver for almost 3 weeks and it as an utter pain to maintain, despite not a single version upgrade and I was plagued with issues that no chat platform would have if the protocol was remotely sane. edit: That is on top of the numerous security issues this hack uncovered. Apparently the matrix.org devs kept a users.txt file with a dump of users + passwords on the server. Signing keys for debian packages were stored unencrypted on the production server. People used unsafe SSH settings (SSH Agent Forwarding), ran outdated servers with known root-priv RCEs for months and root privileges for all users on a server. Why should I ever trust a matrix developer with their protocol or reference implementations ever again if they can't be trusted with the simple task of updating a service when a critical CVE comes out?
  - pferde 8y ago
    While you bring up valid concerns about the Matrix team's security hygiene, the point of an open standard is that anyone can (try to) spot flaws in it, and anyone can (try to) create their own implementation. I myself am waiting for a healthy ecosystem of servers and clients to spring up before starting to rely on Matrix for anything non-ephemeral - even if it takes years. Perhaps I'll even try my hand at writing a client, if I ever run out of things to do. In the meantime, I will dick around with a throwaway matrix.org account to play with it, and to watch progress happen.

deleted 8y ago

[deleted]

fxbl0i 8y ago

Wow, piping the output of cat to another utility. What a n00b. I hope the hacker gets hacked

SpaceInvader 8y ago

As na linux/unix sysadmin with 15+ years of experience my eyes are bleeding everytime I see cat <file> | grep <pattern> instead of just grep <pattern> <file> ;D

jyap 8y ago
Reason why you would do that (and I often do) is that you have further piping options so it becomes standard work flow and muscle memory. What I usually do is cat the file to inspect it, hit Control+C, then up arrow for previous command, then further pipe and head/tail/grep the file. Starting a grep command is fine if you know that's all you're going to be doing.
- laumars 8y ago
  You can use Bash (and I assumed it works in zsh too) variables to save you there. eg grep something !$ `!$` will be replaced with the last parameter of the previous command. Crude example cat /etc/hosts grep 12.34.56.78 $! # will be executed as grep 12.34.56.78 /etc/hosts I find this extremely useful for when I'm cat'ing a file to get a sense of it's output then wanting to do something more meaningful with it in the next command. That all said, I have absent-mindedly done my fair share of stuff like this too: !! | less # eg will run as `cat /etc/hosts | less` # if `cat /etc/hosts` was the previous command run so I certainly wouldn't look down on people who have "abused" cat due to muscle memory.
  - shpx 8y ago
    You can also hit Alt-. to insert the last argument of the previous command.
    - laumars 8y ago
      Unfortunately not on Macs :( That's one of the features I miss the most when using terminals on a Mac.

fxbl0i 8y ago

I implore the hacker to stop. The 7 users of Matrix are very concerned about the future of the service.

subutux 8y ago

It seems he used github for hosting his content on matrix.org https://github.com/matrixnotorg/matrixnotorg.github.io https://github.com/matrixnotorg/matrixnotorg.github.io

F30 8y ago

The (presumed) attacker opened a bunch of issues in Matrix' GitHub issue tracker, explaining the security issues leading to this compromise: https://github.com/matrix-org/matrix.org/issues/created_by/matrixnotorg https://github.com/matrix-org/matrix.org/issues/created_by/m... TL;DR: A collection of inadvertences and suboptimal practices, some (like having GPG signing keys on production systems) more worrying than others. Something that could probably have happened to most orgs without dedicated security resources.

azhenley 8y ago

The hacker seems nice: “Anyways, that's all for now. I hope this series of issues has given you some good ideas for how to prevent this level of compromise in the future. Security doesn't work retroactively, but I believe in you and I think you'll come back from this even stronger than before. Or at least, I hope so -- My own information is in this user table.” https://github.com/matrix-org/matrix.org/issues/365 https://github.com/matrix-org/matrix.org/issues/365

meruru 8y ago
Hopefully, but it could be just posturing.
chatmasta 8y ago
I wonder if he searched for his own credentials on the production server, and if so, is there a log of it?
- legooolas 8y ago
  Just grabbing it to elsewhere and searching there would be trivial, so I'd expect not.
erikbye 8y ago
"Or at least, I hope so -- My own information is in this user table... jk, I use EFNet."
- user9182031 8y ago
  I enjoyed the shout out for EFnet.
DuskStar 8y ago
That gives a 404 now. I'm not enormously happy with GitHub's willingness to completely hide bug reports like this...
- 8y ago

dininski 8y ago

I can see a lot of people trashing on Matrix.org or the "hacker" themselves (the hacker opened a series of issues, detailing how he managed to get in - https://github.com/matrix-org/matrix.org/issues/created_by/matrixnotorg https://github.com/matrix-org/matrix.org/issues/created_by/m...). However everyone seems to be missing the point - matrix seems like a pretty cool and open project. And someone taking over their infrastructure in such an open way is also great for the community. Even though a little dubious on the legal side of things, I believe it's great it was approached with transparency and a dose of humor. Some might argue that this is harmful to matrix as a product and as a brand. But as long as there was no actual harm done and they react appropriately by taking infrastructure security seriously, it could play out well in the end for them. This whole ordeal could end up actually increase trust in the project, if they take swift steps to ensure that something like this does not happen again.

chronogram 8y ago
There are some weird people in those issue threads..
- executesorder66 8y ago
  What did they say? The comments have been deleted.
  - deleted 8y ago
    [deleted]
- ozzmotik 8y ago
  checking in as an internet weird person here, any time you have a platform that synthesizes anonymity with collaboration/social interaction, weirdos like us are gonna pop out like clockwork because we find a safe haven for our, uh, weird stuff. a place to not be judged or whatever. i think the sjw type terminology for it is a safe space. and of course due to the human element being so easily corruptible, many people do also tend to use such things for illicit purposes. hey anybody else remember the days of T-Philez?

growt 8y ago

I'm probably really out of the loop, but what is matrix.org? Looks like an open source slack clone? Why do they have >5 million user accounts? Is that everybody who uses that chat tool?

kzcqt 8y ago
Matrix is what happened when somebody looked at XMPP and yelled "NIH".
- yjftsjthsd-h 8y ago
  Matrix is what happened when somebody looked at XMPP and yelled "wow, this aged poorly and has some major usability issues".
  - kzcqt 8y ago
    And instead of fixing the issues they just went to do a completely new and incompatible thing. That's the very definition of NIH
    - seba_dos1 8y ago
      Especially that, after some period of stagnation, XMPP is doing pretty fine these days with stepping into the modern world.
      - vertex-four 8y ago
        Ehh... not really. I still can't find a good combination of server, desktop client and iOS client that support things like OMEMO, history sharing between clients, and voice/video chat. And the one iOS client (ChatSecure) looks really dodgy and regularly fails while setting up push notifications.
        kzcqt 8y ago
        And instead of adding those features to existing clients, let's create a brand new protocol, server, desktop client and mobile client. Because why not?

beedogs 8y ago

what a mess.

Arathorn 8y ago

Project lead for Matrix.org here - you can see our initial statement on this at http://matrix.org/blog/2019/04/11/security-incident/ http://matrix.org/blog/2019/04/11/security-incident/. It will be updated shortly to reflect the DNS defacement linked here (which was because we failed to rotate a leaked cloudflare API token; we aimed to rotate the master API token but rotated a personal one instead). To our knowledge the rebuilt production infrastructure itself is secure. We've revoked the compromised GPG keys, and are obviously going to do everything we can to improve our production security to avoid a recurrence in future. We can only apologise to everyone caught in the crossfire of this incident.

notyourday 8y ago
Have you implemented hole punching?
- syntonym 8y ago
  What is hole punching and how would it have helped against the attack? I only know the term for UDP firewall transversal.
  - notyourday 8y ago
    The lame version of port knocking that solves 99.9% of issues. 1. Default policy for access to all of the development environment is deny all. 2. A developer triggers a temporary addition of developers current address to the allow list with an idle timer, punching a hole for developer's edge IP to access the infrastructure. 3. When the idle timer expires or when the developer says "i'm done", the allow rule is removed. Obviously, a full blown port knocking with keys and policies would be better for a large organization with hundreds of developers and hundreds of hosts but it is the case where 99.9% of the issues can be solved using a very simple system as in order to get to the vulnerable entry point the attacker would need to do it from an IP address used by a developer at that specific time.
    - peterwwillis 8y ago

odensc 8y ago

TL;DR: Looks like there was a server with an unpatched Jenkins instance running, which allowed RCE. [0] Someone (presumably a developer) was connected to that compromised server via SSH, and had forwarded their SSH agent to it. [1] Apparently that person had root access to the production servers, allowing the attacker to login via the forwarded agent. Yikes. [0]: https://matrix.org/blog/2019/04/11/security-incident/ https://matrix.org/blog/2019/04/11/security-incident/ [1]: https://github.com/matrix-org/matrix.org/issues/358 https://github.com/matrix-org/matrix.org/issues/358

lucb1e 8y ago
Thanks for that summary, the twitter thread that I read on it was not quite as enlightening as this small summary!

nukeop 8y ago

The Github issues created by the hacker just got wiped clean, I guess they want to censor any negative opinions users may have about the quality and security of their service.

snvzz 8y ago

4chan is circulating this picture. It shows the defaced website frontpage. https://i.4cdn.org/g/1555048975736.png https://i.4cdn.org/g/1555048975736.png I have a hard time with the idea that they run the webserver and the matrix server on the same computer. (Regarding users.txt) It seems they do urgently need to hire capable infrastructure people.

josephmx 8y ago
I can't access that image on my corporate network, any chance of an imgur mirror?
- couterSpell 8y ago
  It's just a screenshot of the same info shown on the archive.org page linked by the title. https://web.archive.org/web/20190412060115/http://matrix.org/ https://web.archive.org/web/20190412060115/http://matrix.org... If you can't get to archive.org, just respond and I'll imgur it.
vbrandl 8y ago
They didn't host production and the website on the same server. The attacker had access to the whole network. After that was detected and cleared, the attacker was still able to change DNS records. The domain was redirected to an attacker controlled site (https://github.com/matrixnotorg/matrixnotorg.github.io https://github.com/matrixnotorg/matrixnotorg.github.io) where some logs of the production servers where posted.
snvzz 8y ago
Correction: Not on the same server. They just managed to repoint DNS. See 2019-04-12 update on the incident article[0]. [0]: https://matrix.org/blog/2019/04/11/security-incident/index.html https://matrix.org/blog/2019/04/11/security-incident/index.h...

rglullis 8y ago

The interesting thing for me here is that none of the other homeservers were affected. Despite the weak security on the largest servers, the ecosystem stays alive. Antifragility at its finest.

kzcqt 8y ago
If I paid monthly for a server and then spent time configuring it, now I would be able to talk to people who pay monthly for their servers and spent time configuring them.
- rglullis 8y ago
  Alternatively, if you paid monthly to any hosting provider, you could be talking with customers of any other hosting providers. And if there were any other hosting providers that came doing such shoddy things in their production systems, they would be wiped out of the ecosystem, but the ecosystem would still be alive. Just like email, or phone lines... antifragile.
maxidorius 8y ago
... or is it? https://github.com/matrix-org/matrix-doc/issues/1194 https://github.com/matrix-org/matrix-doc/issues/1194 and https://github.com/matrix-org/matrix-doc/pull/1915 https://github.com/matrix-org/matrix-doc/pull/1915 and https://github.com/matrix-org/synapse/issues/4540 https://github.com/matrix-org/synapse/issues/4540 would tell a different story: Potentially deleting data on remote server just because being matrix.org (or anyone with an access)
- rglullis 8y ago
  I might be misreading it, but it seems that the issues you are pointing out related to 3PID, which is still somewhat centralized. Sure, work on this needs to be done yet the system is evolving to be more independent.

santamarias 8y ago

in order to regain trust in matrix.org, what other options exist than an external security audit?

swalladge 8y ago
Audit the source code, host your own homeserver, and use e2e encryption. The power of open source and decentralisation! :D
ndnxhs 8y ago
I'm far less concerned about this because all my messages were end to end encrypted.

nullc 8y ago

I never use SSH agent forwarding but instead use -D socks5 proxying to directly ssh from my host to the ultimate target hosts.

wink 8y ago

I think it boils down to the fact that infrastructure for projects (no matter the size) is usually a second class citizen at best. Either no one is eager to care for it, or the people who are actually focused on developing the software run it because they need to, or worst case - no contributor is trusted enough to handle infrastructure work, with access being given even more sparsely than commit rights to the whole software. Which is fine by itself, but there are so many (big) projects where infra is kind of terrible because 3 out 100 people involved are doing all the work. Or don't.

proy24 8y ago
It's just a problem with the industry that's like "let's just get something out there and achieve product market first..will worry about infra and security later" that later is just pushed into a backlog and forgotten.
- dijit 8y ago
  As a classically trained sysadmin I've seen this trend going for the last decade. Some developers seem to be pushing that ops shouldn't exist any longer or should be outsourced to google (who don't hire ops) or amazon (who do). Managers see this trend and think that hiring only developers is a good way to save costs and do things the "new way". Traditional ops roles are indeed not as required but security/process/reliability focused people should not be the same people who write new features. They're in contradiction of each other often. If you're a developer who thinks ops shouldn't exist any longer consider this: I can write software and design websites as a sysadmin, does that mean I don't need you now, or that I know everything you do? I argue that it doesn't. A focus on automation is one thing but defenestrating the notion of operations/SRE is going to net you a bad time.
deleted 8y ago
[deleted]

Oblongoid 8y ago

I've been slightly annoyed with matrix ever since they boasted at FOSDEM with the fact that they backdoored their encryption so that the French government could Virus scan sent files. :/

Arathorn 8y ago
we didn't backdoor the encryption. instead, we specced how clients could securely pass attachment keys to an AV server, if they need to. but in practice none of them (other than the french app) do. the whole point was to spell out that we haven't backdoored the encryption, and instead been transparent about how content filtering could be done in the most responsible manner, if it's really needed.

couterSpell 8y ago

I like the fact matrixnotorg decided to alert Matrix to Elasticsearch's existence. But Matrix probably should first figure out how to fix the whole 'all server management ports are open to the internet' problem detailed here: https://github.com/matrix-org/matrix.org/issues/360 https://github.com/matrix-org/matrix.org/issues/360 The last thing we need is another Elasticsearch instance listening on a public IP accessible to the world.

xyproto 8y ago

It's "usless use of cat". He/she should have gone: `grep arathorn users.txt | head -1` Instead of: `cat users.txt | grep arathorn | head -n1` Hackers these days.

draw_down 8y ago
That is fine.
0FDAA764 8y ago
It's a "useless use of head". He/she should have gone: `grep -m1 arathorn users` Instead of: `grep arathorn users.txt | head -1` Commenters these days.
- xyproto 8y ago
  Cool!
- shanth 8y ago
  It's been a while seeing UUoC awards on a random internet discussion.
kqr 8y ago
I still dont get that logic. What if it turns out i want to stick a prerprocessing step before grep? With a "useless" use of cat, thats easy. Without it, i need to do some rearranging. Not convinced.
- xyproto 8y ago
  It's the YAGNI principle.

standinator 8y ago

Stop cat abuse.

humantiy 8y ago

From reading the headline couldn't help but think to myself was it Neo?

trw999 8y ago

As someone running a Matrix homeserver I take this incident as an example of the benefits of decentralization. Unlike in more centralized services, the security lapses of Matrix.org have had no affect on my homeserver.

IronCoderXYZ 8y ago

Looks like all issues created by the "hacker" have been removed? https://github.com/matrix-org/matrix.org/issues?utf8=%E2%9C%93&q=author%3Amatrixnotorg+ https://github.com/matrix-org/matrix.org/issues?utf8=%E2%9C%...

gerogerke 8y ago
Seems like the user itself has been deleted, which might cause Github to remove all content created by that user.
- deleted 8y ago
  [deleted]
- Arathorn 8y ago
  we (Matrix.org) haven't deleted the issues; we were deliberately leaving them up for reference.

NotOscarWilde 8y ago

I lost all my messages with my girlfriend (of course). Can anyone clarify: if I use their "server key backup" and set a passphrase, I am now two passwords away from giving the next hacker read access to all my messages, is that right?