11 ms·
A distributed spam attack across the public Matrix network
- sdflhasjd 5y agoThis is the very immaturity I'm concerned about when people suggest moving our IRC groups to Matrix.
- arp242 5y agoHasn't spam traditionally been a huge issue on IRC? Perhaps it's a bit less now as IRC usage has fallen and/or better systems to stop it have been put in to place, but back in the day when I still used IRC I remember random bots/people spamming ASCII dicks or just offensive text like repeating "nigger" over and over again, and similar "hilarious" stuff. Actually, the only time a server under my management was hacked (CEO installed some random WordPress plugin with RCE, sigh) was to run an IRC spambot to send out exactly this sort of abuse. Hurray for firewalls also blocking outgoing traffic by the way: as far as we could tell this was all just dropped and never impacted anyone. Either way, this doesn't sound like a "Matrix problem".
- darthrupert 5y agoThere's a small difference in that IRC generally is a message passing network whereas Matrix more like a log synchronizer. So spam becomes a part of the log, which means that it'll consume far more resources than just network resources.
- GekkePrutser 5y agoI was on IRC in 93/94 (mainly IRCNet), a couple of years after it all started, and especially in those days it was a total shitfight. Technology around the platform moved at a rapid pace and there was this constant race going on between network admins and malware writers. At that time IRC clients supported a lot of scripting and much of that was used for abusive stuff. Like auto ban/kick, forcing netsplits in order to become admin by abusing protocol issues etc. The problem was bigger than spam alone. It was so bad it put me off IRC for almost a decade before I got back into it. Since then IRC has actually evolved into a more mature userbase but its inherent lack of protection to this and the ease of writing a bot makes a few idiots capable of spamming the entire IRC world. Nevertheless like I said in another post I think this is just a tradeoff we have to learn to live with. Being fully open is great, but comes with drawbacks and implied trust. I still prefer it as it is.
- gerikson 5y agoThis is what makes the recent Freenode takeover so sad - it was a more mature version of IRC that was hijacked by a gang of people with "old IRC" ideals.
- dec0dedab0de 5y agoI haven't used IRC in a long time, and never used Matrix, so I could be way off, please correct me if I am. I think the main difference between matrix and IRC in regards to spam is the federated nature of matrix. In IRC, each organization[1] only has to worry about it's own users. So they can add any kind of verification, bans, filters, etc to manage spam. With matrix, you have to rely on every other organization to police it's own users. Kind of like email, which has way worse of a spam problem than IRC. [1] I said organization, but what is the right word here? Group? Domain? Do IRC and Matrix have their own terms for these things? I wanted to say server at first, but surely there is more than one server.
- ThatGeoGuy 5y agoI think the terminology you're looking for is "homeserver." Homeservers are tied to a single hostname / domain, but may be a collective of servers running the different services that comprise the protocol itself. That said, you _do_ have to worry about spam from other homeservers, but that only matters if you have users from that other homeserver in your room. You can choose to federate / not-federate on a room-by-room basis, or just not federate at all ever. Needless to say, you're right that it is a bit different than IRC in that regard (IRC can just kline hosts / IP ranges from the whole service if you're spamming). However, in practice there's a lot less instances of random spam such as in the article than you would think. For myself, I never noticed any of these spammers at all (anectodal, so mileage may vary). I do have my own homeserver through Element Matrix Services, but I am part of a good number of rooms that exist purely on the Matrix.org server. Certainly I have to "police my users," but that's pretty easy since I run a small homeserver for ~5 users, and we just federate with the greater network when needed. I guess I agree with you in general that there's different failure modes, but I don't think it's as simple as dismissing matrix because it "has the same spam problem as email" compared to IRC. In practice at least, it doesn't seem to be as much of a problem despite how much larger than IRC Matrix has become.
- dec0dedab0de 5y agoThank you for the insight. To be clear I'm not dismissing Matrix, just trying to explain/understand it. I would say that federation is the main reason that email has been so successful for so long, even if it is the same thing that makes spam so hard to control.
- NullPrefix 5y agoIs the sibling comment dead because of the explicit mention of the n word? The points raised in that comment seem valid.
- ThatGeoGuy 5y agoBecause the same behaviour never exists on IRC? All things considered, I can't say that the Matrix.org team responded slower than I've seen any IRCops respond. The main difference is you don't tend to see news about this happening on IRC as much anymore because IRC has had spam problems for decades, and there's hardly as many people who are out and about making it news. Plus, for what its worth, most of us who run our own homeservers never encountered this issue at all, because public registration isn't open in the first place. It seems awfully dismissive to wave off Matrix for purely this one incident as it being "immature" as a protocol / platform.
- dijit 5y agoAs a person who maintains a moderately sized irc network (700 users when I last checked) one of the things that had put me off matrix a couple of years ago was the lack of powerful moderation tools. IRCs are absolutely archaic, for sure. Inspircd’s modules are awkward to use, single characters have special meaning… there’s g/k/u-lines and don’t get them mixed up! But, they’re very powerful and absurdly flexible. We’ve been able to mitigate absolutely huge spam attacks with quite complex logic, and you have strong moderation tools at every level- from the channels with quite rich permissions (voiced, half-ops, ops, admins, founders) and the network level (the aforementioned lines). Maybe matrix has caught up- but it’s hard for me to run a public network without such rich moderation tools. Even if those tools are clunky and awkward.
- seizethegdgap 5y agoModeration tools are slowly improving, but definitely still don't fit all use cases, and it's very unfortunate they don't come natively with synapse. I'm the lead admin of a Matrix ' Space', and our main room now has over 6,000 users (80%+ are inactive or barely active). We use mjolnir(https://github.com/matrix-org/mjolnir https://github.com/matrix-org/mjolnir) to ban and redact users/servers/messages across all of our rooms, which has been a godsend since I used to have to redact all the racist/gore myself message-by-message. It's still reactive instead of proactive, but I'm hoping these tools will mature in time.
- atatatat 5y agoNot sure why sibling comment downvoted: these are problems IRC and others haven't solved without being called "immature".
- undfg 5y agoWhat's pathetic here is that a bot spam wave can cause performance problems in the network.
- stryan 5y agoMost of the network was fine, it only affected the main Matrix server and anyone who federated with the spammed rooms. If you weren't in a room being attacked or left it you were fine.
- theamk 5y agoI wonder if Matrix will get its own "spamhaus" equivalent, and if this will be a single service or a multiple competing ones.
- 0xC0ncord 5y agoThere are public and community-curated banlists than any homeserver admin can subscribe to. Probably not exactly a "spamhaus" equivalent, but it's pretty close.
- RL_Quine 5y agoIt says to email abuse@matrix.org to get “unblocked”, what control do they have to do this?
- detaro 5y agothe control over matrix.org, the homeserver they administrate and they blocked other servers on.
- cube00 5y agoThat's for removing a server level block if it was imposed at the height.
- atatatat 5y agoA federated server, to be clear, that can federate with Matrix.org users as well as the tons of other servers users are joining/hosting.
- handrous 5y agoThe short answer is that, in practice, a huge proportion of Matrix activity is centralized, and they control it.
- prashantsengar 5y agoCan you share more details please?
- handrous 5y agoWhat details would you like? Server population stats? Is it controversial or obscure information that a very high proportion of all Matrix users are on the Matrix.org instance? (seriously, is it?) I thought that fell under the umbrella of common knowledge, so far as Matrix goes. If you've got data indicating otherwise, I'd find that interesting to see. I'm actually finding hard stats very difficult to locate (the few I can find seem incomplete, but corroborate this very strongly, for what they're worth), but again, I thought this was basically assumed to be true by Matrix users. I mean, if you search "matrix protocol" or "matrix chat" or "element chat" and follow the Happy Path of clicks to try it out, you'll have a Matrix.org account. The way you veer off that path? You have to know the address of another Matrix server. You won't be presented with other options, just the ability to type in the address of a different server. That this would result in Matrix.org having the lion's share of users seems a predictable outcome.
- grouphugs 5y agomatrix is so cool, unfortunately i don't have anyone to talk to lol
- beprogrammed 5y agoA prime example of attacks making the target stronger.
- qwertox 5y agoSure, but what if it were an attack which exposes a lot of private information? Just saying that not every kind of attack has a good outcome.
- dane-pgp 5y ago> what if it were an attack which exposes a lot of private information? Then it wouldn't be "a prime example of attacks making the target stronger", it would be a, err, sub-prime example?
- qwertox 5y agoThat's not the only way to interpret the original sentence. It could well have meant that attacks generally make the target stronger.
- GekkePrutser 5y agoYeah that was always going to happen of course.. When you make an open network, it becomes a lot harder to stop this kind of thing from happening. It's something inherent in its very nature. When you're more open you're also more open to abuse. I'm surprised it took this long (which also says a lot of good things about the maturity of the users!) I hope they don't have to tighten things down to the point of interfering with functionality because Matrix is excellent as it is. On the other hand, on IRC which I also use a lot it's just a fact of life.. We just apply some minor workarounds such as +r channels, roll our eyes and move on with life. I'd much rather have a free open network with some spam than a curated one by big tech that's free of spam. PS: Another thing I do foresee happening if Matrix really takes off is big tech datamining it. It's very hard to make a network open and yet hide as much metadata as you can, e.g. how many users are on which servers, who do they communicate with across the network etc. If some big tech entities control some of the servers it becomes really hard to keep that locked away. Of course Matrix's encryption will help but not for all metadata. Again I think it's something we'll have to mitigate rather than prevent outright.
- syzygyhack 5y agoThis is what Cwtch is after right? Doing away with metadata too. They posted here the other day.
- GekkePrutser 5y agoOh I missed that post, I will look for it, thanks! Good to know it's on the radar!
- nighthawk454 5y agoI think I found it here: https://news.ycombinator.com/item?id=27643171 https://news.ycombinator.com/item?id=27643171
- GekkePrutser 5y agoOh I thought this Cwtch was an extension to the matrix protocol to build barriers to metadata mining. One of the things I like about Matrix is its open nature and the many bridges. I would be less interested in yet something else. Especially because Matrix is pretty good at what it aims to do already.
- api 5y agoThe modern Internet is a war zone, like a failed state. Any protocol or system must be armored for battle. Anything that is at all “open” will be destroyed by spam and other forms of exploitation the instant it becomes popular enough to have any value whatsoever as a target.
- beprogrammed 5y agoI don't know about the failed state part, she's open and truely free, at the mercy of it users. Sure is a warzone though. Just humans learning how to behave, maybe one-day it'll all be peace and productive quiet, until then, and on our journey to there, a warzone.
- api 5y agoA better term would be a 'dark forest.' The Internet is a dark forest.
- sitzkrieg 5y agothe lack of admin uis makes stuff like this even more annoying. having to write/scrounge a bunch of scripts to manage users on years old software
- cvwright 5y agoThere is synapse-admin https://github.com/Awesome-Technologies/synapse-admin/ https://github.com/Awesome-Technologies/synapse-admin/ The matrix-docker-ansible-deploy playbook will install it for you very easily. All it takes is enabling one config option.
- jude- 5y agoHonestly, if you're going to clean-slate design an open-membership p2p network, the very first problem you should be tackling is what you're going to do to deal with asshats. Open-membership p2p systems sound nice in theory -- and even tend to work in small-scale settings! -- but once people start to actually use them, the asshats will show up and trash your system for kicks. Plan accordingly.
- an_opabinia 5y ago> the very first problem you should be tackling is what you're going to do to deal with asshats I agree. One solution is "have a real human identity you can't delegate with real stakes if you're banned." What we need is a blue checkmark for everyday people.
- jude- 5y agoKinda surprised no one's used a blockchain for this. The registration flow could be as follows: 1. User registers a non-transferable username (maybe an NFT?) 2. User signs into the client with the username NFT 3. The server admin maintains a list of shadowbanned username NFTs -- if yours gets added, then the server won't relay your messages. Non-transferability means no one will squat usernames (since there's no money to be made by reselling them), and since it costs money to register but costs nothing to shadowban, the asshats only lose money by being asshats (and their spam doesn't even get read).
- southerntofu 5y agoMost dweb startups you hear about are precisely marketing their "solutions" as such. You hear stuff like "but if there's a financial participation for registration you can't have spam so that's why you gotta use our tokens", which is really ridiculous considering the most nefarious actors usually have a lot of resources at hand.
- jude- 5y agoFundamentally, a struggle between asshats and honest users/admins is a struggle of attrition. The honest participants only win the struggle if they can make it too costly for asshats to continue participating. To achieve this, it would be sufficient to make it so the cost of registering a user account is asymptotically more expensive than the cost of shadowbanning an account. If the software required that each additional account a user registers costs more in USD terms than the previous account, while keeping the cost of shadowbanning a username constant, then the honest participants will win in a struggle against asshats who must keep registering accounts to circumvent the shadowban. A strawman approach would be to require all user accounts to be tied to real-world identities. Then, the software can track how many accounts a real-world person has registered, and bill them accordingly. This obviously has other unrelated problems, but it does make it possible to penalize asshats more and more harshly for each infraction. I was suggesting that a blockchain could be a useful building block here, since it already exists, is widely deployed, and costs money to write state. A more practical approach than the strawman could be to implement a username registration system on a blockchain, such that each registrant must burn some of the blockchain's tokens to register a username (thereby imposing a cost to doing so). Crucially, the number of tokens burnt per name would increase as more and more usernames get registered (or as more and more time passes), and in doing so make it more and more costly for asshats to circumvent a shadowban. This would also make it so asshats would lose a war of attrition against admins, and could be implemented today.
- bfrog 5y agoSpam on matrix tends to be racist gore images which is 1000% worse than irc text spam
- motohagiography 5y agoSeems like a high-effort specialized-skill general attack on a privacy and anonymity platform for its own sake. Cui bono? Ah, right...