18 ms·
An update on our security incident
- dboreham 6y agoNo employee should have the power to subvert 130 high value accounts in a short time period.
- djsumdog 6y agoWhy do we even have 'high value' accounts on a centralized platform? Why isn't there a whitehouse.gov ActivityPub instance that no single admin can censor or subvert?
- vaxman 6y agoThis. Just as surely as email, texts and tweets are now admissible in contract law and as we can sign documents online that are binding, we need a national realtime information-streaming infrastructure --text and URLs today, multimedia in the future...all over the (hopefully) secure domestic 5G network. Set the standard, setup a court system for inevitable violators, regulate the gateways allow everyone to connect.
- m463 6y agoWe will do that now. We will start the competitive bidding process, and we expect the RFP paperwork to be returned by October, 2021. After that, if there are no injunctions filed because of the bidding process, preliminary design documents will start being created. Preliminary design review will occur August 2022. ...
- rudolph9 6y agoI can’t tell if this is trolling or a serious comment of how this will roll out?
- Shared404 6y agoIt's a comment on government inefficiency.
- m463 6y agoSeriously I worked on projects subject to government scrutiny and it was ok, but you had to be the right kind of person. Account for your time in 6-minute increments. Milestones I recall off the top of my head were preliminary design, detailed design, 3-5% of your time coding, software integration, hardware software integration, acceptance. It was stable, predictable, and (to me) very soul-crushing.
- lovecg 6y agoNone of that sounds “ok”... Technical question for you, how does this time tracking work in practice? Do you pause every 6 minutes and note what you’re doing? Or just roughly remember at the end of the hour/day?
- m463 6y agoYou had to charge each project you worked on and record your time to .1 hour precision. So recording 1.1 against project 3456 would charge that project for 1 hour and 6 minutes of your time. You had to do the same thing for a dentist appointment. 1.0 hours for an "overhead project". (I should mention this was years ago) Also, lots of the people who worked there were ex-government employees and were fine with it, because software folks got to go home to their family every day at a predictable time, you would get training at regular intervals and although the pay wasn't super competitive it was a good job, indoors and in an air-conditioned office.
- detaro 6y agoThat's what time tracking software is made for. No need to pause, "just" remember to switch the software if you change tasks/projects.
- SilasX 6y agoConsidering that the Trump account had some security measures that protected it from being compromised in this attack, that seems pessimistic.
- brown9-2 6y agoBecause the general public has no clue what ActivityPub is or the desire to learn how to consume feeds from dozen of instances when they could just download a single app onto their smartphone where all the celebrities already are.
- riffic 6y ago> general public has no clue what ActivityPub is The general public doesn't need to know anything about the underlying standards. Does a salesperson care about how SMTP works in order to send and receive emails from their customers?
- brown9-2 6y agoHow do you propose people consume 100s of different ActivityPub feeds without learning the details of how to configure an app to do so, while also staying away from a centralized platform to make it easy to consume the feeds?
- riffic 6y ago> Why isn't there a whitehouse.gov ActivityPub instance that no single admin can censor or subvert? Legislation is sorely needed for public institutions to make public announcement messages (microblog posts, or "tweets") using publicly managed and controlled infrastructure, contributing back to the commons[1]. Stop building into broken commercial services, the standards exist today to rebuild a commons-oriented Internet. [1] https://en.wikipedia.org/wiki/Commons#Digital_commons https://en.wikipedia.org/wiki/Commons#Digital_commons
- deleted 6y ago[deleted]
- deleted 6y ago[deleted]
- 0xUser 6y agoSource (with more details): https://blog.twitter.com/en_us/topics/company/2020/an-update-on-our-security-incident.html https://blog.twitter.com/en_us/topics/company/2020/an-update... > The social engineering that occurred on July 15, 2020, targeted a small number of employees through a phone spear phishing attack. A successful attack required the attackers to obtain access to both our internal network as well as specific employee credentials that granted them access to our internal support tools. Not all of the employees that were initially targeted had permissions to use account management tools, but the attackers used their credentials to access our internal systems and gain information about our processes. This knowledge then enabled them to target additional employees who did have access to our account support tools. Using the credentials of employees with access to these tools, the attackers targeted 130 Twitter accounts, ultimately Tweeting from 45, accessing the DM inbox of 36, and downloading the Twitter Data of 7.
- nodesocket 6y agoWhy are internal employee tools publically accessible? Minimum they should require VPN access, but really go further with Zero Trust.
- bawolff 6y agoThe blog post is vauge but definitely implies that a vpn was in place.
- whoisjuan 6y agoAFIK, in a Zero Trust Architecture a VPN is considered a perimeter and therefore it becomes a vector of attack to access systems of authoritative decision. Many security researchers have already established that the benefits of a VPN especially in the modern distributed world are marginal at best. Basically, yes a VPN makes you a tiny bit safer but it also adds a lot of networking complexity and adds more friction to the job of your employees. It also becomes an attack vector for malicious parties, since once they get VPN access they can theoretically access at least the first layer of protected resources. So in layman's terms an attacker just needs to phish for VPN credentials, maybe steal an OTP token and they will have access to a non-trivial amount of network protected resources. On the other hand if every service you use has its own authentication then the attacker needs to target each service and to know what services to attack they need knowledge that is possibly contained in another system that also requires authentication and is definitely not guaranteed for the attacker that all the systems will have the same password and/or have 2FA disabled. Honestly, in my opinion VPNs are just an excuse to monitor traffic. This is a bit of cynical take, but I'm convinced that companies that use VPNs are more interested in seeing what goes in and out their network than in protecting their resources.
- deleted 6y ago[deleted]
- MoZeu 6y agoIt is inexcusable that Twitter is employing people who are susceptible to social engineering attacks like this. This is simple training and seriousness.
- t-writescode 6y agoIt just takes one mistake to be spearfished.
- mikehollinger 6y agoI will freely admit that I fell for a phishing campaign. I’d just bought something on eBay (this was a while ago). I got an email about something in my account later that day that made it through my spam filters. I clicked on it, signed in, and then realized I’d done the deed. Nothing happened or was lost, but yes - it just takes one quick mistake.
- nevertoolate 6y agoI don’t get it. You know your ebay password?
- Thorrez 6y agoSome password databases involve copy and pasting or autotyping. If you want automatic hostname verification you need a password database integrated with your browser. On mobile many browsers don't support extensions so integrating my password database into the browser would be hard. In short, I do not know my ebay password, but I could have fallen for this phishing attack.
- carstenhag 6y agoOn mobile this is possible even without browser extensions - enpass, lastpass etc work just fine in Chrome or any other app, if it detects a password field.
- patagurbon 6y agoAre account support tools available off premises? I know nothing about security for big companies like Twitter but it seems like tools that enable you to post from any verified user (outside of Trump, someone here once mentioned he had additional account controls) should only be accessible from secure offices regardless of individual credentials.
- bawolff 6y agoThat would probably not work well during the pandemic...
- TwoBit 6y agoIt could with an appropriately secured private VPN.
- bawolff 6y agoOriginal poster said on premise. An appropriately secured vpn may or may not help security, but it is still "virtual" and does not meet the definition of on premise.
- Thorrez 6y agoThe VPN credentials could be phished. Of course appropriately-secured might mean U2F, in which case it likely would have prevented the attack. But they also could have used U2F without a VPN, and that also would have likely prevented the attack. So the VPN isn't really a benefit.
- iamleppert 6y agoThey should require hardware security devices (dongles). Really Twitter should be ashamed of their poor internal security.
- sushshshsh 6y agoDongles are rare here in the US. But I know that bloomberg uses them. I was shocked when I learned that retail banks in Singapore give everyone dongles to log in. In the US that's tyranny Lol
- Thorrez 6y agoRare for employees or rare for consumers? Companies can push much higher security onto employees than onto consumers.
- PopeDotNinja 6y agoHow do they deal with lost dongles?
- marcianx 6y agoEmployees can revoke access for any of their keys/dongles once they realize they are missing. The company can also send an automated warning and auto-expire a key that's not been used for a while.
- skybrian 6y agoSame as other computer equipment, get another one from tech support and revoke the old one. Or at least, that's how it worked when people still went to the office.
- TwoBit 6y agorare? I worked at Facebook and every employee needs one.
- marcan_42 6y ago
- ridaj 6y agoFreaking Twitter needs a serious auth infra upgrade. Unless phishers hijacked employee devices, they accessed the tools remotely, meaning there's no form of client authentication?? Something like U2F which by now is pretty old seems like it would prevent this kind of attack
- zobzu 6y agowhat if.. they used the authentication? But yes u2f/webauthn would probably prevent this. That said keep in mind they also do PR/damage control so we only know what they tell us. For all we know maybe they have u2f and an employee still did bad stuff while a phone was somehow involved. Or whatever else.
- reportgunner 6y agoYes they gained working credentials from an internal slack channel by reading them in plaintext. https://www.dailymail.co.uk/news/article-8536603/British-teenager-denies-Twitter-hack-admits-bought-stolen-account-Bitcoin.html https://www.dailymail.co.uk/news/article-8536603/British-tee...
- woadwarrior01 6y agoJack Dorsey's twitter account getting hacked via a SIM swap attack last year[1], should've been a wake up call for them. The recent hack just shows how lackadisical their attitude to security is. I used to work at FB some years ago and they had U2F for everyone, even back then. Also, regular phishing test drills and red-team exercises. [1]: https://www.wired.com/story/jack-dorsey-twitter-hacked/ https://www.wired.com/story/jack-dorsey-twitter-hacked/
- badrabbit 6y agoAs someone who works to stop these, the most frustrating part is how even infosec people thik enough training or $vendor's email security solution will stop this. It's like boy scouts that think they will stop navy seals. There is too much focus on entry point of an attack,especially by news media.
- iKevinShah 6y agoHey, Any good literature which you'd recommend to read to avoid something like this?
- sudhirj 6y agoI keep trying to read and understand the attacks that have all happened so far... after you understand the vectors that are currently in use, hopefully it'll stoke your imagination to see how you might attack other systems and make up new vectors. All the current attack write-ups will usually link to vulnerabilities as well. See HackerOne disclosures, Google Project Zero or the "How I hacked X" disclosures you see on HN. I don't think published textbooks are very useful — attackers also have access to them, and if the attack has been written down it'll likely be encoded into a firewall software or security process rulebook already (though it might still work for smaller companies lagging behind on the curve).
- badrabbit 6y agoThat's the thing, you can't,not with the way current tech is. But you can read up about having good monitoring/detection and hardening on your endpoints. Microsoft for example recommenda privileged access workstations. If twitter's employees used a separate set of credentials and workstations for privileged twitter moderation than their regular account/machine used for email and day to day stuff I bet the attack wolf have failed.
- Thorrez 6y agoThere are probably Twitter employees whose job it is to reset emails all day long. Having 2 separate computers and accounts, one for for resetting emails (which is done all day) and one for responding to email sounds like quite a burden on employees. How are they going to get the name of the account from one computer to the other? Copy and paste won't work. Retyping from one computer to the other surely will result in typos.
- snoshy 6y agoFrankly, I love Twitter. They've always seemed like the "scrappy" big player in the tech space that's played just the right side of the norm in many cases. An appropriately filtered experience on Twitter as a user is actually quite fun. That said, much of this is atrocious news. For all of their engineering prowess, seeing poor opsec failures combined with the lack of basic security principles like "containment of blast radius" and "fast response to critical failures" is not something you can easily forgive at this scale. But who am I kidding... it would be especially rich if some of these takeovers were enabled by simjacking-like attacks. Not that long ago, the only two-factor auth mechanism that worked for me was SMS.
- babaganooj 6y ago<<ultimately Tweeting from 45>> For a split second I thought they tweeted from Trump's account. But then 36 gave it away. Imagine if they did and decided to declare global thermonuclear war. POTUS should NOT have an account on Twitter. Speaking of POTUS, what about Deep Fakes? There should be an international agreement on how nations should handle such emerging threats to global stability.
- iKevinShah 6y agoSorry if I am taking this on a tanget, but in one of the HN threads regarding this exact security incident, it was recommended that this is why something called as "Blast Radius" needs to be implemented. Anyone here with any literature / sessions one could go through for a good gist of things with respect to Blast Radius?
- AnonC 6y agoI did a quick search on Algolia for HN (hn.algolia.com) and found several comments in this thread and threads related to the recent Cloudflare incident. This could be a starting point in your quest. [1] I don’t have a blog post or article or documentation though. [1]: https://hn.algolia.com/?dateRange=pastYear&page=0&prefix=true&query=Blast%20radius&sort=byDate&type=comment https://hn.algolia.com/?dateRange=pastYear&page=0&prefix=tru...
- ak217 6y ago"Blast radius" is a general term for the worst case impact of a specific type of breach of a given system. The recommendation you read was probably about limiting the blast radius. It's a general security best practice, and you implement it through techniques like federating (compartmentalizing) services away from each other, limited lifetime credentials, attribution, SSO for single point of control for invalidation of credentials, principle of least access (PoLA), privilege separation with role-based access control (RBAC), session logging/audit logging, etc. Most importantly the underlying system needs to have a well-defined and pentested authentication/authorization architecture. The hallmark of systems that limit the blast radius is that they have well-defined limits on how much they trust each other. OWASP (https://owasp.org/ https://owasp.org/) is a great starting point for reading about this stuff.
- ngcc_hk 6y agoWhat they need for you is some Network ai (dark something) to see change in user behaviour eg access accounts and honey pot accounts that any internal staff would raise alert immediately
- praveen9920 6y ago> We will be slower to respond to account support needs So, they limited the account support access to smaller trusted team. This I can understand but it might also cause the delay in identifying such attack, if it happens today.
- IncRnd 6y agoAccount access is one thing, yes, and a hardware 2FA key can help with that. But - what is the reason to allow support personnel to pose as specific users and send tweets from their accounts? There is more than a security issue here. There is a complete security breakdown.
- Thorrez 6y agoI don't believe there was a tool allowing support personnel to pose as users. I believe the tool allows support personnel to reset emails on accounts. Then the attackers used did password resets on the accounts then logged into the accounts and tweeted.
- IncRnd 6y agoI didn't see that in the page for this article. But, that's a good point. The specific text is, "Using the credentials of employees with access to these tools, the attackers targeted 130 Twitter accounts, ultimately Tweeting from 45, accessing the DM inbox of 36, and downloading the Twitter Data of 7." So, this doesn't say what actually happened. If it was employees posing as users in order to post, that is an permission which should not be granted. If it was as you suggest, a password reset, then there is a separate issue with 2fa that would be expected on these accounts. Either way, there are serious security issue. This is similar to Oracle calling itself "Unbreakable" and then getting broken. If Twitter cannot safeguard against so many accounts getting injected with tweets, then something is broken with Twitter's security model.
- Thorrez 6y ago>separate issue with 2fa that would be expected on these accounts. I believe the tool also allowed deleting 2FA from the account. > But because the attackers were able to change the email address tied to the @6 account and disable multi-factor authentication, https://krebsonsecurity.com/2020/07/whos-behind-wednesdays-epic-twitter-hack/ https://krebsonsecurity.com/2020/07/whos-behind-wednesdays-e...
- 6y ago
- heinrichhartman 6y agoWhat I find most problematic about the attac is the incident response by Twitter. As people pointed out here, hijacking Twitter accounts can lead to big stock market crashes, mass panics ("bomb found at XXX") and maybe even military escalations. Under this circumstances, leaving a platform with an unknown number of compromised accounts online, seems irresponsible to me. In such a case you must stop the bleeding ASAP, either by locking up "important" accounts (what they eventually did, after a few hours!) or taking the site offline. Next time this happens, we might not be so lucky.
- bzb3 6y agoNo. We need more things like these to happen so people lose trust on everything they read on the stupid internet. This was good. I hope it keeps happening.
- bananaface 6y agoIf they'd taken the site offline then the event probably would have generated more publicity.
- staticassertion 6y agoAt the same time, it's hard to take down a site that has the type of impact that you're referring to.
- thereyougo 6y ago> We’re acutely aware of our responsibilities to the people who use our service and to society more generally. We’re embarrassed, we’re disappointed, and more than anything, we’re sorry. We know that we must work to regain your trust, and we will support all efforts to bring the perpetrators to justice. We hope that our openness and transparency throughout this process, and the steps and work we will take to safeguard against other attacks in the future, will be the start of making this right. It's not easy to say those words about your own company. I really believe them.
- bananaface 6y agoI think it's a mistake to personify organisations like this.
- baby-yoda 6y agoAgree. That reads like pure PR-corporatespeak to me - oops sorry, wont happen again, business as usual. “They” only “care” to the extent it materially affects the business.
- bananaface 6y agoThe spirit of Mitnick never dies.
- jacquesm 6y agoWhy would it have died? He's very much alive.
- bananaface 6y agoHe's not hacking.
- toyg 6y ago> the attackers targeted 130 Twitter accounts, ultimately Tweeting from 45, accessing the DM inbox of 36, and downloading the Twitter Data of 7 So much effort for so little gain... With proper preparation (i.e. a simple app ready to download everything from an account), they could have made out with the whole data of 130 accounts, silently, before tweeting the hopeless scam message. Instead, this seems like a mostly-manual effort, done in haste. Just dumb thieves pulling off the scam of their life, or cover for a targeted attack of one or two of those 7 they actually siphoned properly? I hope US authorities will figure stuff out beyond the usual "it was a Chinese/Russian/Eastern European gang", which is just code for "fuck knows".
- ACS_Solver 6y agoI imagine that the limited damage the attackers did will help them evade being caught. Ultimately, they just stole some 100k$ in Bitcoin and that's it, so investigations by the FBI et al. will probably not go to great lengths to locate the hackers. On the other hand, attempts to blackmail someone like Tim Cook or Elon Musk would probably be riskier. For the same reason, I suspect the attackers may have intentionally avoided breaching or tweeting from Trump's account, too much risk of attracting serious FBI and Secret Service attention. As it is, the only people who actually lost something are a bunch of Bitcoin owners that got scammed, and it's just not a big deal.
- toyg 6y agoIn the great scheme of things, knowing what Trump (or any other US politician) says to other people is worth relatively little. Having insider knowledge of what certain billionaires say to whom, though, can be immensely valuable. It might well be that a few of them will lit a fire under FBI and friends. We'll see.
- ACS_Solver 6y agoMost of those famous people don't manage their own accounts though. Some do, like Musk. That's why I'd consider stealing his DMs to be more dangerous than running a Bitcoin scam - it could easily upset Musk enough that he bugs the FBI to investigate seriously or, more likely, uses his own considerable resources to run an investigation. See how Bezos hired investigators when his data was leaked, and they did get to the bottom of it. Trump is a bit of a special case because he's POTUS. I'm pretty sure there are no remotely interesting DMs on his account, but if there's one thing the US doesn't like, it's anyone even perceived of messing with national security. An attacker merely logging into Trump's account could be seen as a national security issue, and attract an entirely different level of law enforcement attention.
- gcpwnd 6y agoSlightly OT: I use private tabs and write my banks URL by hand and log in via 2FA. Are there cases or have there been attempts to poison the users address bar history with malicious URLs for fishing?
- info781 6y agoYou should need a PIV card to log into in the admin tools.
- deleted 6y ago[deleted]
- exabrial 6y agoThis is one of the better rcas, but I'd wish they'd punish the details of the fishing attacks so the rest of us could learn.
- ben509 6y ago> We have zero tolerance for misuse of credentials or tools, actively monitor for misuse, regularly audit permissions, and take immediate action if anyone accesses account information without a valid business reason. Okay, so who has been fired? That's what "zero tolerance" means: no excuses, not even "someone tricked me." And no punishment but the maximum. Anything less would involve some degree of tolerance, and when you say "zero" that means no tolerance whatsoever. It's obviously stupid to manage any organization that way, of course. It's a fatuous, dishonest phrase. So stop talking about "zero tolerance" since all it means is "we make hyperbolic claims that we have no intention of living up to."
- Shivetya 6y agoWell besides claims of restricting access to the tools they obviously need to further have restricted access to certain user accounts. We have support persons whose account support is locked from superior accounts or special users and this seems like something Twitter should be doing. Perhaps have another layer where each use of specific functions require unlocking through an incident management tool?
- ytdytvhxgydvhh 6y agoIt’s like when motorcyclists say “safety first” about wearing a helmet and other protective gear. If they really put safety first, they’d choose a safer form of transportation. They mean “given that I’m going to engage in this risky activity, I’m going to try to make this activity as safe as possible”. In this case “zero tolerance” is short for something like, “except for understandable slip-ups that aren’t fully your fault, we’re not going to tolerate any slip-ups”.
- NotSammyHagar 6y agoI think you can honestly say you follow safety first in terms of what is available to safely ride your bike. Just like when I used to rock climb, I felt like we were basically following safe practices - but there was no one to adjudicate them, probably far less testing of various practices with stats than bikes. Also, where we climbed there wasn't expected rockfall. I had barely heard of that being an issue, and we never wore helmets. Later on I realized that was something I might have missed out on. And then the next step was "what other safety practices was I unaware of" ;-) And of course I get that rock climbing is much more dangerous than hiking.
- dirtylowprofile 6y agoThey were hacked by kids on Discord.
- vinniejames 6y agoWhats the final determination around the rumors employees were payed for access to their creds?
- deleted 6y ago[deleted]
- gregors 6y agoSocial engineering will never be stopped, people want to be helpful. And generally speaking the cost for stopping it at non-secure businesses is going to be too high until a security incident happens. Phishing email attacks? Why do employees have business emails at all? Phishing phone attacks? Why would employees have phones with external access? Front of the house (dealing with users) should probably be disconnected from back the of the house (admin access). Before you know it you're in DoD or Bank territory. No Wifi allowed etc, where's your badge buddy?! Things get complex quickly with security.
- tialaramex 6y ago> Social engineering will never be stopped, people want to be helpful. They really do. And so you should design security systems with the assumption that your employees will actively undermine security "to be helpful" to adversaries. > And generally speaking the cost for stopping it at non-secure businesses is going to be too high until a security incident happens. The cost for Yubico's "Security Key" is $20 and there is a volume discount. You should buy each employee a key, and if there's no secure means by which they can be re-authorised when they inevitably lose it, a second one to keep safely for that case. The attackers correctly anticipated that while "Can you get me Jenny in user assistance's phone number?" is just being helpful, "Can you disable Elon Musk's 2FA and give me control over his account?" is a bit... obvious. So they got themselves credentials to do that stuff. But there is no need for Twitter employees to be able to give away those credentials.