4 ms·
Building a threat intelligence feed using the Twitter API and a bit of code
- jsonallen 4y agoIt could be interesting to further enrich the data using info from urlscan.io or something similar.
- Robbedoes 4y agoInteresting! Will have a look on what they can freely offer.
- rashkov 4y agoCould be cool to have a honeypot that automatically visits those links via puppeteer or some other browser automation.
- Robbedoes 4y agoPlan in the future is to visit the links, download malware and upload it to VirusTotal, Malwarebazaar, etc.
- heipei 4y agoCheck out urlscan.io which lets you scan URLs very conveniently and returns a report, including phishing and brand impersonation verdicts. Disclaimer: I’m the founder ;)
- Robbedoes 4y agoAm a fan of your work! Would it be useful if I throw these domains directly to URLScan? For retrieving the results I'll probably have to setup a proper DB and a workflow around parsing the Twitter content. Now, it's just a simple script.
- heipei 4y agoSure, I don't see why not, the worst thing that could happen is that the domain doesn't resolve or the webserver doesn't respond, but at least there will be a record of the domain having been scanned! Thanks for the kind words ;)
- snug 4y agoWith your jq piped to jq, you could just pipe it within jq curl https://twitter.threatintel.rocks/ https://twitter.threatintel.rocks/ --silent | jq -r '.malicious_urls | .[]'
- Robbedoes 4y agoThanks!
- newman314 4y agoIt appears that some URLs are cut off by Twitter. You might have to add some code to expand those URLs. For example: https://xunmaus.… https://xunmaus.xn--rvg In the meantime, I'm using a modified command like so: `curl https://twitter.threatintel.rocks/ https://twitter.threatintel.rocks/ --silent | jq -r '.malicious_urls | .[]' | sort -u | grep -v …`
- deleted 4y ago[deleted]
- Robbedoes 4y agoIt seems this is only the case if it's a tweet that contains a retweet. These are now removed from the stream as they're not of relevance.
- sodality2 4y agoI found several phishing crypto exchange sites that sent wallet recovery codes to a Telegram channel, currently having some fun spamming it :). Wonder if I should create a script that generates plausibly real wallet codes because it seems these scammers are getting these responses in a chat and are checking them by hand!
- hk1337 4y agoThis is interesting. I worked on a threat intelligence platform at HP that shared the same same as a previous threat intelligence platform that ingested twitter a lot but was found to be useless and just a lot of noise.
- alexott 4y agoMaybe for threat intel it wasn't very useful, but when I worked at McAfee 6-7 years ago, I've heavily used Twitter as a source of new sites for URL categorization (content-wise) - it was very useful to get trending stuff, etc., and proactively put classification into URL database
- _wldu 4y agoI'd be hesitant to consume this in a firewall or DNS RPZ. What if someone reported google.com as malicious?
- arboghast 4y agoHappens all the time and those threat feeds aggregation platform that have a black box ML confidence generator for IOCs aren’t trustworthy either. In this day and age, if people don’t put emphasis on TTPs and still only rely on old style threat intel, they will become obsolete.
- brightball 4y agoTTPs?
- milkshakes 4y agohttp://detect-respond.blogspot.com/2013/03/the-pyramid-of-pain.html http://detect-respond.blogspot.com/2013/03/the-pyramid-of-pa...
- CyberMacGyver 4y agoI built something similar 7 years ago. Once I started by crawling for hashtags, I learned soon enough that #backdoor was causing lots of FPs :) that I couldn’t verify at work
- metadat 4y agoWhat's an FP?
- mikequinlan 4y agoFalse Positive
- dontbenebby 4y agoAre use of the api and the website two separate things? I got banned for posting too hard or something, I'm not clear on the specifics, I'm probably just gonna stand up a domain or join mastadon rather than try to crowd into some musky space.