4 ms·

I'm going to offer an opinion not matching the groupthink here. I think this is a great project... many of us gather intelligence on various targets and threat

I'm going to offer an opinion not matching the groupthink here. I think this is a great project... many of us gather intelligence on various targets and threats by hand, because we're used to doing that, and it's the way we've always done it, and it works. A system of automation is more thorough and more complete as well as speeding up the process, allowing for more searches in less time. Sure, the software isn't perfect, but it's better than almost anything else in terms of being more of a product and less of a toolkit. It's open source, so if you think a data source like Facebook is missing, feel free to add it in. Well done, I say.

ArtWomb 7y ago
Yes, I still mine data by hand. Usually inserted directly into sqlite using datagrip. And if I use a script its bespoke, specifically crafted for a single site (or single page) ;) I applaud this effort as well. The web is a chaotic soup of unstructured data. And this is a first step towards agents that can surf the web for us. And return complex queries from natural human language!
- smicallef 7y ago
  Appreciate it, thanks!
  - tonyarkles 7y ago
    Yeah, this looks pretty awesome! I ran it against one of my own domains and there's a fair bit of false positive/flood of information, but the data is really interesting!
smicallef 7y ago
Thanks.. it’s been a ton of work over years, with valuable contributions from others as well, so I’m really glad to see it’s appreciated. If there’s however any pain point you experience with SpiderFoot, I’d be happy to hear it.
- ghostpepper 7y ago
  I am new to OSINT and this looks like a really cool way to get started. Maybe this exists and I missed it but it would nice to have a way not to run scans with a missing API key. Even better would be a way to easily link to instructions to obtain missing API keys, for example each module could have a metadata slot for 'API key generation url'... it would be a lot of work so I understand if it's not in the roadmap. Thanks for your hard work on this project
jamescampbell 7y ago
I've used and setup SpiderFoot for others. It works fine and usually does enough that it helps more than hurts.

ArtWomb 7y ago

Yes, I still mine data by hand. Usually inserted directly into sqlite using datagrip. And if I use a script its bespoke, specifically crafted for a single site (or single page) ;) I applaud this effort as well. The web is a chaotic soup of unstructured data. And this is a first step towards agents that can surf the web for us. And return complex queries from natural human language!

smicallef 7y ago
Appreciate it, thanks!
- tonyarkles 7y ago
  Yeah, this looks pretty awesome! I ran it against one of my own domains and there's a fair bit of false positive/flood of information, but the data is really interesting!

smicallef 7y ago

Thanks.. it’s been a ton of work over years, with valuable contributions from others as well, so I’m really glad to see it’s appreciated. If there’s however any pain point you experience with SpiderFoot, I’d be happy to hear it.

ghostpepper 7y ago
I am new to OSINT and this looks like a really cool way to get started. Maybe this exists and I missed it but it would nice to have a way not to run scans with a missing API key. Even better would be a way to easily link to instructions to obtain missing API keys, for example each module could have a metadata slot for 'API key generation url'... it would be a lot of work so I understand if it's not in the roadmap. Thanks for your hard work on this project

jamescampbell 7y ago

I've used and setup SpiderFoot for others. It works fine and usually does enough that it helps more than hurts.