40 ms·

Wow, we had used the same stack. Neo4j - gave up because of horizontal scaling issues, moved to TitanDB and then our startup folded. Fast forward 3 years and a

Wow, we had used the same stack. Neo4j - gave up because of horizontal scaling issues, moved to TitanDB and then our startup folded. Fast forward 3 years and am back in the graph world, this time dealing with money laundering and fraud rings. It stumps me as to why graphs are not used more in the financial world. Financial transactions are no different from social interactions. So concepts such as community detection etc apply to the financial realm as well and in fact we are using such concepts to determine fraud rings. I may get back to neo4j.

emileifrem 8y ago
Feel free to have a look again and let me know your thoughts. emil@neo4j.com or @emileifrem. 20 of the 25 biggest banks in the world use Neo4j today, and fraud detection (specifically fraud rings) is a really popular use case. (So much so that if you search for "fraud detection" in an incog chrome window, https://neo4j.com/use-cases/fraud-detection/ https://neo4j.com/use-cases/fraud-detection/ is on the first page of google). Btw, we've scaled out horizontally for reads since 2011 but our initial Paxos based implementation was shaky. Since then we've revised it twice and our third generation Raft based architecture is significantly better. If you add multi clustering support + causal consistency and a bunch of other improvements I think our horizontal scalability story is very different today from a few years ago. Lemme know if you try it out and disagree!
- hardwaresofton 8y ago
  Hey if you guys are still considering solutions maybe one of the following derivations of Paxos will help: EPaxos[0] - flexible/minimal quorums + if you're willing to know ops are commutative you can fast-path, and you can flexibly quorum WPaxos[1] - flexible/minimal quorums + dynamic partitioning via object stealing for spreading writes around The main benefit of these is of course getting over raft's weakness of the leader being a SPOF. If you're willing to go with causal consistency there's also CmRDTs (operation/command Replicated Data Types) which are generalizable to "pure operation based CRDTs"[2] which make the merging problem trivial and should in theory boil things down to replication lag while allowing writes at every node. [0]: https://www.cs.cmu.edu/~dga/papers/epaxos-sosp2013.pdf https://www.cs.cmu.edu/~dga/papers/epaxos-sosp2013.pdf [1]: https://cse.buffalo.edu/tech-reports/2017-03.pdf https://cse.buffalo.edu/tech-reports/2017-03.pdf [2]: https://arxiv.org/pdf/1710.04469.pdf https://arxiv.org/pdf/1710.04469.pdf
- deepGem 8y ago
  Thanks Emil. I will definitely get back to you on my findings.
gcoleman 8y ago
We're using neo4j for fraud detection in online gambling and it's very useful for spotting rings of duplicate/related account etc. One benefit I didn't think of before but has become clear: non-technical people find viewing and manipulating the graphs very easy, and explaining the links is much simpler.
JPKab 8y ago
Graph databases have been used for YEARS for fraud detection. There was a company called Metatomix out of Boston that did this back in 2008.... I think they got sold to Ab Initio. Funny to me how tech is constantly a cycle of folks having no idea about the previous incarnations of their ideas.

emileifrem 8y ago

Feel free to have a look again and let me know your thoughts. emil@neo4j.com or @emileifrem. 20 of the 25 biggest banks in the world use Neo4j today, and fraud detection (specifically fraud rings) is a really popular use case. (So much so that if you search for "fraud detection" in an incog chrome window, https://neo4j.com/use-cases/fraud-detection/ https://neo4j.com/use-cases/fraud-detection/ is on the first page of google). Btw, we've scaled out horizontally for reads since 2011 but our initial Paxos based implementation was shaky. Since then we've revised it twice and our third generation Raft based architecture is significantly better. If you add multi clustering support + causal consistency and a bunch of other improvements I think our horizontal scalability story is very different today from a few years ago. Lemme know if you try it out and disagree!

hardwaresofton 8y ago
Hey if you guys are still considering solutions maybe one of the following derivations of Paxos will help: EPaxos[0] - flexible/minimal quorums + if you're willing to know ops are commutative you can fast-path, and you can flexibly quorum WPaxos[1] - flexible/minimal quorums + dynamic partitioning via object stealing for spreading writes around The main benefit of these is of course getting over raft's weakness of the leader being a SPOF. If you're willing to go with causal consistency there's also CmRDTs (operation/command Replicated Data Types) which are generalizable to "pure operation based CRDTs"[2] which make the merging problem trivial and should in theory boil things down to replication lag while allowing writes at every node. [0]: https://www.cs.cmu.edu/~dga/papers/epaxos-sosp2013.pdf https://www.cs.cmu.edu/~dga/papers/epaxos-sosp2013.pdf [1]: https://cse.buffalo.edu/tech-reports/2017-03.pdf https://cse.buffalo.edu/tech-reports/2017-03.pdf [2]: https://arxiv.org/pdf/1710.04469.pdf https://arxiv.org/pdf/1710.04469.pdf
deepGem 8y ago
Thanks Emil. I will definitely get back to you on my findings.

gcoleman 8y ago

We're using neo4j for fraud detection in online gambling and it's very useful for spotting rings of duplicate/related account etc. One benefit I didn't think of before but has become clear: non-technical people find viewing and manipulating the graphs very easy, and explaining the links is much simpler.

JPKab 8y ago

Graph databases have been used for YEARS for fraud detection. There was a company called Metatomix out of Boston that did this back in 2008.... I think they got sold to Ab Initio. Funny to me how tech is constantly a cycle of folks having no idea about the previous incarnations of their ideas.