4 ms·

Trinity: Microsoft Research’s Hypergraph Database

31 points by DanielRibeiro 16y ago

wglb 16y ago
See also http://www.semantic-web.at/index.php?id=1&subid=57&action=resource&item=3290 http://www.semantic-web.at/index.php?id=1&subid=57&a... which is not reference and seems to predate other efforts noted herein.
latch 16y ago
MS focusing on a GraphDB makes a lot of sense to me. Unlike document database or key=>value stores, GraphDBs aren't as mainstream yet (amongst the early adopters). Also, the typical GraphDB use case wouldn't compete with SQL Server. I think it's a problem space that we've only see the tip of the iceberg and Microsoft recognizes this. I also think this is one area where the MS stack doesn't scale well, hence why they'll address it. Sadly, I wouldn't be surprised to see this research get packaged into SQL Server (or worse, Azure).
- shadowfox 16y ago
  > Sadly, I wouldn't be surprised to see this research get packaged into SQL Server (or worse, Azure). Is there a reason for this? I would have thought that finding a practical application for research is actually a good thing?
  - latch 16y ago
    The last time Microsoft did work in the NoSQL space was with Velocity - something akin to memcached. There was a lot of talk, a lot of hype, some CTPs..and then it got bundled into Azure where its potential became a shadow of what it seemed like it could be.
    - MichaelGG 16y ago
      Are you sure about this? Velocity got rolled into "AppFabric" and then Azure did something with the same name. But it's totally different. What you want is Windows Server AppFabric: http://msdn.microsoft.com/en-us/windowsserver/ee695849 http://msdn.microsoft.com/en-us/windowsserver/ee695849 Not to be confused with Windows Azure AppFabric. I'm not sure Velocity really included any interesting research or novel ideas, though.
- snprbob86 16y ago
  Microsoft Research != Microsoft MSR does a lot of very wild, interesting, off the wall, crazy things. They also do a lot of incredibly practical, pragmatic, evolutionary things. It's a large organization, filled with researchers of an incredible range of skills and interests. Any projects coming out of MSR should not be perceived as commitment or desired focus from Microsoft. Some projects simply inspire project teams, others grow into products, but most just die on the drawing board. That said, they are a gravely under appreciated research body, especially by both the open source community and the Microsoft rank and file.
sambeau 16y ago
You can make a nice, simple (and surprisingly fast) graph database from the file system. I had to do this back in 2003 to create dynamic sites for users of a build-your-own-website app (who needed to have their data stored in their local filesystem - and they wouldn't let me use sqlite). It turns out to be really good for expressing websites structures. The downside is that, as you can't create on-the-fly relationships, you need to either store your data in a tree that describes its use, e.g.: blogs/2009/03/05 or you need to add links that point to the same data to create the relationships, e.g.: blogs/2009/03/05 -> stories/a_blog_post Obviously you can add these relationships later in much the same way as you might add indexes to a traditional database. My system also had a notion of arrays. An index to the next auto_increment was held in a file named '.#'. I also has a file called '.class' which would tell my object system how to process a node like this as if it was an object (each class had a a set of scripts that acted as methods). It all worked amazingly well. There were a couple of issues: old Solaris, I recall, used a linear search of directories so the system would crawl if a node got too big (I overcame that first with some simple a/b/c style hashing and second by upgrading Solaris) and directories were 4k each so users (who had file-usage limits) might have had grounds for complaint.
Ruudjah 16y ago
I was thinking... Will every datastructure eventually gets it's own database to facilitate distributed persistence of mass collections? Why, or why not?
- joshu 16y ago
  Seems inevitable.
- joshu 16y ago
  Seems inevitable.
deleted 16y ago
[deleted]
_ques 16y ago
Previously on HN: http://news.ycombinator.com/item?id=2362033 http://news.ycombinator.com/item?id=2362033
joshu 16y ago
I find companies talking about their internal products with no release pretty much the antithesis of hackeryness.
- pedalpete 16y ago
  I completely disagree. They are showing us what is coming and getting our minds going re: what possibilities this opens up for us. Now, I admit, Microsoft is not the first company to create a graph centric db, but them weighing into this market gives the market further credibility. At the moment, I think graph is seen as an edge case. Maybe people will take another look now and start getting more serious about it. Is it sad that it takes a big company jumping on the bandwagon to give it credibility?? Sure, a bit, but that is how the adoption curve goes.
  - joshu 16y ago
    They are talking about stuff they don't plan to allow other people to use. Why is this even interesting? We also had this argument last time this was posted, albeit not surrounded by infoq blogspam.
cabalamat 16y ago
AIUI, graph databases have much less impedance mismatch to objects than SQL databases do. So why are most people still using SQL? Tradition, or efficiency?
- lucifer 16y ago
  Not everything is an object. Also you should try coding against both object and graph dbs from an (OO) domain model. Object identity is a fundamental problem that has nothing to do with the fact that a RDBMS is your persistent back-end. Also, consider the notion of 'dirty' objects as just one of the problems. Finally, you still have to manage (to some extent; per your db ymmv) relationship semantics. Object and graph dbs shine where you wish to traverse your object graph. That is what is painful (and entirely alien) to RDBMS notions. So if your application domain is naturally mapped to traversal semantics, then it is simpler (but note that you certainly can write complex joins that would support the same) and more importantly, the db engine is tuned to be highly efficient for such use cases.
fedd 16y ago
Microsoft is making Trinity after Neo database (http://neo4j.org http://neo4j.org) was made. they even have the name derived!
sblom 16y ago
Seems to me that if a node can have multiple edges pointing to it, and an edge can point to multiple nodes, nodes and edges are effectively the same thing, or at the very least have identical properties. Am I missing something? I suppose it's a useful convention to still partition a hypergraph into nodes and edges.

31 points by DanielRibeiro 16y ago

wglb 16y ago

See also http://www.semantic-web.at/index.php?id=1&subid=57&action=resource&item=3290 http://www.semantic-web.at/index.php?id=1&subid=57&a... which is not reference and seems to predate other efforts noted herein.

latch 16y ago

MS focusing on a GraphDB makes a lot of sense to me. Unlike document database or key=>value stores, GraphDBs aren't as mainstream yet (amongst the early adopters). Also, the typical GraphDB use case wouldn't compete with SQL Server. I think it's a problem space that we've only see the tip of the iceberg and Microsoft recognizes this. I also think this is one area where the MS stack doesn't scale well, hence why they'll address it. Sadly, I wouldn't be surprised to see this research get packaged into SQL Server (or worse, Azure).

shadowfox 16y ago
> Sadly, I wouldn't be surprised to see this research get packaged into SQL Server (or worse, Azure). Is there a reason for this? I would have thought that finding a practical application for research is actually a good thing?
- latch 16y ago
  The last time Microsoft did work in the NoSQL space was with Velocity - something akin to memcached. There was a lot of talk, a lot of hype, some CTPs..and then it got bundled into Azure where its potential became a shadow of what it seemed like it could be.
  - MichaelGG 16y ago
    Are you sure about this? Velocity got rolled into "AppFabric" and then Azure did something with the same name. But it's totally different. What you want is Windows Server AppFabric: http://msdn.microsoft.com/en-us/windowsserver/ee695849 http://msdn.microsoft.com/en-us/windowsserver/ee695849 Not to be confused with Windows Azure AppFabric. I'm not sure Velocity really included any interesting research or novel ideas, though.
snprbob86 16y ago
Microsoft Research != Microsoft MSR does a lot of very wild, interesting, off the wall, crazy things. They also do a lot of incredibly practical, pragmatic, evolutionary things. It's a large organization, filled with researchers of an incredible range of skills and interests. Any projects coming out of MSR should not be perceived as commitment or desired focus from Microsoft. Some projects simply inspire project teams, others grow into products, but most just die on the drawing board. That said, they are a gravely under appreciated research body, especially by both the open source community and the Microsoft rank and file.

sambeau 16y ago

You can make a nice, simple (and surprisingly fast) graph database from the file system. I had to do this back in 2003 to create dynamic sites for users of a build-your-own-website app (who needed to have their data stored in their local filesystem - and they wouldn't let me use sqlite). It turns out to be really good for expressing websites structures. The downside is that, as you can't create on-the-fly relationships, you need to either store your data in a tree that describes its use, e.g.: blogs/2009/03/05 or you need to add links that point to the same data to create the relationships, e.g.: blogs/2009/03/05 -> stories/a_blog_post Obviously you can add these relationships later in much the same way as you might add indexes to a traditional database. My system also had a notion of arrays. An index to the next auto_increment was held in a file named '.#'. I also has a file called '.class' which would tell my object system how to process a node like this as if it was an object (each class had a a set of scripts that acted as methods). It all worked amazingly well. There were a couple of issues: old Solaris, I recall, used a linear search of directories so the system would crawl if a node got too big (I overcame that first with some simple a/b/c style hashing and second by upgrading Solaris) and directories were 4k each so users (who had file-usage limits) might have had grounds for complaint.

Ruudjah 16y ago

I was thinking... Will every datastructure eventually gets it's own database to facilitate distributed persistence of mass collections? Why, or why not?

joshu 16y ago
Seems inevitable.
joshu 16y ago
Seems inevitable.

deleted 16y ago

[deleted]

_ques 16y ago

Previously on HN: http://news.ycombinator.com/item?id=2362033 http://news.ycombinator.com/item?id=2362033

joshu 16y ago

I find companies talking about their internal products with no release pretty much the antithesis of hackeryness.

pedalpete 16y ago
I completely disagree. They are showing us what is coming and getting our minds going re: what possibilities this opens up for us. Now, I admit, Microsoft is not the first company to create a graph centric db, but them weighing into this market gives the market further credibility. At the moment, I think graph is seen as an edge case. Maybe people will take another look now and start getting more serious about it. Is it sad that it takes a big company jumping on the bandwagon to give it credibility?? Sure, a bit, but that is how the adoption curve goes.
- joshu 16y ago
  They are talking about stuff they don't plan to allow other people to use. Why is this even interesting? We also had this argument last time this was posted, albeit not surrounded by infoq blogspam.

cabalamat 16y ago

AIUI, graph databases have much less impedance mismatch to objects than SQL databases do. So why are most people still using SQL? Tradition, or efficiency?

lucifer 16y ago
Not everything is an object. Also you should try coding against both object and graph dbs from an (OO) domain model. Object identity is a fundamental problem that has nothing to do with the fact that a RDBMS is your persistent back-end. Also, consider the notion of 'dirty' objects as just one of the problems. Finally, you still have to manage (to some extent; per your db ymmv) relationship semantics. Object and graph dbs shine where you wish to traverse your object graph. That is what is painful (and entirely alien) to RDBMS notions. So if your application domain is naturally mapped to traversal semantics, then it is simpler (but note that you certainly can write complex joins that would support the same) and more importantly, the db engine is tuned to be highly efficient for such use cases.

fedd 16y ago

Microsoft is making Trinity after Neo database (http://neo4j.org http://neo4j.org) was made. they even have the name derived!

sblom 16y ago

Seems to me that if a node can have multiple edges pointing to it, and an edge can point to multiple nodes, nodes and edges are effectively the same thing, or at the very least have identical properties. Am I missing something? I suppose it's a useful convention to still partition a hypergraph into nodes and edges.