16 ms·
Microsoft open-sources Graph Engine
- wyldfire 10y agoI find that very abstract software packages like this are difficult to visualize without an example. This page does not offer one, but TFM does -- see [1]. Also note that TFM describes the fact that this is Windows-only. That makes it substantially less interesting, IMO. [1] https://www.graphengine.io/docs/manual/index.html#what-is-ge https://www.graphengine.io/docs/manual/index.html#what-is-ge
- v-yadli 10y agoDev here. Cross-platform for Linux/BSD is on the way, and it's already in the repo now.
- webmaven 10y agoDoes that include developing/compiling GE (and GE apps) on Linux/BSD?
- halflings 10y agoThat's great! You should definitely add that to the docs. I just went to the same page and was extremely disappointed when I saw that it required Windows. A benchmark comparing it with similar solutions (esp. Neo4j) would be much welcome.
- v-yadli 10y agoSure we'll definitely update the docs! Or better to work out one way to put the docs on GitHub and keep it synchronized with the site (they're markdown anyway)
- sah2ed 10y agoWhile you are at it, can you guys also do a feature comparison between Graph Engine vs Neo4J vs JanusGraph (vs Apache Giraph)? Thanks
- erichocean 10y agoHow well would this run traditional graph algorithms like PageRank?
- v-yadli 10y agoCheck this out: https://www.graphengine.io/downloads/papers/Trinity.pdf https://www.graphengine.io/downloads/papers/Trinity.pdf
- nl 10y agoDoes this relate somehow to the Trinity graph store[1] which Microsoft have previously published papers about? [1] http://dl.acm.org/citation.cfm?id=2467799 http://dl.acm.org/citation.cfm?id=2467799
- v-yadli 10y agoThis is Trinity.
- dfsegoat 10y agoAwesome. Thanks for being on-top of putting this info out there: I was about to gripe about no mention of linux in the reqs/docs.
- infocollector 10y agoDoes this have a python interface?
- dogma1138 10y agoC# Only IIRC, but you can probably use Python for .NET and since it's open source just write your own interface ;)
- onion2k 10y agoIt provides a RESTful API for getting the data out.
- simonw 10y agoDo you have a link to the docs on that? I didn't spot it.
- onion2k 10y agohttps://www.graphengine.io/docs/manual/index.html#query-serving https://www.graphengine.io/docs/manual/index.html#query-serv... EDIT: Slightly more useful link - https://www.graphengine.io/docs/manual/TSL/MPProtocol.html#http-protocols https://www.graphengine.io/docs/manual/TSL/MPProtocol.html#h...
- sullivant 10y agoI am wondering how it will integrate with R.
- adamnemecek 10y agoI've been running into the idea of computational graphs a lot recently. It's at the core of Tensorflow (and NN in general) but it also comes up for example in Apple's AVFoundation where all audio processing happens in a graph of audio units. Does anyone know what's the theoretical foundation of computational graphs? EDIT: I've created a wiki page for computational graphs. https://en.wikipedia.org/wiki/Computational_Graph https://en.wikipedia.org/wiki/Computational_Graph. Add your input.
- mtrn 10y agoThere is the biological analogue, which has inspired neural networks: > Recall that in our general definition a feed-forward neural network is a computational graph whose nodes are computing units and whose directed edges transmit numerical information from node to node. > Each computing unit is capable of evaluating a single primitive function of its input. In fact the network represents a chain of function compositions which transfor m an input to an output vector (called a pattern). From: https://page.mi.fu-berlin.de/rojas/neural/chapter/K7.pdf https://page.mi.fu-berlin.de/rojas/neural/chapter/K7.pdf (1996) Another ancestor would be the Data-Flow paradigm: > .. programming paradigm that internally represents applications as a directed graph, similarly to a dataflow diagram. Applications are represented as a set of nodes (also called blocks) with input and/or output ports in them. These nodes can either be sources, sinks or processing blocks to the information flowing in the system. Nodes are connected by directed edges that define the flow of information between them. From: https://paginas.fe.up.pt/~prodei/dsie12/papers/paper_17.pdf https://paginas.fe.up.pt/~prodei/dsie12/papers/paper_17.pdf (2012)
- adamnemecek 10y ago> There is the biological analogue, which has inspired neural network I'm somewhat aware but it seems like the idea of a computational graph is the most generic computational idea I can think of and I'm surprised it's not more explored. > Another ancestor would be the Data-Flow paradigm: Oh yeah, data flow is definitely another one.
- 10y ago
- bluejekyll 10y agoAt this rate, are we going to see Windows open-sourced? MS is on a roll. My bias since 1996 is being eroded with each OSS release they have, and multi-platform targeted support. I started using VSCode regularly as my main Rust IDE, and I feel dirty for liking it. It's seemless across macOS and Linux.
- dogma1138 10y agoI don't see MSFT benefiting from open sourcing windows, so likely never.
- bluejekyll 10y agoYeah. I meant that as a little tounge in cheek, I'm pretty sure they never will either.
- dman 10y agoI see them benefiting immensely. A big problem that Microsoft has is that once you start going down rabbit holes on Windows the number of people out there who have a clue is vanishingly small. So for systems programming on Windows there is very little coverage online. Opening Windows up will broaden the number of people who understand the platform on a deep level.
- throwawayish 10y agoEarlier in my life I did a lot of Windows system-level programming, including some minor driver development, and now I'm practically exclusively on the #nix bunch, sometimes doing similar stuff. I can't say that there is a disparity. If anything, the parts that are documented are usually better documented in the Windows world, and there are very good books about most components. (To be fair I never read a book on anything unix apart from Lion's). There is a large amount of stuff online for either, but I often have the feeling that correct examples are less frequently found for, esp., Linux.
- simias 10y ago
- mlmlmasd 10y agoSeems to be just a marketing page with no link to source code or even mention of open source? Here is the github page: https://github.com/Microsoft/GraphEngine https://github.com/Microsoft/GraphEngine
- ygra 10y agoThe repository is linked on the Download page.
- deleted 10y ago[deleted]
- floopidydoopidy 10y agoAlthough astroturfing doesn't seem to be a problem on HN, I'd really appreciate it if you didn't do it.
- whyileft 10y agoThis shouldn't be downvoted. There is definitely at the minimum Microsoft employee vote brigading at this point. HN needs to start making upvotes, downvotes, and time of vote public.
- dang 10y agoAccusations of astroturfing or shillage without evidence are a serious breach of civility on HN. Since you've done it before and ignored our request to stop, we've banned your account. All: both astroturfing and groundless insinuation thereof are against HN's rules. For more, see https://news.ycombinator.com/item?id=13610097 https://news.ycombinator.com/item?id=13610097 upthread.
- whyileft 10y agoCheck the time of votes for users that vote on Microsoft opensource vs normal time of day of votes for those users. You'll notice a bit of difference. But yeah, I really need to get off this site anyway. Its just sad at this point.
- dang 10y agoSince you've done this repeatedly and ignored our request to stop, we've banned your account. Actual astroturfing, when it occurs, is an abuse of HN that we crack down hard on. Defending this community against gaming and abuse is a huge priority for us. Any user who thinks they might be seeing it happen on HN should email us right away (hn@ycombinator.com) so we can investigate. Imaginary astroturfing—the bug that causes some users to be certain that those who disagree with them can only be nefarious shills because otherwise the pure reason of their own point of view would be fully accepted—is also an abuse of HN. This one is orders of magnitude more common, and it is poison. It eats away the heart of civil, substantive discourse, the assumption of good faith on the part of others. Therefore we ban astroturfers, and we also ban users who accuse others of astroturfing or shilling without evidence. An opposing view does not count as evidence, and playing this card as a rhetorical device in an argument breaks the HN guidelines. We detached this subthread from https://news.ycombinator.com/item?id=13607640 https://news.ycombinator.com/item?id=13607640 and marked it off-topic.
- cobookman 10y agoAm I the only one getting SSL cert authority invalid error
- debatem1 10y agoNope, I am as well.
- EGreg 10y agoBut I'm pretty sure you two are the only ones.
- etelej 10y agoNope. Same here. However, it's just Chrome on Android that doesn't recognize the CA, even the native android browser is okay. Bizarre, may be coz they have an incomplete cert chain https://www.ssllabs.com/ssltest/analyze.html?d=www.graphengine.io https://www.ssllabs.com/ssltest/analyze.html?d=www.graphengi... and chrome on android has had issues with that
- EGreg 10y agoI was being sarcastic :)
- mrmondo 10y agoI too had this, forcing the site to load also shows a number of page errors too. *Edit: checking the site on my phone now and it seems to load, pretty poorly designed site though IMO, unbalanced layout but at least it loads now.
- tdhz77 10y agoMe too. Corporate Proxy sometimes acts up though.
- jovdg 10y agoSame here
- zump 10y agoHow is this different to D-Graph?
- tempVariable 10y agoI see that the source code "os.h" file has directives to handle Linux and Apple. Has anyone managed to build it and use it in the even most trivial way?
- v-yadli 10y agoAt the current state you could link Trinity.C with a custom Mono host and then execute your GE assemblies/Trinity.Core. Check out CMakeFile.txt for details. :) We're working to interface Trinity with CoreCLR. .NET standard 2.0 would make this much easier.
- crudbug 10y agoInteresting - Linux Foundation with IBM, Google and others announced last week JanusGraph [0]. Janus provides optional persistent storage option also. What are the good use cases for these ? [0] http://janusgraph.org/ http://janusgraph.org/
- nitrogen 10y agoOld-school AI maybe, fraud detection, social network analysis, recommendation engines, deanonymization, surveillance and tracking...
- crudbug 10y agoI am using a time-series db where each metric measurement requires some static context information to be included, e.g. Event 1 happened in Device A at Node B, Location C. These static entities (A, B, C) are perfect for graph vertices. But, when I am persisting the graph on disk - I store all the data statically, No links just columns - the static entities become dimensional data. Is there a better way to this ? Ideally, we should persist the Object Graph directly without any data conversion. I see a difference in - How we compute and How we store data.
- rodionos 10y agoNot specific to GraphEngine, but you could store all meta-data about the device separate from the time-series itself. The composite metric name (key?) would then include the measurement name and the entity name (device name in this case).
- crudbug 10y agoThat is one possible solution. But, for queries we will need some kind of JOIN at runtime. I do store the entity data separately in a transactional row store. For analytical queries, we use a column store. For this, I have to include the entity data (context) for each fired event. The whole Compute & Storage semantics are different. If only, we had a persistent data-aware compute & storage graph engine that supported transactional & analytical workloads.
- purple-dragon 10y agoI'm a little confused. Is this competitive to something like Spark?
- ronack 10y agoDoes Microsoft use this for anything in production?
- reubenbond 10y agoFrom the Readme, it seems they do: > LIKQ is powering Academic Graph Search API, which is part of Microsoft Cognitive Services.
- vegabook 10y agoIs this in the same space as Storm, Flink, or Spark? Does it do streaming data such as Flink or Storm? Or is it batch-optimized? What languages does the compute engine support?
- brilliantcode 10y agoMicrosoft of 2017 makes me forget about Microsoft of 1997. It's insane how a shakeup of CEO and new cultural shift can seemingly add another major boost to it's brand. I welcome open source, eventually, I believe that it will eat commercial software, if the right economic incentives are in place. Microsoft may be signaling this to the market.
- webmaven 10y ago> Microsoft of 2017 makes me forget about Microsoft of 1997. Never Forget.
- PleaseHelpMe 10y agoTrue! Microsoft still can't gain my respect. Never forget.
- scott_karana 10y agoIf I may paraphrase: "Never forget that large monopolies can wield disgusting influence, but don't let hate for past monopolies blind you to modern monopolies either" :-)
- webmaven 10y ago> don't let hate for past monopolies blind you to modern monopolies either Just because Microsoft is no longer the monopoly-du-jour, doesn't mean they aren't still a monopoly... Just because they have been making (successful) forays into open-source, does not mean they won't still fiercely protect (with whatever means at their disposal) their monopoly cash cows (Windows and Office). The "standartization" of Office Open XML wasn't all that long ago, after all (ECMA in 2006, ISO in 2008).
- greggyb 10y agoWindows is a legitimate cash cow, but because of server installs. Office makes a lot of money; SQL Server and other datacenter products make much more.
- VeejayRampay 10y agoFunny how Microsoft has been rocking hard with their OS releases lately (and good on them for doing so), but there's still a pervasive feeling in the dev community about their true eventual motives and strategy. "Fool me once..." I guess, or as we say in France "Cold water scares the scalded cat".
- sidcool 10y agoFor a noob like me, what are the applications of this framework? What are the use cases? What current tools do this?
- bitwize 10y agoThis is not a cuddly new Microsoft. First comes the embrace (look at all this stuff on our github!), then comes the extend (run your Linux stack on Windows and never have to give up Visual Studio!), I'm sure you know what comes next. Hint: PC manufacturers no longer have to give you the option to disable Secure Boot.
- whodknee 10y ago>PC manufacturers no longer have to give you the option to disable Secure Boot. Then don't buy from those people. Simple
- deleted 10y ago[deleted]
- anon987 10y agoHow the hell does this story have so many upvotes? HN really needs to do something about Microsoft's vote manipulation - it's becoming quite blatant at this point.
- sctb 10y agoPlease don't post baseless accusations of shillage. See more here: https://news.ycombinator.com/item?id=13610097 https://news.ycombinator.com/item?id=13610097.
- rch 10y agoThis era of open source from MS is great, but my reaction is always "OK, here's the Microsoft version of something I've been using for a couple of years already". I'd like to see the Spanner+Cyc GIS-capable global distributed real-time graph engine with FPGA accelerated OLAP support MS Research is probably sitting on, because we can almost / pretty-much hack that together with OSS now.
- dredmorbius 10y agoMIT License, for those curious: https://github.com/Microsoft/GraphEngine/blob/master/LICENSE.md https://github.com/Microsoft/GraphEngine/blob/master/LICENSE...
- kensai 10y agoI still like the neat Pajek. Nifty little piece of software, unknown by most, but really powerful! Especially if you are into social network analysis. Who else uses it? http://mrvar.fdv.uni-lj.si/pajek/ http://mrvar.fdv.uni-lj.si/pajek/
- webmaven 10y agoNever heard of it before, and wow, that page is seriously old-school.
- certchain 10y agoFirefox gives an SSL error: www.graphengine.io uses an invalid security certificate. The certificate is not trusted because the issuer certificate is unknown. The server might not be sending the appropriate intermediate certificates. An additional root certificate may need to be imported. Error code: SEC_ERROR_UNKNOWN_ISSUER And SSL Labs shows the site is vulnerable to the POODLE attack (and has an incomplete cert chain): https://www.ssllabs.com/ssltest/analyze.html?d=www.graphengine.io https://www.ssllabs.com/ssltest/analyze.html?d=www.graphengi...
- zyztem 10y agohttp://www.frankmcsherry.org/graph/scalability/cost/2015/01/15/COST.html http://www.frankmcsherry.org/graph/scalability/cost/2015/01/...
- linux_devil 10y agoI am confused, how is it different from spark graphx?
- deleted 10y ago[deleted]