10 ms·

Facebook and Microsoft introduce ecosystem for interchangeable AI frameworks

335 points by runesoerensen 9y ago

TCM 9y ago
Its interesting that this tool has no PATENTS grant.
- cbuq 9y ago
  Interesting, it is also MIT instead of the usual BSD + Patents Facebook uses. [1] It is also under the 'onnx' namespace, presumably because it was a team effort. It looks like Microsoft often uses MIT (vscode, dotnet, CNTK, ChakraCore), does this mean Microsoft was the driving force (even though it's a Facebook blog)? * just noticed the link to Microsofts blog at the bottom of the page [2]. [1] https://github.com/onnx/onnx https://github.com/onnx/onnx [2] https://www.microsoft.com/en-us/cognitive-toolkit/blog/2017/09/microsoft-facebook-create-open-ecosystem-ai-model-interoperability/ https://www.microsoft.com/en-us/cognitive-toolkit/blog/2017/...
- jiayq84 9y ago
  Yangqing here from facebook. We consciously made it a MIT license as onnx is intended to be widely shared by a lot of participants, and MIT seems to be more widely agreeable among different parties co-owning it. It's also simpler.
  - rofrol 9y ago
    Oh, so now there was no danger of this nasty patent trolls, like with react case?
jeremynixon 9y ago
Automatic productionizability of PyTorch models through Caffe 2 will speed the transition from research to production dramatically - this gives the environment a chance to compete with Tensorflow / Tensorflow Serving. Frameworks (say, DL4J) have been using Keras as a loose way to share models between frameworks. It'll be fascinating to see if Theano / Tensorflow / DL4J / MXNet walk this path as well.
- Q6T46nT668w6i3m 9y ago
  I was curious about their implementation since PyTorch and Caffe2 semantics are very different. Unfortuantly, the authors write: > Currently, our tracer works with many common neural networks, but not some of the more advanced programs in PyTorch such as those with dynamic flow control. Over time, we will enhance ONNX and the tracer to support these programs, so that developers can leverage full flexibility of PyTorch with the high-performance robust deployment capabilities of Caffe2. It is useful, of course. But it's rare for contemporary models to not use dynamic flow. In fact, PyTorch is popular because it encourages this dynamism.
deleted 9y ago
[deleted]
cs702 9y ago
Translation from corporatespeak: "Our AI frameworks are losing developer mindshare to TensorFlow, which is controlled by Google, so we are joining forces." Corporations like Facebook and Microsoft do not like interoperability... until and unless it's in their best interest. The goal here is to make AI frameworks easily interchangeable: build your model with whatever framework you like; it will run unchanged on all other frameworks/platforms/stacks. Regardless of the motivations, I actually think this is GREAT NEWS, and I hope that Amazon and maybe Apple, and eventually hopefully Google too will introduce and promote compatibility with this standard. We all benefit from more interoperability.
- jononor 9y ago
  The corporations that got data are also very interested in open development of machine learning algorithms. They know that the data is key for getting value out of the algorithms anyway... EDIT: I second that regardless of motivation, this is a step forward.
- bluetwo 9y ago
  The implication is that this will make your ML better somehow. Not sure why that would be.
  - nametube 9y ago
    from the article > We developed ONNX together with Microsoft to bridge this gap and to empower AI developers to choose the framework that fits the current stage of their project and easily switch between frameworks as the project evolves. ML workflows tend to be very heterogenous with people using everything from sklearn to DL4J to play with data. standardized serialization formats make it possible to bring results from those experiments back into a hardened productionized environment for serving the model.
    - bluetwo 9y ago
      Thanks.
- 9y ago
nametube 9y ago
While this is awesome I'm personally wondering when big companies like FB and MS are going to standardize on a format for predictive models in general. [0] PMML exists but there are very few companies that are standing behind it. [0] http://dmg.org/pmml/v4-3/GeneralStructure.html http://dmg.org/pmml/v4-3/GeneralStructure.html
sanxiyn 9y ago
Now, it would be lovely if coremltools could support ONNX and convert ONNX model to CoreML.
- jiayq84 9y ago
  Yangqing here (created Caffe and Caffe2) - we are much interested in enabling this path. Historically CoreML has provided Caffe and Keras interfaces, and having ONNX / CoreML interop would help a lot for everyone to ship models more easily. Earlier in the year we provided compatibility between Caffe2 and Qualcomm's SNPE library, which follows this similar philosophy.
  - Q6T46nT668w6i3m 9y ago
    Hi, Yangqing! Nice project. B) I want to clarify that Apple advertises Keras support for use with CoreML, but the converter uses a graph from the TensorFlow backend. It begs the question, have you spoken with anybody from the TensorFlow (or Keras) communities about collaborating?
    - fchollet 9y ago
      CoreML supports Keras but not TensorFlow because Keras models form a well-structured subset of all possible TensorFlow graphs. It would be quite difficult to support completely arbitrary TensorFlow graphs, but supporting every Keras layer is relatively straightforward. To answer your question: I had no knowledge of this ONIX project before the public announcement today. Speaking purely for myself, if I wanted to develop a universal model exchange format, the first step I would take would be to get in touch with the makers of the frameworks that sum to 80-90% of the market share -- TF, Keras, MXNet. But maybe such a strategy was thought to be superfluous in this case -- for instance, because ONIX may not actually be intended as a universal model exchange format.
      - liuliu 9y ago
        To be fair, CNTK (BrainScript) has quite impressive list of features to support dynamic control structure (in a symbolic fashion, comparing to PyTorch which delegated much of the dynamic control structure to underlying language Python). I think Tensorflow and CNTK probably the only two frameworks pursued such implementation strategy. IMHO, looking back, supporting control structures may not be that useful (see the recent attention based models, all of them can be unroll'ed to ordinary graphs), but it is so interesting to implement!
liuliu 9y ago
Doesn't seem to have support for control structure (If, While etc.).
nicodjimenez 9y ago
Great move by Facebook to look beyond Caffe's legacy prototext file format which has become somewhat standard in computer vision, and unite Pytorch and Caffe2. Unfortunately there are limitations to the neural network logic that can be represented in configuration files, especially when dealing with dynamic networks. Most networks used in vision are static so that's where I expect this will add the most value. My only wish is that Caffe2 followed Pytorch's interface. After spending 5 minutes with Pytorch it's pretty clear that they got the interface design right.
Markoff 9y ago
can we send messages through it?
sandGorgon 9y ago
Why a new format ? Why couldn't protobuf, hdf5 or the Apache Feather/Arrow format (which counts Hadley and McKinney as contributors). I understand there could be something unique in this format, but really keen to understand what.
- ezyang 9y ago
  PyTorch/ONNX dev here. ONNX is a proto2 format https://github.com/onnx/onnx/blob/master/onnx/onnx.proto https://github.com/onnx/onnx/blob/master/onnx/onnx.proto -- we definitely wanted to make it easy for people to parse and load ONNX exported graphs :)
- squarecog 9y ago
  hdf5, Feather, Arrow, protobufs, json, xml -- all solve the problem of binary representation of data on disk. They all leave the question of how to map said data to a specific problem domain up to the developer. Projects like ONNX define said mapping for a specific domain (in ONNX's case, by agreeing on a proto schema for ML models, and its interpretation). To use a simplistic metaphor: protobufs are the .docx format; onnx is a resume template you can fill out in Word.
- jiayq84 9y ago
  Yangqing here (caffe2 and ONNX). We did use protobuf and we have an extensive discussion about its versions even, from our experience with the Caffe and Caffe2 deployment modes. Here is a snippet from the codebase: // Note [Protobuf compatibility] // ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ // Based on experience working with downstream vendors, we generally can't // assume recent versions of protobufs. This means that we do not use any // protobuf features that are only available in proto3. // // Here are the most notable contortions we have to carry out to work around // these limitations: // // - No 'map' (added protobuf 3.0). We instead represent mappings as lists // of key-value pairs, where order does not matter and duplicates // are not allowed.
  - sandGorgon 9y ago
    pretty cool and thanks for that reply! Did you look at something like Arrow/Feather, which is looking to get adopted as the interoperable format in R/Pandas ... and maybe even Spark. There's been quite a bit of momentum behind it to optimize it for huge usecases - https://thenewstack.io/apache-arrow-designed-accelerate-hadoop-spark-columnar-layouts-data/ https://thenewstack.io/apache-arrow-designed-accelerate-hado... It is based on Google Flatbuffers, but is undergoing enough engineering specifically from a big data/machine learning perspective. Instead of building directly over Protobuf, it might be interesting to build it on top of Arrow (in exactly the same way that Feather is based on top of Arrow https://github.com/wesm/feather https://github.com/wesm/feather).
megamindbrian 9y ago
Why would you want to pile another black box on top of the black box?
- starshadowx2 9y ago
  It's black boxes all the way down.
- deleted 9y ago
  [deleted]
electrograv 9y ago
> To directly export this code, ONNX would have to support conditionals [...] Does ONNX aim to serve as a human-readable and human-maintainable network description for experimentation and research, or does ONNX aim to serve only as a model export interchange format (primarily to be machine-interpreted)? This could serve very well in the latter case, but would be quite problematic in the former -- where conditionals, for loops, etc. are essential to a compact and readable representation of many network architectures. Reading a static neural network architecture descriptor of a very deep neural network is akin to reading assembly code from a compiler that does loop unrolling. Static network descriptions work great when pushing a model to production, but are much less suitable for the research and development stages (IMO).
- jsgo 9y ago
  my guess is it is the latter. If I have my own framework and I want to operate with this standard, I need to have the ability to map to and from ONNX's format. I don't think it is intended for people to write directly in ONNX's format to maintain compatibility. If it were, I'd imagine it would fall flat very quickly as people wouldn't want to rewrite everything in the hopes that their framework of choice will adopt it. Seems the intent is more like .odt to me than, say, Markdown.
MikkoFinell 9y ago
Just imagine the world we will live in, when Facebook or Microsoft wins the race to general AI.
- aianus 9y ago
  I imagine it will be like the Manhattan Project, impossible to contain to one company or country for long.
lolive 9y ago
I have heard about something similar in the open source community: https://deepdetect.com https://deepdetect.com Doesn't it sound like a better long-term option than corporate-driven political initiative?
pgroves 9y ago
Predictive Model Markup Language (PMML) was released in 1997 to solve a very similar problem (but for lots of predictive models). I don't know that it ever really caught on. That was also promoted by the tool builders but not by the users. While checking the wikipedia page, I'm surprised to see that it is still being developed (with a 2016 release): https://en.wikipedia.org/wiki/Predictive_Model_Markup_Language https://en.wikipedia.org/wiki/Predictive_Model_Markup_Langua... Please note that in 1997 it was NOT ridiculous to use XML for this.
- nametube 9y ago
  You might also be interested in openscoring https://github.com/openscoring/openscoring https://github.com/openscoring/openscoring . It lets you run PMML models and score examples through a rest API.
- bayonetz 9y ago
  PMML is alive and well for sure. It really only covers simpler model types so something aimed at deep nets models is a welcome addition. However, PMML is still alive and well because plenty of ML will continue to be based on simpler model types and modelers don't want to be tied down on choice of modeling tech even for simple model types.
- mindcrime 9y ago
  PMML is definitely in use. There are occasional compatibility issues, but for the present, it is the standard for exchanging models.
naturalgradient 9y ago
Any TensorFlow devs who might be able to comment on plans to interoperate with TensorFlow/reasons why it's not on the cards?
- Q6T46nT668w6i3m 9y ago
  I am a regular TensorFlow (and Keras) contributor and I saw the announcement when you did. In either situation, Google contributors partcipating without speaking to the TensorFlow community or the Facebook and Microsoft authors not reaching out to the TensorFlow community, it's a bad look. However, it's entirely possible we'll hear more next week at the TensorFlow symposium.
pishpash 9y ago
So "AI" is equivalent to "neural networks" nowadays, eh?
- dgaaaaaaaaaa 9y ago
  pretty much
- jeremynixon 9y ago
  That's not implied by the title. Neural Networks is a member of Machine Learning, which is a member of AI. PyTorch, Caffe 2 and CNTK are Deep Learning Frameworks. All Deep Learning Frameworks are AI frameworks, but not all AI frameworks are neural network / deep learning frameworks.
- Q6T46nT668w6i3m 9y ago
  I realize you are joking, but in some fields, like computer vision, yes.
khana 9y ago
Big corporate is wasting NO TIME HERE. Like kids in a sandbox jockeying for position. Ahh the parallels of scale.
khana 9y ago
SkyNet or bust bro.
jamesblonde 9y ago
Michelangelo, announced by Uber 2 days ago (https://eng.uber.com/michelangelo/ https://eng.uber.com/michelangelo/), use another format for storing models/features and for serving data. It would be interesting to hear why they didn't use protobufs for storing features instead of their own system. My guess would be that they have a mix of scikit learn and tensorflow, and they didn't want to be too tightly coupled to tensorflow. Still, this move by FB and Microsoft is what happens when you're desperate - your frameworks are not being adopted, last chance role of the dice.
protomok 9y ago
I'm curious if FB/Microsoft considered Khronos Neural Network Exchange Format (NNEF) - https://www.khronos.org/nnef https://www.khronos.org/nnef?
Nocturnals 9y ago
Does that mean that we can see the AI implementations facebook uses? They invented the format so I'm sure we'll see everything facebook does in the new ONNX format. From the placement of ads to whatever... (sorry for the sarcastic comment)
agibsonccc 9y ago
Hi, Adam from DL4j here. I'm really excited to see this. We will work on supporting this format as well. We have a new pytorch like api coming out (dynamic graphs etc), so this fits nicely in to that. I know a lot of cynics are looking at this and are thinking this is all politics. Honestly: I am glad something like this exists as a framework developer. We would like to build an easy to maintain set of benchmarks for perf comparisons and have found that to be hard to do. We were originally going to lean on keras with our model import for this, but if we can get say: dynet and chainer on board with this as well, I think it will help the ecosystem as a whole move forward. (If you're asking "why" it's hard, main reason for us was the primary java interface, we are fixing this with a new python interface based on pyjnius) Overall, I don't think the OpenCL folks will be able to pull this off (the field moves too fast). I feel like this is something the people who actually build the tools should do. Ultimately, I would be surprised if tensorflow didn't hop on board at some point. A lot of ML models are being built in pytorch first now. No one loses having that easily accessible.
- hedgehog 9y ago
  Hey Adam, I don't know about the "OpenCL folks" but we're able to use OpenCL to exceed TensorFlow + cuDNN throughput on the same hardware for real networks. We (Vertex.AI) mostly care about embedded platforms and we currently use Keras as our research front end but we're open to collaborations.
  - agibsonccc 9y ago
    I am talking about their neural net format khronos is working on. We likely dont overlap. I am mostly focused on banking gov and telco. We are supporting this out of interest in being able to run these neural nets in our distribution. We have a few embedded use cases but we will likely write our own graph executioner for arm and the like.
    - hedgehog 9y ago
      Ah, I don't know too much about what Khronos is doing. We have thought about your work from the standpoint of supporting DL4j on a range of datacenter accelerators, we do tune for NVIDIA etc with good results. We're pretty framework and hardware neutral.
evc123 9y ago
ONNX plans to support TensorFlow as well: https://github.com/onnx/onnx/issues/3 https://github.com/onnx/onnx/issues/3

335 points by runesoerensen 9y ago

TCM 9y ago

Its interesting that this tool has no PATENTS grant.

cbuq 9y ago
Interesting, it is also MIT instead of the usual BSD + Patents Facebook uses. [1] It is also under the 'onnx' namespace, presumably because it was a team effort. It looks like Microsoft often uses MIT (vscode, dotnet, CNTK, ChakraCore), does this mean Microsoft was the driving force (even though it's a Facebook blog)? * just noticed the link to Microsofts blog at the bottom of the page [2]. [1] https://github.com/onnx/onnx https://github.com/onnx/onnx [2] https://www.microsoft.com/en-us/cognitive-toolkit/blog/2017/09/microsoft-facebook-create-open-ecosystem-ai-model-interoperability/ https://www.microsoft.com/en-us/cognitive-toolkit/blog/2017/...
jiayq84 9y ago
Yangqing here from facebook. We consciously made it a MIT license as onnx is intended to be widely shared by a lot of participants, and MIT seems to be more widely agreeable among different parties co-owning it. It's also simpler.
- rofrol 9y ago
  Oh, so now there was no danger of this nasty patent trolls, like with react case?

jeremynixon 9y ago

Automatic productionizability of PyTorch models through Caffe 2 will speed the transition from research to production dramatically - this gives the environment a chance to compete with Tensorflow / Tensorflow Serving. Frameworks (say, DL4J) have been using Keras as a loose way to share models between frameworks. It'll be fascinating to see if Theano / Tensorflow / DL4J / MXNet walk this path as well.

Q6T46nT668w6i3m 9y ago
I was curious about their implementation since PyTorch and Caffe2 semantics are very different. Unfortuantly, the authors write: > Currently, our tracer works with many common neural networks, but not some of the more advanced programs in PyTorch such as those with dynamic flow control. Over time, we will enhance ONNX and the tracer to support these programs, so that developers can leverage full flexibility of PyTorch with the high-performance robust deployment capabilities of Caffe2. It is useful, of course. But it's rare for contemporary models to not use dynamic flow. In fact, PyTorch is popular because it encourages this dynamism.

deleted 9y ago

[deleted]

cs702 9y ago

Translation from corporatespeak: "Our AI frameworks are losing developer mindshare to TensorFlow, which is controlled by Google, so we are joining forces." Corporations like Facebook and Microsoft do not like interoperability... until and unless it's in their best interest. The goal here is to make AI frameworks easily interchangeable: build your model with whatever framework you like; it will run unchanged on all other frameworks/platforms/stacks. Regardless of the motivations, I actually think this is GREAT NEWS, and I hope that Amazon and maybe Apple, and eventually hopefully Google too will introduce and promote compatibility with this standard. We all benefit from more interoperability.

jononor 9y ago
The corporations that got data are also very interested in open development of machine learning algorithms. They know that the data is key for getting value out of the algorithms anyway... EDIT: I second that regardless of motivation, this is a step forward.
bluetwo 9y ago
The implication is that this will make your ML better somehow. Not sure why that would be.
- nametube 9y ago
  from the article > We developed ONNX together with Microsoft to bridge this gap and to empower AI developers to choose the framework that fits the current stage of their project and easily switch between frameworks as the project evolves. ML workflows tend to be very heterogenous with people using everything from sklearn to DL4J to play with data. standardized serialization formats make it possible to bring results from those experiments back into a hardened productionized environment for serving the model.
  - bluetwo 9y ago
    Thanks.
9y ago

nametube 9y ago

While this is awesome I'm personally wondering when big companies like FB and MS are going to standardize on a format for predictive models in general. [0] PMML exists but there are very few companies that are standing behind it. [0] http://dmg.org/pmml/v4-3/GeneralStructure.html http://dmg.org/pmml/v4-3/GeneralStructure.html

sanxiyn 9y ago

Now, it would be lovely if coremltools could support ONNX and convert ONNX model to CoreML.

jiayq84 9y ago
Yangqing here (created Caffe and Caffe2) - we are much interested in enabling this path. Historically CoreML has provided Caffe and Keras interfaces, and having ONNX / CoreML interop would help a lot for everyone to ship models more easily. Earlier in the year we provided compatibility between Caffe2 and Qualcomm's SNPE library, which follows this similar philosophy.
- Q6T46nT668w6i3m 9y ago
  Hi, Yangqing! Nice project. B) I want to clarify that Apple advertises Keras support for use with CoreML, but the converter uses a graph from the TensorFlow backend. It begs the question, have you spoken with anybody from the TensorFlow (or Keras) communities about collaborating?
  - fchollet 9y ago
    CoreML supports Keras but not TensorFlow because Keras models form a well-structured subset of all possible TensorFlow graphs. It would be quite difficult to support completely arbitrary TensorFlow graphs, but supporting every Keras layer is relatively straightforward. To answer your question: I had no knowledge of this ONIX project before the public announcement today. Speaking purely for myself, if I wanted to develop a universal model exchange format, the first step I would take would be to get in touch with the makers of the frameworks that sum to 80-90% of the market share -- TF, Keras, MXNet. But maybe such a strategy was thought to be superfluous in this case -- for instance, because ONIX may not actually be intended as a universal model exchange format.
    - liuliu 9y ago
      To be fair, CNTK (BrainScript) has quite impressive list of features to support dynamic control structure (in a symbolic fashion, comparing to PyTorch which delegated much of the dynamic control structure to underlying language Python). I think Tensorflow and CNTK probably the only two frameworks pursued such implementation strategy. IMHO, looking back, supporting control structures may not be that useful (see the recent attention based models, all of them can be unroll'ed to ordinary graphs), but it is so interesting to implement!

liuliu 9y ago

Doesn't seem to have support for control structure (If, While etc.).

nicodjimenez 9y ago

Great move by Facebook to look beyond Caffe's legacy prototext file format which has become somewhat standard in computer vision, and unite Pytorch and Caffe2. Unfortunately there are limitations to the neural network logic that can be represented in configuration files, especially when dealing with dynamic networks. Most networks used in vision are static so that's where I expect this will add the most value. My only wish is that Caffe2 followed Pytorch's interface. After spending 5 minutes with Pytorch it's pretty clear that they got the interface design right.

Markoff 9y ago

can we send messages through it?

sandGorgon 9y ago

Why a new format ? Why couldn't protobuf, hdf5 or the Apache Feather/Arrow format (which counts Hadley and McKinney as contributors). I understand there could be something unique in this format, but really keen to understand what.

ezyang 9y ago
PyTorch/ONNX dev here. ONNX is a proto2 format https://github.com/onnx/onnx/blob/master/onnx/onnx.proto https://github.com/onnx/onnx/blob/master/onnx/onnx.proto -- we definitely wanted to make it easy for people to parse and load ONNX exported graphs :)
squarecog 9y ago
hdf5, Feather, Arrow, protobufs, json, xml -- all solve the problem of binary representation of data on disk. They all leave the question of how to map said data to a specific problem domain up to the developer. Projects like ONNX define said mapping for a specific domain (in ONNX's case, by agreeing on a proto schema for ML models, and its interpretation). To use a simplistic metaphor: protobufs are the .docx format; onnx is a resume template you can fill out in Word.
jiayq84 9y ago
Yangqing here (caffe2 and ONNX). We did use protobuf and we have an extensive discussion about its versions even, from our experience with the Caffe and Caffe2 deployment modes. Here is a snippet from the codebase: // Note [Protobuf compatibility] // ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ // Based on experience working with downstream vendors, we generally can't // assume recent versions of protobufs. This means that we do not use any // protobuf features that are only available in proto3. // // Here are the most notable contortions we have to carry out to work around // these limitations: // // - No 'map' (added protobuf 3.0). We instead represent mappings as lists // of key-value pairs, where order does not matter and duplicates // are not allowed.
- sandGorgon 9y ago
  pretty cool and thanks for that reply! Did you look at something like Arrow/Feather, which is looking to get adopted as the interoperable format in R/Pandas ... and maybe even Spark. There's been quite a bit of momentum behind it to optimize it for huge usecases - https://thenewstack.io/apache-arrow-designed-accelerate-hadoop-spark-columnar-layouts-data/ https://thenewstack.io/apache-arrow-designed-accelerate-hado... It is based on Google Flatbuffers, but is undergoing enough engineering specifically from a big data/machine learning perspective. Instead of building directly over Protobuf, it might be interesting to build it on top of Arrow (in exactly the same way that Feather is based on top of Arrow https://github.com/wesm/feather https://github.com/wesm/feather).

megamindbrian 9y ago

Why would you want to pile another black box on top of the black box?

starshadowx2 9y ago
It's black boxes all the way down.
deleted 9y ago
[deleted]

electrograv 9y ago

> To directly export this code, ONNX would have to support conditionals [...] Does ONNX aim to serve as a human-readable and human-maintainable network description for experimentation and research, or does ONNX aim to serve only as a model export interchange format (primarily to be machine-interpreted)? This could serve very well in the latter case, but would be quite problematic in the former -- where conditionals, for loops, etc. are essential to a compact and readable representation of many network architectures. Reading a static neural network architecture descriptor of a very deep neural network is akin to reading assembly code from a compiler that does loop unrolling. Static network descriptions work great when pushing a model to production, but are much less suitable for the research and development stages (IMO).

jsgo 9y ago
my guess is it is the latter. If I have my own framework and I want to operate with this standard, I need to have the ability to map to and from ONNX's format. I don't think it is intended for people to write directly in ONNX's format to maintain compatibility. If it were, I'd imagine it would fall flat very quickly as people wouldn't want to rewrite everything in the hopes that their framework of choice will adopt it. Seems the intent is more like .odt to me than, say, Markdown.

MikkoFinell 9y ago

Just imagine the world we will live in, when Facebook or Microsoft wins the race to general AI.

aianus 9y ago
I imagine it will be like the Manhattan Project, impossible to contain to one company or country for long.

lolive 9y ago

I have heard about something similar in the open source community: https://deepdetect.com https://deepdetect.com Doesn't it sound like a better long-term option than corporate-driven political initiative?

pgroves 9y ago

Predictive Model Markup Language (PMML) was released in 1997 to solve a very similar problem (but for lots of predictive models). I don't know that it ever really caught on. That was also promoted by the tool builders but not by the users. While checking the wikipedia page, I'm surprised to see that it is still being developed (with a 2016 release): https://en.wikipedia.org/wiki/Predictive_Model_Markup_Language https://en.wikipedia.org/wiki/Predictive_Model_Markup_Langua... Please note that in 1997 it was NOT ridiculous to use XML for this.

nametube 9y ago
You might also be interested in openscoring https://github.com/openscoring/openscoring https://github.com/openscoring/openscoring . It lets you run PMML models and score examples through a rest API.
bayonetz 9y ago
PMML is alive and well for sure. It really only covers simpler model types so something aimed at deep nets models is a welcome addition. However, PMML is still alive and well because plenty of ML will continue to be based on simpler model types and modelers don't want to be tied down on choice of modeling tech even for simple model types.
mindcrime 9y ago
PMML is definitely in use. There are occasional compatibility issues, but for the present, it is the standard for exchanging models.

naturalgradient 9y ago

Any TensorFlow devs who might be able to comment on plans to interoperate with TensorFlow/reasons why it's not on the cards?

Q6T46nT668w6i3m 9y ago
I am a regular TensorFlow (and Keras) contributor and I saw the announcement when you did. In either situation, Google contributors partcipating without speaking to the TensorFlow community or the Facebook and Microsoft authors not reaching out to the TensorFlow community, it's a bad look. However, it's entirely possible we'll hear more next week at the TensorFlow symposium.

pishpash 9y ago

So "AI" is equivalent to "neural networks" nowadays, eh?

dgaaaaaaaaaa 9y ago
pretty much
jeremynixon 9y ago
That's not implied by the title. Neural Networks is a member of Machine Learning, which is a member of AI. PyTorch, Caffe 2 and CNTK are Deep Learning Frameworks. All Deep Learning Frameworks are AI frameworks, but not all AI frameworks are neural network / deep learning frameworks.
Q6T46nT668w6i3m 9y ago
I realize you are joking, but in some fields, like computer vision, yes.

khana 9y ago

Big corporate is wasting NO TIME HERE. Like kids in a sandbox jockeying for position. Ahh the parallels of scale.

khana 9y ago

SkyNet or bust bro.

jamesblonde 9y ago

Michelangelo, announced by Uber 2 days ago (https://eng.uber.com/michelangelo/ https://eng.uber.com/michelangelo/), use another format for storing models/features and for serving data. It would be interesting to hear why they didn't use protobufs for storing features instead of their own system. My guess would be that they have a mix of scikit learn and tensorflow, and they didn't want to be too tightly coupled to tensorflow. Still, this move by FB and Microsoft is what happens when you're desperate - your frameworks are not being adopted, last chance role of the dice.

protomok 9y ago

I'm curious if FB/Microsoft considered Khronos Neural Network Exchange Format (NNEF) - https://www.khronos.org/nnef https://www.khronos.org/nnef?

Nocturnals 9y ago

Does that mean that we can see the AI implementations facebook uses? They invented the format so I'm sure we'll see everything facebook does in the new ONNX format. From the placement of ads to whatever... (sorry for the sarcastic comment)

agibsonccc 9y ago

Hi, Adam from DL4j here. I'm really excited to see this. We will work on supporting this format as well. We have a new pytorch like api coming out (dynamic graphs etc), so this fits nicely in to that. I know a lot of cynics are looking at this and are thinking this is all politics. Honestly: I am glad something like this exists as a framework developer. We would like to build an easy to maintain set of benchmarks for perf comparisons and have found that to be hard to do. We were originally going to lean on keras with our model import for this, but if we can get say: dynet and chainer on board with this as well, I think it will help the ecosystem as a whole move forward. (If you're asking "why" it's hard, main reason for us was the primary java interface, we are fixing this with a new python interface based on pyjnius) Overall, I don't think the OpenCL folks will be able to pull this off (the field moves too fast). I feel like this is something the people who actually build the tools should do. Ultimately, I would be surprised if tensorflow didn't hop on board at some point. A lot of ML models are being built in pytorch first now. No one loses having that easily accessible.

hedgehog 9y ago
Hey Adam, I don't know about the "OpenCL folks" but we're able to use OpenCL to exceed TensorFlow + cuDNN throughput on the same hardware for real networks. We (Vertex.AI) mostly care about embedded platforms and we currently use Keras as our research front end but we're open to collaborations.
- agibsonccc 9y ago
  I am talking about their neural net format khronos is working on. We likely dont overlap. I am mostly focused on banking gov and telco. We are supporting this out of interest in being able to run these neural nets in our distribution. We have a few embedded use cases but we will likely write our own graph executioner for arm and the like.
  - hedgehog 9y ago
    Ah, I don't know too much about what Khronos is doing. We have thought about your work from the standpoint of supporting DL4j on a range of datacenter accelerators, we do tune for NVIDIA etc with good results. We're pretty framework and hardware neutral.

evc123 9y ago

ONNX plans to support TensorFlow as well: https://github.com/onnx/onnx/issues/3 https://github.com/onnx/onnx/issues/3