4 ms·

Microsoft acquires deep learning startup Maluuba

182 points by damvigilante 10y ago

roymurdock 10y ago
More interesting info on Maluuba's 2 actual, recently-released datasets: http://datasets.maluuba.com/ http://datasets.maluuba.com/ Their "News QA dataset" contains 120k Q&As collected from CNN articles: Documents are CNN news articles. Questions are written by human users in natural language. Answers may be multiword passages of the source text. Questions may be unanswerable. NewsQA is collected using a 3-stage, siloed process. Questioners see only an article's headline and highlights. Answerers see the question and the full article, then select an answer passage. Validators see the article, the question, and a set of answers that they rank. NewsQA is more natural and more challenging than previous datasets. Their "Frames" dataset contains 1369 dialogues for vacation scheduling: With this dataset, we also present a new task: frame tracking. Our main observation is that decision-making is tightly linked to memory. In effect, to choose a trip, users and wizards talked about different possibilities, compared them and went back-and-forth between cities, dates, or vacation packages. Current systems are memory-less. They implement slot-filling for search as a sequential process where the user is asked for constraints one after the other until a database query can be formulated. Only one set of constraints is kept in memory. For instance, in the illustration below, on the left, when the user mentions Montreal, it overwrites Toronto as destination city. However, behaviours observed in Frames imply that slot values should not be overwritten. One use-case is comparisons: it is common that users ask to compare different items and in this case, different sets of constraints are involved (for instance, different destinations). Frame tracking consists of keeping in memory all the different sets of constraints mentioned by the user. It is a generalization of the state tracking task to a setting where not only the current frame is memorized. Adding this kind of conversational memory is key to building agents which do not simply serve as a natural language interface for searching a database but instead accompany users in their exploration and help them find the best item. --- Can anyone with experience in ML/AI comment on how novel/complex these projects are, and how expensive it would be to build out these datasets? Would be interesting to see what it takes to publish a few datasets trained on 20 day conversations between real people, and get acquired by Microsoft/Apple/Google.
- ma2rten 10y ago
  Maluuba does more than publish a few datasets.
- strebler 10y ago
  I doubt they were acquired based on their datasets! The "Frames" dataset states 12 users over 20 days. But if one of the next 12 users was a Microsoft executive who liked what they saw, well then that's an important data point :) Having Bengio as an advisor definitely helps too.
  - roymurdock 10y ago
    I think you may be confused...they had 12 "wizard" participants converse with each other for 20 days in order to build out the logic needed for the dataset...not that 12 people used the dataset for their own projects/products or as customers.
    - strebler 10y ago
      You're probably right! It just states "12 particpants over a period of 20 days" on their site - hard to tell what that means.
- deepnotderp 10y ago
  Maluuba has real researchers from MILA and university of Toronto. It was an acquihire.
  - roymurdock 10y ago
    Understood, I'm looking for an analysis - it would cost x to hire the same size team of researchers with comparable research experience for the same amount of time to build out these datasets as proof-of-concept products, plus whatever other costs an ML research team accrues.
    - paulsutter 10y ago
thewhitetulip 10y ago
MS has always shown that they can do research, halolens is one such product and the recent ML enablement spree is another example. It is a good thing that they are going beyond their comfort zone of milking Windows. What is interesting is to see if their efforts make any difference in the status quo. I say this after using AzureML, which I liked, there is no such thing on the internet which allows you to write a ML model without knowing a programming language! It is a webapp which asks you to put data inside it, click a few buttons and it generates python or R code for you. Just brilliant.
- dforrestwilson1 10y ago
  Thanks for the comment I had not seen this before. https://azure.microsoft.com/en-us/services/machine-learning/ https://azure.microsoft.com/en-us/services/machine-learning/ ^Basic version is free to use!
  - thewhitetulip 10y ago
    Yes, basic version is free to use and it truly is an amazing application. MS feels like a company with a direction :-) Had they had this direction ten years ago, probably android would not have existed!
- oedenfield 10y ago
  It also has a Jupyter Notebook service piece currently in preview. https://notebooks.azure.com/ https://notebooks.azure.com/ (full disclosure: employee here)
  - thewhitetulip 10y ago
    I'll love it when Jupyter 2.0 is supported if it isnt currently supported. Great product!
EternalData 10y ago
Go Montreal! woot.
- rocky1138 10y ago
  I thought this company was located in Waterloo.
  - mindcruzer 10y ago
    Yeah last time I checked it was at Columbia & Phillip, right next to UW.
    - EternalData 10y ago
      One of the co-founders is pretty big in the Montreal startup scene.
clickok 10y ago
Here's a presentation from one of their researchers, Harm van Seijen[0], to give you an idea of what sort of work they do. Applying reinforcement learning to dialogue systems seems incredibly difficult, but if Maluuba (or others) can get a handle on the problem it would not be unreasonable to expect another revolution in the vein of applying convolutional nets to vision. 0: https://www.youtube.com/watch?v=s-8WkKhHYqA https://www.youtube.com/watch?v=s-8WkKhHYqA
Rainymood 10y ago
Interesting ... honest question: what kind of business models do these ML firms have?
redtrackker 10y ago
not sure if this was a successful exit. LinkedIn shows the employee count being cut in half over the last little while. Also 2 of the other cofounders left?
- jpantony 10y ago
  Out of curiosity what made you think 2 cofounders left?

182 points by damvigilante 10y ago

More interesting info on Maluuba's 2 actual, recently-released datasets: http://datasets.maluuba.com/ http://datasets.maluuba.com/ Their "News QA dataset" contains 120k Q&As collected from CNN articles: Documents are CNN news articles. Questions are written by human users in natural language. Answers may be multiword passages of the source text. Questions may be unanswerable. NewsQA is collected using a 3-stage, siloed process. Questioners see only an article's headline and highlights. Answerers see the question and the full article, then select an answer passage. Validators see the article, the question, and a set of answers that they rank. NewsQA is more natural and more challenging than previous datasets. Their "Frames" dataset contains 1369 dialogues for vacation scheduling: With this dataset, we also present a new task: frame tracking. Our main observation is that decision-making is tightly linked to memory. In effect, to choose a trip, users and wizards talked about different possibilities, compared them and went back-and-forth between cities, dates, or vacation packages. Current systems are memory-less. They implement slot-filling for search as a sequential process where the user is asked for constraints one after the other until a database query can be formulated. Only one set of constraints is kept in memory. For instance, in the illustration below, on the left, when the user mentions Montreal, it overwrites Toronto as destination city. However, behaviours observed in Frames imply that slot values should not be overwritten. One use-case is comparisons: it is common that users ask to compare different items and in this case, different sets of constraints are involved (for instance, different destinations). Frame tracking consists of keeping in memory all the different sets of constraints mentioned by the user. It is a generalization of the state tracking task to a setting where not only the current frame is memorized. Adding this kind of conversational memory is key to building agents which do not simply serve as a natural language interface for searching a database but instead accompany users in their exploration and help them find the best item. --- Can anyone with experience in ML/AI comment on how novel/complex these projects are, and how expensive it would be to build out these datasets? Would be interesting to see what it takes to publish a few datasets trained on 20 day conversations between real people, and get acquired by Microsoft/Apple/Google.

ma2rten 10y ago
Maluuba does more than publish a few datasets.
strebler 10y ago
I doubt they were acquired based on their datasets! The "Frames" dataset states 12 users over 20 days. But if one of the next 12 users was a Microsoft executive who liked what they saw, well then that's an important data point :) Having Bengio as an advisor definitely helps too.
- roymurdock 10y ago
  I think you may be confused...they had 12 "wizard" participants converse with each other for 20 days in order to build out the logic needed for the dataset...not that 12 people used the dataset for their own projects/products or as customers.
  - strebler 10y ago
    You're probably right! It just states "12 particpants over a period of 20 days" on their site - hard to tell what that means.
deepnotderp 10y ago
Maluuba has real researchers from MILA and university of Toronto. It was an acquihire.
- roymurdock 10y ago
  Understood, I'm looking for an analysis - it would cost x to hire the same size team of researchers with comparable research experience for the same amount of time to build out these datasets as proof-of-concept products, plus whatever other costs an ML research team accrues.
  - paulsutter 10y ago

thewhitetulip 10y ago

MS has always shown that they can do research, halolens is one such product and the recent ML enablement spree is another example. It is a good thing that they are going beyond their comfort zone of milking Windows. What is interesting is to see if their efforts make any difference in the status quo. I say this after using AzureML, which I liked, there is no such thing on the internet which allows you to write a ML model without knowing a programming language! It is a webapp which asks you to put data inside it, click a few buttons and it generates python or R code for you. Just brilliant.

dforrestwilson1 10y ago
Thanks for the comment I had not seen this before. https://azure.microsoft.com/en-us/services/machine-learning/ https://azure.microsoft.com/en-us/services/machine-learning/ ^Basic version is free to use!
- thewhitetulip 10y ago
  Yes, basic version is free to use and it truly is an amazing application. MS feels like a company with a direction :-) Had they had this direction ten years ago, probably android would not have existed!
oedenfield 10y ago
It also has a Jupyter Notebook service piece currently in preview. https://notebooks.azure.com/ https://notebooks.azure.com/ (full disclosure: employee here)
- thewhitetulip 10y ago
  I'll love it when Jupyter 2.0 is supported if it isnt currently supported. Great product!

EternalData 10y ago

Go Montreal! woot.

rocky1138 10y ago
I thought this company was located in Waterloo.
- mindcruzer 10y ago
  Yeah last time I checked it was at Columbia & Phillip, right next to UW.
  - EternalData 10y ago
    One of the co-founders is pretty big in the Montreal startup scene.

clickok 10y ago

Here's a presentation from one of their researchers, Harm van Seijen[0], to give you an idea of what sort of work they do. Applying reinforcement learning to dialogue systems seems incredibly difficult, but if Maluuba (or others) can get a handle on the problem it would not be unreasonable to expect another revolution in the vein of applying convolutional nets to vision. 0: https://www.youtube.com/watch?v=s-8WkKhHYqA https://www.youtube.com/watch?v=s-8WkKhHYqA

Rainymood 10y ago

Interesting ... honest question: what kind of business models do these ML firms have?

redtrackker 10y ago

not sure if this was a successful exit. LinkedIn shows the employee count being cut in half over the last little while. Also 2 of the other cofounders left?

jpantony 10y ago
Out of curiosity what made you think 2 cofounders left?