5 ms·

Deep learning architectures built by machines (so we no longer have to design architecture to solve problems) https://arxiv.org/abs/1611.01578 https://arxiv.org

Deep learning architectures built by machines (so we no longer have to design architecture to solve problems) https://arxiv.org/abs/1611.01578 https://arxiv.org/abs/1611.01578 Transfer Learning (so we need less data to build models) http://ftp.cs.wisc.edu/machine-learning/shavlik-group/torrey.handbook09.pdf http://ftp.cs.wisc.edu/machine-learning/shavlik-group/torrey... Generative adversarial networks (so computers can get human like abilities at generating content) https://papers.nips.cc/paper/5423-generative-adversarial-nets https://papers.nips.cc/paper/5423-generative-adversarial-net...

petra 10y ago
>> Generative adversarial networks (so computers can get human like abilities at generating content) Does this only apply to artistic content, or also to engineering content ? Say PCB layouts, architectural plans, mechanical designs, etc ?
- ktta 10y ago
  Artistic only. You'd need an exact verifier in case of engineering designs, and something built with neural networks is not one. To get a better understanding (other than reading a paper) read this excellent blog post https://openai.com/blog/generative-models/ https://openai.com/blog/generative-models/
  - rryan 10y ago
    GANs are a general tool -- they just happen to get a lot of attention for generating images of stuff. Here's an example for generating sequences [1]. The example is language oriented, but ultimately GANs are interesting because you can use them to build a generator for an arbitrary data distribution. This can have many applications in engineering (to take a random example -- generating plausible looking chemical structures under a certain set of constraints). As with any ML application, you need to quantify your tolerance for "inaccuracy" (in a generative setting, how well the generated distribution matches the true data distribution). This is simply an engineering trade-off and will vary based on the application. [1] https://arxiv.org/abs/1609.05473 https://arxiv.org/abs/1609.05473
- sarthakjain 10y ago
  The approach was applied without any real knowledge of art, even though it has been applied to the domains you mentioned I don't see why not. [edit]: it is a lot harder to build a NN when there are very constraint rules. But it is also a lot easier to verify and penalize it and generate synthetic data.
- deleted 10y ago
  [deleted]
Phait 10y ago
All these are definitely cool, but I think we're still a long way from leaving the "look at this cool toy" status and stepping into the "I can add value to society" status. Furthermore, if we consider that most of these DL paper completely ignore the fact that the nets must run for days on a GPU to get decent results, then everything appears way less impressive. But that's just my opinion. I love working in deep learning, but we still have LOTS of work to do.
- alook 10y ago
  It's true that there aren't too many practical applications of GAN's yet, but I'd argue that transfer learning is already pretty powerful. It's fairly commonplace to approach a compute vision task by starting with VGG/AlexNet/etc and fine-tuning it on a relatively small dataset.
- iandanforth 10y ago
  Could you elaborate? After running for days / weeks/ months the output is a net that can do inference in seconds, or with some now-common techniques milliseconds with only small reductions in accuracy. These nets can then be deployed to phones to solve a rapidly increasing number of identification tasks, everything from plants to cancer. The time from theoretical paper to widely deployed app is smaller in DL than in any other field I have experience with.
- rybosome 10y ago
  There is a LOT of investment in model training right now, with frameworks, specialized hardware (like Google's TPU), cloud services, etc., not to mention the GPU vendors themselves scrabbling like mad to develop chipsets that accommodate this more efficiently. It's going to take less and less time and money to train a useful model.
emcq 10y ago
What's new in transfer learning?
TwoBit 10y ago
I have a hard time reading most research papers on deep learning, as they are geared towards mathematicians who already know that lingo.
pinouchon 10y ago
I read the infoGAN paper yesterday. It blew my mind. https://arxiv.org/pdf/1606.03657v1.pdf https://arxiv.org/pdf/1606.03657v1.pdf. This is a way to do disentangled feature representation learning without supervision.

petra 10y ago

>> Generative adversarial networks (so computers can get human like abilities at generating content) Does this only apply to artistic content, or also to engineering content ? Say PCB layouts, architectural plans, mechanical designs, etc ?

ktta 10y ago
Artistic only. You'd need an exact verifier in case of engineering designs, and something built with neural networks is not one. To get a better understanding (other than reading a paper) read this excellent blog post https://openai.com/blog/generative-models/ https://openai.com/blog/generative-models/
- rryan 10y ago
  GANs are a general tool -- they just happen to get a lot of attention for generating images of stuff. Here's an example for generating sequences [1]. The example is language oriented, but ultimately GANs are interesting because you can use them to build a generator for an arbitrary data distribution. This can have many applications in engineering (to take a random example -- generating plausible looking chemical structures under a certain set of constraints). As with any ML application, you need to quantify your tolerance for "inaccuracy" (in a generative setting, how well the generated distribution matches the true data distribution). This is simply an engineering trade-off and will vary based on the application. [1] https://arxiv.org/abs/1609.05473 https://arxiv.org/abs/1609.05473
sarthakjain 10y ago
The approach was applied without any real knowledge of art, even though it has been applied to the domains you mentioned I don't see why not. [edit]: it is a lot harder to build a NN when there are very constraint rules. But it is also a lot easier to verify and penalize it and generate synthetic data.
deleted 10y ago
[deleted]

Phait 10y ago

All these are definitely cool, but I think we're still a long way from leaving the "look at this cool toy" status and stepping into the "I can add value to society" status. Furthermore, if we consider that most of these DL paper completely ignore the fact that the nets must run for days on a GPU to get decent results, then everything appears way less impressive. But that's just my opinion. I love working in deep learning, but we still have LOTS of work to do.

alook 10y ago
It's true that there aren't too many practical applications of GAN's yet, but I'd argue that transfer learning is already pretty powerful. It's fairly commonplace to approach a compute vision task by starting with VGG/AlexNet/etc and fine-tuning it on a relatively small dataset.
iandanforth 10y ago
Could you elaborate? After running for days / weeks/ months the output is a net that can do inference in seconds, or with some now-common techniques milliseconds with only small reductions in accuracy. These nets can then be deployed to phones to solve a rapidly increasing number of identification tasks, everything from plants to cancer. The time from theoretical paper to widely deployed app is smaller in DL than in any other field I have experience with.
rybosome 10y ago
There is a LOT of investment in model training right now, with frameworks, specialized hardware (like Google's TPU), cloud services, etc., not to mention the GPU vendors themselves scrabbling like mad to develop chipsets that accommodate this more efficiently. It's going to take less and less time and money to train a useful model.

emcq 10y ago

What's new in transfer learning?

TwoBit 10y ago

I have a hard time reading most research papers on deep learning, as they are geared towards mathematicians who already know that lingo.

pinouchon 10y ago

I read the infoGAN paper yesterday. It blew my mind. https://arxiv.org/pdf/1606.03657v1.pdf https://arxiv.org/pdf/1606.03657v1.pdf. This is a way to do disentangled feature representation learning without supervision.