3 ms·

I've often seen many fixed-input posts like this. I've been looking for something that will allow me to train a model that will take a varying number of inputs

I've often seen many fixed-input posts like this. I've been looking for something that will allow me to train a model that will take a varying number of inputs and produce a single output. So for instances: [list of 10 items] -> Some Number [list of 500 items] -> Some Number These items are not reducible to a single scalar value. They have too many widely varying meanings. Can anyone point me in the right direction?

deleted 9y ago
[deleted]
michaf 9y ago
I am not an expert, but maybe look into recurrent neural networks [0]? These things can turn a sequence of inputs of possibly arbitrary length into a fixed size internal representation, and can output a single value derived from this representation. [0] https://en.wikipedia.org/wiki/Recurrent_neural_network https://en.wikipedia.org/wiki/Recurrent_neural_network
- anon1253 9y ago
  most implementations are not arbitrary length, but instead rely on padding to make it a fix length sequence.
candiodari 9y ago
TLDR: This is what LSTMs do. E.g. http://machinelearningmastery.com/sequence-classification-lstm-recurrent-neural-networks-python-keras/ http://machinelearningmastery.com/sequence-classification-ls... More detailed answer. LSTMs have 2/3 modes : sequence-to-sequence (immediately), sequence-to-single-output, and sequence-to-sequence (delayed). Sequence-to-sequence immediate is generally referred to as "seq2seq models", if you want to google it. This is used, for instance, in Deep Speech. Essentially the network takes in a sequence and immediately generates a new sequence from it. Sequence-to-single-output is called "sequence classification", and is used in text sentiment analysis. The network takes in a sequence of items and comes up with some number. Sequence-to-sequence (delayed) is called sequence generation. An example use would be translation. The network takes in a sequence, thinks about it a bit, and then outputs a new sequence. There are other things that may work well. For instance I've found "windowed convolution" over a sequence to work well, even better than LSTMs (and certainly easier and quicker), for sentiment analysis. You essentially make a "window" of items, and have it output "bits" (1 = true, 0 = false). For every window frame you generate these bits and then you maxpool (or just add) them over the entire length of the sequence. Of course this will never detect anything longer than the window.

deleted 9y ago

[deleted]

michaf 9y ago

I am not an expert, but maybe look into recurrent neural networks [0]? These things can turn a sequence of inputs of possibly arbitrary length into a fixed size internal representation, and can output a single value derived from this representation. [0] https://en.wikipedia.org/wiki/Recurrent_neural_network https://en.wikipedia.org/wiki/Recurrent_neural_network

anon1253 9y ago
most implementations are not arbitrary length, but instead rely on padding to make it a fix length sequence.

candiodari 9y ago

TLDR: This is what LSTMs do. E.g. http://machinelearningmastery.com/sequence-classification-lstm-recurrent-neural-networks-python-keras/ http://machinelearningmastery.com/sequence-classification-ls... More detailed answer. LSTMs have 2/3 modes : sequence-to-sequence (immediately), sequence-to-single-output, and sequence-to-sequence (delayed). Sequence-to-sequence immediate is generally referred to as "seq2seq models", if you want to google it. This is used, for instance, in Deep Speech. Essentially the network takes in a sequence and immediately generates a new sequence from it. Sequence-to-single-output is called "sequence classification", and is used in text sentiment analysis. The network takes in a sequence of items and comes up with some number. Sequence-to-sequence (delayed) is called sequence generation. An example use would be translation. The network takes in a sequence, thinks about it a bit, and then outputs a new sequence. There are other things that may work well. For instance I've found "windowed convolution" over a sequence to work well, even better than LSTMs (and certainly easier and quicker), for sentiment analysis. You essentially make a "window" of items, and have it output "bits" (1 = true, 0 = false). For every window frame you generate these bits and then you maxpool (or just add) them over the entire length of the sequence. Of course this will never detect anything longer than the window.