4 ms·
Which begs the question what the added benefit would be from such an approach, i.e. what could we even learn from that compared to other methods? To quote from
by qayxc 4y ago
Which begs the question what the added benefit would be from such an approach, i.e. what could we even learn from that compared to other methods?
To quote from an article in The Royal Society:
> [...] bird song evolution can also be understood as a response to natural selection, as when oscine passerine species in urban habitats raise the frequency of their song as a means to overcome traffic noise.
Things like changes in the vocalisation can just be adjustments to the environment or simply due to sexual selection. How would an LLM be able to capture this, given that it would lack such additional information?
Then there's this quote from the same article:
> In contrast to the expectation of gradual change through cultural or genetic drift over time, the results instead demonstrate that plastic traits such as song can exhibit punctuated evolution, with bursts of trait divergence interrupting extended periods of stasis.
So even if a model was trained on (regional) bird dialects, the data could become obsolete basically over night, as the structure and patterns might change dramatically between breeding seasons. The researchers used statistical modelling (classical Gaussian mixture models) to analyse the data. I'm a bit lost as to how LLMs could be applied in this context (in that I have no idea what kind of scientific questions they could answer).
The full study can be found here: https://royalsocietypublishing.org/doi/10.1098/rspb.2021.2062 https://royalsocietypublishing.org/doi/10.1098/rspb.2021.206...
- nerdponx 4y agoIt's not unreasonable to expect that a big enough model could learn some kind of useful feature space for bird vocalizations. You are essentially replacing a parsimonious quasi-parametric model (Gaussian process) with an enormous fully nonparametric one. Transformer units should be as well-suited for this as for any other sequence learning task.