6 ms·
In some ways, this is a kind of "no shit, sherlock" moment that all machine learning faces. It becomes a dead end, feedback loop like holding a microphone too
by smackeyacky 4y ago
In some ways, this is a kind of "no shit, sherlock" moment that all machine learning faces. It becomes a dead end, feedback loop like holding a microphone too close to a speaker.
Funnily enough, genomic powered genetics in livestock production has the same characteristics. Without the flow of new phenotypic data the genomic models rot rapidly and become useless for prediction.
Predictive language models powered by older predictive language models will quickly become nonsense.
You have to remember just how deceptive something simple as Eliza was for people who didn't know how it worked.
- visarga 4y agoChanges to language are smaller. GPT-4 has a cutoff date in 2021 and is plenty smart and useful. It just doesn't know about the war in Ukraine.
- smackeyacky 4y agoThere are at least two things that make information wrong. One is "right, but not right now" i.e time. The right answer delivered late is wrong c.f. real time systems. The other is "garbage in, garbage out" which is related but a bit wider. While these models cannot distinguish input from older crap models, you cannot consider their output reliably right.
- visarga 4y agoI have high hopes in dataset engineering. Scaling the model is already reaching the limits of hardware and funding, the training corpus is almost all the text written on the internet (by some accounts just 10% of it, so maybe we got a couple more years left there). So then what can we do? We turn to the quality of the data. Input examples can be enriched, facts can be checked for support. Based on existing corpus of text we can generate new training examples. Some of them, like code, can also contain execution data and tests.
- gibolt 4y agoInformation is not just the public internet. Every text message, school paper, phone call, and recording of an individual could be future input. Then expand that to photos and videos. Then consume all of YouTube. Each of these sources is still rapidly growing. We are far from drying up the well of the human experience.
- smackeyacky 4y agoWe are a long way from capturing the richness of human experience. Your assumption is that everybody records everything all the time but they don't. Like linkedin never has anybody failing at anything. It's going to end up a simulation of what people think other people want to hear, rather than what actually happened. The experiment has already failed because the assumption that the internet has captured everything is wrong.
- gibolt 4y agoAsserting that all our audio won't be captured is yet another assumption. The amount of data being collected is ever increasing, and it is conceivable that near unlimited microphone/camera access and storage could lead to this in the near future. Either way, it isn't static or decreasing.
- berkle4455 4y ago> It just doesn't know about the war in Ukraine. Neither does most of Russia for that matter.
- braymundo 4y agoThey do, and they (russians) mostly support it. It's not putin's war.
- vasco 4y agoThe war in Ukraine started in 2014.
- typon 4y agothat goes against the narrative
- PoignardAzur 4y agoThis is more inflammatory than it is informative.
- capableweb 4y agoHow so? Inflammatory would mean it's either irrelevant, unfair or harmful in some way. Stating a correction to a fact is hardly either. If someone stated that the attack on Pearl Harbor happened in 1945 and someone else corrected them that it happened in 1941, is that inflammatory? Edit: Maybe you're referring to it being called a "war" in the first place is inflammatory? I'd give you that, and we'd be better off calling it a conflict as none of the parties formally declared war, but then we start veering into semantics instead...
- weberer 4y agoThe parent commenter is correct. You're the one mistaken. https://en.wikipedia.org/wiki/Russo-Ukrainian_War https://en.wikipedia.org/wiki/Russo-Ukrainian_War
- jackmott42 4y agoThis is a form of pedantry that people consider insufferable. The reason is because everyone is well aware that the Ukraine was in conflict with russia since 2014, and even before really. Imagine your irritation at someone pointing that out, when you already know it also. We are all comfortable calling the recent full scale invasion "The war".
- kolinko 4y agoBefore 2022 it was annexation and supplying separatists in Donbas. And sending some special ops. It was a conflict, but not The same way Europe is in conflict with Russia now - we send supplies and all the support we can do without starting an actual war. I love in Poland, spoke with many Ukrainians, and these seem to be commonly accepted definitions.
- ThrowawayTestr 4y agoMaybe the next step in ML will be teaching them how to forget.
- caddemon 4y agoI think continual learning (and incidentally curriculum learning) are going to be some of the more important subtopics within deep learning pretty soon. These things seem under-researched to me right now, despite the fact it's a space where a small academic lab could still compete. I think part of the problem is most papers are supposed to show an engineering feat or a proven mathematical result. Not enough room for developing knowledge on how networks learn scientifically IMO.