4 ms·
How many days does it take before the turkey realizes it’s going to get its head cut off on its first thanksgiving? Less glibly I think models will follow the
by arcticbull 3y ago
How many days does it take before the turkey realizes it’s going to get its head cut off on its first thanksgiving?
Less glibly I think models will follow the same sigmoid as everything else we’ve developed and at some point it’ll start to taper off and the amount of effort required to achieve better results becomes exponential.
I look at these models as a lossy compression logarithm with elegant query and reconstruction. Think JPEG quality slider. The first 75% of the slider the quality is okay and the size barely changes, but small deltas yield big wins. And like an ML hallucination the JPEG decompressor doesn’t know what parts of the image it filled in vs got exactly right.
But to get from 80% to 100% you basically need all the data from the input. There’s going to be a Shannon’s law type thing that quantifies this relationship in ML by someone who (not me) knows what they’re talking about. Maybe they already have?
These models will get better yes but only when they have access to google and bing’s full actual web indices.
- drw85 3y agoI don't think access to bing or google solves this problem. Right now, there are many questions that the internet gives unclear answers to. Try to find out if a plant is toxic for cats via google. Many times the results say both yes and no and it's impossible to assume which one is true based on the count of the results. Feeding the models more garbage data will not make the results any better.
- tudorw 3y agoquite, look at Google offering a prize pot for 'forgetting', also, sorry, but typical engineer think that this comes after the creation, like forever plastics or petroleum, for some reason, great engineers often seem to struggle with second and third order consequences or believe externalities to be someone else's problem. Perhaps if they had started with how to forget, they could have built the models from the ground up with this capability, not tacked on after once they realise the volume of bias and wrongness their models have ingested...