4 ms·
I get that this is technically interesting, for certain, but the sheer amount of energy and associated global warming risk needed to do something with >=99% acc
by Sophira 8mo ago
I get that this is technically interesting, for certain, but the sheer amount of energy and associated global warming risk needed to do something with >=99% accuracy that we've been able to do easily for decades with a guaranteed 100% accuracy seems to me to be wasteful to the extreme.
- coolsunglasses 8mo ago>Hacker News not any more, eh?
- thereisnospork 8mo agoYou need to recalibrate your sense of scale if you think that this is a geologically relevant usage of energy.
- nradov 8mo agoWait until you see the quantum computer that it takes to factor the integer 15.
- vrighter 8mo ago* to println("3") It didn't even run the full algorithm, but a compiled version that requires knowledge of the result beforehand. They've also factored 21... it also has a factor of 3.
- Lerc 8mo agoWhat would be an acceptable amount of energy to spend on something that someone has done in a different manner before? Would you rather we stick with all of the current known ways to do things. Does this boil down to a condemnation of all scientific endeavours if they use resources? Would it change things if the people who did it enjoyed themselves? Would they have spent more energy playing a first person shooter to get the same degree of enjoyment? How do you make the calculation of the worth of a human endeavour? Perhaps the greater question is why are you making a calculation of the worth of a human endeavour.
- ratmeadow 8mo agoOk I don't really care either way but to play devil's advocate, what exactly is this specific challenge of adding numbers with a transformer model demonstrating/advancing? The pushpack from people, albeit a little aggressive, does have a grain of truth. We're demonstrating that a model which uses preexisting addition instructions can add numbers? I mean yeah you can do it with arbitrarily few parameters because you don't need a machine learning model at all. Not exactly groundbreaking so I reckon the debate is fair. Now if you said this proof of addition opens up some other interesting avenue of research, sure.
- Lerc 8mo ago>what exactly is this specific challenge of adding numbers with a transformer model demonstrating/advancing? Well for starters, it puts the lie to the argument that a transformer can only output examples it has seen before. Performing the calculation on examples that haven't been seen demonstrates generalisation of the principles and not regurgitation. While this misconception persists in a large number of people, counterexamples can always serve a useful purpose.
- qsera 8mo ago>it puts the lie to the argument But it does not, right? You can either show it something, or modify the parameters in a way that resemble the result of showing it something. You can claim that the model didn't see the thing, but that would mean nothing, because you are making the same effect with parameter tweaks indirectly.
- Lerc 8mo agoThat's a counterargument to a different thing. Iteratively measuring loss is a way to reconstruct values. That's trivial to show for a single value If 5 gives you a loss of 2 and 9 gives you a loss of 2 then you know the missing value is 7. A model with enough parameters can memorise the training set in a similar manner. Technically the model hasn't seen that data by direct input either, but the mechanism provides the means to determine the what the data was. In that respect it is reasonable to say the model has seen the data. Performing well on examples not in the training set is doing something else. Any attempt to characterise that as having been seen before negates any distinction between taking in data and reasoning about that data.
- mapontosevenths 8mo ago> the sheer amount of energy and associated global warming risk I think that's one very good reason to make them more efficient, and that's part of the point of contests like this one.
- tovej 8mo agoMaking things more efficient in a market setting just means they're used more. Which means we eventually use more resources with efficient methods, not less.
- userbinator 8mo agoBecause it's fun. Life is meant to be enjoyed. Those who worry about an imaginary risk and live their lives in constant fear have turned into nothing more than machines enslaved by propaganda.