13 ms·
How to fit any dataset with a single parameter
- aappleby 5y agovery silly. :)
- zwieback 5y agomore digits than I care for
- asimjalis 5y agoThe word “single parameter” is doing a lot of work in this statement.
- llimllib 5y agointentionally: > In addition to casting doubts on the validity of parameter-counting methods and highlighting the importance of complexity bounds based on Occam’s razor such as minimum description length (that trade off goodness-of-fit with expressive power), we hope that fα may also serve as entertainment for curious data scientists
- jldugger 5y ago>The purpose of this paper is to show that all the samples of any arbitrary dataset X can be reproduced by a simple differentiable equation: fα(x) = sin^2 (2^xτ arcsin √α) So... use a PRNG?
- andi999 5y agoDoesn't this exclude negative numbers?
- caterama 5y agoThis is beautiful. The real numbers are uncountable, so seems intuitive that could approximate any other space with them... basically a hash function. But this one has an inverse! It's so cool that someone actually implemented this.
- Sharlin 5y agoYou don't need uncountability. Any nontrivial interval of rational numbers contains every finite string that exists, in fact infinite copies of each!
- blauditore 5y agoIt's not about uncountability, but simply that their space is infinite. The same would work with (unbounded) integer numbers.
- bflesch 5y agoThis sounds like it could explain many ML models ;)
- mr_toad 5y agoIt’s actually possible in theory for part of a neural net to simply encode the training data and use it to regurgitate responses. In practice, more efficient parts of the network should win in the end.
- NavinF 5y agoHas nobody in this thread heard of https://en.m.wikipedia.org/wiki/Cross-validation_(statistics) https://en.m.wikipedia.org/wiki/Cross-validation_(statistics... Every ml tutorial start with splitting your data in into a training set and validation set.
- idiotsecant 5y agoFirst, this is a fun implementation and I love it. Second, you could as easily embed an infinite size dataset into an infinitely long binary string and say that you've reproduced your dataset with a 'single' parameter! That's sort of what this is doing, with some extra steps.
- laumars 5y agoYour second point is what I first assumed this paper was doing before I read it.
- cobookman 5y agoUp until ARG_MAX is reached, or you run out of RAM that is... https://serverfault.com/a/844666 https://serverfault.com/a/844666
- hnlmorg 5y agoThis reminds me of the fun thought experiment of using /dev/random as storage. Given an infinite amount of time you'll find every file you need.
- kirubakaran 5y agoInfinite storage, horrendous seek time.
- amelius 5y agoThe index might need more bits than the file.
- hnlmorg 5y agoConsiderably more in all probability :)
- alisonkisk 5y agoπfs https://github.com/philipl/pifs https://github.com/philipl/pifs
- kzrdude 5y agoHow to have fun with mpmath :)
- civilized 5y agoToo easy. Now fit any dataset using a substring of digits from the binary expansion of pi
- air7 5y agoThis reminds me of a joke idea I read somewhere: You can encode the entire Encyclopaedia Britannica using a single mark on a simple stick! Just encode the text as a ascii codes after the decimal dot of a zero. (0.656168.. etc). Then just mark that ratio of the sticks length and you're done...
- thehappypm 5y agoYou can do it easier than that -- just make a stick of length .65168 meters. (Same idea, just the "full 1 meter stick" is defined elsewhere, not by the length of the stick).
- tshaddox 5y agoThat’s way harder, because you have to cut a stick to write and have a meter ruler to read.
- gh02t 5y agoAnd of course in a footnote of the documentation for this encoding "Sufficiently precise measurement of this mark left as an exercise to the reader."
- mjburgess 5y agoI don't see this as a joke, but a radical and important point. Reality, in being geometrical, is infinitely informationally dense (with a discrete conception of information). This distinction between geometrical space and time, and discrete algorithmic computability is unbridgeable. And hence there is an extemely firm footing on which to reject: AI, brain scanning readers, teleporters, etc and most sci-fi computationalim. Almost nothing can be simulated, as in, realised by a merely computational system.
- idiotsecant 5y agoReality isn't quite infinitely dense though, it's hard to imagine how you could encode data geometrically over a distance smaller than the planck length, for example. Reality just has a very, very fine resolution.
- michaelcampbell 5y agoThis reminds me of the "worst way to ask a user for a telephone number" UI, after one UI forced the user to select each digit in a 0-9 dropdown. The specific one I'm thinking of is just spit out a scrollable numeric string of Pi, and make the user scroll until their phone number was the digits of Pi that matched it. My (7 digit) phone number occurs after digit position 9_000_000, and occurs 21 times in the first 200_000_000 digits.
- albertzeyer 5y agoI think it is still not proven that Pi contains all possible strings. https://math.stackexchange.com/questions/216343/does-pi-contain-all-possible-number-combinations https://math.stackexchange.com/questions/216343/does-pi-cont...
- eru 5y agoBut it contains all possible phone numbers. (Assuming that phone numbers are less than, say, 100 digits.)
- throwawaytemp27 5y agoI wonder if you gave away your phone number there? Or how many 7 digit numbers match those criteria?
- michaelcampbell 5y agoMy phone number is already "out" in terms of being on the internet, and I can change it pretty easily if it became an ACTUAL security issue. But it would be an interesting exercise to do, I guess.
- genewitch 5y ago7 digits is barely a phone number in most of the United States anymore, most places outside of rural areas require 10 digit dialing and hide it behind carrier dial plans - I mean in the way a rural person can dial one, two, or three digits because all land lines in an area have the same area code, prefix, and possibly additional digits in common.
- cs702 5y agoFun read. But if we allow ourselves as much precision as we need, we don't even need a parameter. Any constant that is a normal number should suffice. Such constants already contain every possible sequence of digits you could muster -- i.e., they already contain every possible dataset. EDIT: I replaced "transcendental" with "normal" after reading Scarblac's comment below: https://news.ycombinator.com/item?id=28699622 https://news.ycombinator.com/item?id=28699622 -- many important transcendental numbers, including π (Pi), are thought to be (but have not been proven to be!) normal.
- Scarblac 5y agoI don't believe just being transcendental is sufficient for that property. E.g. I can't imagine that pi with all the '1' digits replaced by '2' isn't transcendental, but it clearly doesn't contain every sequence of digits. I think you mean normal numbers.
- cs702 5y agoYou're right: normal, not transcendental. IIRC, most real numbers are normal, and many important transcendental numbers are thought to be (but have not been proven to be) normal. I edited my comment. Thank you for the correction!
- amelius 5y ago> most real numbers are normal Which means that most real numbers violate all present and future copyrights.
- KerrickStaley 5y agoIt's not true that any transcendental number would work. A transcendental number is a number which isn't the root of any polynomial with rational coefficients. This property doesn't imply that it contains all number sequences. It hasn't even been proven that pi contains all number sequences. See https://math.stackexchange.com/questions/216343/does-pi-contain-all-possible-number-combinations https://math.stackexchange.com/questions/216343/does-pi-cont... for more details.
- axegon_ 5y agoHypothetically you could achieve something similar using this[1]. Note the word "hypothetically". [1] https://github.com/cakenggt/Library-Of-Pybel https://github.com/cakenggt/Library-Of-Pybel
- darawk 5y agoI've thought about this for a while. I didn't go to the trouble of actually inventing the function, but just the fact that it ought to be possible in principle is sufficient to change your perspective on some things. Most notably, parameter count based information criteria. AIC, BIC, et al. need to be reformulated for each model in which they're used, and not all parameters can be treated as fungible.
- deleted 5y ago[deleted]
- a-dub 5y agothis made me smile. :) oh, uncountable infinities.
- shmageggy 5y agoPrior work: "One parameter is always enough", Piantadosi (2018) https://www.google.com/search?q=one%20parameter%20piantadosi https://www.google.com/search?q=one%20parameter%20piantadosi Haven't read the posted article but sounds like the same idea and motivation
- shoto_io 5y agoIt is the first reference in this article. So I guess they have built on it.
- ww520 5y agoLet me guess before reading the article. Infinite precision float point number?
- lnyan 5y agoIt reminds me of the pifs [1], which stores data in π - [1] πfs - the data-free filesystem https://github.com/philipl/pifs https://github.com/philipl/pifs
- Tomminn 5y agoI love this as a verifiable implementation of a mathematically trivial, but easy to forget point. So I'm sad that when I tried to recreate the elephant scatter plot, I haven't been able to. Anyone find exact parameters that work for tau and alpha? Just wish they'd given a complete list of the decimal places for those animals. It would've been something to plot them yourself.
- pietroppeter 5y agoIn the paper you find a link to a GitHub repo that should have all results being reproducible
- pietroppeter 5y agoAnd here is the relevant notebook for animal shapes: https://github.com/Ranlot/single-parameter-fit/blob/a065af75b1a1885de297aba0d39f0ae33a098ee1/animalShapes.ipynb https://github.com/Ranlot/single-parameter-fit/blob/a065af75...
- Tomminn 5y agoEpic, thank you:)
- felixleungsc 5y agoNow I see why the elephant. From the references: > [2] “I’m not very impressed with what you’ve been doing.” As recounted the famous physicist Freeman Dyson himself, this is how Nobel laureate Enrico Fermi started their 1953 meeting. “Well, what do you think of the numerical agreement?”, Dyson countered. To which, Fermi replied “You know, Johnny von Neumann always used to say: With four parameters I can fit an elephant, and with five I can make him wiggle his trunk. So I don’t find the numerical agreement impressive either.”
- ur-whale 5y agoThe question being of course: what happens when you lerp of two such "single parameters".
- raoa 5y agoWould this be a good compression tool in any instances?