10 ms·
Fitting an elephant with four non-zero parameters
- EdwardCoffin 2y agoFreeman Dyson recounts the episode [1] that inspired this paper in his web of life interviews (prepositioned to the fitting an elephant bit) [2] [1] https://youtu.be/hV41QEKiMlM https://youtu.be/hV41QEKiMlM [2] https://youtu.be/hV41QEKiMlM?t=118 https://youtu.be/hV41QEKiMlM?t=118
- parker-3461 2y agoThanks for linking these, I was not very familiar of these works/discussions taking place in the past, but these really helped establish the context. Very grateful that these videos are readily available.
- EdwardCoffin 2y agoI listened to the whole series with Dyson some time in the past year. It was well worth it. I also listened to the series with Murray Gell-Mann [1] and Hans Bethe [2]. All time well worth spending, and I've been thinking of downloading all the bits, concatenating them into audio files, and putting them on my phone for listening to when out on walks (I'm pretty sure the videos do not add anything essential: it's just a video of the interviewee talking - no visual aids). [1] https://www.youtube.com/playlist?list=PLVV0r6CmEsFxKFx-0lsQDs6oLP3SZ9BlA https://www.youtube.com/playlist?list=PLVV0r6CmEsFxKFx-0lsQD... [2] https://www.youtube.com/watch?v=LvgLyzTEmJk&list=PLVV0r6CmEsFyUDSroBQVEcbnNud7I9xom&index=1&t=0s https://www.youtube.com/watch?v=LvgLyzTEmJk&list=PLVV0r6CmEs...
- lazamar 2y agoLol. Loved it. This was a lovely passage from Dyson’s Web of Stories interview, and it struck a chord with me, like it clearly did with the authors too. It happened when Dyson took the preliminary results of his work on the Pseudoscalar theory of Pions to Fermi and Fermi very quickly dismissed the whole thing. It was a shock to Dyson but freed him from wasting more time on it. Fermi: When one does a theoretical calculation, either you have a clear physical module in mind or a rigorous mathematical basis. You have neither. How many free parameters did you use for your fitting? Dyson: 4 Fermi: You know, Johnny Von Neumann always used to say ‘with four parameters I can fit an elephant; and with five I can make him wiggle his trunk’.
- dheera 2y agoI wish there was more humor on arXiv. If I could make a discovery in my own time without using company resources I would absolutely publish it in the most humorous way possible.
- btown 2y agoThere's plenty of humor on arXiv, and that's part of why it's so incredible! Some lists: https://academia.stackexchange.com/questions/86346/is-it-ok-to-upload-joke-papers-to-arxiv https://academia.stackexchange.com/questions/86346/is-it-ok-... https://www.ellipsix.net/arxiv-joke-papers.html https://www.ellipsix.net/arxiv-joke-papers.html
- azeemba 2y agoConsider posting this as a new post! It seems like a fun list to read through
- mananaysiempre 2y agoJoke titles and/or author lists are also quite popular, e.g. the Greenberg, Greenberger, Greenbergest paper[1], a paper with a cat coauthor whose title I can’t seem to recall (but I’m sure there’s more than one I’ve encountered), or even the venerable, unfortunate in its joke but foundational in its substance Alpher, Bethe, Gamow paper[2]. Somewhat closer to home, I think computer scientist Conor McBride[3] is the champion of paper titles (entries include “Elimination with a motive”, “The gentle art of levitation”, “I am not a number: I am a free variable”, “Clowns to the left of me, jokers to the right”, and “Doo bee doo bee doo”) and sometimes code in papers: letmeB this (F you) | you == me = B this | otherwise = F you letmeB this (B that) = B that letmeB this (App fun arg) = letmeB this fun `App` letmeB this arg (Yes, this is working code; yes, it’s crystal clear in the context of the paper.) [1] https://arxiv.org/abs/hep-ph/9306225 https://arxiv.org/abs/hep-ph/9306225 [2] https://en.wikipedia.org/wiki/Alpher%E2%80%93Bethe%E2%80%93Gamow_paper https://en.wikipedia.org/wiki/Alpher%E2%80%93Bethe%E2%80%93G... [3] http://strictlypositive.org/ http://strictlypositive.org/
- 2y ago
- boywitharupee 2y agowhat's the purpose of this? is it one of those 'fun' problems to solve?
- jfoutz 2y agoThis quote might help - https://en.wikipedia.org/wiki/Von_Neumann%27s_elephant#History https://en.wikipedia.org/wiki/Von_Neumann%27s_elephant#Histo... yes, a fun problem, but also a criticism of using to many parameters.
- Steuard 2y agoSadly, the constant term (the average r_0) is never specified in the paper (it seems to be something in the neighborhood of 180?): getting that right is necessary to produce the image, and I can't see any way not to consider it a fifth necessary parameter. So I don't think they've genuinely accomplished their goal. (Seriously, though, this was a lot of fun!)
- rsfern 2y agoThey say in the text that it’s the average value of the data points they fit to. I think whether to count it as a parameter depends on whether you consider standardization to be part of the model or not
- Steuard 2y agoI see your point, that it's really just an overall normalization for the size rather than anything to do with the shape. I can accept that, and I'll grant them the "four non-zero parameters" claim. Though in that case, I would have liked for them to make it explicit. Maybe normalize it to "1", and scale the other parameters appropriately. (Because as it stands, I don't think you can reproduce their figure from their paper.)
- lupire 2y agoIIUC: A real-parameter (r(theta) = sum(r_k cos(k theta))) Fourier series can only draw a "wiggly circle" figure with one point on each radial ray from the origin. A compex parameter (z(theta) = sum(e^(z_ theta))) can draw more squiggly figures (epicycles) -- the pen can backtrack as the drawing arm rotates, as each parameter can move a point somewhere on a small circle around the point computed from the previous parameter (and recursively). Obligatory 3B1B https://m.youtube.com/watch?v=r6sGWTCMz2k https://m.youtube.com/watch?v=r6sGWTCMz2k Since a complex parameter is 2 real parameters, we should compare the best 4-cosine curve to the best 2-complex-exponential curve.
- elijahbenizzy 2y agoThis is humorous (and well-written), but I think its more than that. I'm always making the joke (observation) that ML (AI) is just curve-fitting. Whether "just curve-fitting" is enough to produce something "intelligent" is, IMO, currently unanswered, largely due to differing viewpoints on the meaning of "intelligent". In this case they're demonstrating some very clean, easy-to-understand curve-fitting, but it's really the same process -- come up with a target, optimize over a loss function, and hope that it generalizes, (this one, obviously, does not. But the elephant is cute.) This raises the question Neumann was asking -- why have so many parameters? Ironically (or maybe just interestingly), we've done a lot with a ton of parameters recently, answering it with "well, with a lot of parameters you can do cool things".
- visarga 2y ago> Whether "just curve fitting" is enough to produce something "intelligent" is, IMO, currently unanswered Continual "curve fitting" to the real world can create intelligence. What is missing is not something inside the model. It's missing a mechanism to explore, search and expand its experience. Our current crop of LLMs ride on human experience, they have not largely participated in creating their own experiences. That's why people call it imitation learning or parroting. But once models become more agentic they can start creating useful experiences on their own. AlphaZero did it.
- maitola 2y agoI love the ironic side of the article. Perhaps they should add the reason for it, from Fermi's and Neumann's. When you are building a model of reality in Physics, If something doesn’t fit the experiments, you can’t just add a parameter (or more) variate it and fit the data. The model should have zero parameters, ideally, or the least possible, or, even at a more deeper level, the parameters should emerge naturally from some simple assumptions. With 4 parameters you don’t know whether you are really capturing a true aspect of reality of just fitting the data of some experiment.
- jampekka 2y agoThis was mentioned in the first paragraph of the paper. The paper is mostly humoristic. That said, the wisdom of the quip has been widely lost in many fields. In many fields data is "modeled" with huge regression models with dozens of parameters or even neural networks with billions of parameters. > In 1953, Enrico Fermi criticized Dyson’s model by quoting Johnny von Neumann: “With four parameters I can fit an elephant, and with five I can make him wiggle his trunk.”[1]. This quote is intended to tell Dyson that while his model may appear complex and precise, merely increasing the number of parameters to fit the data does not necessarily imply that the model has real physical significance.
- karmakaze 2y agoThat's how I feel about dark matter. Oh this galaxy is slower than this other similar one. The first one must have less dark matter then. What can't be fit by declaring the amount of dark matter that must be present fits the data? It's unfalsifiable, just because we haven't found it, doesn't mean it doesn't exist. Even worse than string/M-theory which at least has math.
- edflsafoiewq 2y agoIt's easy to say "Epicycles! Epicycles!", but people are going to continue using their epicycles until a Copernicus comes along.
- bee_rider 2y agoYa know, in academic writing I tend to struggle with making it sound nice and formal. I try not to use the super-stilted academic style, but it is still always a struggle to walk the line between too loose and too jargony. Maybe this sort of thing would be a really good tradition. Everyone must write a very silly article with some mathematical arguments in it. Then, we can all go forward with the comfort of knowing that we aren’t really at risk of breaking new grounds in appearing unserious. It is well written and very understandable!
- xpe 2y agoOne take away: Don’t count parameters. Count bits.
- Scene_Cast2 2y agoBetter yet, count entropy.
- xpe 2y agoWhy “better”? Entropy in the information theoretic sense is usually quantified in bits.
- xpe 2y agoAnother take away (not directly stated in the article but implied): Counting the information content of a model is more than just the parameters; the structure of the model itself conveys information.
- will1am 2y agoI think often underappreciated insight
- pharmacy7766 2y agoOne parameter is enough: https://aip.scitation.org/doi/10.1063/1.5031956 https://aip.scitation.org/doi/10.1063/1.5031956
- danbruc 2y agoThe number of parameters is just the wrong metric, it should be the amount of information contained in the parameter values, their entropy, Kolmogorov complexity or something along that line.
- dweinus 2y ago"This single parameter model provides a large improvement over the prior state of the art in fitting an elephant" Lol
- Nition 2y agoNice. This is like how you can achieve unlimited compression by storing your data in a filename instead of in the file.
- redox99 2y agoThat's like saying your entire hard drive is a single number.
- comex 2y agohttps://en.wikipedia.org/wiki/Illegal_number https://en.wikipedia.org/wiki/Illegal_number
- gowld 2y agohttps://github.com/philipl/pifs https://github.com/philipl/pifs > πfs: Never worry about data again! > πfs is a revolutionary new file system that, instead of wasting space storing your data on your hard drive, stores your data in π! You'll never run out of space again - π holds every file that could possibly exist! They said 100% compression was impossible? You're looking at it!
- zellyn 2y ago
- aqme28 2y ago> It only satisfies a weaker condition, i.e., using four non-zero parameters instead of four parameters. Why would that be a harder problem? In the case that you get a zero parameter, you could inflate it by some epsilon and the solution would basically be the same.
- Sesse__ 2y agoThey also, effectively, fit information in the indexes of the parameters. I.e., _which_ of the parameters are nonzero carries real information. In a sense, they have done their fitting using nine parameters, of which five are zero.
- aqme28 2y agoI didn’t read enough to catch that. How the heck did they justify that?
- nyssos 2y ago> In the case that you get a zero parameter, you could inflate it by some epsilon and the solution would basically be the same. Not everything is continuous. Add an epsilon worth of torsion to GR and you don't get almost-GR, you get a qualitatively different theory in which potentially arbitrarily large violations of the equivalence principle are possible.
- aqme28 2y agoThat's not relevant here though, because their function is continuous and they're fitting to an arbitrary shape. It's not a "perfect science," so there would be wiggle room.
- deleted 2y ago[deleted]
- LisaST 2y ago[dead]
- ggm 2y agoNo love for D'Arcy Thompson on growth and form? His parametric models for organisms were quite nice (if very simplistic) https://en.wikipedia.org/wiki/On_Growth_and_Form https://en.wikipedia.org/wiki/On_Growth_and_Form
- classified 2y agoWhat is that horizontal bar above r0 in the last equation?
- pietroppeter 2y agoLove how they misspelled Piantadosi as Paintadosi :)
- deleted 2y ago[deleted]
- foobarian 2y agoReminds me of an old joke: "What is the difference between an elephant and an aspirin?" - "There isn't any, except the elephant is large, wrinkly and grey."
- tagami 2y agoATCG
- bertm 2y agoI have to plug Dr Octave Levenspiel. Levenspiel was a professor emeritus when I did my undergrad. He did much of the work on industrial fluidized beds among other things. The elephant curve discussions were a criticism of the complex multi-parameter fitting for heterogenous catalysis of the time. https://levenspiel.com/elephants/ https://levenspiel.com/elephants/ He tried for a while to get an aerodynamics paper published on the flight of dinosaurs. http://levenspiel.com/wp-content/uploads/2016/02/DinosaurW.pdf http://levenspiel.com/wp-content/uploads/2016/02/DinosaurW.p... This intellectual curiosity reminded me a bit of Feynman and his plate spinning.