5 ms·
Paper2Code: Automating Code Generation from Scientific Papers
- colkassad 1y agoIt would be neat to run their pdf through their implementation[1] and compare results. https://github.com/going-doer/Paper2Code https://github.com/going-doer/Paper2Code
- endofreach 1y agoDamn, i was hoping the link was your result of that. Please do that. I can't start another project currently. But i'd love the short result as an anecdote. But if you don't do it, i might have to. Please let me know. Great idea, really.
- omneity 1y agoIf I was the paper author I would have done it and include the results as an appendix or a repo.
- JackYoustra 1y agohaha would that itself be a product of the paper then?
- omneity 1y agoMaybe by doing it enough times o3-mini will end up reimplementing itself?
- endofreach 1y agoImagine, this was actually the consequence— against all odds. The true power of AI is about to be discovered... through this silly experiment... and you are the one... all you gotta do— is do it. And imagine you don't do it, because you think it can't lead to such a serious result... and if we miss this great leap forward... go, throw your life away and do this. now. the universe is waiting on you, my friend.
- barotalomey 1y agoThat's how nothing ever works outside fiction.
- colkassad 1y agoIt's be something like this that creates the Singularity, just you see
- protolyticmind 1y agoI thought it would be humorously ironic :D
- polygot 1y agoI decided to do that, and made Paper2Code2Code: https://github.com/alexyorke/Paper2Code2Code/tree/main https://github.com/alexyorke/Paper2Code2Code/tree/main
- IshKebab 1y agoAnd... does it work?
- bjourne 1y agoIt relies on OpenAI's o3-mini model which (I think) you have to pay for.
- sitkack 1y agoI have had good results doing bidirectional programming in Tex <=> Python.
- somethingsome 1y agoCan you give more details? I'm curious
- sitkack 1y agoGive it a try, have it teach you tex for the summation notation, have it write the code, modify the code, have it translate back to tex. Repeat. You can do a quick test to see which models have been trained on tex. Keep a tex visualizer handy.
- somethingsome 1y agoI like the idea of having automatic code creation from papers, but I’m scared of it. Suppose you get a paper, you automatically implement the code, and then modify it a bit with a novel idea, and publish your paper. Then somebody else does that with your paper, and does the same.. at some point, we will have a huge quantity of vibe coded code on github, and two similar papers will have very different underlying implementations, so hard to reason about and hard to change. From a learning perspective, you try to understand the code, and it's all spaghetti, and you loose more time understanding the code than it would take to just reimplement it. You also learn a lot by not only reading the paper but reading the authors code where most of the small details reside. And I'm not even talking about the reliability of the code, test to know that it's the correct implementation. Authors try to make papers as close as possible to the implementation but sometimes subtle steps are removed, sometimes from inadvertance, sometimes because the number of pages is lionmited. A paper and an implementation are not one-to-one mappings
- tomrod 1y ago> we will have a huge quantity of vibe coded code on github That may actually be an improvement over much of the code that is generated for papers.
- Narew 1y agoHonestly, the code from my interns have greatly improve since they use AI. And there is lots of really ugly and hard to read code from papers. So I don't think it will be an obvious loss of readability to have code completely generated by AI :)
- somethingsome 1y agoVery interesting, do you have a specific approach to educate them in how to use LLMs? Or they do it free wheel? Do you give advices? If so which kind? I would love to have a structured approach to help students learn to use better LLMs. What I have observed (for uni students) is that they produce better code overall, but have no idea how it works, and would not be able to reproduce it without LLMs. (this is from first to last year)
- ks2048 1y agoSo who has a code2paper model that we can hook up in a loop?
- brundolf 1y agoNot what OP is about, but idea I just had: We should have the technology now to hand-write pseudocode on a piece of paper (or whiteboard or chalkboard), and have it translated and executed. Maybe you even hook up a projector, and project the output back onto the board
- wzdd 1y agoI did this recently with a forward-mode AD paper, by just pasting the PDF into Claude. Like everyone, I've had mixed results with Claude coding, so I wouldn't bet my life on the output, but Claude was able to produce something for Pytorch that worked first go, had appropriate performance characteristics, and it was able to convincingly explain the connection between parts of the generated code and the paper. I was impressed.
- joejoo 1y ago[dead]