catgary

8 ms·

91.
by catgary 2y ago
I kind of thought Neural Operators were slotting into the some problem domains where FEM is used (based on recent work in weather modelling, cloth modelling, etc) and thought there was some sort of FEM -> NO lineage. Did I completely mis
92.
by catgary 2y ago
Yeah my wife briefly worked with a team on Bing, she said her team was basically operating like horror stories she’d been told about outsourcing sweatshops, complete with verbal abuse from managers and general disrespect from developers for
93.
by catgary 2y ago
Yeah whenever I’ve spoken to people who work on stuff like IREE or OpenXLA they gave me the impression that understanding how to use those compilers/runtimes is an entire job.
94.
by catgary 2y ago
I think tensorflow-datasets and tensorflow-serving are great, but for model development I think most people use JAX and then export it to a tensorflow SavedModel with Orbax.
95.
by catgary 2y ago
There is a team of dedicated people working on Koka. They say the language isn’t production ready, and they don’t seem to be rushing. But I don’t think they’d bother with VSCode/IDE support if they didn’t feel like they were getting cl
96.
by catgary 2y ago
You’re probably right. I just think it’s the only real candidate for a functional language that could enter the zeitgeist like Rust or Swift did, it’s a research language that has been percolating at Microsoft for some time. A new language
97.
by catgary 2y ago
I kind of agree that Haskell missed its window, and a big part of the problem is the academic-heavy ecosystem (everyone is doing great work, but there is a difference between academic and industrial code). I’m personally quite interested in
98.
by catgary 2y ago
Ok, your Haskell example is basically drawing your opponent in the wojack meme and declaring yourself the winner.
99.
by catgary 2y ago
Well, yeah, I don’t know what you expect me to say, it’s sloppy work.
100.
by catgary 2y ago
I feel like Song et al characterized diffusion models as SDEs pretty unambiguously, and it connects to Optimal Transport in a pretty unambiguous manner. I understand the desire to give different perspectives, but once you start using multip
101.
by catgary 2y ago
Right, basically the generalization goes: Lie groups -> Lie group actions -> Lie groupoids. This is not a new observation (in fact Arnold’s fluid mechanics can be rephrased using lie groupoids https://tspace.library.utoront
102.
by catgary 2y ago
Oh, I mean, the most obvious problem with that idea is there’s no way to ensure that whenever you update your weights that they will still satisfy those constraints. This idea has been well studied in mathematical physics, going back to Poi
103.
by catgary 2y ago
I think the point was they (probably) used the abstract math to prove some desirable properties about the trading strategy?
104.
by catgary 2y ago
“A new kind of deep learning.” Edit: I need to think more before I comment, “A new kind of data science” was right there.
105.
by catgary 2y ago
I was definitely on a group chat of ML/category theory people roasting this paper when it came out, I’m sure I wasn’t the only one.
106.
by catgary 2y ago
Ehh, I’m an ML scientist with a PhD in category theory and I really don’t see it going anywhere. The comparison to geometric deep learning is especially misplaced, because I think you’ll find that Bronstein’s school sets things us as a grou
107.
by catgary 2y ago
This is an old-ish article about Julia, but from what I can tell the core issues with autograd were never fixed: https://kidger.site/thoughts/jax-vs-julia/
108.
by catgary 2y ago
I’m only really thinking about ML/scientific computing workflows where all the heavy lifting happens in jax/torch/polars.
109.
by catgary 2y ago
I think you’re right, so it is now as shape polymorphic as any framework that with an XLA backend can be. I work with edge devices, so I have also been experimenting with IREE for deployment, that can handle dynamic shapes (at times, it sto
110.
by catgary 2y ago
It’s part of model export/serialization, it is documented here: https://jax.readthedocs.io/en/latest/export/export.html#supp... Edit: I think you need to look here as well, the Exported objects do in fac
111.
by catgary 2y ago
I’m jealous of your experience with its autograd if it “just worked” for you. It was a huge pain for me to get it to do anything non-trivial.
112.
by catgary 2y ago
Yeah I was pretty enthusiastic about Julia for a year or two, even using it professionally. But honestly, JAX gives you (almost) everything Julia promises and its automatic differentiation is incredibly robust. As Python itself becomes a pr
113.
by catgary 2y ago
Yeah I’m pretty well-versed in free constructions of various algebraic objects and how this would interact with things like a quadratic form, etc, but couldn’t sort out GA (I think the authors of “GA4CS” had a very different sort of compute
114.
by catgary 2y ago
Right, but you get that is really nothing like coaching high school math students, right?
115.
by catgary 2y ago
I don’t think there’s any real cost to failure in this case - the worst case scenario is a high school student does a bit more extracurricular mathematics before going into a different field, where they will likely still benefit from that e
116.
by catgary 2y ago
I generally think math competitions do more harm than good for math outreach, especially if only 60(!!) students for a country the size of the US are being given any actual development after the initial tests.
117.
by catgary 2y ago
Eh, I think that’s setting students up for failure once they enter graduate studies or more open ended problems that don’t come from a problem bank. Productive struggle is a perfectly valid approach to teaching, it’s just less pleasant in t
118.
by catgary 2y ago
There’s a middle ground in animation where you use techniques like text2motion with pre-existing assets that, I think, has a lot of appeal for games.
119.
by catgary 2y ago
I think this is pretty on par with telling someone they need to understand category theory if they’re playing around with Haskell/FP. I have lots of experience using quaternions in robotics/animation, but I never found geometric a
120.
by catgary 2y ago
I’m ready to switch over to fleet for my ML/data science work load once they support plugins (so I can use Ruff) and Jupyter notebooks (the notebooks are mainly for debugging).

More ›