8 ms·
Francois Chollet is leaving Google
- tadeegan 2y agoI guess they realized muilti-backend keras is futile? I never liked the tf.keras apis and the docs always promosed multi backend but then I guess they were never able to deliver that without breaking keras 3 changes. And even now.... "Keras 3 includes a brand new distribution API, the keras.distribution namespace, currently implemented for the JAX backend (coming soon to the TensorFlow and PyTorch backends)". I don't believe it. They are too different to reconcile under 1 api. And even if you could, I dont really see the benefit. Torch and Flax have similar goals to Keras and are imo better.
- modeless 2y agoWhy would you interpret this as Google disliking Keras? Seems a lot more likely he was poached by Anthropic.
- blackeyeblitzar 2y agoWhere did you see that he was poached by Anthropic?
- modeless 2y agoI am not suggesting that I know it for a fact. I do recall some speculation on X to that effect but I can't find it now. Maybe just because Anthropic has been getting a lot of people lately.
- modeless 2y agoMystery solved: he's founding a startup: https://x.com/fchollet/status/1857012265024696494 https://x.com/fchollet/status/1857012265024696494
- hedgehog 2y agoMulti-backend Keras was great the first time around and it might be a more widely used API today if the TF team hadn't pulled that support and folded Keras into TF. I'm sure they had their reasons but I suspect that decision directly increased the adoption of PyTorch.
- fchollet 2y agoActually, `keras.distribution` is straightforward to implement in TF DTensor and with the experimental PyTorch SPMD API. We haven't done it yet first because these APIs are experimental (only JAX is mature) and second because all the demand for large-model distribution at Google was towards the JAX backend.
- max_ 2y agoI wonder what he will be working on? Maybe he figured out a model that beats ARC-AGI by 85%?
- trott 2y ago> Maybe he figured out a model that beats ARC-AGI by 85%? People have, I think. One of the published approaches (BARC) uses GPT-4o to generate a lot more training data. The approach is scaling really well so far [1], and whether you expect linear scaling or exponential one [2], the 85% threshold can be reached, using the "transduction" model alone, after generating under 2 million tasks ($20K in OpenAI credits). Perhaps for 2025, the organizers will redesign ARC-AGI to be more resistant to this sort of approach, somehow. --- [1] https://www.kaggle.com/competitions/arc-prize-2024/discussion/543953#3036186 https://www.kaggle.com/competitions/arc-prize-2024/discussio... [2] If you are "throwing darts at a board", you get exponential scaling (the probability of not hitting bullseye at least once reduces exponentially with the number of throws). If you deliberately design your synthetic dataset to be non-redundant, you might get something akin to linear scaling (until you hit perfect accuracy, of course).
- thrw42A8N 2y ago> If you are "throwing darts at a board", you get exponential scaling (the probability of not hitting bullseye reduces exponentially with the number of throws). Honest question - is that so, and why? I thought you have to calculate the probability of each throw individually as nothing fundamentally connects the throws together, only that long term there will be a normal distribution of randomness.
- trott 2y ago> The probability of not hitting bullseye at least once ... I added a clarification.
- fastball 2y agoI like the idea of ARC-AGI and think it was worth a shot. But if someone has already hit the human-level threshold, I think the entire idea can be thrown out. If the ARC-AGI challenge did not actually follow their expected graph[1], I see no reason to believe that any benchmark can be designed in a way where it cannot be gamed. Rather, it seems that the existing SOTA models just weren't well-optimized for that one task. The only way to measure "AGI" is in however you define the "G". If your model can only do one thing, it is not AGI and doesn't really indicate you are closer, even if you very carefully designed your challenge. [1] https://static.supernotes.app/ai-benchmarks-2.png https://static.supernotes.app/ai-benchmarks-2.png
- minimaxir 2y agoGenuine question: who is using Keras in production nowadays? I've done a few work projects in Keras/TensorFlow over the years and it created a lot of technical debt and lost time debugging it, with said issues disappearing once I switched to PyTorch. The training loop with Keras for simple model is indeed easier and faster than PyTorch oriented helpers (e.g. Lightning AI, Hugging Face accelerate) but much, much less flexible.
- dools 2y agoFTA "With over two million users, Keras has become a cornerstone of AI development, streamlining complex workflows and democratizing access to cutting-edge technology. It powers numerous applications at Google and across the world, from the Waymo autonomous cars, to your daily YouTube, Netflix, and Spotify recommendations."
- mistrial9 2y agosure -- all true in 2018; right about then pyTorch passed TensforFlow in the raw numbers of research papers using it.. grad students later make products and product decisions.. currently, pyTorch is far more popular, the bulk of that is with LLMs source: pyTorch Foundation, news
- paxys 2y agoThe existence of a newer, hotter framework doesn't mean all legacy applications in the world instantly switch to it. Quite the opposite in fact.
- magicalhippo 2y agoAs someone who hasn't really used either, what's pytorch doing that's so much better?
- jwjohnson314 2y agoPyTorch is just much more flexible. Implementing a custom loss function, for example, is straightforward in PyTorch and a hassle in Keras (or was last time I used it, which was several years ago).
- _giorgio_ 2y ago[flagged]
- deleted 2y ago[deleted]
- ClassyJacket 2y agoKeras is not a language
- _giorgio_ 2y agoKeras eras random asymptotic sh*
- deleted 2y ago[deleted]
- synergy20 2y agoI read somewhere TF will not be developed actively down the road, Google switched to JAX internally and TF pretty much lost the war to Pytorch.
- sakex 2y agoJax is really nice
- geor9e 2y agoIf I were to speculate, I would guess he quit Google. 2 days ago, his $1+ million Artificial General Intelligence competition ended. Chollet is now judging the submissions and will announce the winners in a few weeks. The timing there can't be a coincidence.
- paxys 2y agoMore generally, there is unlimited opportunity in the AI space today, especially for someone of his stature, and staying tied to Google probably isn't as enticing. He can walk into any VC office and raise a hundred million dollars by the end of the day to build whatever he wants.
- hiddencost 2y ago$100M isn't enough capital for an AI startup that's training foundation models, sadly. A ton of folks of similar stature who raised that much burnt it within two years and took mediocre exits.
- versteegen 2y agoChollet is a leading skeptic of the generality of LLMs (see arcprize.org). He surely isn't doing a startup to train another one.
- NitpickLawyer 2y agoI think we'll start to see a differentiation soon. The likes of Ilya will raise money to do whatever, including foundation models / new arch, while other startups will focus on post-training, scaling inference, domain adaptation and so on. I don't think the idea of general foundational model from scratch is a good path for startups anymore. We're already seeing specialised verticals (cursor, codeium, both at ~100-200m funding rounds) and they're both focused on specific domains, not generalist. There's probably enough "foundation" models out there to start working on post-training stuff already, no need to reinvent the wheel.
- zxexz 2y ago
- bearcollision 2y agoI've always wondered how fchollet had authority to force keras into TF... https://github.com/tensorflow/community/pull/24 https://github.com/tensorflow/community/pull/24
- tbalsam 2y agoI remember this post as the day that Keras died. Very strange political powerplay on the part of fchollet, and did immeasurable damage to the community and code that used TF, not just in that PR but also in the precedent it set for other stuff. People legitimately were upset by the attempt to move tensorflow under an unnecessary Keras namespace, and he locked the PR and said that Reddit was brigading it (despite it being pretty consistently disliked as a change, among other changes). People tried to reason with him in the PR thread, but to no avail, the Keras name had to live on, whether or not TF died with it (and it very well did, unfortunately). There were other things working against TF but this one seemed to be the final nail in the coffin, from what I can tell. I ended up minimizing engagement with the work he's done since as a result.
- choppaface 2y agonotably that link shows “@tensorflow tensorflow deleted a comment from fchollet on Nov 21, 2018” as well as other deleted comments
- osm3000 2y agoI loved Keras at the beginning of my PhD, 2017. But it was just the wrong abstraction: too easy to start with, too difficult to create custom things (e.g., custom loss function). I really tried to understand TensorFlow, I managed to make a for-loop in a week. Nested for-loop proved to be impossible. PyTorch was just perfect out of the box. I don't think I would have finished my PhD in time if it wasn't for PyTorch. I loved Keras. It was an important milestone, and it made me believe deep learning is feasible. It was just...not the final thing.
- rd11235 2y ago> it was just the wrong abstraction: too easy to start with, too difficult to create custom things Couldn’t agree with this more. I was working on custom RNN variants at the time, and for that, Keras was handcuffs. Even raw TensorFlow was better for that purpose (which in turn still felt a bit like handcuffs after PyTorch was released).
- hooloovoo_zoo 2y agoKeras was a miracle coming from writing stuff in Theano back in the day though.
- singhrac 2y agoAnd PyTorch was a miracle after coming from LuaTorch (or Torch7 iirc). We’ve made a lot of strides over the years.
- V1ndaar 2y agoI didn't realize Keras was actually released before Tensorflow, huh. I used Theano quite a bit in 2014 and early 2015, but then went a couple years without any ML work. Compared to the modern libraries Theano is clunky, but it taught one a bit more about the models, heh.
- blaufuchs 2y agoWow that gives me flashbacks to learning Theano/Lasagne, which was a breath of fresh air coming from Caffe. Crazy how far we've come since then.
- _giorgio_ 2y ago[flagged]
- uptownfunk 2y agoARC is a bigger contribution than Keras. It’s great he had two major contributions. I can’t wait to see how they crack ARC
- tgma 2y agoStrange. Had never read blog posts about individual engineers leaving Google on official Google Developers Blog before. Is this a first? Every day someone prominent leaves Google... Sounds like a big self-own if Google starts to post this kind of stuff. Looks like sole post by either of the (both new to Google) authors in the byline.
- mi_lk 2y agoSame, what point does the post serve? And it's not like Keras is the hottest thing in DL world.
- tgma 2y agoEven if it were, the article is written like a farewell email the employee sends to their group, not from Keras standpoint. I bet a couple rando VPs are writing self-promotional material to increase their visibility and they had nothing better to publish. Both are only 1yr in there. Google needs a DOGE (Department of Google Efficiency).
- 12345hn6789 2y agoGoogle is no longer the hot place to be. These blog posts are just soft launches of the engineers new companies. They're googlers, they know you gotta repeat yourself over and over to get mind share going :)
- fchollet 2y agoHi HN, Francois here. Happy to answer any questions! Here's a start -- "Did you get poached by Anthropic/etc": No, I am starting a new company with a friend. We will announce more about it in due time! "Who uses Keras in production": Off the top of my head the current list includes Midjourney, YouTube, Waymo, Google across many products (even Ads started moving to Keras recently!), Netflix, Spotify, Snap, GrubHub, Square/Block, X/Twitter, and many non-tech companies like United, JPM, Orange, Walmart, etc. In total Keras has ~2M developers and powers ML at many companies big and small. This isn't all TF -- many of our users have started running Keras on JAX or PyTorch. "Why did you decide to merge Keras into TensorFlow in 2019": I didn't! The decision was made in 2018 by the TF leads -- I was a L5 IC at the time and that was an L8 decision. The TF team was huge at the time, 50+ people, while Keras was just me and the open-source community. In retrospect I think Keras would have been better off as an independent multi-backend framework -- but that would have required me quitting Google back then. Making Keras multi-backend again in 2023 has been one of my favorite projects to work on, both from the engineering & architecture side of things but also because the product is truly great (also, I love JAX)!
- blixt 2y agoWill you come back to Europe?
- fchollet 2y agoI will still be US-based for the time being. I'm seeing great things happening on the AI scene in Paris, though!
- hashtag-til 2y agoCongratulations Francois! Thanks for maintaining Keras for such a long time and overcoming the corporate politics to get it where it is now. I've been using it since early 2016 and it has been present all my career. It is something I use as the definitive example of how to do things right in the Python ecosystem. Obviously, all the best wishes for you and your friend in the new venture!!
- 2y ago
- kreyenborgi 2y agoHis tweets are quite interesting. E.g. https://x.com/fchollet/status/1638057646602489856 https://x.com/fchollet/status/1638057646602489856 https://x.com/fchollet/status/1840486105118015901 https://x.com/fchollet/status/1840486105118015901 https://x.com/fchollet/status/1845103528806662258 https://x.com/fchollet/status/1845103528806662258
- MasterScrat 2y agoVery insightful to have a number from him here: > LLMs are trained on much more than the whole Internet -- they also consume handcrafted answers produced by armies of highly qualified data annotators (often domain experts). Today approximately 20,000 people are employed full-time to produce training data for LLMs.
- Skylyz 2y agoThese are comforting for sure if you're scared about your future as a SWE.
- retinaros 2y agoFrancois good luck with your new beginning. Your book (and aurelien one) greatly helped me entering this field
- knbknb 2y agoDo you plan to write more "fundamental" AI papers such as "On the measure of intelligence", do you plan to refine your ARC-AGI benchmark again?
- fchollet 2y agoYes to both.
- flamby54 2y agoHello François, thank you for your great work to the Open Source community. Aren't you worried that your work may only be profitable to some US-based interests that may backfire to your home country ? given the actual political situation... France needs you, come back home. This is not a judgment, just wondering about your opinion on it.
- sidcool 2y agoIs PyTorch genuinely so good that most people have stopped using Keras/TF?
- Skylyz 2y agoHi ! Thanks for ARC it's lots of fun. Did you think about expanding ARC beyond the current 32x32 relatively low dsl depth format ? Do you think there's anything to gain from it ?