8 ms·

Hi HN, Francois here. Happy to answer any questions! Here's a start -- "Did you get poached by Anthropic/etc": No, I am starting a new company with a friend.

• blixt 2y ago
  Will you come back to Europe?
  • fchollet 2y ago
    I will still be US-based for the time being. I'm seeing great things happening on the AI scene in Paris, though!
• hashtag-til 2y ago
  Congratulations Francois! Thanks for maintaining Keras for such a long time and overcoming the corporate politics to get it where it is now. I've been using it since early 2016 and it has been present all my career. It is something I use as the definitive example of how to do things right in the Python ecosystem. Obviously, all the best wishes for you and your friend in the new venture!!
  • fchollet 2y ago
    Thank you!
• satyanash 2y ago
  > "Why did you decide to merge Keras into TensorFlow in 2019": I didn't! The decision was made in 2018 by the TF leads -- I was a L5 IC at the time and that was an L8 decision. The TF team was huge at the time, 50+ people, while Keras was just me and the open-source community. In retrospect I think Keras would have been better off as an independent multi-backend framework -- but that would have required me quitting Google back then. The fact that an "L8" at Google ranks above an OSS maintainer of a super-popular library "L5" is incredibly interesting. How are these levels determined? Doesn't this represent a conflict of interest between the FOSS library and Google's own motivations? The maintainer having to pick between a great paycheck or control of the library (with the impending possibility of Google forking).
  • darkwizard42 2y ago
    L8 at Google is not a random pecking order level. L8s generally have massive systems design experience and decades of software engineering experience at all levels of scale. They make decisions at Google which can have impacts on the workflows of 100s of engineers on products with 100millions/billions of users. There are less L8s than there are technical VPs (excluding all the random biz side VP roles) L5 here designates that they were a tenured (but not designated Senior) software engineer. It doesn't meant they don't have a voice in these discussions (very likely an L8 reached out to learn more about the issue, the options, and ideally considered Francois's role and expertise before making a decision), it just means its above their pay grade. I'll let Francois provide more detail on the exact situation.
    • deleted 2y ago
      [deleted]
    • belter 2y ago
      The history of the company does not seem to demonstrate such a semi-genius are capable of producing successful products. Can hardly be third on Cloud.
  • fchollet 2y ago
    This is just the standard Google ladder. Your initial level when you join is based on your past experience. Then you gain levels by going through the infamous promo process. L8 represents the level of Director. Yes, there are conflicts of interests inherent to the fact that OSS maintainers are usually employed by big tech companies (since OSS itself doesn't make money). And it is often the case that big tech companies leverage their involvement in OSS development to further their own strategic interests and undermine their competitors, such as in the case of Meta, or to a lesser extent Google. But without the involvement of big tech companies, you would see a lot less open-source in the world. So you can view it as a trade off.
• openrisk 2y ago
  > I was a L5 IC at the time and that was an L8 decision omg, this sounds like the gigantic, ossified and crushing bureaucracy of a third world country. It must be saying something profound about the human condition that such immense hierarchies are not just functioning but actually completely dominating the landscape.
  • Barrin92 2y ago
    Bureaucracy as per Weber is simply 'rationally organized action'. It dominates because this is the appropriate way to manage hundreds of thousands of people in a impersonal, rule based and meritocratic way. Third world countries work the other way around, they don't have professional bureaucracies, they only have clans and families. It's not ossified but efficient. If a company like Google with about ~180.000 employees were to make decisions by everyone talking to everyone else you can try to do the math on what the complexity of that is.
    • agos 2y ago
      effective, maybe. efficient... I would not be so sure.
      • yazaddaruvala 2y ago
        Depends on what you’re trying to achieve. Small organizations define efficiency based on time to make number go up/down. Meanwhile, if something bad happens at 2am and no one wakes up - whatever there we’re likely no customers impacted. Larger organizations are really efficient at ensuring the p10 (ie worst) hires are not able to cause any real damage. Every other thing about the org is set up to most cost effectively ensure least damage. Meanwhile, numbers should also go up is a secondary priority.
    • dbspin 2y ago
      Bureaucracies are certainly impersonal, but you'd be at a loss to find one that's genuinely rule based and meritocratic. To the extent that they become remain rule based they are no longer effective and get routed around. To the extent that they're meritocratic, the same thing happens with networks of influence. Once you get high enough, or decentralised enough bureaucracies work like any other human tribes. Bureaucracies may sometimes be effective ways to cut down on nepotism (although they manifestly fail at that in my country), but they're machines for manifesting cronyism.
• dkga 2y ago
  Hi François, just wanted to take the opportunity to tell you how much your work has been important for me. Both at the start, getting into deep learning (both keras and the book) and now with keras3 as I'm working to spread DL techniques in economics. The multi-backend is really a massive boon, as it also helps ensure that the API would remain both standardised and simple, which is very helpful to evangelise new users that are used to higher-level scripting languages as my crowd is. In any case, I just want to say how much an inspiration the keras work has been and continues to be. Merci, François !
  • fchollet 2y ago
    Thanks for the kind words -- glad Keras has been useful!
• trott 2y ago
  Congrats, François, and good luck! Q: The ARC Prize blog mentions that you plan to make ARC harder for machines and easier for humans. I'm curious if it will be adapted to resist scaling the training dataset (Like what BARC did -- see my other comment here)? As it stands today, I feel like the easiest approach to solving it would be BARC x10 or so, rather than algorithmic inventions.
  • fchollet 2y ago
    Right, one rather uninteresting line of approaches to ARC consists of trying to anticipate what might be in the test set, by generating millions of synthetic tasks. This can only work on relatively simple tasks, since the chance of task collision (between the test set and what you generate) is very low for any sophisticated task. ARC 2 will improve on ARC 1 by making tasks less brute-forceable (both in the sense of making in harder to find the solution program by generating random programs built on a DSL, and in the sense of making it harder to guess the test tasks via brute force task generation). We'll keep the human facing difficulty roughly constant, which will be controlled via human testing.
    • versteegen 2y ago
      Hi! As someone who spent the last month pouring myself into the ARC challenge (which has been lots of fun, thanks so much for creating it), I'm happy to see it made harder, but please make it harder by requiring more reasoning, not by requiring more human-like visual perception! ARC is almost perfect as a benchmark for analogical reasoning, except for the need for lots of image processing as well. [Edit: however, I've realised that perception is representation, so requiring it is a good thing.] Any plan for more validation data to match the new harder testset?
      • Skylyz 2y ago
        I had never thought about how close perception and reasoning are from a computational point of view, the parts of ARC that we call "reasoning" seem to just be operations that the human brain is not predisposed to solve easily. A very interesting corollary is that the first AGIs might be way better thinkers than humans by default because of how they can seamlessly integrate new programs into their cognition in a perfect symbiosis with computers.
• imfing 2y ago
  just wanna take this chance to say a huge thank you for all the amazing work you’ve done with Keras! back in 2017, Keras was my introductory framework to deep learning. it’s simple, Pythonic interface made finetuning models so much easier back then. also glad to see Keras continue to thrive after getting merged into TF, especially with the new multi-backend support. wishing you all the best in your new adventure!
• bootywizard 2y ago
  Hi Francois, congrats on leaving Google! ARC and On the Measure of Intelligence have both had a phenomenal impact on my thinking and understanding of the overall field. Do you think that working on ARC is one of the most high leverage ways an individual can hope to have impact on the broad scientific goal of AGI?
  • fchollet 2y ago
    That's what I plan on doing -- so I would say yes :)
    • Skylyz 2y ago
      François as a contestant of ARC prize ?! For real ?
      • fchollet 2y ago
        I will never enter ARC Prize myself, since I'm organizing it. But the reason I made ARC in the first place was to work on it myself! I intend to solve it (outside of the context of the competition).
• schmorptron 2y ago
  Hey,I really liked your little book of deep learning, even though I didn't understand everything in it yet. Thanks for writing it!
  • fchollet 2y ago
    Enjoy the book!
  • Philpax 2y ago
    Er, isn't that by François Fleuret, not by François Chollet?
    • schmorptron 2y ago
      you... are correct. Shame on me. Still a good book!
• c1b 2y ago
  Hi Francois, I'm a huge fan of your work! In projecting ARC challenge progress with a naive regression from the latest cycle of improvement (from 34% to 54%), it seems that a plausible estimate as to when the 85% target will be reached is sometime between late 2025 & mid 2026. Supposing ARC challenge target is reached in the coming years, does this update your model of 'AI risk'? // Would this cause you to consider your article on 'The implausibility of intelligence explosion' to be outdated?
  • fchollet 2y ago
    This roughly aligns with my timeline. ARC will be solved within a couple of years. There is a distinction between solving ARC, creating AGI, and creating an AI that would represent an existential risk. ARC is a stepping stone towards AGI, so the first model that solves ARC should have taught us something fundamental about how to create truly general intelligence that can adapt to never-seen-before problem, but it will likely not itself be AGI (due to be specialized in the ARC format, for instance). Its architecture could likely be adapted into a genuine AGI, after a few iterations -- a system capable of solving novel scientific problems in any domain. Even this would not clearly lead to "intelligence explosion". The points in my old article on intelligence explosion are still valid -- while AGI will lead to some level of recursive self-improvement (as do many other systems!) the available evidence just does not point to this loop triggering an exponential explosion (due to diminishing returns and the fact that "how intelligent one can be" has inherent limitations brought about by things outside of the AI agent itself). And intelligence on its own, without executive autonomy or embodiment, is just a tool in human hands, not a standalone threat. It can certainly present risks, like any other powerful technology, but it isn't a "new species" out to get us.
    • YeGoblynQueenne 2y ago
      ARC as a stepping-stone for AGI? For me, ARC has lost all credibility. Your white paper that introduced it claimed that core knowledge priors are needed to solve it, yet all the systems that have any non-zero performance on ARC so far have made no attempt to learn or implement core knowledge priors. You have claimed at different times and in different forms that ARC is protected against memorisation-based Big Data approaches, but the systems that currently perform best on ARC do it by generating thousands of new training examples for some LLM, the quintessential memorisation-based Big Data approach. I too, believe that ARC will soon be solved: in the same way that the Winograd Schema Challenge was solved. Someone will finally decide to generate a large enough dataset to fine-tune a big, deep, bad LLM and go to town, and I do mean on the private test set. If ARC was really, really a test of intelligence and therefore protected against Big Data approaches, then it wouldn't need to have a super secret hidden test set. Bongard Problems don't and they still stand undefeated (although the ANN community has sidestepped them in a sense, by generating and solving similar, but not identical, sets of problems, then claiming triumph anyway). ARC will be solved and we won't learn anything at all from it, except that we still don't know how to test for intelligence, let alone artificial intelligence. The worst outcome of all this is the collateral damage to the reputation of symbolic program synthesis which you have often name-dropped when trying to steer the efforts of the community towards it (other times calling it "discrete program search" etc). Once some big, compensating, LLM solves ARC, any mention of program synthesis will elicit nothing but sneers. "Program synthesis? Isn't that what Chollet thought would solve ARC? Well, we don't need that, LLMs can solve ARC just fine". Talk about sucking out all the air from the room, indeed.
• cowsaymoo 2y ago
  I’m really going through it, trying to get legacy Theano and TensorFlow 1.x models from 2016 running on modern GPUs due to compatibility headaches due to OS, NVIDIA CUDA, CuDNN, drivers, docker, python, and package/image hubs all contributing their own roadblocks to actually coding. Ideally we would abandon this code, but we kind of need it running if we want to thoroughly understand our new model's performance on unseen old data, and/or understand Kappa scores between models. Will the move towards freeing Keras from TF again potentially reintroduce version chaos, or will it future proof it from that? Do you see a potential for something like this to once again befall tomorrow's legacy code relying on TF 1.x and 2.x?
  • fchollet 2y ago
    Keras is now standalone and multi-backend again. Keras weights files from older versions are still loadable and Keras code from older versions are still runnable (on any backend as long as they only used Keras APIs)! In general the ability to move across backends makes your code much longer-lived: you can take your Keras models with you (on a new backend) after something like TF or PyTorch stops development. Also, it reduces version compatibility issues, since tf.keras 2.n could only work with TF 2.n, but each Keras 3 version can work with a wide range of older and newer TF versions.
• fransje26 2y ago
  From one François to an other, thank you for you work, and all the best with your next endeavor! Your various tutorials and your book "Deep Learning with Python" have been invaluable in helping me get up to speed in applied deep learning and in learning the ropes of a field I knew nothing about.
• gama843 2y ago
  Hi Francois, any chance to work or at least intern (remote, unpaid) with you directly? Would be super interesting and enriching.
• deleted 2y ago
  [deleted]
• mFixman 2y ago
  > I was a L5 IC at the time Kudos to Google for hiring extremely competent people, but I'm surprised that the creator and main architect of Keras hadn't been promoted to Staff Engineer at minimum.
  • oooyay 2y ago
    Down leveling is a pretty common strategy larger companies use to retain engineers.
    • Centigonal 2y ago
      Could you elaborate on this? how does being down-leveled make an engineer less likely to leave?
      • toomuchtodo 2y ago
        It’s gaslighting to make you work harder to achieve the promo.
    • dekhn 2y ago
      Google in particular often downlevelled incoming engineers by one level from what their "natural" level should be- IE, a person who should have been an L6 would often be hired at L5 and then have to "prove themself" before getting that promo.
  • xyst 2y ago
    at certain levels in the corporate ladder, it's all about who or whom you glaze to get to that next level. actual hard skills are irrelevant
  • toxik 2y ago
    Hierarchy aside, I am surprised the literal author and maintainer of the project, on Google’s payroll no less, was not consulted on such a decision. Seems borderline arrogant.
    • 2y ago
• raverbashing 2y ago
  Thanks for that, and thanks for Keras Another happy Keras user here (under TF - but even before with Theano)
• cynicalpeace 2y ago
  What are some AI frameworks you really like working with? Any that go overlooked by others?
  • fchollet 2y ago
    My go-to DL stack is Keras 3 + JAX. W&B is a great tool as well. I think JAX is generally under-appreciated compared to how powerful it is.
• Borchy 2y ago
  Hello, Francois! My question isn't related directly to the big news, but to a lecture you gave recently https://www.youtube.com/watch?v=s7_NlkBwdj8&ab_channel=MachineLearningStreetTalk https://www.youtube.com/watch?v=s7_NlkBwdj8&ab_channel=Machi... At 20:45 you say "So you cannot prepare in advance for ARC. You cannot just solve ARC by memorizing the solutions in advance." And at 24:45 "There's a chance that you could achieve this score by purely memorizing patterns and reciting them." Isn't that a contradiction? The way I understand it on one hand you are saying ARC can't be memorized on the other you are saying it can?
• harisec 2y ago
  Congrats, good luck with your new company! I have one question regarding your ARC Prize competition: The current leader from the leaderboard (MindsAI) seems not to be following the original intention of the competition (fine tune a model with millions of tasks similar with the ARC tasks). IMO this is against the goal/intention of the competition, the goal being to find a novel way to get neural networks to generalize from a few samples. You can solve almost anything by brute-forcing it (fine tunning on millions of samples). If you agree with me, why is the MindsAI solution accepted?
  • versteegen 2y ago
    > the goal being to find a novel way to get neural networks to generalize from a few samples Remove "neural networks". Most ARC competitors aren't using NNs or even machine learning. I'm fairly sure NNs aren't needed here. > why is the MindsAI solution accepted? I hope you're not serious. They obviously haven't broken any rule. ARC is a benchmark. The point of a benchmark is to compare differing approaches. It's not rigged.
    • Borchy 2y ago
      I also don't understand why MindsAI is included. ARC is supposed to grade LLMs on their ability to generalize i.e. the higher score the more useful they are. If MindsAI scores x2 than the current SOTA then why are we wasting our $20 on inferior LLMs like ChatGPT adn Claude when we could be using the one-true-god MindsAI? If the answer is "because it's not a general-purpose LLM" then why is ARC marketed as the ultimate benchmark, the litmus test for AGI (I know I know, passing ARC doesn't mean AGI, but the opposite is true, I know)?
      • fchollet 2y ago
        ARC was never supposed to grade LLMs! I designed the ARC format back when LLMs weren't a thing at all. It's a test of AI systems' ability to generalize to novel tasks.
  • fchollet 2y ago
    I believe the MindsAI solution does feature novel ideas that do indeed lead to better generalization (test-time fine-tuning). So it's definitely the kind of research that ARC was supposed to incentivize -- things are working as intended. It's not a "hack" of the benchmark. And yes, they do use a lot of synthetic pretraining data, which is much less interesting research-wise (no progress on generalization that way...) but ultimately it's on us to make a robust benchmark. MindsAI is playing by the rules.
• danielthor 2y ago
  Thank you for Keras! Working with Tensorflow before Keras was so painful. When I first read the news I was just thinking you would make a great lead for the tools infra at a place like Anthropic, but working on your own thing is even more exciting. Good luck!