7 ms·

Instead of learning the latest workarounds for the kinks and quirks of a beta AI product, I'm going to wait 3 weeks for the advice to become completely obsolete

• thornewolf 2y ago
  To be fair, the article basically says "ask the LLM for what you want in detail"
  • fullstackwife 2y ago
    great advice, but difficult to apply given very small context window of o1 models
• jameslk 2y ago
  The churn is real. I wonder if so much churn due to innovation in a space can prevent enough adoption such that it actually reduces innovation
  • miltonlost 2y ago
    A constantly changing "API" coupled with a inherently unreliable output is not conducive to stable business.
    • b112 2y ago
      Unless your business is customer service reps, with no ability to do anything but read scripts, who have no real knowledge of how things actually work. Then current AI is basically the same, for cheap.
      • dartos 2y ago
        Many service reps do have some expertise in the systems they support. Once you get past the tier 1 incoming calls, support is pretty specialized.
        • b112 2y ago
          Many service reps do have some expertise in the systems they support. I said "Unless your business is customer service reps, with...". It's a conditional. It doesn't mean all service reps are clueless, or scripted. But I bet you've encountered the ones that are!
    • ithkuil 2y ago
      It's interesting that despite all these real issues you're pointing out a lot of people nevertheless are drawn to interact with this technology. It looks as if it touches some deep psychological lever: have an assistant that can help to carry out tasks that you don't have to bother learning the boring details of a craft. Unfortunately lead cannot yet be turned into gold
      • dartos 2y ago
        > a lot of people nevertheless are drawn to interact with this technology. To look at this statement cynically, a lot of people are drawn to anything with billions of dollars behind it… like literally anything. Not to mention the amount companies spend on marketing AI products. > It looks as if it touches some deep psychological lever: have an assistant that can help to carry out tasks That deep lever is “make value more cheaply and with less effort” From what I’ve seen, most of the professional interest in AI is based on cost cutting. There are a few (what I would call degenerate) groups who believe there is some consciousness behind these AI, but theyre very small group.
        • pixl97 2y ago
          When I was a kid I messed with computers because they were new, fun, and interesting. At the time I never realized they'd be my source of a living in the future. Current AI has brought back a lot of that wonder and interest for me, and I'm sure the same is true for a lot of other computer nerds.
          • dartos 2y ago
            I’d consider myself one of those nerds. I’ve been in love with programming since I was 9 (20+ years ago now) I was mystified by LLMs a couple years ago. But after really understanding how they work and running into their limitations, a lot of that sheen was lost. There’s not a ton of interesting technology happening with LLM, more a ton of interesting math. (Math, especially linalg, is not the part of computer science I, personally, fell in love with.) The outputs of LLMs, unlike programming languages, is pretty random and trial and error based. There’s never any real skill or expertise being built by playing with these tools. My control over the output isn’t as direct or understandable as with programming. There’s no joy of discovery, only joy of getting the slot machine to give me what I want once in a while. I’ve regained a lot of that wonder, recently, by doing graphics programming and learning lisp. Going against industry trends in my recreational programming has helped the field feel fresh to me. Regardless, I don’t think the extreme minority of people who are truly nerdily passionate about tech are the “a lot of people” OC or I was talking about.
  • dartos 2y ago
    It’s churn because every new model may or may not break strategies that worked before. Nobody is designing how to prompt models. It’s an emergent property of these models, so they could just change entirely from each generation of any model.
    • kyle_grove 2y ago
      IMO the lack of real version control and lack of reliable programmability have been significant impediments to impact and adoption. The control surfaces are more brittle than say, regex, which isn’t a good place to be. I would quibble that there is a modicum of design in prompting; RLHF, DPO and ORPO are explicitly designing the models to be more promptable. But the methods don’t yet adequately scale to the variety of user inputs, especially in a customer-facing context. My preference would be for the field to put more emphasis on control over LLMs, but it seems like the momentum is again on training LLM-based AGIs. Perhaps the Bitter Lesson has struck again.
      • dartos 2y ago
        I’d agree with your quibble. People are trying to design how to prompt, but it’s very different in both implementation and result than designing a programming language or a visual language, ofc.
• raincole 2y ago
  There was a debate over whether to integrate Stable Diffusion into the curriculum in a local art school here. Personally while I consider AI a useful tool, I think it's quite pointless to teach it in school, because whatever you learn will be obsolete next month. Of course some people might argue that the whole art school (it's already quite a "job-seeking" type, mostly digital painting/Adobe After Effect) will be obsolete anyway...
  • londons_explore 2y ago
    All knowledge degrades with time. Medical books from the 1800's wouldn't be a lot of use today. There is just a different decay curve for different topics. Part of 'knowing' a field is to learn it and then keep up with the field.
  • swyx 2y ago
    > whatever you learn will be obsolete next month this is exactly the kind of attitude that turns university courses into dinosaurs with far less connection to the “real world” industry than ideal. frankly its an excuse for laziness and luddism at this point. much of what i learned about food groups and economics and politics and writing in school is obsolete at this point, should my teachers not have bothered at all? out of what? fear? the way stable diffusion works hasn’t really changed, and in fact people have just built comfyui layers and workflows on top of it in the ensuing 3 years, and the more you stick your head in the sand because you already predetermined the outcome you are mostly piling up the debt that your students will have to learn on their own because you were too insecure to make a call without trusting that your students can adjust as needed
    • loktarogar 2y ago
      The answer in formal education is probably somewhere in the middle. The stuff you learn shouldn't be obsolete by the time you graduate but at the same time they should be integrating new advancements sooner. The problem has also always been that those who know enough about cutting edge stuff are generally not interested in teaching for a fraction of what they can get doing the stuff.
  • 2y ago
• icpmacdo 2y ago
  Modern AI both shortens the useful lifespan of software and increases the importance of development speed. Waiting around doesn’t seem optimal right now.
• QuantumGood 2y ago
  Great summary of how AI compresses the development (and hype) product cycle
• gwern 2y ago
  What people are discovering with the latest models is that often their errors are due to entirely reasonable choices and assumptions... which happen to be wrong in your specific case. They call a library you don't have installed, or something like that. Short of inventing either telepathy or spice which can allow LLMs to see the future, it will increasingly be the case that you cannot use the best models efficiently without giving them extensive context. Writing 'reports' where you dump in everything even tangentially relevant is the obvious way to do so, and so I would expect future LLMs to be even more so than o1-preview/pro.
  • sheepscreek 2y ago
    I get much better output from o1* models when I dump a lot of context + leave a detailed but tightly scoped prompt with minimal ambiguity. Sometimes I even add - don’t assume, ask me if you are unsure. What I get back is usually very very high quality. To the point that I feel my 95th percentile coding skills have diminishing returns. I find that I am more productive researching and thinking about the what and leaving the how (implementation details) to the model - nudging it along. One last thing, anecdotally - I find that it’s often better to start a new chat after implementing a chunky bit/functionality.
    • gwern 2y ago
      Yes, I've tried out both: ordering it to ask me questions upfront, and sometimes restarting with an edited 'report' and a prototype implementation for a 'clean start'. It feels like it sometimes helps... but I have no numbers or rigorous evidence on that.
  • pizza 2y ago
    Maximum likelihood training tinges, nay, corrupts, everything it touches. That’s before you pull apart the variously-typed maximum likelihood training processes that the artifice underwent.. Your model attempts to give you a reasonably maximum likelihood output (in terms of kl-ball constrained preference distributions not too far from language), and expects you to be the maximum likelihood user (since its equilibriation is intended for the world in which you the user are just like the people who ended up in the training corpus) for which the prompt that you gave would be a maximum likelihood query (implying that there are times it’s better to ignore you-specific contingencies in your prompt to instead rather re-envision your question instead as being a noisily worded version of a more normal question). I think there are probably some ways to still use maximum likelihood but you switch out over the ‘what’ that is being assumed as likely - eg models that attenuate dominant response strategies as needed by the user, and easy ux affordances for the user to better and more fluidly align the model with their own dispositional needs.
• AbstractH24 2y ago
  Out of curiosity, where do you look for the advice?