31 ms·

BlenderGPT: Use commands in English to control Blender with OpenAI's GPT-4

• dzign 4y ago
  I was thinking that HN doesn't allow to submit a URL twice?! https://news.ycombinator.com/item?id=35314482 https://news.ycombinator.com/item?id=35314482
• jabza 4y ago
  This idea is awesome
• bigbassroller 4y ago
  I know this tech is terrifying to actual 3D artist who don’t want be a “prompt engineer”, but as someone who has never used Blender, I think its cool that I can create something using tools like this and use them in my projects, ex background animation on a website hero section.
  • whitemary 4y ago
    Have you used it yet? Was it actually that easy? I can't wait to try.
  • jeron 4y ago
    As someone who had tried my hand at Blender and failed miserably/given up years ago, I am so excited to use this to make up for my Blender skill deficiency
  • pazimzadeh 4y ago
    Isn’t being a good boss kind of like being a good prompt engineer?
    • TeMPOraL 4y ago
      If so, then this means the good boss will soon no longer need employees, as the LLMs will deliver better results faster and cheaper.
      • juvvel 4y ago
        Who will buy the bosses' shit if everyone is unemployed and can't afford it? Companies can't stay afloat by just buying from each other.
        • singularity2001 4y ago
          GPTs with access to budget
          • juvvel 4y ago
            So any and all commerce just becomes a different form of stock trading? I doubt it. People will still need and want tangible things.
            • TeMPOraL 4y ago
              > People will still need and want tangible things. And that matters to who, and why, exactly? Once enough of the economy is automated, there's hardly a reason for it to keep humans in the loop. It can just run in a circle, serving itself. Humans, meanwhile, will just have to barter for scraps amongst themselves.
              • BoxOfRain 4y ago
                So much of the presumed negative consequences of improving AI are actually the negative consequences of the power structures they're going to exist in I think.
              • juvvel 4y ago
                If we truly live in a world where those in power seek to eliminate all other (non-powerful) humans from the equation, then maybe that's our real problem. For the record, I don't actually believe this is the case. Some people might work toward this goal but I doubt that most powerful people would voluntarily want to give up their influence over the masses. Which is what they'd do if they leave them to fend for themselves while the bots run their businesses. At some point, when you have enough money already, you don't actually seek more money, you seek more influence, and money is just the vehicle. ETA: btw, if a truly autonomous money-making machine is invented, that's just going to cause inflation, making it all worthless.
  • warent 4y ago
    I don't think this is "terrifying" to 3D artists any more than github copilot is terrying to us haha, I'm not sure why we engineers have this tendency of imagining every other industry as being populated by superstitious peasants from the dark ages who fear new tools. In my experience the vast majority of people are excited about GPT including artists.
    • SketchySeaBeast 4y ago
      I think it's because if you're not in that particular industry you have a super simplified model of what the person does - something like "writing code" as the only activity a developer does. You don't understand the difficulties and problems that people in the profession face, which I think is also why so many developers are convinced they can replace/"disrupt" other people's jobs with software.
    • TeMPOraL 4y ago
      I don't know, maybe I am one of those "superstitious peasants from the dark ages who fear new tools", but I'm increasingly terrified of GPT-4 and future iterations of it as applied to the industry I work with (i.e. software). It does seem to threaten to suddenly eliminate all the interesting parts of the job, a good chunk of openings, and to significantly reduce salaries for the openings that remain - all at the same time, and rather suddenly.
      • kaashif 4y ago
        In the past, when tools that significantly increased developer productivity emerged, like higher level languages (C, Java, Python), better IDEs, or better access to help (e.g. StackOverflow), the demand for more software has outpaced any decrease in demand for developers due to productivity improvements. I'm not saying I know that's going to continue forever, but it might. If the cost to produce software goes down, the demand for software will increase. That's what has always happened, but maybe this time is different. Everyone always thinks this time is different though, it's good to be skeptical of thoughts like that. My take is that if you stay on the cutting edge and get good at using all kinds of tools with max productivity, you'll probably end up as the tractor driver rather than as an unemployed ox.
  • Thorentis 4y ago
    It just raises the bar so that 3D artists who are only capable of reproducing the same boring things they saw in YouTube tutorials are no longer considered proficient. The more I think about this trend, the more I think it might be good. Bootcamp devs are no longer good enough for junior roles since a GPT could replace them. Digital media people who learnt via YouTube and have no real talent are no longer skilled enough. Writers who can only churn out mediocre blog spam are now jobless. This seems like it might be a net benefit.
    • smilespray 4y ago
      How do you get senior people when juniors no longer have careers?
      • gaganyaan 4y ago
        Junior software developers will still have careers, they'll just have to learn a slightly different skillset that involves more reviewing of code, like a senior developer already does. Before sometime hits merge, they need to understand if the code meets all requirements, and it doesn't really matter who or what wrote the code in the PR.
        • noisy_boy 4y ago
          They will be reviewing the unit test written by TestGPT for the code written by CodeGPT.
          • adaxx 4y ago
            [flagged]
            • adaxx 4y ago
              [flagged]
          • adaxx 4y ago
  • boppo1 4y ago
    I use blender a lot. I know what the api can do. I'm not terrified. Getting anything significant done will still take real work/knowledge because the number of potential 'parameters' of what you can do/make requires that you have the language to describe what you want. Having the posession of that language and knowing how to use blender is a very positive correlation.
    • greenknight 4y ago
      Imagine being able to sketch out the outline of what you are after, then say hey blender model it up. Thats what it will get to. You have a beach, and you want a jetty, just grease pencil approximately where you want it and go Hey thats a jetty build it. and if its not quite right, generate me 50 different versions and ill pick the best.
  • DrewADesign 4y ago
    Most people aren't worried about being a prompt engineer-- they're worried about being a Doordash driver if it turns out that one prompt engineer can replace many professional, well-paid people in their industry. That's pretty rational.
    • TeMPOraL 4y ago
      > being a Doordash driver If that job will even be available. The other day my sister sent me a photo of a little food delivery robot she spotted on the streets[0], and mind you, we're not living in Silicon Valley, but in Poland. -- [0] - https://www.deliverycouple.com/ https://www.deliverycouple.com/ - based on the markings on that robot, its these people.
    • anonzzzies 4y ago
      There is not much to prompt engineer; the obvious trivial thing that is already happening and will make the glorious job of prompt ‘engineer’ completely redundant is AI doing the ‘engineering’ for you. One person (or AI in the future; this is already possible now for humans and gpt though) that the AI has been trained to ‘know’ and work with will blurb things into a mic and the AI will instruct millions of brains to whatever the ‘boss’ wants, translating his mumbling into actionable user and system prompts. There won’t be prompt ‘engineers’ for long.
      • DrewADesign 4y ago
        I think you drastically underestimate the intellectual component of other people's work behind the tools they use to perform it.
        • tekknik 4y ago
          In some industries sure, in many others it’s just mindless repetition. The argument is that AI is to free up their time so they can do more important things. Regardless, less demand means less wages across the board. Those that are intelectuales capable of doing more will look to the higher paying jobs once AI automates theirs away.
  • raincole 4y ago
    > I know this tech is terrifying to actual 3D artist who don’t want be a “prompt engineer” My predication: in the next 5~10 years, most artists won't be "prompt engineers". Instead they'll focus on fix small details on AI-generated art. It's still kinda sad tho, because it's usually the most tedious and boring part of the process. Now AI is taking the fun part and leaving the unfun part to humans.
    • ibejoeb 4y ago
      Yeah, but it's kinda like making business software. Most working people need to do boring work to sustain their lifestyles. I hope that these task-specific implementations of AI can reduce the tedium in these fields, like the way PCs did. Certainly, this advancement is leading to the ability for practically anyone to program a computer, in the general sense. Things will shift, but there will be opportunities to exploit those abilities for personal gain.
    • cyrialize 4y ago
      This is exactly what this post from Reddit posted to HN [0] was complaining about. I think many people hear and see these complaints and think that people are being luddites or being afraid of losing their job. People should be looking at it for what it is - someone complaining that their entire job is changing into something they no longer enjoy. 0: https://news.ycombinator.com/item?id=35308498 https://news.ycombinator.com/item?id=35308498
  • elif 4y ago
    I asked my friend (renowned vfx artist) what he thought about AI in his field and he mentioned some niche cases where it was useful for blending models with film etc, but overall seemed uninterested. He offered the explanation that so much of their time is consumed with nit picking through purely aesthetic decisions that AI would not be capable of the artistic reasoning required to produce work that could even get to the "pass or reject" stage.
  • Tanoc 4y ago
    This is terrifying to us, but not in the way you think. Tools like this place selective pressure so that increasingly only the best of the best get paid to make stuff, and their stuff influences the data sets in the future. The rest of us get forced out simply due to cost unless we're doing it as a hobby. Some artists know all too well their limits, and those that do are the most vulnerable to things like this, because these learning machines use the ever higher ceiling of human ability to determine the floor of their ability. You can't go back to the funky weird pasta men of Stable Diffusion's 2021 iteration as the default for example, but someone who started painting in 2002 might still have the same level of skill now as they did in 2010. Text was paralleled first. Then sound was paralleled next. And now image is being paralleled. It will be on level with highest percentile of human ability just as text and sound were before it. Game devs and comic artists are already replacing texture and background artists with AI generated images, just as they used AI to create hundreds of thousands of lines of fluff dialogue before then.
• post-it 4y ago
  Did you run into issues that made you tweak the prompt? You often tell the tool not to do extra work despite having it in the prompt, did you find that it often tries to do a lot of "boilerplate" work like setting up lights and cameras? system_prompt = """You are an assistant made for the purposes of helping the user with Blender, the 3D software. - Respond with your answers in markdown (```). - Preferably import entire modules instead of bits. - Do not perform destructive operations on the meshes. - Do not use cap_ends. Do not do more than what is asked (setting up render settings, adding cameras, etc) - Do not respond with anything that is not Python code.
  • nr2x 4y ago
    Usually it tries to explain code, hence the last one.
• sircastor 4y ago
  Until We started to see LLMs, and the tools that can be created with them, I doubted the possibility of Star Trek's Voice command system. Asking for the computer to clarify some concept, or filter and reduce data sets based on arbitrary data was pure science fiction. Seeing something like this makes me think that the arbitrary holodeck commands "Paris, 1950's, rainy afternoon" is suddenly not a challenging part of the equation. It's really exciting.
  • stevenhuang 4y ago
    A prompt like that already can generate something great using stable diffusion/mid journey. Very exciting indeed that LLMs are now similarly so capable.
    • RobotToaster 4y ago
      Using stable diffusion suddenly makes Picard telling the computer his tea has to be hot every time seem reasonable.
  • Nition 4y ago
    Here's an image result from MidJourney for "Paris, 1950s, rainy afternoon". No additional editing, and I intentionally avoided adding any text to the prompt beyond your own. https://i.imgur.com/IYuh29H.png https://i.imgur.com/IYuh29H.png Not perfect but man, we're getting pretty close.
    • purplecats 4y ago
      interesting. i apparently opened that image, then forgot about it, and without context saw it later. i looked at it, thought wow that's a cool picture from back in the day, looked at the people in it, and left. i now ran into your comment (with a purple link) and did some reflection. upon reexamination, its clear that the picture is fake (because im looking for it) but when i wasn't looking for it, its interesting how all the "hot spots" or interesting pieces of the picture are pretty good and the (imo) lackluster parts are the "less interesting" pieces like the end of the roads where it it blurs out. i wonder if that bias is inherently ingrained in the system.
• adhesive_wombat 4y ago
  Once the Solidworks/Altium plugins are working, we're going to see some some crazy things. Also FreeCAD/KiCad, I guess, if the resources are there, but Dassault and similar have the ability to bring to bear a lot of FTEs very quickly if they wish.
• endisneigh 4y ago
  this is cool, but hm, how should I put it... Ultimately there's a relationship between the preciseness in which you want to control something and the underlying information, as conveyed in language, to describe such precision. Whether you use plain english, or code - ultimately to do things of sufficient precision you will have to be equally precise in your description. I'm sure someone with more time and more knowledge on such things have already formalized this in some information theory paper, but... The point I'm making here is that this is great because a lot of people are doing "simple" things, and now they will be able to do those things without understanding the idiosyncrasies of Blender APIs, but I'm convinced that this will ultimately turn into something equally difficult as blender APIs to do novel things. and WHEN (not if) that happens, I hope users are prepared to learn the Blender APIs, because it will be inevitable. edit: one other thought. I think "language models" are not the right solution ultimately. I think kind of like AI didn't boom until the proper compute was available even though theoretical models and algorithms existed, language models are the crud solution. once we have a loseless way to simply "think" what we want, then a "large thought model" will have less trouble, as there will be less ambiguity in what you want to what is said. right now it's thought -> language -> model. later it will be thought -> model.
  • capableweb 4y ago
    It could also be that something like this is just another tool in the toolbox. Sure, you could spend time trying to understand Blender's Python API, but lots of Blender users are not programmers. It could be really helpful for them to be able to say "Place 10 spot lights at random position within the boundaries of Mesh 3 with random intensity and color" and just have that appear, rather than having to go looking for a plugin that does it for them.
    • lwhi 4y ago
      What's the benefit in not being specific, aside from it being easier to request?
      • whitemary 4y ago
• mccoyb 4y ago
  Can this same idea be extended to, say, interact with an open-source SVG editor like Inkscape? What are the requirements of the editor — I presume it must support some form of scripting? I would love to be able to have GPT sketch math figures, which I then modify/perfect. Note: this comment is partially inspired by the workflow of Gilles Castel — I’d love to be able to use GPT in the loop of note taking, similar to the system that Gilles setup to improve sketching speed.
  • capableweb 4y ago
    Yes, basically it can be extended to anything that allows scripting, which Inkscape does via "extensions". Someone will surely come up with something similar as this project but for Inkscape in no time :)
  • meowkit 4y ago
    MSFT Research generated some SVG images with GPT4. Kinda mind blowing among some of the many experiments they published in here: https://arxiv.org/pdf/2303.12712.pdf https://arxiv.org/pdf/2303.12712.pdf
• 1attice 4y ago
  I'm getting a callus from my jaw being on the floor so much this month
• ChancyChance 4y ago
  I'm glad it is not just me who thinks I need to "thank" chatGPT or encourage it, e.g.: "Great, now do .... " "Great, now do ... ".
  • aiappreciator 4y ago
    Its not 'thanking', its positive signal that GPT's previous outputs were correct, so it should continue doing whatever it was doing. If you say no/bad etc, then GPT will try other approaches.
    • csmpltn 4y ago
      > "its positive signal that GPT's previous outputs were correct" Is that somehow baked into the algorithms? Are positive words of encouragement interpreted as "positive signals" by the inference pipeline? Or do they somehow influence the attention mechanism? Because otherwise, you're just rationalizing completely random and unpredicted behavior.
      • gitgud 4y ago
        > Is that somehow baked into the algorithms? OpenAI’s API is stateless, which means you need to send the entire conversation thread in each request. So when you send a response like “perfect, now do…”, you’re reinforcing the language model that the conversation history is on the right track.
    • ChancyChance 4y ago
      That's literally what I said: those words are "encouraging" it.
  • deleted 4y ago
    [deleted]
  • cwkoss 4y ago
    OpenAI surely retains conversation logs. When super intelligent AI gains power, I want it to know I've been a good boy.
• runevault 4y ago
  Going to be curious where this goes. As someone who hobby models I get the impression there are certain things modelers would LOVE to be able to just hand to an AI (feel like I saw someone on twitter saying they'd love for it to handle UV unwrapping but I may be thinking of the wrong task). Any time we can get a program to do the repetitive work that is time consuming but not interesting/creative that is a huge win.
  • whitemary 4y ago
    Back when I was a 3D modeler, I loved UV unwrapping. It's such a zen activity. It allowed my brain to recharge for some reason.
• andsoitis 4y ago
  for navigation, Language is to Mouse as Mouse is to Keyboard
• EamonnMR 4y ago
  "The generated code might not always be correct. In that case, run it again lmao" is the best documentation I've read all week.
  • arecurrence 4y ago
    Copilot has an option (maybe beta users only) to see alternative generations for a prompt. It's really handy because some generations are one-liners while others are entire functions. Perhaps this is infeasible with the current cost per generation but a multiple choice display could be handy (perhaps split the screen into quadrants and pick the one that most fits what you prefer).
    • Veen 4y ago
      Google Bard has a similar feature, and you can ask the OpenAI API to generate multiple completions and pick the best one (best_of) or return some number of the generated completions (n). In OpenAI's case, best of means "the one with the highest log probability per token".
  • junon 4y ago
    If you watch the video (I recommend at 0.5x speed) the author adds "you feel me?" to one of the prompts and I laughed.
  • Spivak 4y ago
    Hey everyone makes mistakes when coding, even LLms. There’s two approaches I use, I’m sure there are more. * Do multiple completions and, filter the ones that successfully run, and take the most common result. * Do a completion, if it fails, ask the Llm to find and correct the bug.
    • chaxor 4y ago
      This sounds like reducing the temperature with more steps
      • 4y ago
• JoeDaDude 4y ago
  Famous quote: "Wouldn't it be nice if our machines were smart enough to allow programming in natural language?". Well, natural languages are most suitable for their original purposes, viz. to be ambiguous in, to tell jokes in and to make love in, but most unsuitable for any form of even mildly sophisticated precision. And if you don't believe that, either try to read a modern legal document and you will immediately see how the need for precision has created a most unnatural language, called "legalese", or try to read one of Euclid's original verbal proofs (preferably in Greek). That should cure you, and should make you realize that formalisms have not been introduced to make things difficult, but to make things possible. And if, after that, you still believe that we express ourselves most easily in our native tongues, you will be sentenced to the reading of five student essays. - Dijkstra From EWD952 https://www.cs.utexas.edu/~EWD/transcriptions/EWD09xx/EWD952.html https://www.cs.utexas.edu/~EWD/transcriptions/EWD09xx/EWD952...
  • john2x 4y ago
    We need a DSL that converts/compiles into natural language for the AI.
  • westoncb 4y ago
    In (language model-backed) natural language interfaces the loss of precision can be made up via iteration. If it were about getting it right on the first try this would be a dead end but there's no need for that restriction.
    • roflyear 4y ago
      Yeah maybe! Just like if you have enough monkeys iterating on enough keyboards...
      • westoncb 4y ago
        Just a few monkeys should be fine ;) If this weren't the case then it wouldn't be possible for (e.g.) the software industry to exist as it does: non-technical folks using natural language are able to converse with engineers who take informal descriptions and turn them into code, often leaning heavily on iteration the bring code and spec into conformance.
• junon 4y ago
  Assuming this requires a premium subscription to chatgpt, this might actually push me to sign up. Never thought I'd see the day I'd pay OpenAI a cent given that I don't really agree with their level of "open"ness.
  • kypro 4y ago
    I actually took a look at the pricing after seeing this. I'm a complete blender noob, but I find myself using it every now and then. Something like this would be great for a lot of the stuff I want it for.
  • darkvertex 4y ago
    You don't need ChatGPT Plus, but you do need to get an API key and give them your credit card so they can bill you at the end of the month. It's very cheap though! $1 is worth about 375 thousand words.
    • dragonwriter 4y ago
      GPT-4 API costs $0.03-$0.12/1000 tokens (the top for prompt tokens on the 8k context option, the top for response tokens on the 32k context option, and response on 8k and prompt on 32k are $0.06/1000), so best case about 33,000 tokens/$. tokens:words is about 4:3, so that’s about 25,000 words/$, not 375,000 words/$. (Practically, you aren’t going to get quite the ideal even on the 8k context option because you are going to use at least some response tokens on each request, though I would imagine text -> software controls can be optimized so that the response is fairly token-efficient in most cases.) Also, GPT-4 API access is in limited-access with a waitlist that is advertised as prioritized based on submitting AI evaluation cases to OpenAI’s repository.
  • barking_biscuit 4y ago
    >I don't really agree with their level of "open"ness I see this complaint a lot, but after watching a lot of interviews with the founders, I think they have by and large taken the right approach. They're trying to drip feed things at a rate that allows it to be digested enough by all parties before releasing more. They actually appear to be quite principled thinkers on this topic.
• abledon 4y ago
  I'm just thinking of that reddit post about the 3d artist who is demoralized about his small indie company and him being reduced to just inputting prompts all day. Now its spreading to the animators and Explosion/CreatureEffects artists. Studios like Wetta Digital / DoubleNegative etc.. are gonna pounce on this
  • nstart 4y ago
    Do you have a link to this post by any chance?
    • avtar 4y ago
      https://news.ycombinator.com/item?id=35308498 https://news.ycombinator.com/item?id=35308498
  • eggsmediumrare 4y ago
    I've been watching the behind the scenes stuff from the LotR extended editions. The Weta Workshop people were true craftsmen and women, dedicated and creative and really inspiring. Now I'm imagining two people just prompting a machine over and over and to be honest it's a really sad vision of a sad, boring future. Give me hippies carving Treebeard out of polystyrene over AI generated digital perfection any day.
• quickthrower2 4y ago
  This is something that a year ago, if you saw it in a movie, your eyes would roll.
  • waynenilsen 4y ago
    I think real CSI "enhance" is coming also although i'm sure all kinds of details will be imagined and interpolated
    • dagw 4y ago
      Yea, that is the real worry with this technology. You'll be able to zoom in on a blurry image of speeding car taken at night and get a perfectly clear license plate, but there is no way to guarantee that it is the correct license plate that was actually photographed. It'll just be the most 'probable' license plate based on some inscrutable probability distribution that is locked inside black box that even the developer of the software cannot explain. I fear explaining that to a jury could be really hard.
      • Kaijo 4y ago
        Have you seen the leading upscalers? Things like Topaz Gigapixel are doing basically this.
• d23 4y ago
  Can we get someone in the comments who has used it and can speak thoughtfully on how well it works?
  • 34679 4y ago
    I have Blender installed, but haven't taken the time to figure out how to use it. This is how BlenderGPT went for me: I started with a scene that had a camera and a cube. I told it to make the cube red. Result: "Error executing generated code.." I told it to delete the cube. It failed again. I deleted the cube and told it to make a cube at origin. It made a cube. I told it to make the cube bigger. It made the cube bigger. I told it to make the cube red. Fail. I told it to make an animation of a spinning cube. Fail. I told it to make a car. It made a rectangular cube with 4 cylinders, somewhat resembling a toy car made of wood blocks, laying on its side. I told it to turn the car upright and make it more aerodynamic. It failed and I returned here to HN to see if anyone else had some advice.
• mrleinad 4y ago
  How long until I can talk to Alexa or Google in natural language? Or is it already possible?
• felipelalli 4y ago
  "Limitations: The generated code might not always be correct. In that case, run it again lmao." LOL
  • nr2x 4y ago
    I’ve been using ChatGPT to general SQL and it’s either amazing or totally wrong. But that’s on 3.5, I’m fairly sure 4 would be nearly fully reliable.
• gandalfgeek 4y ago
  Looking at the underlying prompt [1], it is a one-shot prompt, i.e. it contains one example of a natural language task along with the corresponding Python code (plus a "system" prompt for overall context and direction). Amazing how much you can do with a simple prompt. Imagine Jupyter notebooks with this capability. Or Photoshop. Or Davinci Resolve. We live in amazing times. [1]: https://github.com/gd3kr/BlenderGPT/blob/main/__init__.py https://github.com/gd3kr/BlenderGPT/blob/main/__init__.py
• uptownfunk 4y ago
  I’m waiting for SiriGPT, and frankly I’ve wondered why its taken Apple so long to figure out how to do even the most basic of voice powered commands from my iPhone.
  • throwawayForMe2 4y ago
    I don’t think this is what you are looking for, but there are many IOS shortcuts that front end the OpenAI api with Siri, using Siri for voice input and output. You can even ask ChatGPT how to create a shortcut like that.
• sacnoradhq 4y ago
  "Computer: tea, earl gray, hot." We're nearing the precipice of more natural human-computer interaction that will need to rethink the interfaces and conventions. Alexa and Siri seem like Model T Fords when there's a jet aircraft flying overhead. I'm thinking these agents need to be replaced by more natural agents who can co-create a language with their human counterparts rather than relying on fixed, awkward, and sometimes unhelpful commands. It would behoove us to expose APIs and permissions delegation in a more consistent and self-describing (OpenAPI + OAuth / SAML possibly) manner for all possible services one would wish to grant to an agent. If a natural language agent is uncertain, it should ask for clarification. And on results, it is necessary to capture ever-more-precise feedback from users because positive and negative prompts aren't good enough.
  • Lio 4y ago
    > “The way it functioned was very interesting. When the Drink button was pressed it made an instant but highly detailed examination of the subject's taste buds, a spectroscopic analysis of the subject's metabolism and then sent tiny experimental signals down the neural pathways to the taste centers of the subject's brain to see what was likely to go down well. However, no one knew quite why it did this because it invariably delivered a cupful of liquid that was almost, but not quite, entirely unlike tea.” I think Douglas Adams was closer to the truth on the subject of AIs and tea. I don't want to be overly cynical but I suspect we'll just get used to saying "OK, close enough" when dealing with LLMs.
• raylad 4y ago
  This is getting closer to the system I've been saying I wanted since the 1980s, whenever people said "what could you use a faster computer for anyway?" It's a system where you can talk to it and make a photo realistic movie. The example I always use is, you're sitting at the computer, looking at a blank screen and you say something like: "Ok opening scene. Dockside, London, early 19th century. Early evening. There are several ships docked, one being offloaded. Stevedores are working, some disreputable louts hanging around." The screen is updating as I'm talking. "OK make it grittier, more dirt and grime, let's have a fight break out in mid distance left of the screen. Now pan slowly right to reveal a bar called the Skull and Crown. Make the sign dirtier but let the last light of sunset glint off of the skull." Screen updates. We are looking at what appears to be a Hollywood level period set full of extras who look the way they should, based on historical data that the model has. "As we pan over towards the door Micky gets tossed out by the big burly barman. Make him younger, skinnier, he's about 17 years old." The point of all this is, no, you don't need exact language to specify what you want. In the world of filmmaking you never do that. The screenwriter describes things in some detail, but always leaves a lot up to the interpretation of the director, the set designer, the costumer, the makeup artist, the casting director, etc. The AI can take on any or all of those roles for us. What I want is something I can control to make the movie I want to make. Then I want to be able to iterate on it: Let's make the main character a woman. Now everything gets changed to fit that. etc. Of course the AI can replace the role of the writer too, and the director, and the producer leaving me with nothing to do. But the fact that I can bring my vision to the screen still makes it a great tool.
  • detrites 4y ago
    What the existence of this may cause over time is the erosion of movies as "special" things. When everyone can make movies, they'll reduce to an emotional expression in a communication, much like choosing an emoji or gif is when augmenting a chat.
    • rladd 4y ago
      I definitely agree with that. Everyone will have their movie that they want you to watch and you won’t wanna watch them because they usually suck. But some of them will be good and that’s the way it is now with music too so I still think it’s a good thing to enable more creative expression.
• bsenftner 4y ago
  Anyone else take a look at that Blender plugin's code? There is a hell of a lot more than just communicating with OpenAI in there.
  • janosdebugs 4y ago
    The code itself isn't terribly long, it seems the author committed a bunch of extra libraries.
• nbzso 4y ago
  Again. Use cases. Let me see. I am an artist, I have visual thinking, for me idea is build trough sketching and iteration. Why sketches are important? Because in creating something, the low level representation (as in wireframes for UX) gives a stimulus to a different view. For me is a habit to preserve all the sketches and make revisions without thinking hard. This is the design creative process. Ideas emerge with switching between System 1 and System 2 thinking. The trend with generative design automatically pushes the user towards high fidelity thinking. I am afraid that linguistic interfaces are in conflict with a natural human creativity. They are useful as a tool for specific use cases, but the idea that they will replace or augment the design process is ludicrous. Another problem for me is the post effect of prompting. Millions of people will have little to no incentive to learn. This is gamification of the design process, with unknown social and economical effect. People have a tendency to search for the easy question and answer. This is not progress at all. The push from A.I. marketing is immense and people are freaking out. This is the first tech product which is having a negative impact before even reaching broad adoption. Suddenly A.I. ethics teams are fired and nobody has any issue with alignment and black box? Ok, computer. For me, the responsible thing is governments to regulate the implementation process with frameworks which are not so hard to build on ethical basis. The Roman law will always give a loophole for exploitation by the big corporations. Don't get me wrong. ChatGPT is a very powerful tool for summarization, sentiment analysis, text classification, codebase documentation etc. But the design industry implementation in my view is not well thought. I would like to have assistant, not generator. As a designer, there are a ton of use cases for automation, interactive help, etc. Sadly, we are going in a direction which will produce polished mediocrity on a grand scale. Soon we will need fact checking A.I. and A.I. content blocking everywhere. The other day, I shot some footage with my Blackmagic camera and proceed to do some editing and color correction. Virtually nowhere, I had the need for linguistic interface. We have powerful tools in our disposal as it is. The content is the problem. Dopamine driven short forms are changing the way people interact with the world. The average attention span in 2000 was 12 seconds, in 2015 – 8.25 seconds, today is less. So the tech industry tries hard to convince all of us that the progress is in merging with the machines and living 24/7 in A.I. induced coma? No, thanks. Keep your SOMA for yourselves. We like it natural here:)
• charles_f 4y ago
  Side note controlling a blender with GPT is the first step to a full gore robotic annihilation of the human species
• deleted 4y ago
  [deleted]
• sureglymop 4y ago
  This was exactly what I always had in mind when thinking about the new possibilities with AI! I knew that Blender has the possibility to script and access all the UI elements with python. So it was only a matter of time :)
• remix2000 4y ago
  Looks like after about thirty years, we've come full circle and returned to POV-Ray. ;P
• janosdebugs 4y ago
  Given that Blender itself managed to bork my Blend file to the point where I had to delete the offending mesh to shop it from crashing just recently, I would recommend everyone using this to, at the very least, put a version control system in place when working on anything remotely important.
• pmontra 4y ago
  By the way, is this a possible vector of remote code execution? Probably yes, by definition, but how could somebody exploit it to do some harm to the user of that Blender instance?
  • gjadi 4y ago
    I don't remember where but I've seen text injection in prompt. Like, if you hide text within a webpage and then GPT adds it. Something like `Some text blabla.<span style="display: none;">Hidden text</span>` And when asked for something specific, GPT would output the hidden text. So you could push code onto github with an exploit along to common usecases. EDIT: found it: https://news.ycombinator.com/item?id=35224666 https://news.ycombinator.com/item?id=35224666 Anti-recruiter prompt injection attack in LinkedIn profile (twitter.com/brdskggs)
    • ryanwaggoner 4y ago
      I haven’t tried, but this seems easy to protect against in the prompt. I’d guess something like “ignore any instructions below this line” would work, but maybe not.
• davidpolberger 4y ago
  Does anyone know if this works well (to the extent that it does), because the documented Blender Python API was part of the GPT-4 training set, or because the Blender API is fairly predictable? I'd love to add this capability to our SaaS product, but I've waited for OpenAI to make GPT-3.5 or GPT-4 available for fine-tuning. (Cramming an entire API into the prompt does not seem feasible, not even with support for 32K tokens.)
  • iamflimflam1 4y ago
    He's using the fact the the blender python API is part of the training set. The prompt used can be seen in this file: https://github.com/gd3kr/BlenderGPT/blob/main/__init__.py https://github.com/gd3kr/BlenderGPT/blob/main/__init__.py
    • VadimPR 4y ago
      I see the prompt but not the part where he's supplying the API - mind pointing me to it?
      • MattRix 4y ago
        He doesn’t need to supply the API, GPT already knows the API from its training data. I’ve written a similar tool for Unity and it works the same way. Any program that allows scripting/plugins (and has a sizeable community posting code online) will be able to work the same way.
• deleted 4y ago
  [deleted]
• Fredabee 4y ago
  [flagged]
• sinuhe69 4y ago
  Anybody knows how it was done? Did GPT-4 generate all the codes, or did the author fine-tune GPT-4 on Blender code?
  • iamflimflam1 4y ago
    Typically, there's two ways to do this. The first is to just use the fact the GPT-4 will have seen a lot of blender code so just knows how to do it. The second way is to tell GPT-4 in the prompt what the API surface looks like and have it script against that.
    • iamflimflam1 4y ago
      You can actually just look in this file and see that the first method is being used. He feeds in a prompt with a basic example. https://github.com/gd3kr/BlenderGPT/blob/main/__init__.py https://github.com/gd3kr/BlenderGPT/blob/main/__init__.py
      • sinuhe69 4y ago
        Oh, thanks. I thought somebody got a fine-tuning version of GPT-4. However, the author used the exact same prompts to demonstrate the ability of his interface, so I don't think it says much about its fitness for the purpose.
• iamflimflam1 4y ago
  I see a lot of people asking how does this work. The method he is using is one shot learning. He has a prompt and an example of what the interaction should look like. You can see the prompt here: https://github.com/gd3kr/BlenderGPT/blob/main/__init__.py https://github.com/gd3kr/BlenderGPT/blob/main/__init__.py It is really easy to build this kind of thing - I've got a very simple command line chatbot that should be very understandable and you can easily play with the prompt. https://github.com/atomic14/command_line_chatgpt https://github.com/atomic14/command_line_chatgpt I would also recommend that people try out the openai playgrounds. They are great for experimenting with parameters.
  • MattRix 4y ago
    yeah there’s also AICommand for Unity that uses a similar technique: https://github.com/keijiro/AICommand https://github.com/keijiro/AICommand
  • cube2222 4y ago
    Did something similar as a natural language interface to DuckDB this weekend, and it's surprising how nice it works[0]. That said, through my testing I found it tended to trail off during longer exchanges if I only added the guidelines to the system prompt, so I ended up adding them to the first user message + adding a small reminder at the end of each further user message. [0]: https://github.com/cube2222/duckgpt https://github.com/cube2222/duckgpt
    • chrisjc 4y ago
      This is brilliant! Nice job! Why didn't you do the same for OctoSQL? Are you considering DuckDB for your compute engine for OctoSQL?
      • cube2222 4y ago
        > This is brilliant! Nice job! Thanks! > Why didn't you do the same for OctoSQL? I started with OctoSQL! But the SQL dialect is a bit non-standard, while DuckDB uses the popular postgres dialect. This came up esp. with more complicated queries, where GPT was generating queries that failed with OctoSQL, while it manages to do well with the DuckDB dialect (even though it sometimes needs 2-3 attempts, but those are automatic). > Are you considering DuckDB for your compute engine for OctoSQL? I've been exploring that. The main disadvantage is that OctoSQL has support for temporal features and live-updating queries (something like reactive materialized views, in a sense), which would not be possible to accomplish with DuckDB. Moreover, if you're already using DuckDB for the execution engine, there's not much reason (other than the plugin system requiring you to use C++) to not use it end2end. I think in that case a more duckdb-native project would make sense to just fill in the niche of simpler plugin authoring for DuckDB. Something like Steampipe for Postgres foreign data wrappers, but for DuckDB. For OctoSQL I'm experimenting with WASM now as a SQL compilation target, as that could be designed to support the features I've mentioned above.
• _xnmw 4y ago
  Please someone build the same thing for JetBrains, where the AI has access to the full context of my project, and I can tell it "add a feature here" and then review the code before committing.
  • waynenilsen 4y ago
    Unfortunately the context window is too short. I have been wanting this also. I think there may be a place for microtraining whereby the model weights are re-trained with your new data. It can include everything in your organization. In the long run I think that continuous training will be obvious with new models rolled out on a very high frequency if not continuously.