9 ms·

Using Lua coroutines to create an RPG dialogue system

82 points by stevekemp 10y ago

SomeCallMeTim 10y ago
I've done this kind of thing in Lua before, but what troubles me about it is the fact that a lot of the state of your game ends up encoded in the Lua stack. What happens when the user wants to save? There's Pluto [1], which is somewhat slow for complex state. And it doesn't work on LuaJIT code -- and I tend to like to run games on LuaJIT. And it's opaque: If you're debugging some kind of interaction in the game, you can't just look at the save state to see what's where. I used to be a big fan of Lua, and this kind of AI and behavior coding was a component of that. But ... I think that, on balance, actually creating a data format that represents the state and "coding" using that plus state machines is actually a more robust and useful way to accomplish the same thing. [1] http://lua-users.org/wiki/PlutoLibrary http://lua-users.org/wiki/PlutoLibrary
- stevekemp 10y ago
  I've certainly hit situations in the past where the Lua stack became too large, and caused all kinds of problems in debugging. But despite that I think coroutines have their place, and this was a nice example that I found when I was looking for inspiration. (I've been working on a Lua-scripted mail client for the past couple of years, and I'm starting to think more seriously about async-behaviour.)
  - SomeCallMeTim 10y ago
    Coroutines are awesome, don't get me wrong. BUT, contrary to what I believed when I was still working with Lua, making the coroutine explicit (like async/await in C# and JavaScript ES2017) is actually even better. In particular, when your coroutines are explicit, you can say "queue up this task, this task, and this task, and continue here when they're ALL done." So when you're dealing with several tasks that might take different amounts of time to complete, they can do the work in parallel. I went from being a major Lua fanboy and posting frequently to the mailing list to drifting away from involvement to abandoning the language, in a relatively short period of time. I even wrote a blog entry about it. [1] I still think Lua has better "bones" than JavaScript, but with TypeScript, async/await, and JavaScript JIT engines everywhere, TypeScript just wins for me. [1] https://realmensch.org/2016/05/28/goodbye-lua/ https://realmensch.org/2016/05/28/goodbye-lua/
    - daurnimator 10y ago
      The problem with making coroutines (like async/await in JS) is http://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/ http://journal.stuffwithstuff.com/2015/02/01/what-color-is-y... This is one of the reasons I love lua's coroutines.
      - unscaled 10y ago
        I assume this is exactly what SomeCallMeTim meant. Yes, explicitly awaitable futures/promises are slightly less ergonomic than coroutines, but this is a very reasonable trade-off. You give up: 1. Extra cognitive load every time you need to await. 2. You've got to be more careful when designing some generic higher-order functions (you need to decide which color their function arguments should be). In return you get: 1. Explicit concurrency. No function that you call unexpectedly suspends behind your back. In some cases this explicitness could be actually important (e.g. when using per-thread/process shared memory). 2. Futures/promises/tasks are first-class values. You could pass them along, save them for later, await on all or any of them and so forth. 3. Theoretically speaking, promises can be more memory efficient than coroutines since you don't need to allocate a full stack for each of them - just the state you need. "What Color Is Your Function" is misleading: there is no strictly superior solution for this tradeoff. Futures/promises encode concurrency as a monad and like every monad they are viral and once you start propagating them around, they can be a pain. But the alternative for not encoding some concern of your program as a monad is to let it pass through an implicit side-channel (I/O, error-handling, global state, built-in scheduler) that is harder to control and can't be treated as first-class object in the language. Go itself chose to go the explicit with error-handling. It doesn't even uses Monads, so it gets the worst of both worlds in my opinion, but the designers' main concern with exceptions which they managed to avert is still valid: with (unchecked) exceptions you never know which function you call is going to raise an error on you somewhere down the line. So yes, with Go you'll never have with unexpected throws, but you'll have to deal with unexpected goroutine suspension. In Go all of your functions will be happily colorless with regards to concurrency but will be colored with regards to error-handling. Again, both choices here are valid, but they are not so brain-dead simple or universal as this article is trying to put them.
- deleted 10y ago
  [deleted]
- buzzybee 10y ago
  On the other hand, it's certainly more engineering to have to build a custom format. I implemented a story engine that compiles to behavior trees, and then subsequently to a small instruction set in this past year and, well, we're talking about three months to go from having no technology to having a pipeline capable of complex scenarios. But I came out of it with a much more competent strategy for addressing assets than anything I've done in the past, so I'm not complaining. It is the core technology of the game(choice driven, text simulation) so it needed a big investment. And the game is going to have content updates, which makes save games terrifying; In the end, I designed it so that it has episodes and restarting the game starts it from that episode so that I don't have to worry about massive amounts of persistent state getting permanently corrupted.
- gravypod 10y ago
  You could do this in C with something like libdill as they give you a handle to a coroutine. All you need to do then is create a state_t that holds the coroutine reference managing the object, the channels for it, and all of the state needed for the computation. During saving you prune dead and save only the computation state. During loading you recreate all of your Objects. It wouldn't be all that slow and I think it would get over most of the overhead in some implementations I've seen of UI/Interaction code for games.
  - SomeCallMeTim 10y ago
    I actually hacked together my own "coroutine" scripting library in C once upon a time. Lots of macros. Same problem though. When I said slow, I meant slow to save the entire state of the Lua VM. You can't even do that in C, so it doesn't really apply to the problem I was pointing out.
    - gravypod 10y ago
      All saveing your object state should be is memcpy'ing your array of all objects into some sort of static memory and on load looping through every object and starting a coroutine for every object. I don't think that's too much overhead.
      - SomeCallMeTim 10y ago
        Sorry, but you're totally missing the key point. Say you have an AI that's made 3 decisions and you're in the MIDDLE of a coroutine where it's waiting for the next criteria that will shift it to its next state. So the AI code looks like: local player = lookForAPlayer() if navigateToPlayerLocation() then local quest = givePlayerQuest(); while (!quest:done()) do waitWithStandardReply("Waiting for you to finish the quest!"); end if (quest:resolved()) then giveQuestReward(); while (true) do waitWithStandardReply("Thanks for finishing that quest for me!"; end else -- quest abandonded while (true) do waitWithStandardReply("Thanks for nothing!"); end end end This is rough example code, and not how I'd probably handle this case specifically, but it gives you the general idea. Each of those functions (except the quest member functions) has, somewhere inside it, one or more "yield()" statements, and some logic to make its goal happen incrementally. Yes, it's awesome to be able to code AI logic that way. But how do you save the coroutine state? The actual execution state of the coroutine itself, including what line it will resume next, the values of any bound variables, the values of any local variables, are all part of the state of the AI in that case. In C, you've got functions that may be relocated somewhere else in RAM and potentially stacks to save (if you're using a "real" coroutine library), so just doing a memcpy won't work. What would you memcpy to save the line that the script is paused on? In Lua, it can be done with Pluto, but then you can't use LuaJIT. I see libdill has the ability to use a user-supplied stack, but it doesn't appear to have a "resume" call that takes such a stack memory block, so the best that would happen is that the coroutine would start over from the beginning. Meaning that if you saved the above coroutine in the middle of a quest, the character involved would offer you a new quest when you saw them instead of accepting the result of a successfully allocated quest. If your object states are all being handled in traditional objects, then you're not using the coroutines in the way described by the article.
- pcwalton 10y ago
  > I think that, on balance, actually creating a data format that represents the state and "coding" using that plus state machines is actually a more robust and useful way to accomplish the same thing. Manually coding up a "state machine" is kind of reinventing the Lua interpreter, no?
  - mmjaa 10y ago
    As a long-term Lua fan, I'm with you. I think that the OP is really complaining about not understanding the Lua VM well enough to push it to its limits - in fact, coding up a 'proper state machine' that can compete with what the Lua VM, itself, has to offer is going to be very, very difficult. That said, I'm in favour of other approaches to this problem - using (i.e. generating) Lua bytecode itself as a way of maintaining state, and then pushing things around while having a good understanding of llimits.h seems to be quite important ..
    - SomeCallMeTim 10y ago
      Sorry, but you're pretty profoundly wrong. I have written an entire (popular) game engine that tightly integrated Lua and used custom-generated C++ bindings. I had to dig pretty deep into the Lua VM to accomplish that. Heck, I found and fixed a bug in the LuaJIT 1.x series -- my fix integrated by Mike Pall, with a credit on the changes page. The thing is if you encode game state into the program state, like using coroutines for complex, long-running AI tasks, for instance, then there's no way to save or restore that state if you're using LuaJIT. And Pluto saves the state in an opaque manner. And as another commenter pointed out, if you want to update the game with new behaviors, god knows how that will interact with save states that have huge call stacks in them. And bug fixes? Yeah, since you're saving the whole program state at this point, bug fixes won't be incorporated in the restored image.
      - optionalparens 10y ago
        Agreed. I've seen things like this tried in Smalltalk images and it was usually a disaster long-term, meaning that the state was simply thrown out and a new image spun up because you're SOL. Either that, or you need tons of logic to handle the old program state and it becomes a convoluted mess. Most of the games and engines I've ever worked on that have dialog systems expressly avoid encoding things into the program state as much as reasonably possible. This is not only for restoring state, but also making for a saner experience editing and changing things during development. Very rarely does the format you encode things like dialog stay the same during a dev cycle unless you're using a canned engine and don't make any serious changes. Using Lua program state itself seems like a way to drive yourself mad by the end of the dev cycle, and make it incredible unfriendly to work with writers and other content creators. People who work with asset pipelines often do a lot of cruel and unusual data conversions and munging to bridge how different people and systems need to work, but eventually things in the actual game engine are best kept as simple data that can be read in and reproduced at any time. Moreover, as a developer, I should be able to load up any state of my game at any point in the game just by providing the necessary input data as possible, and like a pure function, it should produce the same game state every time if possible. Let's also not forget other things like internationalization and localization. As far as interfacing with Lua, I've mostly done it in the last few decades in a simple way where we handed entity component-style data that is just pure data, i.e. no function pointers and crazy stuff, and either let Lua do the rest with that state, or send pure data back to the C++ when needed. As for the data format itself, I've seen everything from XML to custom binary formats to CSV and Excel used for dialog. I think I'd sooner murder someone if game dialog lived in code, even in a Lua script.
- jamesu 10y ago
  An additional problem is what happens when you need to patch the game? If you need to modify some form of quest or dialogue script written as a coroutine, how do you migrate the state? What if you've changed the script 5 times in a series of patches, and the user is still on version X? Then again you still have this problem even if you power your quest/dialog with data instead. Without sufficient information it's practically impossible to make a smooth migration.
- leggomylibro 10y ago
  That is definitely an issue with this sort of thing; I think that you'd need to organize your conversations in a tree structure, and have 'save/load' methods for each node that could transcribe between a serialized data format like json or yaml, and functions in lua. With a format like that, you have the option of writing your dialogues in code or plain text, and easily switching between the two. The only major downside is that you can't easily define arbitrary behavior that way; every unique action that a dialogue event could trigger would need to be transcribed in your 'save/load' methods, which may feel limiting if you have a lot of hard-to-generalize logic.
hypothete 10y ago
Here's a similar implementation on the PICO-8 platform, which uses a subset of Lua: http://www.lexaloffle.com/bbs/?tid=3833 http://www.lexaloffle.com/bbs/?tid=3833 This author demonstrates a multiple choice ask() function as well.
EekSnakePond 10y ago
Robust RPG dialogue and trigger system for JavaScript: https://github.com/codeotter/thusspokenpc https://github.com/codeotter/thusspokenpc
dividuum 10y ago
For simple dialogs that don't have to be saved (see the other discussions), this seems like an easy way to quickly put any format of dialog system together without reinventing your own DSL to encode those dialogs as state machine(s). You can easily compose dialogs just by calling functions and thanks to Luas tail call optimization you could even jump between dialogs without the fear of running out of stack space. I'm not saying it's the best solution but it's pretty simple and probably easy to understand. I've used a similar approach way back in a networked multiplayer game where players could telnet into the game server and use text based menus to control the game. You can see the relevant Lua code here: https://github.com/dividuum/infon/blob/master/server.lua https://github.com/dividuum/infon/blob/master/server.lua. I don't want to imagine how this code would look like without the use of coroutines.
Pica_soO 10y ago
Im wondering, could one auto-convert JavaScript- librarys to lua-code?
- veli_joza 10y ago
  The trend is to compile anything under the sun into JS, not the other way around. Although they are quite similar under the hood, Lua programmers take pride in Lua's elegant and minimalistic design, along with JIT speed. JS warts would not be very welcome. That said, there was a project called lua-colony that attempted the conversion.
- etiene 10y ago
  There are some JS to Lua transpilers around, but I'm not sure how robust they are. I want to try some of them later to try to convert some handy JS libraries and feed them into LuaRocks.
  - Pica_soO 10y ago
    The Git-hub link, please?
optionalparens 10y ago
I can see how it might be fun to do something like this for a blog post or simple project, but in a real game this just doesn't work for many reasons, most listed here already. There are tons of ways of doing dialog system. Mostly it comes down to your specific game and I highly recommend against generic dialog systems. Rather, the most you can do at a generic level is try to come up with a good way to store dialog and retrieve it in the ways you need. This isn't the best thing I've seen or worked with, but here's an easy to understand and documented method and presentation about it form Valve: http://www.gdcvault.com/play/1015317/AI-driven-Dynamic-Dialog-through http://www.gdcvault.com/play/1015317/AI-driven-Dynamic-Dialo... Anyway, I've been programming games and game engines for several decades, and coroutines are always in the list of things that you can throw in the junk pile along with other cool things like continuations, various theoretical data structures, fancy dispatch mechanisms, events/signals, application-wide functional purity, etc. Of course all of these things are useful, but they tend to have problems that make them of limited use in professional game development. Here's a brief checklist of show-stoppers off the top of my head I tend to go through before introducing anything "creative" into a game, and especially a game engine. Most of the aforementioned items fail one or more of these. - Performs awful in common cases or when applied to actual game-like conditions vs. theoretical - Murders cache lines - Hard or impossible to debug - Hard to save, load, serialize, deserialize, or snapshot - Doesn't work over networks - Doesn't play nice with libraries or data coming from other libraries or languages (ex: 3rd party physics lib) - Unpredictable execution start or end times - Non-determinant memory usage or allocation patterns - Risk of stack-overflows - Non-portable or problems on specific target architectures - Requires "religion" meaning it infects all your code or forces you to write all of your code a certain way, everywhere - Fragments memory - Long blocking or uninterruptable processing Conversely, besides the obvious opposites of most of the above, I look for the following when introducing some major data structure or conceptual item like coroutines to a game or game engine: - Leads to predictable, consistent code and resource usage - Data-driven - Editor friendly (as a corollary to the above) - Clear debugging story - Both simple and easy if possible. No, they are not the same. - Plays well with the world around it - Portable - Fast - Loved by the CPU, GPU, and/or compiler, and produces reasonable assembly on target hardware - Easy for someone else to jump in and understand the flow and how it works, assuming they are not a moron. Pretty much these two lists mean that games tend to be made of meat and potatoes things. Simple data structures like arrays, custom allocators, predictable branching and execution patterns, up-front memory allocation, and CPU and GPU loving goodness. Pretty much anything stashing things away and building up massive states or jumping around all over the place, pointer chasing, and so-on should always be a no-go. Anyway, if you're building a truly simple game, as in a 2 day sort of affair, do whatever you want. If you want to build something even remotely complex that you will work on for awhile, it's best to do things the right way which means detach yourself from everything you think you know about most of computer science and think in terms of pipelines, CPU instructions, caches, and predictability among other things. This often means programming against the grain of your language a bit, throwing out sugar, std libraries, and all kinds of things. It really sucks, but that's the essence of good game programming. I hope one day it will be different, but as long as there's still need to push gameplay and graphical boundaries, professional developers almost always need to squeeze out what they can.
managun 10y ago
That's interesting. One of my pet projects is a compiler for Bethesda games quests - having source in text/plain file(s), that's the motivation - and as I am designing the underlying language, maybe I should consider coroutines as well...
shinymark 10y ago
I was the producer and an engineer on Grim Fandango Remastered, a remaster of one of the first games to use Lua (2.5, later 3.0 alpha). And the lead engine programmer on the original game in 1998, Bret Mogilefsky, added coroutines to the Lua language in the game engine before they were officially added as an official language feature. They are very important for scripting adventure gameplay in an intuitive way for non-engineers and helped the original team ship Grim Fandango in 1998. It was similar to what LucasArts had done with SCUMM, their adventure game domain specific language, previously. To reference SomeCallMeTim's comment - the way the save game system worked in Grim Fandango was to serialize the entire Lua environment! Stack, all code, etc! And too jamesu - when you patch the game you patch the code that was serialized into the save game! The story behind all of this in Grim is pretty interesting. If anyone is interested in more details please come to the Game Developers Conference this year as I'm giving a talk on the technical details of how the original game and the remaster works. Oliver Franzke and I will cover both the Grim Fandango and Day of the Tentacle remasters.
- veli_joza 10y ago
  I'd love to hear all the gritty details about game internals, but I cannot attend GDC (wrong continent). Please submit the talk to HN once it's available.
- anonymoushn 10y ago
  I hope there's a video of this talk!

82 points by stevekemp 10y ago

I've done this kind of thing in Lua before, but what troubles me about it is the fact that a lot of the state of your game ends up encoded in the Lua stack. What happens when the user wants to save? There's Pluto [1], which is somewhat slow for complex state. And it doesn't work on LuaJIT code -- and I tend to like to run games on LuaJIT. And it's opaque: If you're debugging some kind of interaction in the game, you can't just look at the save state to see what's where. I used to be a big fan of Lua, and this kind of AI and behavior coding was a component of that. But ... I think that, on balance, actually creating a data format that represents the state and "coding" using that plus state machines is actually a more robust and useful way to accomplish the same thing. [1] http://lua-users.org/wiki/PlutoLibrary http://lua-users.org/wiki/PlutoLibrary

stevekemp 10y ago
I've certainly hit situations in the past where the Lua stack became too large, and caused all kinds of problems in debugging. But despite that I think coroutines have their place, and this was a nice example that I found when I was looking for inspiration. (I've been working on a Lua-scripted mail client for the past couple of years, and I'm starting to think more seriously about async-behaviour.)
- SomeCallMeTim 10y ago
  Coroutines are awesome, don't get me wrong. BUT, contrary to what I believed when I was still working with Lua, making the coroutine explicit (like async/await in C# and JavaScript ES2017) is actually even better. In particular, when your coroutines are explicit, you can say "queue up this task, this task, and this task, and continue here when they're ALL done." So when you're dealing with several tasks that might take different amounts of time to complete, they can do the work in parallel. I went from being a major Lua fanboy and posting frequently to the mailing list to drifting away from involvement to abandoning the language, in a relatively short period of time. I even wrote a blog entry about it. [1] I still think Lua has better "bones" than JavaScript, but with TypeScript, async/await, and JavaScript JIT engines everywhere, TypeScript just wins for me. [1] https://realmensch.org/2016/05/28/goodbye-lua/ https://realmensch.org/2016/05/28/goodbye-lua/
  - daurnimator 10y ago
    The problem with making coroutines (like async/await in JS) is http://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/ http://journal.stuffwithstuff.com/2015/02/01/what-color-is-y... This is one of the reasons I love lua's coroutines.
    - unscaled 10y ago
      I assume this is exactly what SomeCallMeTim meant. Yes, explicitly awaitable futures/promises are slightly less ergonomic than coroutines, but this is a very reasonable trade-off. You give up: 1. Extra cognitive load every time you need to await. 2. You've got to be more careful when designing some generic higher-order functions (you need to decide which color their function arguments should be). In return you get: 1. Explicit concurrency. No function that you call unexpectedly suspends behind your back. In some cases this explicitness could be actually important (e.g. when using per-thread/process shared memory). 2. Futures/promises/tasks are first-class values. You could pass them along, save them for later, await on all or any of them and so forth. 3. Theoretically speaking, promises can be more memory efficient than coroutines since you don't need to allocate a full stack for each of them - just the state you need. "What Color Is Your Function" is misleading: there is no strictly superior solution for this tradeoff. Futures/promises encode concurrency as a monad and like every monad they are viral and once you start propagating them around, they can be a pain. But the alternative for not encoding some concern of your program as a monad is to let it pass through an implicit side-channel (I/O, error-handling, global state, built-in scheduler) that is harder to control and can't be treated as first-class object in the language. Go itself chose to go the explicit with error-handling. It doesn't even uses Monads, so it gets the worst of both worlds in my opinion, but the designers' main concern with exceptions which they managed to avert is still valid: with (unchecked) exceptions you never know which function you call is going to raise an error on you somewhere down the line. So yes, with Go you'll never have with unexpected throws, but you'll have to deal with unexpected goroutine suspension. In Go all of your functions will be happily colorless with regards to concurrency but will be colored with regards to error-handling. Again, both choices here are valid, but they are not so brain-dead simple or universal as this article is trying to put them.
deleted 10y ago
[deleted]
buzzybee 10y ago
On the other hand, it's certainly more engineering to have to build a custom format. I implemented a story engine that compiles to behavior trees, and then subsequently to a small instruction set in this past year and, well, we're talking about three months to go from having no technology to having a pipeline capable of complex scenarios. But I came out of it with a much more competent strategy for addressing assets than anything I've done in the past, so I'm not complaining. It is the core technology of the game(choice driven, text simulation) so it needed a big investment. And the game is going to have content updates, which makes save games terrifying; In the end, I designed it so that it has episodes and restarting the game starts it from that episode so that I don't have to worry about massive amounts of persistent state getting permanently corrupted.
gravypod 10y ago
You could do this in C with something like libdill as they give you a handle to a coroutine. All you need to do then is create a state_t that holds the coroutine reference managing the object, the channels for it, and all of the state needed for the computation. During saving you prune dead and save only the computation state. During loading you recreate all of your Objects. It wouldn't be all that slow and I think it would get over most of the overhead in some implementations I've seen of UI/Interaction code for games.
- SomeCallMeTim 10y ago
  I actually hacked together my own "coroutine" scripting library in C once upon a time. Lots of macros. Same problem though. When I said slow, I meant slow to save the entire state of the Lua VM. You can't even do that in C, so it doesn't really apply to the problem I was pointing out.
  - gravypod 10y ago
    All saveing your object state should be is memcpy'ing your array of all objects into some sort of static memory and on load looping through every object and starting a coroutine for every object. I don't think that's too much overhead.
    - SomeCallMeTim 10y ago
      Sorry, but you're totally missing the key point. Say you have an AI that's made 3 decisions and you're in the MIDDLE of a coroutine where it's waiting for the next criteria that will shift it to its next state. So the AI code looks like: local player = lookForAPlayer() if navigateToPlayerLocation() then local quest = givePlayerQuest(); while (!quest:done()) do waitWithStandardReply("Waiting for you to finish the quest!"); end if (quest:resolved()) then giveQuestReward(); while (true) do waitWithStandardReply("Thanks for finishing that quest for me!"; end else -- quest abandonded while (true) do waitWithStandardReply("Thanks for nothing!"); end end end This is rough example code, and not how I'd probably handle this case specifically, but it gives you the general idea. Each of those functions (except the quest member functions) has, somewhere inside it, one or more "yield()" statements, and some logic to make its goal happen incrementally. Yes, it's awesome to be able to code AI logic that way. But how do you save the coroutine state? The actual execution state of the coroutine itself, including what line it will resume next, the values of any bound variables, the values of any local variables, are all part of the state of the AI in that case. In C, you've got functions that may be relocated somewhere else in RAM and potentially stacks to save (if you're using a "real" coroutine library), so just doing a memcpy won't work. What would you memcpy to save the line that the script is paused on? In Lua, it can be done with Pluto, but then you can't use LuaJIT. I see libdill has the ability to use a user-supplied stack, but it doesn't appear to have a "resume" call that takes such a stack memory block, so the best that would happen is that the coroutine would start over from the beginning. Meaning that if you saved the above coroutine in the middle of a quest, the character involved would offer you a new quest when you saw them instead of accepting the result of a successfully allocated quest. If your object states are all being handled in traditional objects, then you're not using the coroutines in the way described by the article.
pcwalton 10y ago
> I think that, on balance, actually creating a data format that represents the state and "coding" using that plus state machines is actually a more robust and useful way to accomplish the same thing. Manually coding up a "state machine" is kind of reinventing the Lua interpreter, no?
- mmjaa 10y ago
  As a long-term Lua fan, I'm with you. I think that the OP is really complaining about not understanding the Lua VM well enough to push it to its limits - in fact, coding up a 'proper state machine' that can compete with what the Lua VM, itself, has to offer is going to be very, very difficult. That said, I'm in favour of other approaches to this problem - using (i.e. generating) Lua bytecode itself as a way of maintaining state, and then pushing things around while having a good understanding of llimits.h seems to be quite important ..
  - SomeCallMeTim 10y ago
    Sorry, but you're pretty profoundly wrong. I have written an entire (popular) game engine that tightly integrated Lua and used custom-generated C++ bindings. I had to dig pretty deep into the Lua VM to accomplish that. Heck, I found and fixed a bug in the LuaJIT 1.x series -- my fix integrated by Mike Pall, with a credit on the changes page. The thing is if you encode game state into the program state, like using coroutines for complex, long-running AI tasks, for instance, then there's no way to save or restore that state if you're using LuaJIT. And Pluto saves the state in an opaque manner. And as another commenter pointed out, if you want to update the game with new behaviors, god knows how that will interact with save states that have huge call stacks in them. And bug fixes? Yeah, since you're saving the whole program state at this point, bug fixes won't be incorporated in the restored image.
    - optionalparens 10y ago
      Agreed. I've seen things like this tried in Smalltalk images and it was usually a disaster long-term, meaning that the state was simply thrown out and a new image spun up because you're SOL. Either that, or you need tons of logic to handle the old program state and it becomes a convoluted mess. Most of the games and engines I've ever worked on that have dialog systems expressly avoid encoding things into the program state as much as reasonably possible. This is not only for restoring state, but also making for a saner experience editing and changing things during development. Very rarely does the format you encode things like dialog stay the same during a dev cycle unless you're using a canned engine and don't make any serious changes. Using Lua program state itself seems like a way to drive yourself mad by the end of the dev cycle, and make it incredible unfriendly to work with writers and other content creators. People who work with asset pipelines often do a lot of cruel and unusual data conversions and munging to bridge how different people and systems need to work, but eventually things in the actual game engine are best kept as simple data that can be read in and reproduced at any time. Moreover, as a developer, I should be able to load up any state of my game at any point in the game just by providing the necessary input data as possible, and like a pure function, it should produce the same game state every time if possible. Let's also not forget other things like internationalization and localization. As far as interfacing with Lua, I've mostly done it in the last few decades in a simple way where we handed entity component-style data that is just pure data, i.e. no function pointers and crazy stuff, and either let Lua do the rest with that state, or send pure data back to the C++ when needed. As for the data format itself, I've seen everything from XML to custom binary formats to CSV and Excel used for dialog. I think I'd sooner murder someone if game dialog lived in code, even in a Lua script.
jamesu 10y ago
An additional problem is what happens when you need to patch the game? If you need to modify some form of quest or dialogue script written as a coroutine, how do you migrate the state? What if you've changed the script 5 times in a series of patches, and the user is still on version X? Then again you still have this problem even if you power your quest/dialog with data instead. Without sufficient information it's practically impossible to make a smooth migration.
leggomylibro 10y ago
That is definitely an issue with this sort of thing; I think that you'd need to organize your conversations in a tree structure, and have 'save/load' methods for each node that could transcribe between a serialized data format like json or yaml, and functions in lua. With a format like that, you have the option of writing your dialogues in code or plain text, and easily switching between the two. The only major downside is that you can't easily define arbitrary behavior that way; every unique action that a dialogue event could trigger would need to be transcribed in your 'save/load' methods, which may feel limiting if you have a lot of hard-to-generalize logic.

hypothete 10y ago

Here's a similar implementation on the PICO-8 platform, which uses a subset of Lua: http://www.lexaloffle.com/bbs/?tid=3833 http://www.lexaloffle.com/bbs/?tid=3833 This author demonstrates a multiple choice ask() function as well.

EekSnakePond 10y ago

Robust RPG dialogue and trigger system for JavaScript: https://github.com/codeotter/thusspokenpc https://github.com/codeotter/thusspokenpc

dividuum 10y ago

For simple dialogs that don't have to be saved (see the other discussions), this seems like an easy way to quickly put any format of dialog system together without reinventing your own DSL to encode those dialogs as state machine(s). You can easily compose dialogs just by calling functions and thanks to Luas tail call optimization you could even jump between dialogs without the fear of running out of stack space. I'm not saying it's the best solution but it's pretty simple and probably easy to understand. I've used a similar approach way back in a networked multiplayer game where players could telnet into the game server and use text based menus to control the game. You can see the relevant Lua code here: https://github.com/dividuum/infon/blob/master/server.lua https://github.com/dividuum/infon/blob/master/server.lua. I don't want to imagine how this code would look like without the use of coroutines.

Pica_soO 10y ago

Im wondering, could one auto-convert JavaScript- librarys to lua-code?

veli_joza 10y ago
The trend is to compile anything under the sun into JS, not the other way around. Although they are quite similar under the hood, Lua programmers take pride in Lua's elegant and minimalistic design, along with JIT speed. JS warts would not be very welcome. That said, there was a project called lua-colony that attempted the conversion.
etiene 10y ago
There are some JS to Lua transpilers around, but I'm not sure how robust they are. I want to try some of them later to try to convert some handy JS libraries and feed them into LuaRocks.
- Pica_soO 10y ago
  The Git-hub link, please?

optionalparens 10y ago

I can see how it might be fun to do something like this for a blog post or simple project, but in a real game this just doesn't work for many reasons, most listed here already. There are tons of ways of doing dialog system. Mostly it comes down to your specific game and I highly recommend against generic dialog systems. Rather, the most you can do at a generic level is try to come up with a good way to store dialog and retrieve it in the ways you need. This isn't the best thing I've seen or worked with, but here's an easy to understand and documented method and presentation about it form Valve: http://www.gdcvault.com/play/1015317/AI-driven-Dynamic-Dialog-through http://www.gdcvault.com/play/1015317/AI-driven-Dynamic-Dialo... Anyway, I've been programming games and game engines for several decades, and coroutines are always in the list of things that you can throw in the junk pile along with other cool things like continuations, various theoretical data structures, fancy dispatch mechanisms, events/signals, application-wide functional purity, etc. Of course all of these things are useful, but they tend to have problems that make them of limited use in professional game development. Here's a brief checklist of show-stoppers off the top of my head I tend to go through before introducing anything "creative" into a game, and especially a game engine. Most of the aforementioned items fail one or more of these. - Performs awful in common cases or when applied to actual game-like conditions vs. theoretical - Murders cache lines - Hard or impossible to debug - Hard to save, load, serialize, deserialize, or snapshot - Doesn't work over networks - Doesn't play nice with libraries or data coming from other libraries or languages (ex: 3rd party physics lib) - Unpredictable execution start or end times - Non-determinant memory usage or allocation patterns - Risk of stack-overflows - Non-portable or problems on specific target architectures - Requires "religion" meaning it infects all your code or forces you to write all of your code a certain way, everywhere - Fragments memory - Long blocking or uninterruptable processing Conversely, besides the obvious opposites of most of the above, I look for the following when introducing some major data structure or conceptual item like coroutines to a game or game engine: - Leads to predictable, consistent code and resource usage - Data-driven - Editor friendly (as a corollary to the above) - Clear debugging story - Both simple and easy if possible. No, they are not the same. - Plays well with the world around it - Portable - Fast - Loved by the CPU, GPU, and/or compiler, and produces reasonable assembly on target hardware - Easy for someone else to jump in and understand the flow and how it works, assuming they are not a moron. Pretty much these two lists mean that games tend to be made of meat and potatoes things. Simple data structures like arrays, custom allocators, predictable branching and execution patterns, up-front memory allocation, and CPU and GPU loving goodness. Pretty much anything stashing things away and building up massive states or jumping around all over the place, pointer chasing, and so-on should always be a no-go. Anyway, if you're building a truly simple game, as in a 2 day sort of affair, do whatever you want. If you want to build something even remotely complex that you will work on for awhile, it's best to do things the right way which means detach yourself from everything you think you know about most of computer science and think in terms of pipelines, CPU instructions, caches, and predictability among other things. This often means programming against the grain of your language a bit, throwing out sugar, std libraries, and all kinds of things. It really sucks, but that's the essence of good game programming. I hope one day it will be different, but as long as there's still need to push gameplay and graphical boundaries, professional developers almost always need to squeeze out what they can.

managun 10y ago

That's interesting. One of my pet projects is a compiler for Bethesda games quests - having source in text/plain file(s), that's the motivation - and as I am designing the underlying language, maybe I should consider coroutines as well...

shinymark 10y ago

I was the producer and an engineer on Grim Fandango Remastered, a remaster of one of the first games to use Lua (2.5, later 3.0 alpha). And the lead engine programmer on the original game in 1998, Bret Mogilefsky, added coroutines to the Lua language in the game engine before they were officially added as an official language feature. They are very important for scripting adventure gameplay in an intuitive way for non-engineers and helped the original team ship Grim Fandango in 1998. It was similar to what LucasArts had done with SCUMM, their adventure game domain specific language, previously. To reference SomeCallMeTim's comment - the way the save game system worked in Grim Fandango was to serialize the entire Lua environment! Stack, all code, etc! And too jamesu - when you patch the game you patch the code that was serialized into the save game! The story behind all of this in Grim is pretty interesting. If anyone is interested in more details please come to the Game Developers Conference this year as I'm giving a talk on the technical details of how the original game and the remaster works. Oliver Franzke and I will cover both the Grim Fandango and Day of the Tentacle remasters.

veli_joza 10y ago
I'd love to hear all the gritty details about game internals, but I cannot attend GDC (wrong continent). Please submit the talk to HN once it's available.
anonymoushn 10y ago
I hope there's a video of this talk!