nickwalton00

13 ms·

1. 31.▲
   by nickwalton00 7y ago

   You'd need a GPU as good as the ones google colab offers which is a 12GB NVIDIA Tesla K80. You'd also need cuda and tensorflow installed.
2. 32.▲
   by nickwalton00 7y ago

   Thanks!
3. 33.▲
   by nickwalton00 7y ago

   Yeah I've thought a lot about that. It's not too difficult to go from game data -> model output, but it's not as clear how to go from model output -> game data.
4. 34.▲
   by nickwalton00 7y ago

   That's amazing.
5. 35.▲
   by nickwalton00 7y ago

   I used a DGX 1 to train, I trained for somewhere around 12-16 hours down to a loss of something around ~2.2-2.3.
6. 36.▲
   by nickwalton00 7y ago

   Had to go check my training file to remember. Datasize: Around 30 MB, so around ~8000000 token? Can't remember exactly Learning Rate: was 1e-4, so I guess not that slow. I trained for around 1000 steps, but ended up liking the model fr
7. 37.▲
   by nickwalton00 7y ago

   Haven't seen this before, but if you click on the boxes where the code is and press Shift+Enter a few times it should work
8. 38.▲
   by nickwalton00 7y ago

   Sometimes the colab notebook freezes, I'm not totally sure why it happens, but if you stop the cell you can get an ID to reload your game. Then just restart the cell and enter the load <ID> for an action and you should be able to
9. 39.▲
   by nickwalton00 7y ago

   Hmm I was doing that before and was still paying decent costs without a huge increase in download speed.
10. 40.▲
    by nickwalton00 7y ago

    I couldn't find a good adventure game text corpus so I ended up scraping my own from chooseyourstory.com
11. 41.▲
    by nickwalton00 7y ago

    I did have access to a cluster of GPUs through my professor's lab so compute wasn't as much of an issue. And it may be true that 774M is equally effective. I haven't played around with 774M enough to know. I had a decent amou
12. 42.▲
    by nickwalton00 7y ago

    It's actually reacting to what you say. Your action is appended onto the previous story and fed into the model to generate the next part of the result. Sometimes it does fail to tie the result close enough to the action you had, perhap
13. 43.▲
    by nickwalton00 7y ago

    Yeah there's definitely times where it's not as coherent as you would hope, other times though it blows me away with certain adventures.
14. 44.▲
    by nickwalton00 7y ago

    Yeah that's a really good future/research question. I've thought some about how you could do more structured generation. Maybe you could do some kind of reverse summarization, where you take a plot outline and expand out the
15. 45.▲
    by nickwalton00 7y ago

    Yeah definitely. It does lack the same sense of progression that you have with a game someone carefully crafted, it's almost more like collaborative storytelling than a game in that sense. It's pretty cool to see the adventures yo
16. 46.▲
    by nickwalton00 7y ago

    This is a project I've been working on for the past several months and it has blown me away with what it can do. Definitely recommend trying it out. I think the coolest thing about it is in any other game you are constrained in what yo
17. 47.▲
    Show HN: AI Dungeon 2 – AI-generated text adventure built with 1.5B param GPT-2(aidungeon.io)

    584 points by nickwalton00 7y ago|220 comments
18. 48.▲
    by nickwalton00 7y ago

    Best would be to run it in a colab notebook yourself. https://colab.research.google.com/github/ilopezfr/gpt-2/blob...
19. 49.▲
    by nickwalton00 7y ago

    Thanks! I just used the vanilla GPT-2 model, but I'm planning on fine tuning it on some fantasy data. If OpenAI releases their large model I would love to try it out and see how much it improves it.
20. 50.▲
    by nickwalton00 7y ago

    I just used the vanilla GPT-2 model which was trained on large swathes of the internet so that explains the random Donald Trump stuff haha.
21. 51.▲
    by nickwalton00 7y ago

    ...so it's threatening because it's better haha.
22. 52.▲
    by nickwalton00 8y ago

    Thanks!
23. 53.▲
    by nickwalton00 8y ago

    Thanks haha! It's sometimes incoherent, but it's fun!
24. 54.▲
    by nickwalton00 8y ago

    AI Dungeon is an AI generated text adventure that uses deep learning to create each adventure. It uses OpenAI's new GPT-2 model, which has 117 million parameters, to generate each story block and possible action. I started making it as
25. 55.▲
    AIDungeon, a deep learning generated text adventure (browser playable)(aidungeon.io)

    11 points by nickwalton00 8y ago|5 comments
26. 56.▲
    by nickwalton00 8y ago

    I did the same thing. Unfollowed everyone but super close family and friends. It does wonders to clear addiction. Now I check it for maybe 5-10 minutes once a week and that's it.
27. 57.▲
    by nickwalton00 8y ago

    This is just my two cents but I think lots of practice with various companies will help you get more confident. Apply to a bunch of jobs you probably wouldn't even want to do. You'll be less stressed about them because you don&#x2
28. 58.▲
    by nickwalton00 8y ago

    Yeah I agree. I like knowing rough averages of what similar roles are making but when it comes to individual people's salaries it can be difficult.
29. 59.▲
    by nickwalton00 8y ago

    Well said, I think where all things indicate it will be healthy normal births should be the default, but when medical risks indicate otherwise than c sections can provide life saving help.
30. 60.▲
    by nickwalton00 8y ago

    I've found that the most effective strategy was to unfollow (not unfriend) everyone except close friends and family. Now my Facebook feed only has people I care about and takes me max 5 min a day before it's the same content.