4 ms·
I go into it more in the README but yes, GPT-2 overfits Magic cards very quickly despite my best efforts (usually it shifts the mana cost/rarity a bit more thou
by minimaxir 7y ago
I go into it more in the README but yes, GPT-2 overfits Magic cards very quickly despite my best efforts (usually it shifts the mana cost/rarity a bit more though than your example).
- YeGoblynQueenne 7y agoHm. Sorry to hear that. I'm looking into the pregenerated dump you posted on github earlier and I'm seeing a lot of cards that are basically copies of existing cards. These are from mtg-gpt-2-cloud-run/generated_card_dumps/temp_0_7/cards_0.txt and just the top few rows (also, only the ones I immediately recognised): Shrieking Specter {6}{B} (common) Creature ~ Specter (4/4) Flying Whenever Shrieking Specter attacks, defending player discards a card. Luminous Bonds {2}{W} (common) Enchantment ~ Aura Enchant creature Enchanted creature can't attack or block. Mwonvuli Acid-Moss {3}{G} (uncommon) Sorcery Destroy target land. Search your library for a forest card and put that card onto the battlefield tapped. Then shuffle your library. Druid'S Deliverance {1}{G} (common) Instant Prevent all combat damage that would be dealt this turn. Populate. And so on. Don't ask me why I immediately recognised Shrieking Specter. It would take a bit of work to figure out which of the generated cards are mashups of existing cards, or the characteristics of existing cards with new names, and which ones are original. Do you have a way to figure this out? Hey, this is an interesting project regardless. At least with M:tG cards it's very easy to see when GPT-2's output is identical to its input. Imagine trying to figure that out with arbitrary text... Edit: Oh, the README. Reading it now.
- mxwsn 7y agoTo avoid copying the training set you can increase the temperature at sampling time. This should trade off between accuracy (maximum likelihood) and quirkiness (more stochastic background).
- minimaxir 7y agoSee the dumps for various sampling temperatures. Even at 1.2 (versus 0.7), there is some evidence of overfitting. The app uses a random temperature between 0.7 and 1.2 to prevent it from going off-the-rails.
- ackbar03 7y agoI'd imagine it'd be hard to avoid? There isn't exactly a lot of text on these cards, with so many parameters in the model it wouldn't be surprising that it overfits
- SilasX 7y agoI think I found another bug of some kind. For "Demonic trade 1B", I got the following, but black shouldn't have artifact destroyers. Demonic Trade {1}{B} (common) Sorcery Buyback {2} Destroy target artifact.
- minimaxir 7y agoColor pie violations are not an inherent bug, just an outcome of the inherent random nature of AI. This AI does a better job than previous ones of following the color pie.