4 ms·

I really like these tests for what its worth and I see them on youtube sometimes. I would like to ask a few things though TLDR: Basically focusing on recreatin

I really like these tests for what its worth and I see them on youtube sometimes. I would like to ask a few things though TLDR: Basically focusing on recreating pay to win (mobile or otherwise) games and recreating them non pay to win perhaps instead of focusing on recreation of minecraft for benchmarks could have a genuinely meaningful impact, and making these games portable as well could be another interesting idea. [so it can be played on any operating system/device so using web or if native then for (Android/IOS/Linux/Windows/MacOS) using game engines like (Preferably godot)/Unity/UE.] Could the test focus more on pay to win games with unique dynamics. For example: I literally wanted to create a clash royale recreation because clash royale is a highly pay to win game. The game is unique enough to have memories but is pay to win enough that it ragebaits me as to what its current situation is, its so pay to win now. A recreation would have genuine effect whereas yet another minecraft clone wouldn't. I have some fond memories of the game and my brother and I used to play it (my brother moreso than me). Also clash of clans and clash royale famously prevented windows users. So I remember downloading bluestacks to play it on laptop but it required 2GB of ram and back then we only had 1GB. (Ironic that we might come back to that time) Another question that I have for you which I have been genuinely curious is: who is footing the bill for these benchmarks and youtube videos. What are the economics surrounding it? I imagine the bill to run quite hot sometimes and I find running these benchmarks to be quite unaffordable personally. I also wish to ask if you have any theories as to why not people on Youtube share their videos. I found this [0] Minecraft clone by Fable 5.1 extremely good yet they haven't shared the source. I am unsure as to what exact reasons might be behind most youtube videos on recreation with AI to not share the actual code. I don't find much rationale in not sharing AI generated code of a recreation of a game especially if one is making a video about it. So thanks for once for actually sharing the output code as well as I surprisingly found it to be a bit rare! [0]: https://youtu.be/I0do_vbnMBI?t=361 https://youtu.be/I0do_vbnMBI?t=361

senko 26d ago
I'd make a distinction between youtubers who are incentivized to dial their reactions to the max for everything[0] and people doing silly tests but keeping their expectations and reactions real (most famously, Simon's pelican test). As to why the prompts aren't shared, for these more complex things it's most likely a somewhat messy process (ie. not a single-prompt one-shot creation; some back & forth) that would make the whole thing seem less spectacular. Likewise for the end result - it's probably cherry-picked what works well. For example, in Claude models' resuts I always get stuck in water (something about height/jump calculations is off), where with Astra I didn't have that problem. These sorts of issues you can only spot if you try to playthrough yourself. This is just my speculation tho. As for me, I do the tests because I'm interested in the results (easy comparison across time & models) - then I started sharing them because people asked. I have OpenAI and Anthropic subscriptions so testing these is not an extra cost for me (I'm sloppy around recording the tokens & API-equivalent cost tho - have to improve on this). For the other models, it's total a few bucks per month or so. So if you're careful about the cost, it's not too much, especially for a serious youtuber who's doing it for commercial reasons. Finally regarding your comment about cloning the popular enshittified games - I don't think people are going to be doing that for testing, but if you want to have a different spin (or do a close-enough clone for yourself) on a game you loved, the modern AI systems can often deliver! [0] from the link you posted "i am in disbelief, this is insane, bro what is this, you can't believe it's ai" - yeah...umm, it's not that good :)

senko 26d ago

I'd make a distinction between youtubers who are incentivized to dial their reactions to the max for everything[0] and people doing silly tests but keeping their expectations and reactions real (most famously, Simon's pelican test). As to why the prompts aren't shared, for these more complex things it's most likely a somewhat messy process (ie. not a single-prompt one-shot creation; some back & forth) that would make the whole thing seem less spectacular. Likewise for the end result - it's probably cherry-picked what works well. For example, in Claude models' resuts I always get stuck in water (something about height/jump calculations is off), where with Astra I didn't have that problem. These sorts of issues you can only spot if you try to playthrough yourself. This is just my speculation tho. As for me, I do the tests because I'm interested in the results (easy comparison across time & models) - then I started sharing them because people asked. I have OpenAI and Anthropic subscriptions so testing these is not an extra cost for me (I'm sloppy around recording the tokens & API-equivalent cost tho - have to improve on this). For the other models, it's total a few bucks per month or so. So if you're careful about the cost, it's not too much, especially for a serious youtuber who's doing it for commercial reasons. Finally regarding your comment about cloning the popular enshittified games - I don't think people are going to be doing that for testing, but if you want to have a different spin (or do a close-enough clone for yourself) on a game you loved, the modern AI systems can often deliver! [0] from the link you posted "i am in disbelief, this is insane, bro what is this, you can't believe it's ai" - yeah...umm, it's not that good :)