4 ms·

2. Translating between two different coding languages (migration) I have a game written in XNA 100% of the code is there, including all the physics that I han

2. Translating between two different coding languages (migration) I have a game written in XNA 100% of the code is there, including all the physics that I hand-wrote. All the assets are there. I tried to get Gemini and Claude to do it numerous times, always with utter failure of epic proportions with anything that's actually detailed. 1 - my transition from the lobby screen into gameplay? 0% replicated on all attempts 2 - the actual physics in gameplay? 0% replicated none of it works 3 - the lobby screen itself? non-functional Okay so what did it even do? Well it put together sort of a boilerplate main menu and barebones options with weird looking text that isn't what I provided (given that I provided a font file), a lobby that I had to manually adjust numerous times before it could get into gameplay, and then nonfunctional gameplay that only handles directional movement and nothing else with sort of half-working fish traveling behavior. I've tried this a dozen times since 2023 with AI and as late as late last year. ALL of the source code is there every single thing that could be translated to be a functional game in another language is there. It NEVER once works or even comes remotely close. The entire codebase is about 20,000 lines, with maybe 3,000 of it being really important stuff. So yeah I don't really think AI is "really good" at anything complex. I haven't really been proven wrong in my 4 years of using it now.

KronisLV 7mo ago
I crave to see people saying "Here's the repo btw: ..." and others trying out porting it over, just so we see all of the ways how AI fails (and how each model does) and maybe in the middle there a few ways to improve its odds. Until it eventually gets included in training data, a bit like how LLMs are oddly good at making SVGs of pelicans on bicycles nowadays. And then, maybe someone slightly crazy comes along and tries seeing how much they can do with regular codegen approaches, without any LLMs in the mix, but also not manual porting.
- Madmallard 7mo ago
  I'm sure someone who regularly programs games in the destination language I want who also has worked with XNA in the past as a game developer could port it in a week or something yeah
- jb_hn 7mo ago
  Agreed -- coding agents / LLMs are definitely imperfect, but it's always hard to contextualize "it failed at X" without knowing exactly what X was (or how the agent was instructed to perform X)
sheeshkebab 7mo ago
In those situations you basically need to guide llm to do it properly. It rarely one shots complex problems like this, especially in non web dev, but could make it faster than doing it manually.
- Madmallard 7mo ago
  Oh believe me I broke it down super finely, down to single files and even single functions in some places It still is completely and utterly hopeless
  - sigseg1v 7mo ago
    I've done this multiple times in various codebases, both medium sized personal ones (approx 50k lines for one project, and a smaller 20k line one earlier) and am currently in the process of doing a similar migration at work (~1.4 million lines, but we didn't migrate the whole thing, more like 300k of it). I found success with it pretty easily for those smaller projects. They were gamedev projects, and the process was basically to generate a source of truth AST and diff it vs a target language AST, and then do some more verifier steps of comparing log output, screenshot output, and getting it to write integration tests. I wrote up a bit of a blog on it. I'm not sure if this will be of any use to you, maybe your case is more difficult, but anyway here you go: https://sigsegv.land/blog/migrating-typescript-to-csharp-accurately/ https://sigsegv.land/blog/migrating-typescript-to-csharp-acc... For me it worked great, and I would (and am) using a similar method for more projects.
    - mettamage 7mo ago
      super cool, don't have the time to read it right now but to think in terms of ASTs is pretty handy!
    - Madmallard 7mo ago
      "I also wanted to build a LOT of unit tests, integration tests, and static validation. From a bit of prior experience I found that this is where AI tooling really shines, and it can write tests with far more patience that I ever could. This lets it build up a large hoard of regression and correctness tests that help when I want to implement more things later and the codebase grows." The tests it writes in my experience are extremely terrible, even with verbose descriptions of what they should do. Every single test I've ever written with an LLM I've had to modify manually to adjust it or straight up redo it. This was as recent as a couple months ago for a C# MAUI project, doing playwright-style UI-based functionality testing. I'm not sure your AST idea would work for my scenario. I'd be wanting to convert XNA game-play code to PhaserJS. It wouldn't even be close to 95% similar. Several things done manually in XNA would just be automated away with PhaserJS built-ins.
PacificSpecific 7mo ago
This matches my experience. Unless it's been done to death online (crud etc) it falls on its face every time.
- Madmallard 7mo ago
  It's okay shills and those with stock in the AI copies brainwashed themselves inside out and will spam forever on this website that if you just introduce the right test conditions agents can do anything. Never mind engineering considerations if it passes the tests it's good homie! Just spent an extra few hundred or thousand a month on it! Especially on the company I have stock in! Give me money!
KellyCriterion 7mo ago
- Split it in different modules / tasks - Do not say: "just convert this" - On critical sections you do a method-per-method-translation - Dont forget: your 20.000 lines source at a whole will make any model to be distracted on longer tasks (and sessions, for sure) - Do dedicated projects within Claude per each sub-module
vb7132 7mo ago
Yes, you are right: amongst the four points, migration is the most contentious one. You need to be fairly prudent about migration and depending on the project complexity, it may or may not work. But I do feel this is a solvable problem long term.

KronisLV 7mo ago

I crave to see people saying "Here's the repo btw: ..." and others trying out porting it over, just so we see all of the ways how AI fails (and how each model does) and maybe in the middle there a few ways to improve its odds. Until it eventually gets included in training data, a bit like how LLMs are oddly good at making SVGs of pelicans on bicycles nowadays. And then, maybe someone slightly crazy comes along and tries seeing how much they can do with regular codegen approaches, without any LLMs in the mix, but also not manual porting.

Madmallard 7mo ago
I'm sure someone who regularly programs games in the destination language I want who also has worked with XNA in the past as a game developer could port it in a week or something yeah
jb_hn 7mo ago
Agreed -- coding agents / LLMs are definitely imperfect, but it's always hard to contextualize "it failed at X" without knowing exactly what X was (or how the agent was instructed to perform X)

sheeshkebab 7mo ago

In those situations you basically need to guide llm to do it properly. It rarely one shots complex problems like this, especially in non web dev, but could make it faster than doing it manually.

Madmallard 7mo ago
Oh believe me I broke it down super finely, down to single files and even single functions in some places It still is completely and utterly hopeless
- sigseg1v 7mo ago
  I've done this multiple times in various codebases, both medium sized personal ones (approx 50k lines for one project, and a smaller 20k line one earlier) and am currently in the process of doing a similar migration at work (~1.4 million lines, but we didn't migrate the whole thing, more like 300k of it). I found success with it pretty easily for those smaller projects. They were gamedev projects, and the process was basically to generate a source of truth AST and diff it vs a target language AST, and then do some more verifier steps of comparing log output, screenshot output, and getting it to write integration tests. I wrote up a bit of a blog on it. I'm not sure if this will be of any use to you, maybe your case is more difficult, but anyway here you go: https://sigsegv.land/blog/migrating-typescript-to-csharp-accurately/ https://sigsegv.land/blog/migrating-typescript-to-csharp-acc... For me it worked great, and I would (and am) using a similar method for more projects.
  - mettamage 7mo ago
    super cool, don't have the time to read it right now but to think in terms of ASTs is pretty handy!
  - Madmallard 7mo ago
    "I also wanted to build a LOT of unit tests, integration tests, and static validation. From a bit of prior experience I found that this is where AI tooling really shines, and it can write tests with far more patience that I ever could. This lets it build up a large hoard of regression and correctness tests that help when I want to implement more things later and the codebase grows." The tests it writes in my experience are extremely terrible, even with verbose descriptions of what they should do. Every single test I've ever written with an LLM I've had to modify manually to adjust it or straight up redo it. This was as recent as a couple months ago for a C# MAUI project, doing playwright-style UI-based functionality testing. I'm not sure your AST idea would work for my scenario. I'd be wanting to convert XNA game-play code to PhaserJS. It wouldn't even be close to 95% similar. Several things done manually in XNA would just be automated away with PhaserJS built-ins.

PacificSpecific 7mo ago

This matches my experience. Unless it's been done to death online (crud etc) it falls on its face every time.

Madmallard 7mo ago
It's okay shills and those with stock in the AI copies brainwashed themselves inside out and will spam forever on this website that if you just introduce the right test conditions agents can do anything. Never mind engineering considerations if it passes the tests it's good homie! Just spent an extra few hundred or thousand a month on it! Especially on the company I have stock in! Give me money!

KellyCriterion 7mo ago

- Split it in different modules / tasks - Do not say: "just convert this" - On critical sections you do a method-per-method-translation - Dont forget: your 20.000 lines source at a whole will make any model to be distracted on longer tasks (and sessions, for sure) - Do dedicated projects within Claude per each sub-module

vb7132 7mo ago

Yes, you are right: amongst the four points, migration is the most contentious one. You need to be fairly prudent about migration and depending on the project complexity, it may or may not work. But I do feel this is a solvable problem long term.