4 ms·

I was a little shocked that they could get it fully working in a week to be honest. My side project is a very similar ambition (https://tsz.dev https://tsz.dev)

by mohsen1 5mo ago

I was a little shocked that they could get it fully working in a week to be honest. My side project is a very similar ambition (https://tsz.dev https://tsz.dev) but I am in no way claiming success. i keep adding more and more tests to ensure things works. Even after all of TypeScript's own tests pass I am finding bugs which I was totally expecting. The bar for matching tsc's behavior is really _really_ high. see: https://github.com/type-challenges/type-challenges https://github.com/type-challenges/type-challenges I'm not against using LLMs to write a lot of code. But verification should be 100x more robust now that we can output code at this rate.

tapirl 5mo ago
Are there any evidences which prove the process was done in a week?
cedws 5mo ago
I'm stunned that it went from 'this is an experiment' to merging a ~million lines of (likely) unreviewed code in a week. I have nothing against using agents but to rush something like this and leave the community blindsided seems extremely ameteurish. Like something you'd expect a bright eyed graduate engineer to do.
- mohsen1 5mo ago
  tsz for me is an experiment to see how can this kind of work be done better. With a slight difference that tsz is not a direct port and it's a different architecture. I'm also not claiming to have answers but I've learned a ton. A few things that works - Test before code, Bun had lots of test so that's good but maybe they could start by asking Mythos to write like 20k additional tests that pass on Zig Bun first. - Deterministic anti-slop features. LLMs love to solve the problem in the wrong abstraction layer or place. There are many ways to catch this with deterministic tests. I do this in tsz a lot - Roadmap that constantly evolving by humans. - Taking a pause and looking how the progress is going and undoing slop - Fuzztest(https://github.com/google/fuzztest https://github.com/google/fuzztest) style "trying to break things" with the powers of LLM
- singpolyma3 5mo ago
  Blindsided? Has there even been a release yet?
  - koteelok 5mo ago
    Yes, they released a canary version.
    - edenstrom 5mo ago
      True enough. Every merge to main releases a canary though
  - silverwind 5mo ago
    Knowing them, I expect a release probably next week, have fun in production with it.
hayd 5mo ago
I suspect they've been planning this and experimenting for many months. Along with the large existing test suite, they have lots of tooling for parallelizing agents and an unlimited token budget. So don't feel too bad..
- jaapz 5mo ago
  The author was on here claiming it was an experiment just last week. Didn't look like they have been planning much of this, but they did have some trouble with zig to the tune of having to maintain their own fork that the zig maintainers didn't want to merge. So people probably could have seen this coming If I was the bun maintainers I wouldn't want to have to maintain a language fork either, when there are viable alternatives. Now with LLM's getting stronger and stronger a rewrite like this is getting more and more viable, and it looks like they are just approaching this like the "how do you eat an elephant? One bite at a time"-method. Not sure why people thing this is an elaborate PR campaign, we already know LLM's are increasingly powerful for coding tasks.
jwilliams 5mo ago
tsz looks impressive. Congrats!
nextaccountic 5mo ago
https://tsz.dev/sound-mode/ https://tsz.dev/sound-mode/ This is awesome. Typescript really needs more of this. I hope this gets more publicized and perhaps get adopted by Microsoft. I am not sure you should call it sound mode though. > It is not a mathematical proof of soundness, and it does not make third-party .d.ts files truthful. Here there are two completely unrelated things First, soundness is a mathematical thing (sorry). If something is sound, it's really really sound. Soundness is the thing that says we don't need to manually check the details - we can trust the compiler to do the right thing. It will do the wrong thing if there's a bug, but the implementation can be fixed. Soundness means there's is no bug in the spec, no bug in the type system that would make things go wrong even in theory. Second, and here's the important thing.. it's entirely okay and expected that real world languages need to have unchecked features where we trust the humans to do the right thing, rather than trusting the compiler to verify. That's like Haskell's unsafeCoerce or Java's sun.misc.unsafe. The problem really is when the type system breaks even when we don't do any of those
- mohsen1 5mo ago
  Thank you for the feedback. I think I should rename the "Sound Mode" to something more honest. "Strict" is already taken so I should think of a better name. Making tsz compatible with tsc and making it sound is mathematically impossible. The history of this stuff (https://hegel.js.org https://hegel.js.org) is telling. If you really make a sound JS checker, nobody will really use it unless you build an entire ecosystem as big as npm with it. TypeScript's decision to embrace existing npm ecosystem and layering .d.ts on top was a very practical decision that made it so successful.
  - tialaramex 5mo ago
    > TypeScript's decision to embrace existing npm ecosystem and layering .d.ts on top was a very practical decision that made it so successful. This is basically the same argument made for C++ and I think it's not a very solid one. We know this worked, but that's not evidence that alternatives couldn't have succeeded. I think in a world where Queen never happened the prevailing wisdom would be that someone like Farrokh Bulsara (Freddie Mercury) can't be a rockstar because that seems ridiculous. Actually speaking of royalty, Prince is an even more extreme case. You cannot use numeral-letter substitutions in song titles, that's something for kids and won't... oh unless you're Prince. Yeah, no, "Nothing Compares 2 U" is fine Prince, thanks.
endospore 5mo ago
Calling it as working is a bit of an exaggeration. Looked at the code for a few minutes I'd expect it to crash and burn if they ever dare to turn the optimization on.
yarn_ 5mo ago
> I was a little shocked that they could get it fully working in a week to be honest It shouldn't be shocking, it was done using only compute, and the codebase is owned by the company who owns the compute, you literally just turn the dial up and it will be done faster. Efficacy of LLMs aside - anything they could do in 30 days they could also do in 3, if you spend more money.
archargelod 5mo ago
> verification should be 100x more robust now that we can output code at this rate. If you believe that having enough tests can save an LLM codebase - I have bad news for you. You can write a test for every single interaction ever written in typescript and it'd not be even close to 10% coverage.

by mohsen1 5mo ago

tapirl 5mo ago

Are there any evidences which prove the process was done in a week?

cedws 5mo ago

I'm stunned that it went from 'this is an experiment' to merging a ~million lines of (likely) unreviewed code in a week. I have nothing against using agents but to rush something like this and leave the community blindsided seems extremely ameteurish. Like something you'd expect a bright eyed graduate engineer to do.

mohsen1 5mo ago
tsz for me is an experiment to see how can this kind of work be done better. With a slight difference that tsz is not a direct port and it's a different architecture. I'm also not claiming to have answers but I've learned a ton. A few things that works - Test before code, Bun had lots of test so that's good but maybe they could start by asking Mythos to write like 20k additional tests that pass on Zig Bun first. - Deterministic anti-slop features. LLMs love to solve the problem in the wrong abstraction layer or place. There are many ways to catch this with deterministic tests. I do this in tsz a lot - Roadmap that constantly evolving by humans. - Taking a pause and looking how the progress is going and undoing slop - Fuzztest(https://github.com/google/fuzztest https://github.com/google/fuzztest) style "trying to break things" with the powers of LLM
singpolyma3 5mo ago
Blindsided? Has there even been a release yet?
- koteelok 5mo ago
  Yes, they released a canary version.
  - edenstrom 5mo ago
    True enough. Every merge to main releases a canary though
- silverwind 5mo ago
  Knowing them, I expect a release probably next week, have fun in production with it.

hayd 5mo ago

I suspect they've been planning this and experimenting for many months. Along with the large existing test suite, they have lots of tooling for parallelizing agents and an unlimited token budget. So don't feel too bad..

jaapz 5mo ago
The author was on here claiming it was an experiment just last week. Didn't look like they have been planning much of this, but they did have some trouble with zig to the tune of having to maintain their own fork that the zig maintainers didn't want to merge. So people probably could have seen this coming If I was the bun maintainers I wouldn't want to have to maintain a language fork either, when there are viable alternatives. Now with LLM's getting stronger and stronger a rewrite like this is getting more and more viable, and it looks like they are just approaching this like the "how do you eat an elephant? One bite at a time"-method. Not sure why people thing this is an elaborate PR campaign, we already know LLM's are increasingly powerful for coding tasks.

jwilliams 5mo ago

tsz looks impressive. Congrats!

nextaccountic 5mo ago

https://tsz.dev/sound-mode/ https://tsz.dev/sound-mode/ This is awesome. Typescript really needs more of this. I hope this gets more publicized and perhaps get adopted by Microsoft. I am not sure you should call it sound mode though. > It is not a mathematical proof of soundness, and it does not make third-party .d.ts files truthful. Here there are two completely unrelated things First, soundness is a mathematical thing (sorry). If something is sound, it's really really sound. Soundness is the thing that says we don't need to manually check the details - we can trust the compiler to do the right thing. It will do the wrong thing if there's a bug, but the implementation can be fixed. Soundness means there's is no bug in the spec, no bug in the type system that would make things go wrong even in theory. Second, and here's the important thing.. it's entirely okay and expected that real world languages need to have unchecked features where we trust the humans to do the right thing, rather than trusting the compiler to verify. That's like Haskell's unsafeCoerce or Java's sun.misc.unsafe. The problem really is when the type system breaks even when we don't do any of those

mohsen1 5mo ago
Thank you for the feedback. I think I should rename the "Sound Mode" to something more honest. "Strict" is already taken so I should think of a better name. Making tsz compatible with tsc and making it sound is mathematically impossible. The history of this stuff (https://hegel.js.org https://hegel.js.org) is telling. If you really make a sound JS checker, nobody will really use it unless you build an entire ecosystem as big as npm with it. TypeScript's decision to embrace existing npm ecosystem and layering .d.ts on top was a very practical decision that made it so successful.
- tialaramex 5mo ago
  > TypeScript's decision to embrace existing npm ecosystem and layering .d.ts on top was a very practical decision that made it so successful. This is basically the same argument made for C++ and I think it's not a very solid one. We know this worked, but that's not evidence that alternatives couldn't have succeeded. I think in a world where Queen never happened the prevailing wisdom would be that someone like Farrokh Bulsara (Freddie Mercury) can't be a rockstar because that seems ridiculous. Actually speaking of royalty, Prince is an even more extreme case. You cannot use numeral-letter substitutions in song titles, that's something for kids and won't... oh unless you're Prince. Yeah, no, "Nothing Compares 2 U" is fine Prince, thanks.

endospore 5mo ago

Calling it as working is a bit of an exaggeration. Looked at the code for a few minutes I'd expect it to crash and burn if they ever dare to turn the optimization on.

yarn_ 5mo ago

> I was a little shocked that they could get it fully working in a week to be honest It shouldn't be shocking, it was done using only compute, and the codebase is owned by the company who owns the compute, you literally just turn the dial up and it will be done faster. Efficacy of LLMs aside - anything they could do in 30 days they could also do in 3, if you spend more money.

archargelod 5mo ago

> verification should be 100x more robust now that we can output code at this rate. If you believe that having enough tests can save an LLM codebase - I have bad news for you. You can write a test for every single interaction ever written in typescript and it'd not be even close to 10% coverage.