3 ms·

If you read the reddit thread, it was coded by hand then only bug checked with ai.

by panja 8mo ago

toyg 8mo ago
It was benchmarked with AI. Benchmarks being the main reason for this thing existing...
- Imustaskforhelp 8mo ago
  I mean, its open source so people can create benchmark and independently verify if the AI was wrong and then have the claims be passed to the author. I haven't read the reddit thread or anything but If the author coded it by hand or is passionate about this project, he will probably understand what we are talking about. But I don't believe its such a big deal to have a benchmark be written by AI though? no?
  - CyberDildonics 8mo ago
    Why would someone spend their time checking someone else's AI slop when that person couldn't even be bothered to write the basic checks that prove their project was worthwhile?
  - gwern 8mo ago
    > I mean, its open source so people can create benchmark and independently verify if the AI was wrong and then have the claims be passed to the author. Thank you for volunteering. I look forward to your results.
    - Imustaskforhelp 8mo ago
      > Thank you for volunteering. I look forward to your results. Sure can you wait a few weeks tho? I know nothing about benchmarking so gonna learn it first and I have a few tests to prepare for irl. I do feel like someone else more passionate about the project should try to pick the benchmarking though. I don't mind benchmarking it but I only know tools like hyper for benchmarks & I have played with my fair share of zip archives and their random access retrieval but I feel like even that would depend from source to source. There are some experienced people in here who are really cool at what they do, I just wanted to say that if someone's interested and already has the Domain Specific knowledge to benchmark & they enjoy it in the first place, this having AI benchmark shouldn't be much of a problem in comparison.
- usefulcat 8mo ago
  After reading the reddit comments, it looks like a primary problem is that the author doesn't (didn't?) understand how to benchmark it correctly. Like comparing the time to mmap() a file with the time to actually read the same file. Not at all the same thing. For example: https://old.reddit.com/r/selfhosted/comments/1qi64pr/i_got_into_an_argument_on_discord_about_how/o0r4fdg/ https://old.reddit.com/r/selfhosted/comments/1qi64pr/i_got_i...

If you read the reddit thread, it was coded by hand then only bug checked with ai.

by panja 8mo ago

If you read the reddit thread, it was coded by hand then only bug checked with ai.

toyg 8mo ago

It was benchmarked with AI. Benchmarks being the main reason for this thing existing...

Imustaskforhelp 8mo ago
I mean, its open source so people can create benchmark and independently verify if the AI was wrong and then have the claims be passed to the author. I haven't read the reddit thread or anything but If the author coded it by hand or is passionate about this project, he will probably understand what we are talking about. But I don't believe its such a big deal to have a benchmark be written by AI though? no?
- CyberDildonics 8mo ago
  Why would someone spend their time checking someone else's AI slop when that person couldn't even be bothered to write the basic checks that prove their project was worthwhile?
- gwern 8mo ago
  > I mean, its open source so people can create benchmark and independently verify if the AI was wrong and then have the claims be passed to the author. Thank you for volunteering. I look forward to your results.
  - Imustaskforhelp 8mo ago
    > Thank you for volunteering. I look forward to your results. Sure can you wait a few weeks tho? I know nothing about benchmarking so gonna learn it first and I have a few tests to prepare for irl. I do feel like someone else more passionate about the project should try to pick the benchmarking though. I don't mind benchmarking it but I only know tools like hyper for benchmarks & I have played with my fair share of zip archives and their random access retrieval but I feel like even that would depend from source to source. There are some experienced people in here who are really cool at what they do, I just wanted to say that if someone's interested and already has the Domain Specific knowledge to benchmark & they enjoy it in the first place, this having AI benchmark shouldn't be much of a problem in comparison.
usefulcat 8mo ago
After reading the reddit comments, it looks like a primary problem is that the author doesn't (didn't?) understand how to benchmark it correctly. Like comparing the time to mmap() a file with the time to actually read the same file. Not at all the same thing. For example: https://old.reddit.com/r/selfhosted/comments/1qi64pr/i_got_into_an_argument_on_discord_about_how/o0r4fdg/ https://old.reddit.com/r/selfhosted/comments/1qi64pr/i_got_i...