4 ms·

Not a comment on the quality of this application, but rather a tangential topic: there should be some way of auditing the accuracy of emulators in general. If a

Not a comment on the quality of this application, but rather a tangential topic: there should be some way of auditing the accuracy of emulators in general. If a genuine homebrew scene wants to exist (not just getting old games to work), new programmers who have little-to-no access to the original hardware need some practical way of knowing their software will work properly across different emulators. In the old days, most platforms had one good emulator that everyone used, and these emulators were supported by communities like AtariAge. In the Web 2.0 world, those communities are fractured or even gone completely, and often not easy to find in search engines. There are now much more platforms that emulators can be run on. Hardware emulation isn’t a solution. An FPGA emulator isn’t intrinsically more accurate than a software one, and documentation and schematics that are used to implement emulators can be incorrect and don’t specify ‘undefined behavior’ which was very much used reliably by software developers. The approach adopted by MAME has proven to be the only reliable: you have to measure some output on a real working system and compare it to the emulator behavior, but this can be impractical and becomes moreso over time as systems become more difficult to operate. (E.g. I can’t even operate most 8-bit computers without some kind of mod because I don’t have a CRT television anymore).

yarg 4y ago
> there should be some way of auditing the accuracy of emulators in general. Not really (and you don't always want perfect accuracy anyway). You don't only need to validate the operations of the machine, but also the timing with which it operates. And I don't mean just the clock/frequency/operations-per-second - it's everything. Every little chip that performs a calculation, every little wire that sends a message.
- anikom15 4y ago
  Timing can only be as accurate and precise as the system that is running the emulation. Accuracy can be expressed in relation to limited bounds. For example, loading software through a filesystem instead of a physical cartridge slot is inaccurate, but practically meaningless, so we don’t take that account in a metric for accuracy.
  - squeaky-clean 4y ago
    Until you find that one weird game that relied on some obscure or undefined behavior that only exists with the cartridge ;p Not a popular game, but the cartridge for Morita Shogi 64 had a built in RJ-11 jack and allowed online multiplayer. Plenty of SNES games included a SuperFX graphics coprocessor in the cartridge. Star Fox is probably the most popular example. SuperFX games were popular enough that it was quickly emulated successfully. But you can't treat the cartridge as just a file system abstraction on SNES.
    - anikom15 4y ago
      Whether the cartridge contains ROM, a processor, or a connector, we can’t emulate the physical aspects of it (unless we use some kind of VR). Loading a file is a simulation of inserting a cartridge. There is fundamentally a difference. That’s what I was trying to get at. The question is if the difference matters. We usually can’t answer that. For example, we are finding that many video systems have to be displayed on the scanline level, rather than as a framebuffer, to be correct. Something like the Apple II will probably have to be done on the pixel clock level (at least). There always seem to be corner cases that require you to go ‘lower level’.
      - alrlroipsp 4y ago
        Your observations is correct, but it also depends on other factors. The instruction timing and such fiddly low-level details is most visible in systems programmed in low level, such as older consoles. Modern systems tend to be programmed in higher level languages, which works much better to HLE (high-level emulate), since they tend to be programmed in a way that cares less/not at all about underlying hardware fiddly details. So when you are emulating the NES, these details matter way more than when you are emulating the Switch for example.
      - deleted 4y ago
        [deleted]
flatiron 4y ago
There are roms that test accuracy. Hardware will return 100% and emulators will receive less. I believe there are some psx test Roms that no emulator gets 100%. I know for a fact there are n64 test Roms that don’t get close on an emulator. You point on hardware emulation stands regarding it doesn’t automatically mean more accurate emulation. Although it lends itself to be more accurate due to you “can’t make a software thread for each discrete component” software problem as well as “how do I take controller input and when” as well which are wayyyy easier on fpga than software.
distantsounds 4y ago
there are emulation test suites. the fact you throw out 'there should be auditing' without acknowledging the literal _years_ of documentation and research leads me to believe you know nothing on the topic, rather just want complain on hacker news about something you have no involvement in. MAME literally scans the dies to read the chips and understand their logic. FPGAs faithfully recreate this as well. and no, you can still run a 8bit micro on whatever display you want. Projects like dolphin and BSNES documented the hell out of everything they do with a goal of accuracy in mind.
- soulofmischief 4y ago
  Rest in peace, Near
- anikom15 4y ago
  No disrespect intended. I just want to make sure people are aware of these issues. Recognizing a problem doesn’t put down the developers who will have to work on it, rather it highlights how important their jobs are and how valuable they are as a resource!
ohgodplsno 4y ago
Web 2.0 has made it so fractured that massive communities exist on /r/EmuDev, Emulator 101 is a well known source for new learners, nesdev.org is the reference for NES stuff, a shitton of the data we have is from the Web 1.0 days and is just as fractured, and we even have repos like https://github.com/marethyu/awesome-emu-resources https://github.com/marethyu/awesome-emu-resources that collect a shitload of useful stuff (including the test ROMs that other commenters have mentioned, that will only work if you perfectly emulate the target system). As well as that, many people are decapping entire chips and games, just for exact emulation. So, no, you're talking out your ass, sorry.
- anikom15 4y ago
  It’s harder to find those communities through search engines now though.

yarg 4y ago

> there should be some way of auditing the accuracy of emulators in general. Not really (and you don't always want perfect accuracy anyway). You don't only need to validate the operations of the machine, but also the timing with which it operates. And I don't mean just the clock/frequency/operations-per-second - it's everything. Every little chip that performs a calculation, every little wire that sends a message.

anikom15 4y ago
Timing can only be as accurate and precise as the system that is running the emulation. Accuracy can be expressed in relation to limited bounds. For example, loading software through a filesystem instead of a physical cartridge slot is inaccurate, but practically meaningless, so we don’t take that account in a metric for accuracy.
- squeaky-clean 4y ago
  Until you find that one weird game that relied on some obscure or undefined behavior that only exists with the cartridge ;p Not a popular game, but the cartridge for Morita Shogi 64 had a built in RJ-11 jack and allowed online multiplayer. Plenty of SNES games included a SuperFX graphics coprocessor in the cartridge. Star Fox is probably the most popular example. SuperFX games were popular enough that it was quickly emulated successfully. But you can't treat the cartridge as just a file system abstraction on SNES.
  - anikom15 4y ago
    Whether the cartridge contains ROM, a processor, or a connector, we can’t emulate the physical aspects of it (unless we use some kind of VR). Loading a file is a simulation of inserting a cartridge. There is fundamentally a difference. That’s what I was trying to get at. The question is if the difference matters. We usually can’t answer that. For example, we are finding that many video systems have to be displayed on the scanline level, rather than as a framebuffer, to be correct. Something like the Apple II will probably have to be done on the pixel clock level (at least). There always seem to be corner cases that require you to go ‘lower level’.
    - alrlroipsp 4y ago
      Your observations is correct, but it also depends on other factors. The instruction timing and such fiddly low-level details is most visible in systems programmed in low level, such as older consoles. Modern systems tend to be programmed in higher level languages, which works much better to HLE (high-level emulate), since they tend to be programmed in a way that cares less/not at all about underlying hardware fiddly details. So when you are emulating the NES, these details matter way more than when you are emulating the Switch for example.
    - deleted 4y ago
      [deleted]

flatiron 4y ago

There are roms that test accuracy. Hardware will return 100% and emulators will receive less. I believe there are some psx test Roms that no emulator gets 100%. I know for a fact there are n64 test Roms that don’t get close on an emulator. You point on hardware emulation stands regarding it doesn’t automatically mean more accurate emulation. Although it lends itself to be more accurate due to you “can’t make a software thread for each discrete component” software problem as well as “how do I take controller input and when” as well which are wayyyy easier on fpga than software.

distantsounds 4y ago

there are emulation test suites. the fact you throw out 'there should be auditing' without acknowledging the literal _years_ of documentation and research leads me to believe you know nothing on the topic, rather just want complain on hacker news about something you have no involvement in. MAME literally scans the dies to read the chips and understand their logic. FPGAs faithfully recreate this as well. and no, you can still run a 8bit micro on whatever display you want. Projects like dolphin and BSNES documented the hell out of everything they do with a goal of accuracy in mind.

soulofmischief 4y ago
Rest in peace, Near
anikom15 4y ago
No disrespect intended. I just want to make sure people are aware of these issues. Recognizing a problem doesn’t put down the developers who will have to work on it, rather it highlights how important their jobs are and how valuable they are as a resource!

ohgodplsno 4y ago

Web 2.0 has made it so fractured that massive communities exist on /r/EmuDev, Emulator 101 is a well known source for new learners, nesdev.org is the reference for NES stuff, a shitton of the data we have is from the Web 1.0 days and is just as fractured, and we even have repos like https://github.com/marethyu/awesome-emu-resources https://github.com/marethyu/awesome-emu-resources that collect a shitload of useful stuff (including the test ROMs that other commenters have mentioned, that will only work if you perfectly emulate the target system). As well as that, many people are decapping entire chips and games, just for exact emulation. So, no, you're talking out your ass, sorry.

anikom15 4y ago
It’s harder to find those communities through search engines now though.