3 ms·
One reason you sometimes don't get responses on this sort of thing is that the claims in the paper are sometimes just fake. Not totally fake-- e.g. they'll do
by nullc 5y ago
One reason you sometimes don't get responses on this sort of thing is that the claims in the paper are sometimes just fake.
Not totally fake-- e.g. they'll do part of the work so that they feel like they can extrapolate to the rest. Like they might implement a field multiply and benchmark it, then just count how many of those they need for the whole circuit.
Then no one wants to waste the column inches explaining what they actually did or dealing with reviewers that want to nitpick it ... at least not if they can help it. I know I've personally regretted including experimental details for that reason. ("oh great, a reviewer wants me to re-run 20 CPU years of number crunching because they've taken issue with some irrelevant detail that I could have just left out. Fine.")
The "more approximate than they let on" doesn't just come from sketchy and obscure research groups but this kind of work happens under well known names too. If you read a lot of the lit you can get an impression on which groups/people are actually competent implementers because they publish them or at least also solve sub-problems that you'd only encounter when creating a full implementation.
There is a lot of pressure to go this route because of time constraints and because the engineering to actually implement this stuff is often too far outside the authors expertise-- the people who really know about implementations can get highly paid in industry and many do.
For anyone who isn't faking their research this should be all the more incentive to publish your code! But so far it seems the incentives haven't lined up that way yet.