4 ms·
> If you can’t reproduce results, is it really research, or just show and tell? You can't start with a statement about clinical trials not being perfectly repr
by IanCal 3y ago
> If you can’t reproduce results, is it really research, or just show and tell?
You can't start with a statement about clinical trials not being perfectly reproducible and that's fine, then say this.
> what you’re actually testing could change without you knowing
If people are lying about an extremely important part of their product, which they have little reason to. But then this applies to pretty much everything. Starting with the assumption that people are lying about everything and nothing is as it seems may technically make things more reproducible but it's going to require unbelievable effort for very little return.
> There’s no way to know if a paper is reproducible at all.
This is a little silly because these models are available extremely easily and at a pay-as-you-go pricing. And again, it requires an assumption that openai is lying about a specific feature of a product.
- starbugs 3y ago> You can't start with a statement about clinical trials not being perfectly reproducible and that's fine, then say this. Nobody said that to begin with. Re-read their comment. > If people are lying about an extremely important part of their product [...] Nobody is alleging that anyone is lying. It's just that we cannot be sure what the research actually refers to, because of the nature of a proprietary/closed model. > This is a little silly because these models are available extremely easily and at a pay-as-you-go pricing. What does this have to do with the parent comment? I don't think it's appropriate to call anyone here silly, just because you don't like their comment and don't have good counter arguments.
- IanCal 3y ago> Nobody is alleging that anyone is lying. Let's be clear, you have made an explicit claim that openai are lying. > What does this have to do with the parent comment? Because many other fields would kill for this level of reproducibility, grab an API key, spend a few quid running a script and you can get the results yourself.
- deleted 3y ago[deleted]