3 ms·
Good points. But I feel like even with the cities article it could still ‘cheat’ by recognising what the data is supposed to be and filling in the blanks. Does
by cal85 2y ago
Good points. But I feel like even with the cities article it could still ‘cheat’ by recognising what the data is supposed to be and filling in the blanks. Does it even need to be real though? What about generating a fake article to use as a test so it can’t possibly recognise the contents? You could even get GPT to generate it, just give it the ‘Largest cities’ HTML and tell it to output identical HTML but with all the names and statistics changed randomly.
- wizzwizz4 2y ago> You could even get GPT to generate it This isn't a good idea, if you want a fair test. See https://gwern.net/doc/reinforcement-learning/safe/2023-krakovna-specificationgamingexamplesinai-masterlist.html https://gwern.net/doc/reinforcement-learning/safe/2023-krako..., specifically https://arxiv.org/abs/1712.02950 https://arxiv.org/abs/1712.02950.