Havn't got access to the OPT-175 models yet as they are prioritising researchers, bloom is going to be huge but these OPT models that they did release IMO are n

Havn't got access to the OPT-175 models yet as they are prioritising researchers, bloom is going to be huge but these OPT models that they did release IMO are not a step forward from research done in GPT-NEO/EleutherAI and AI21. Its a great step forward it being shared/open but the models themselves i don't see having big impact. They just seem to loop around a lot i don't know why... i think they trained on datasets with duplicate content or lots of repeating characters. Check out https://text-generator.io https://text-generator.io which is orders of magnitude cheaper than GPT-3 and still makes creative writing/code autocomplete without too much looping issues that youd see with OPT models. For that repeating you can dial up the repetition penalty or N to generate more sequences in a single request (and you are only charged by the request not by characters/tokens which helps), often generating N results is much more creative than generating a long result as the generated output gets fed into the input when generating long text and often causes that kind of repetitive looping. Some tactics to mitigate that repetitiveness are: * Dont use OPT... * repetition_penalty/retries/seed * Generate N results and combine instead of doing one big generate as you dont know how many results you'll really get and they are less creative/likely contain shared info/repetitiveness. * Creative input prompts There's probably other creative ways of getting variety by splitting the generation into different calls with higher repetition penalty as you get longer or looping detection etc. Its easy to detect repetition in the structured case like chat but hard when doing longer text generation/creative writing