It is likely a combination of direct goal-driven human dialogue corpus they sourced from their mechanical turk-like contractors and a subset of commoncrawl (mor

It is likely a combination of direct goal-driven human dialogue corpus they sourced from their mechanical turk-like contractors and a subset of commoncrawl (more specifically stackoverflow, stackexchange, reddit, and maybe even twitter) bootstrapped from it with text classification. Read their recent arxiv papers, there is enough to get the picture of their pipeline, specifically the "InstructGPT" paper.