>We are currently mining just about all the internet data that's available. If AI starts to become economically important, there will be an incentive to create

>We are currently mining just about all the internet data that's available. If AI starts to become economically important, there will be an incentive to create more data. If I'm OpenAI, why not pay a company to put microphones around their office, transcribing everything everyone says for training data. Buy troves of corporate (or personal) emails. If there are one hundred million office workers in the world, and you can convince/pay 10% of companies to let you spy on them, and each says or writes 5,000 words per day (these are all low estimates, in my opinion), you'd be getting ten billion more words of data per day. You'd double the size of the Chincilla training data in about a month. If you incorporate video for a truly multimodal model, now you're talking about not only the entirety of Youtube, but potentially also CCTV footage. If you could get the data from every surveillance camera in the world, you'd generate one YouTube worth of video every ten minutes (based on 150 million hours of video on YouTube).