2 ms·
A lot of training data was curated in Kenya[0]. I would imagine if LLM data was curated in Japan our LLMs would sound a lot like the authors of their most popu
by Timpy 10mo ago
A lot of training data was curated in Kenya[0]. I would imagine if LLM data was curated in Japan our LLMs would sound a lot like the authors of their most popular English text books. Maybe other common Japanese idioms would leak in to the training data, like "ね" or "でしょう", ChatGPT would say "Don't you agree?" at the end of every message.
[0] https://www.theverge.com/features/23764584/ai-artificial-intelligence-data-notation-labor-scale-surge-remotasks-openai-chatbots https://www.theverge.com/features/23764584/ai-artificial-int...
- m4rtink 10mo agoYou are completely right dajou~ ^_^ !
- delis-thumbs-7e 10mo agoMaybe we all should start writing Japanglish to show our authenticity? Or rather, ”Maybe we all should start writing the Japanglish, so that peoples can feel our real soul, you know?”
- erikig 10mo agoThe Indian-born textbook author mentioned (Malkiat Singh [0]) had an inordinate influence on many Kenyan students because his textbooks were the de-facto standard for years. Its interesting how this influence extends as his students get to curate the LLMs on which the world has come to rely. [0] https://en.wikipedia.org/wiki/Malkiat_Singh https://en.wikipedia.org/wiki/Malkiat_Singh
- jojobas 10mo agoSo twists of training data procurement bring us the best of doing the needful through Africa.
- bakugo 10mo agoI guess it can't be helped.
- koakuma-chan 10mo agoIt's not because I like you or anything.
- bpodgursky 10mo agoThis is a wild misunderstanding of LLMs. Data labeling has nothing to do with generating the astronomical text corpus used to train modern LLMs.
- heavyset_go 10mo agoThe HF part of RLHF to refine the output of LLMs also happens in these places
- astrange 10mo agoNote RLHF can only perform selection on existing model outputs, adding new data is SFT or else just more pretraining. ChatGPT speaking African English was mostly just 3.5. 4o speaks like a TikTok user from LA. 5 seems kind of generic.
- casey2 10mo ago樣 is just setting us up for ChatGPT :| ChatGPT (japan) XD