5 ms·

I suppose the vast majority of training data used for cutting edge models was created after 1900.

by imjonse 9mo ago

dogma1138 9mo ago
Ofc they are because their primary goal is to be useful and to be useful they need to always be relevant. But considering that Special Relativity was published in 1905 which means all its building blocks were already floating in the ether by 1900 it would be a very interesting experiment to train something on Claude/Gemini scale and then say give in the field equations and ask it to build a theory around them.
- p1esk 9mo ago
  How can you train a Claude/Gemini scale model if you’re limited to <10% of the training data?
- famouswaffles 9mo ago
  His point is that we can't train a Gemini 3/Claude 4.5 etc model because we don't have the data to match the training scale of those models. There aren't trillions of tokens of digitized pre-1900s text.
kopollo 9mo ago
I don't know if this is related to the topic, but GPT5 can convert an 1880 Ottoman archival photograph to English, and without any loss of quality.
- ddxv 9mo ago
  My friend works in that period of Ottoman archives. Do you have a source or something I can share?
  - kopollo 9mo ago
    Yes here: https://katalog.devletarsivleri.gov.tr/ https://katalog.devletarsivleri.gov.tr/ Here you will find all official resources provided by the Turkish state. You can register as a foreign citizen and conduct research. However, knowing Turkish is a great advantage. You can find more accurate documents by searching in the summaries, because all documents are archived with a short Turkish explanation - a summary. Also, most scanned images you want to download are paid.
    - ddxv 9mo ago
      Oh yeah I guess he researches at the one you're talking about in Istanbul. I actually was asking about a source for where you saw it translating ottoman, I thought you'd seen an article or something I could share to him. Sounds like you're the primary source though and I guess you already know him if you're both working there! Haha

I suppose the vast majority of training data used for cutting edge models was created after 1900.

by imjonse 9mo ago

I suppose the vast majority of training data used for cutting edge models was created after 1900.

Ofc they are because their primary goal is to be useful and to be useful they need to always be relevant. But considering that Special Relativity was published in 1905 which means all its building blocks were already floating in the ether by 1900 it would be a very interesting experiment to train something on Claude/Gemini scale and then say give in the field equations and ask it to build a theory around them.

p1esk 9mo ago
How can you train a Claude/Gemini scale model if you’re limited to <10% of the training data?
famouswaffles 9mo ago
His point is that we can't train a Gemini 3/Claude 4.5 etc model because we don't have the data to match the training scale of those models. There aren't trillions of tokens of digitized pre-1900s text.

kopollo 9mo ago

I don't know if this is related to the topic, but GPT5 can convert an 1880 Ottoman archival photograph to English, and without any loss of quality.

ddxv 9mo ago
My friend works in that period of Ottoman archives. Do you have a source or something I can share?
- kopollo 9mo ago
  Yes here: https://katalog.devletarsivleri.gov.tr/ https://katalog.devletarsivleri.gov.tr/ Here you will find all official resources provided by the Turkish state. You can register as a foreign citizen and conduct research. However, knowing Turkish is a great advantage. You can find more accurate documents by searching in the summaries, because all documents are archived with a short Turkish explanation - a summary. Also, most scanned images you want to download are paid.
  - ddxv 9mo ago
    Oh yeah I guess he researches at the one you're talking about in Istanbul. I actually was asking about a source for where you saw it translating ottoman, I thought you'd seen an article or something I could share to him. Sounds like you're the primary source though and I guess you already know him if you're both working there! Haha