A model trained only on text written before 1500 would tell you metal is shiny because of the mercurial principle. It would be articulate, internally consistent, and completely wrong.
This is a real research area now. The models are called vintage LLMs, or historical LLMs, and they come in two forms that are not equivalent.
MonadGPT, released in 2023 by Pierre-Carl Langlais, is a seven billion parameter fine tune of Mistral-Hermes 2 trained on roughly eleven thousand early modern texts in English, French and Latin spanning 1400 to 1700, drawn from Early English Books Online and Gallica. It answers in period language and reaches for dated references. But it is a costume. The modern physics still sits in the base weights and leaks through.
TimeCapsuleLLM takes the harder route. Hayk Grigorian trains from scratch on London sources published between 1800 and 1875. Over seven thousand books, legal documents and newspapers. A custom tokeniser strips modern vocabulary entirely. He calls th...
Suggested Credits
Tags, Events, and Projects