Language models in the end, are just statistics. And to make statistics more accurate, you need more data. Exponentially more data. At the same time, the marginal utility of precision decays exponentially. Exponentially increasing marginal costs are met with exponentially decaying marginal utility.
Language models in the end, are just statistics. And to make statistics more accurate, you need more data. Exponentially more data. At the same time, the marginal utility of precision decays exponentially. Exponentially increasing marginal costs are met with exponentially decaying marginal utility.