3 ms·
Studies show relatively conclusively that using primarily synthetic data woven intentionally with seeded real-world data is an effective strategy for training f
by blackqueeriroh 1y ago
Studies show relatively conclusively that using primarily synthetic data woven intentionally with seeded real-world data is an effective strategy for training frontier LLMs: https://consensus.app/search/synthetic-data-effectiveness-frontier/hNdPhq9FSEy9lvpsG82oyw https://consensus.app/search/synthetic-data-effectiveness-fr...
- BriggyDwiggs42 1y agoYou can only take that so far before model collapse. Even if it were equivalent to multiplying currently available data by four, would that be enough? Might still look like an incremental improvement.