Similarly HN is already an important data source for large language model training. For example, the RefinedWeb paper lists HN as one of only 12 websites that

Similarly HN is already an important data source for large language model training. For example, the RefinedWeb paper lists HN as one of only 12 websites that were excluded. From what I understand, it was excluded because it went into the final dataset unvetted. RefinedWeb was used for the Falcon model. https://arxiv.org/pdf/2306.01116.pdf https://arxiv.org/pdf/2306.01116.pdf