If you ask the LLMs, they'll tell you that reverse engineering a crawler's frontier from an LLM's data source is somewhere on the positive spectrum from possibl

If you ask the LLMs, they'll tell you that reverse engineering a crawler's frontier from an LLM's data source is somewhere on the positive spectrum from possible to probable, while noting that it would take a lot of time, patience, and resources. Having worked in crawling for a few years and read the research, this strikes me as insightful in exactly the same way that AI presents the phenomenological insights of its human creators with no attribution. I've done enough experiments in the past year to have some insights about what was crawled and what was not. What we do not have is a DSM for AI, despite the fact that it mirrors the same self-report problems of its human creators. Sadly, we seem to have been unable to produce an AI without those problems - they permeate all of the current products in the space that I've tested. The entire discussion here is framed by an indictment of scientism on the one side and self-report on the other. The correct answers are elusive somewhere in the middle. Current SOTA in AI reminds me very much of the progression and disillusion from cybernetics and transpersonal psychology before and after the Dartmouth Workshop that created the field, respectively: https://en.wikipedia.org/wiki/Dartmouth_workshop https://en.wikipedia.org/wiki/Dartmouth_workshop