3 ms·
The prompt is probably simple, but the bigger challenge is that even a minified html of a typical web page would be more than the 4k gpt token limit
by malborodog 4y ago
The prompt is probably simple, but the bigger challenge is that even a minified html of a typical web page would be more than the 4k gpt token limit
- genmon 4y agoI extract the main content div, which includes various other divs and assorted HTML cruft from 25 years of content management systems. Then convert that to Markdown, which GPT groks happily, and it preserves the right balance of discarding meaningless structure but preserving some semantics (italics, headings, etc). The best tool I've found for that process is aaronsw's html2text, amazing that it's still so valuable after all these years.
- malborodog 4y agoThanks for explaining — very helpful!