3 ms·

> HTML is about 5%-20% more tokens than a similar Markdown. I'm not seeing this at all. I've got a small search engine I made that strips HTML back to Markdown

> HTML is about 5%-20% more tokens than a similar Markdown. I'm not seeing this at all. I've got a small search engine I made that strips HTML back to Markdown for its full-text indexing. HTML is typically 10x bigger than the Markdown of the actual content, but that's because the majority of HTML out there is truly terrible. I personally like HTML, and my own webpages are all hand-coded HTML. In that case, it's probably a closer ratio to what you describe. I'd suggest it's much higher than 20% more, but it's not likely more than double. But that's assuming someone paying attention to the efficiency of the HTML, and most people / websites just don't. Markdown is even more readable without tools than HTML - it's essentially a plain text document - but I agree that HTML is better for actual semantic structure.

honr 1mo ago
We are comparing different things. I am comparing expressing information in semantic HTML or in Markdown. HTML doesn't bloat it much, and it is easier to work with (programmatically in python, ts, go, java, etc.; including parsing and composing and formatting). I am not talking about web apps, which commonly have more "app" than "information". And the ratio is really closer to 1.1x tokens than 1.2x, unless the information you are expressing is highly structured. Typically 1 structural element (html tag, markdown marks such as ###) ends up being each 1 token. HTML often has the matching closing tag, and that is really what the difference in number of tokens boils down to. Feel free to run your own tests and compare (I ran a few and it was within 1.09-1.14x).
- SyneRyder 1mo ago
  I'll give a concrete example, but I think we might be talking at cross-purposes. A reasonably randomly chosen news article: https://www.abc.net.au/news/2026-08-27/shania-twain-interview-take-5-zan-rowe/107057510 https://www.abc.net.au/news/2026-08-27/shania-twain-intervie... This HTML page is 360523 bytes of raw HTML. After it has gone through my Markdown parser, it is just 9915 bytes of plain text / Markdown, about a 97% reduction. Only 3% of the HTML is actual content. The point of Accept Markdown is to save web hosts bandwidth. An AI harness can (and already does) download the HTML & parses out a Markdown version so that it is only minimal tokens before it hits the context window. But I still need to download the 360KB of HTML from the server in order to extract the 9KB I actually need. By serving Markdown versions of your page to AI agents, you can save 97% of the bandwidth that AI agents might be incurring. There's no reason this page needs to be 360KB of HTML. A handcoded / handoptimized HTML file might be only 12KB - converting from Markdown to HTML is only going to minimal file increase. But that isn't what the web is. It's full of slop generated by CMS applications & Bootstrap & web frameworks and relics of Frontpage edited WYSIWYG HTML editors. Accept Markdown is trying to get webhosts to save everyone bandwidth by serving the Markdown from their side. Maybe it has a chance if it gets baked in at the web server level, or because Cloudflare is applying it to sites that flow through their network. But I think it's ultimately futile - the same people who don't know their Wordpress output is garbage, also won't know how to configure Markdown on their server.
  - honr 1mo ago
    I am still not getting the point or we are talking past each other. When I am providing information for humans, I CAN go with semantic HTML (that 9kB Markdown becoming maybe 10KB semantic HTML). I can also go with "framework" layers and turn that into a massive 1MB crap which when rendered by the end user's browser maybe shows most of my content plus ads plus supply chain attacks I unintentionally pulled through with one of the "framework" layers, ahem, nodeJS, ahem. When I am providing data for both humans and AI agents, XML and other formal data formats might be good choices. If the data happens to be mostly text, then HTML and Markdown both work, but HTML is easier to compose (with programs). If you don't need composition at all, markdown or plain text are obviously easier to hand-craft. Only in the last scenario, where you don't need to combine information and are directly writing snippets of text, Markdown comes out as a decent option. I honestly don't have a clue how often that happens in practice. Maybe for casual writers (which may be a big market that I am blind to)? As soon as I want to put in a plain table in the data, Markdown gets in the way.

honr 1mo ago

We are comparing different things. I am comparing expressing information in semantic HTML or in Markdown. HTML doesn't bloat it much, and it is easier to work with (programmatically in python, ts, go, java, etc.; including parsing and composing and formatting). I am not talking about web apps, which commonly have more "app" than "information". And the ratio is really closer to 1.1x tokens than 1.2x, unless the information you are expressing is highly structured. Typically 1 structural element (html tag, markdown marks such as ###) ends up being each 1 token. HTML often has the matching closing tag, and that is really what the difference in number of tokens boils down to. Feel free to run your own tests and compare (I ran a few and it was within 1.09-1.14x).

SyneRyder 1mo ago
I'll give a concrete example, but I think we might be talking at cross-purposes. A reasonably randomly chosen news article: https://www.abc.net.au/news/2026-08-27/shania-twain-interview-take-5-zan-rowe/107057510 https://www.abc.net.au/news/2026-08-27/shania-twain-intervie... This HTML page is 360523 bytes of raw HTML. After it has gone through my Markdown parser, it is just 9915 bytes of plain text / Markdown, about a 97% reduction. Only 3% of the HTML is actual content. The point of Accept Markdown is to save web hosts bandwidth. An AI harness can (and already does) download the HTML & parses out a Markdown version so that it is only minimal tokens before it hits the context window. But I still need to download the 360KB of HTML from the server in order to extract the 9KB I actually need. By serving Markdown versions of your page to AI agents, you can save 97% of the bandwidth that AI agents might be incurring. There's no reason this page needs to be 360KB of HTML. A handcoded / handoptimized HTML file might be only 12KB - converting from Markdown to HTML is only going to minimal file increase. But that isn't what the web is. It's full of slop generated by CMS applications & Bootstrap & web frameworks and relics of Frontpage edited WYSIWYG HTML editors. Accept Markdown is trying to get webhosts to save everyone bandwidth by serving the Markdown from their side. Maybe it has a chance if it gets baked in at the web server level, or because Cloudflare is applying it to sites that flow through their network. But I think it's ultimately futile - the same people who don't know their Wordpress output is garbage, also won't know how to configure Markdown on their server.
- honr 1mo ago
  I am still not getting the point or we are talking past each other. When I am providing information for humans, I CAN go with semantic HTML (that 9kB Markdown becoming maybe 10KB semantic HTML). I can also go with "framework" layers and turn that into a massive 1MB crap which when rendered by the end user's browser maybe shows most of my content plus ads plus supply chain attacks I unintentionally pulled through with one of the "framework" layers, ahem, nodeJS, ahem. When I am providing data for both humans and AI agents, XML and other formal data formats might be good choices. If the data happens to be mostly text, then HTML and Markdown both work, but HTML is easier to compose (with programs). If you don't need composition at all, markdown or plain text are obviously easier to hand-craft. Only in the last scenario, where you don't need to combine information and are directly writing snippets of text, Markdown comes out as a decent option. I honestly don't have a clue how often that happens in practice. Maybe for casual writers (which may be a big market that I am blind to)? As soon as I want to put in a plain table in the data, Markdown gets in the way.