3 ms·

That is the current standard. But it is hard for agents to read efficiently. To access JSON-LD, an agent must download the entire HTML page. This creates a hays

by tsazan 10mo ago

That is the current standard. But it is hard for agents to read efficiently. To access JSON-LD, an agent must download the entire HTML page. This creates a haystack problem where you download 2MB of noise just to find 5KB of data. Even then, you pay a syntax tax. JSON is verbose. Brackets and quotes waste valuable context window. Furthermore, the standard lacks behavior. JSON-LD lists facts but lacks instructions on how to sell (like @SEMANTIC_LOGIC). CommerceTXT is a fast lane. It does not replace JSON-LD. It optimizes it.

inerte 10mo ago
Wouldn't be easier on everybody (servers and clients) to just expose Structured Data in a text file then? And add the 1 or 2 things it doesn't have?
- tsazan 10mo ago
  That solves bandwidth. It fails on tokens. JSON syntax is heavy. Brackets and quotes consume context window. More importantly, Schema.org is a dictionary of facts. It lacks behavior. It defines what a product is, but not how to sell it. It has no concept of @SEMANTIC_LOGIC or @BRAND_VOICE. We need a format that carries both data and instructions efficiently. JSON-LD is too verbose and too static for that.
  - reddalo 10mo ago
    > JSON syntax is heavy. I'd say it's not heavy. JSON syntax is pretty lean compared to XML.
    - tsazan 10mo ago
      JSON is lean for data exchange between machines. But in the LLM economy, the currency is tokens, not bytes. To an LLM tokenizer, every bracket and quote is a distinct cost. In our tests, this 'syntax tax' accounts for up to 30% of the payload. We chose a line-oriented format to minimize overhead and maximize the context window for actual commerce data.
tjhorner 10mo ago
Who says you need to pipe the entire document with JSON-LD directly into the context window? I agree, that is very wasteful. You can just parse the relevant bits out and convert the JSON-LD data into something like your txt format before presenting it to the LLM. Bake that right into whatever tool it uses to scrape websites.
- tsazan 10mo ago
  That solves the Token Tax. It fails the Bandwidth Tax. To get that JSON-LD, you still download 2MB of HTML. You execute JS. You parse the DOM. You are buying a haystack to find a needle, then cleaning the needle. We propose serving just the needle. Furthermore, JSON-LD is strictly for facts. It cannot express @SEMANTIC_LOGIC. It lacks the instructions on how to sell.

by tsazan 10mo ago

inerte 10mo ago

Wouldn't be easier on everybody (servers and clients) to just expose Structured Data in a text file then? And add the 1 or 2 things it doesn't have?

tsazan 10mo ago
That solves bandwidth. It fails on tokens. JSON syntax is heavy. Brackets and quotes consume context window. More importantly, Schema.org is a dictionary of facts. It lacks behavior. It defines what a product is, but not how to sell it. It has no concept of @SEMANTIC_LOGIC or @BRAND_VOICE. We need a format that carries both data and instructions efficiently. JSON-LD is too verbose and too static for that.
- reddalo 10mo ago
  > JSON syntax is heavy. I'd say it's not heavy. JSON syntax is pretty lean compared to XML.
  - tsazan 10mo ago
    JSON is lean for data exchange between machines. But in the LLM economy, the currency is tokens, not bytes. To an LLM tokenizer, every bracket and quote is a distinct cost. In our tests, this 'syntax tax' accounts for up to 30% of the payload. We chose a line-oriented format to minimize overhead and maximize the context window for actual commerce data.

tjhorner 10mo ago

Who says you need to pipe the entire document with JSON-LD directly into the context window? I agree, that is very wasteful. You can just parse the relevant bits out and convert the JSON-LD data into something like your txt format before presenting it to the LLM. Bake that right into whatever tool it uses to scrape websites.

tsazan 10mo ago
That solves the Token Tax. It fails the Bandwidth Tax. To get that JSON-LD, you still download 2MB of HTML. You execute JS. You parse the DOM. You are buying a haystack to find a needle, then cleaning the needle. We propose serving just the needle. Furthermore, JSON-LD is strictly for facts. It cannot express @SEMANTIC_LOGIC. It lacks the instructions on how to sell.