3 ms·

The conversion is still very error-prone. It can't convert a lot of packages, and the last paper I read, StarVector, half the HTML version is just missing. (I t

by gwern 3y ago

The conversion is still very error-prone. It can't convert a lot of packages, and the last paper I read, StarVector, half the HTML version is just missing. (I think it hit an error at a figure of some sort.) I reported an error, but I've been reporting errors against the ar5iv and abstracts for years now and the long tail of problems just seems like an incredible slog.

KRAKRISMOTT 3y ago
Where are the computer vision people? This is the perfect type of problem for multi modal LLMs
- IlliOnato 3y ago
  Except that the errors made by an LLM might be harder to spot then converter errors that typically are very blatant, and don't usually alter text (perhaps just drop parts of it). Also, a bug in a converter is conceptually much easier to fix than to re-train your LLM. I am not sure that AI in it's current state is useful when "high fidelity" is required.
dginev 3y ago
Can confirm. From an ar5iv standpoint, 2.56% articles currently fail to convert entirely, and 22.9% have known errors to the converter. That leaves 74.5% of nominally usable articles. This success rate is noticeably lower for the newest batches of arXiv submissions, as the converter hasn't caught up with the most recent package innovations. We have a plan in place to meaningfully fall back for unknown packages, but that will take at least another year to put in place, and likely another couple of years to stabilize. Meanwhile, there is some hope that with arXiv launching the HTML Beta we will get more contributions for package support (LaTeXML is an open source project, with public domain licensing, everybody benefits). But again the original point is spot on. Coverage will be hit-or-miss for a while longer yet, for an arbitrary arXiv submission. The good news is that authors could work towards better support for their articles, if they wanted to.

by gwern 3y ago

KRAKRISMOTT 3y ago

Where are the computer vision people? This is the perfect type of problem for multi modal LLMs

IlliOnato 3y ago
Except that the errors made by an LLM might be harder to spot then converter errors that typically are very blatant, and don't usually alter text (perhaps just drop parts of it). Also, a bug in a converter is conceptually much easier to fix than to re-train your LLM. I am not sure that AI in it's current state is useful when "high fidelity" is required.

dginev 3y ago

Can confirm. From an ar5iv standpoint, 2.56% articles currently fail to convert entirely, and 22.9% have known errors to the converter. That leaves 74.5% of nominally usable articles. This success rate is noticeably lower for the newest batches of arXiv submissions, as the converter hasn't caught up with the most recent package innovations. We have a plan in place to meaningfully fall back for unknown packages, but that will take at least another year to put in place, and likely another couple of years to stabilize. Meanwhile, there is some hope that with arXiv launching the HTML Beta we will get more contributions for package support (LaTeXML is an open source project, with public domain licensing, everybody benefits). But again the original point is spot on. Coverage will be hit-or-miss for a while longer yet, for an arbitrary arXiv submission. The good news is that authors could work towards better support for their articles, if they wanted to.