2 ms·
Incorrect. I originally wrote: To modify the page, we need to read and parse the HTML to find elements that need changing. Since the very early days
by jgrahamc 10y ago
Incorrect.
I originally wrote:
To modify the page, we need to read and parse the HTML to find
elements that need changing. Since the very early days of Cloudflare,
we’ve used a parser written using Ragel. A single .rl file contains
an HTML parser used for all the on-the-fly HTML modifications that
Cloudflare performs.
About a year ago we decided that the Ragel parser had become too
complex to maintain and we started to write a new parser, named cf-html,
to replace it. This streaming parser works correctly with HTML5 and
is much, much faster and easier to maintain.
We first used this new parser for the Automatic HTTP Rewrites feature
and have been slowly migrating functionality that uses the old Ragel
parser to cf-html.
Both cf-html and the old Ragel parser are implemented as NGINX modules
compiled into our NGINX builds. These NGINX filter modules parse buffers
(blocks of memory) containing HTML responses, make modifications as
necessary, and pass the buffers onto the next filter.
It turned out that the underlying bug that caused the memory leak had
been present in our Ragel-based parser for many years but no memory was
leaked because of the way the internal NGINX buffers were used.
Introducing cf-html subtly changed the buffering which enabled the
leakage even though there were no problems in cf-html itself.
The Ragel author contact me on Twitter and I changed the text to (stars around changes):
To modify the page, we need to read and parse the HTML to find
elements that need changing. Since the very early days of Cloudflare,
we’ve used a parser written using Ragel. A single .rl file contains
an HTML parser used for all the on-the-fly HTML modifications that
Cloudflare performs.
About a year ago we decided that the *Ragel-based* parser had become too
complex to maintain and we started to write a new parser, named cf-html,
to replace it. This streaming parser works correctly with HTML5 and
is much, much faster and easier to maintain.
We first used this new parser for the Automatic HTTP Rewrites feature
and have been slowly migrating functionality that uses the old Ragel
parser to cf-html.
Both cf-html and the old Ragel parser are implemented as NGINX modules
compiled into our NGINX builds. These NGINX filter modules parse buffers
(blocks of memory) containing HTML responses, make modifications as
necessary, and pass the buffers onto the next filter.
*For the avoidance of doubt: the bug is not in Ragel itself. It is in
Cloudflare's use of Ragel. This is our bug and not the fault of Ragel.*
It turned out that the underlying bug that caused the memory leak had
been present in our Ragel-based parser for many years but no memory was
leaked because of the way the internal NGINX buffers were used.
Introducing cf-html subtly changed the buffering which enabled the
leakage even though there were no problems in cf-html itself.
https://twitter.com/jgrahamc/status/835056414062161920 https://twitter.com/jgrahamc/status/835056414062161920