4 ms·

My side project kind of escalated quickly into a main project. I've been working on my own browser for the last couple months, and decided that I can improve a

My side project kind of escalated quickly into a main project. I've been working on my own browser for the last couple months, and decided that I can improve a lot when it comes to using the web for automating and acquiring knowledge (i.e. the semantic aspect of it). Currently on the verge of founding a (possibly viable) startup with it, but the browser itself is totally alpha for now. Been working on parsers and protocols for a while now, and had to switch to TDD to keep my sanity together. Needed to write my own test runner that can simulate network behaviours (2G slow fragmentation is real) and peer to peer scenarios. Most servers out there don't comply with specifications, so making my own client- or peer-side implementations work was a hard task. Currently writing my own SGML parser and optimizer, so that the browser receives only "linted and upgraded" html that is free of malicious parts, whilst embracing the idea of disallowing everything that could be potentially misused, including CDNs that do cache busting all the time. The idea behind the browser concept is that trust is not established by default, and users should decide what website to trust, and match that with what kind of content they'd expect the website to deliver. [1] https://github.com/cookiengineer/stealth https://github.com/cookiengineer/stealth

urbleflan 6y ago
This looks really cool.
Jemaclus 6y ago
This is awesome. I can't imagine working on something of that scale by myself. I'm so impressed. Keep up the good work!
- cookiengineer 6y ago
  Thanks much, really appreciate it ^_^
walterbell 6y ago
> It is built by a former contributor to both Chromium and Firefox, and is built out of personal opinion on how Web Browsers should try to understand the Semantic Web. Could you share more about this vision? > writing my own SGML parser How did you land on SGML? What do you think of a browser/mode that parses markdown, so we can have a "markdown web" with less complex clients?
- cookiengineer 6y ago
  > Could you share more about this vision? Phew, tough question. As I went into web development when XHTML 1.1 strict was the "cool shit", I kind of valued the aspect of using the web for acquiring and distributing knowledge. Not only for me, but also for publishing or other forms of media (e.g. by offering print stylesheets), screen readers, and semantic extraction of that kind of knowledge. (I was also working on project(s) that were using DAISY to automatically convert websites into hearable formats to be consumable by blind people.) Somehow from then (around 2000ish) to now, everything went to shit and nobody cares about that aspect anymore. News websites are too busy displaying ads and pushing subscription dialogs in my face (before I read a single line of their article) - rather than being readable or consumable. And I kind of disagree with that. I want to make the web an automatable tool to acquire knowledge in an easy manner. And I hope I can do that programming-free. Currently, programmers can easily build scrapers - but imagine the possibilities once any person or kid can do that with a few mouse clicks. I know there are a lot of proprietary scrapy-based solutions out there already, but honestly I think they're crappy. They see the web as DOM and not as a statistical model that a neural network "could" learn once you have a different way of rendering/parsing/modelling things. > How did you land on SGML? The reason why I am currently building my HTML(5) compatible parser with SGML ideas is because nobody closes tags. The spec is very complicated (especially while having an eye on what can be abused in the XSS sense or related security issues with CORS), so currently I'm kind of looking at a lot of parsers out there and try to find my own way of making this into a statistical model, so that in future my neural net adapters can optimize old HTML code into new, clean, HTML5 code. > What do you think of a browser/mode that parses markdown, so we can have a "markdown web" with less complex clients? Actually this was my first idea to build this. I wanted to convert all html to markdown and back, so that it's easier and cleaner. The issue I realized is that most markup and meta information that comes with a website is lost in markdown (or commonmark), and layouting sometimes implies structure, too - due to how websites in wordpress (or any user-friendly CMS) are being built. Code-wise you usually cannot imply meaning by only looking at HTML, sadly, that's why I switched to a "filtering proxy-like" approach, whereas the Browser UI simply receives the upgraded, clean HTML, CSS (and webfonts or other assets).
  - 6y ago
hoten 6y ago
Did you consider building this on top of Chromium? It has to be so much more work to recreate a browser, securely. I mean, certainly, it is more than one dozen people can handle in a lifetime. Was there something about Chromium that doesn't work on a basic level?
- cookiengineer 6y ago
  > Did you consider building this on top of Chromium? Currently, the Browser UI is actually just a PWA pointing to the nodejs instance and is reusing whatever rendering engine is available. I want to have a clean codebase, so everything is babelfree es2018 and will only run in edgium and safari 12+ (and chrome 70 or webkitgtk or webkitqt or firefox etc). For mobile my plan is to bundle nodejs-mobile and just use a webview there, which is based on chakra (so it is JIT free and is technically allowed on iOS). For desktop I will probably unclutter servo modules and try to have a minimal fork that doesn’t have all the web apis I don’t want or need...but I’m not sure, as I’m not yet familiar enough with the servo codebase. One thing is sure: I can’t create a competing rendering and layouting engine, so I gotta reuse an existing one.
StillBored 6y ago
That is awesome. I've thought for a while that a worthwhile project would be a new ground up browser. I've been put off it because I'm aware of my own limitations. Back a couple decades ago I wrote an os (well toy os is a better description because like many projects you can write an os in a weekend and then spend the rest of your life and your 1000 best friends finishing it). Anyway, IMHO, you should really focus on code clarity and hitting the high points with a good modular system. Ignore all the edge cases and if/when you open source it, that will allow people to focus on narrow pieces and make them more compliant. The world doesn't need another rats nest like firefox and chromium have become. AKA you need to reinvent the konqueror of 1999 that spawned webkit/chromium.
- cookiengineer 6y ago
  I totally agree with you. Stealth currently is a PWA and reuses an existing Browser as a rendering engine or runtime. As the codebase is babelfree es2018 I am currently dependent on a modern Browser being preinstalled on the system (aka edgium or safari 12 and later) in order to use ESM modules. I have no chance competing with google, so I’m probably gonna reuse as much of the servo project as possible when it comes to runtime and layouting/rendering. Currently a bit unplanned, on Android and iOS I have an experimental prototype up and running that’s just bundling nodejs-mobile and using a webview to localhost. The browser UI (pwa ui) is served on port 65432 in order to allow userspace usage (ephermal ports can also be used by anyone on Windows).

urbleflan 6y ago

This looks really cool.

Jemaclus 6y ago

This is awesome. I can't imagine working on something of that scale by myself. I'm so impressed. Keep up the good work!

cookiengineer 6y ago
Thanks much, really appreciate it ^_^

walterbell 6y ago

> It is built by a former contributor to both Chromium and Firefox, and is built out of personal opinion on how Web Browsers should try to understand the Semantic Web. Could you share more about this vision? > writing my own SGML parser How did you land on SGML? What do you think of a browser/mode that parses markdown, so we can have a "markdown web" with less complex clients?

cookiengineer 6y ago
> Could you share more about this vision? Phew, tough question. As I went into web development when XHTML 1.1 strict was the "cool shit", I kind of valued the aspect of using the web for acquiring and distributing knowledge. Not only for me, but also for publishing or other forms of media (e.g. by offering print stylesheets), screen readers, and semantic extraction of that kind of knowledge. (I was also working on project(s) that were using DAISY to automatically convert websites into hearable formats to be consumable by blind people.) Somehow from then (around 2000ish) to now, everything went to shit and nobody cares about that aspect anymore. News websites are too busy displaying ads and pushing subscription dialogs in my face (before I read a single line of their article) - rather than being readable or consumable. And I kind of disagree with that. I want to make the web an automatable tool to acquire knowledge in an easy manner. And I hope I can do that programming-free. Currently, programmers can easily build scrapers - but imagine the possibilities once any person or kid can do that with a few mouse clicks. I know there are a lot of proprietary scrapy-based solutions out there already, but honestly I think they're crappy. They see the web as DOM and not as a statistical model that a neural network "could" learn once you have a different way of rendering/parsing/modelling things. > How did you land on SGML? The reason why I am currently building my HTML(5) compatible parser with SGML ideas is because nobody closes tags. The spec is very complicated (especially while having an eye on what can be abused in the XSS sense or related security issues with CORS), so currently I'm kind of looking at a lot of parsers out there and try to find my own way of making this into a statistical model, so that in future my neural net adapters can optimize old HTML code into new, clean, HTML5 code. > What do you think of a browser/mode that parses markdown, so we can have a "markdown web" with less complex clients? Actually this was my first idea to build this. I wanted to convert all html to markdown and back, so that it's easier and cleaner. The issue I realized is that most markup and meta information that comes with a website is lost in markdown (or commonmark), and layouting sometimes implies structure, too - due to how websites in wordpress (or any user-friendly CMS) are being built. Code-wise you usually cannot imply meaning by only looking at HTML, sadly, that's why I switched to a "filtering proxy-like" approach, whereas the Browser UI simply receives the upgraded, clean HTML, CSS (and webfonts or other assets).
- 6y ago

hoten 6y ago

Did you consider building this on top of Chromium? It has to be so much more work to recreate a browser, securely. I mean, certainly, it is more than one dozen people can handle in a lifetime. Was there something about Chromium that doesn't work on a basic level?

cookiengineer 6y ago
> Did you consider building this on top of Chromium? Currently, the Browser UI is actually just a PWA pointing to the nodejs instance and is reusing whatever rendering engine is available. I want to have a clean codebase, so everything is babelfree es2018 and will only run in edgium and safari 12+ (and chrome 70 or webkitgtk or webkitqt or firefox etc). For mobile my plan is to bundle nodejs-mobile and just use a webview there, which is based on chakra (so it is JIT free and is technically allowed on iOS). For desktop I will probably unclutter servo modules and try to have a minimal fork that doesn’t have all the web apis I don’t want or need...but I’m not sure, as I’m not yet familiar enough with the servo codebase. One thing is sure: I can’t create a competing rendering and layouting engine, so I gotta reuse an existing one.

StillBored 6y ago

That is awesome. I've thought for a while that a worthwhile project would be a new ground up browser. I've been put off it because I'm aware of my own limitations. Back a couple decades ago I wrote an os (well toy os is a better description because like many projects you can write an os in a weekend and then spend the rest of your life and your 1000 best friends finishing it). Anyway, IMHO, you should really focus on code clarity and hitting the high points with a good modular system. Ignore all the edge cases and if/when you open source it, that will allow people to focus on narrow pieces and make them more compliant. The world doesn't need another rats nest like firefox and chromium have become. AKA you need to reinvent the konqueror of 1999 that spawned webkit/chromium.

cookiengineer 6y ago
I totally agree with you. Stealth currently is a PWA and reuses an existing Browser as a rendering engine or runtime. As the codebase is babelfree es2018 I am currently dependent on a modern Browser being preinstalled on the system (aka edgium or safari 12 and later) in order to use ESM modules. I have no chance competing with google, so I’m probably gonna reuse as much of the servo project as possible when it comes to runtime and layouting/rendering. Currently a bit unplanned, on Android and iOS I have an experimental prototype up and running that’s just bundling nodejs-mobile and using a webview to localhost. The browser UI (pwa ui) is served on port 65432 in order to allow userspace usage (ephermal ports can also be used by anyone on Windows).