7 ms·

Ask HN: If I wanted to make a web browser, where would I start?

45 points by avindroth 10y ago

deleted 10y ago
[deleted]
wslh 10y ago
You have mainly two options: do everything from scratch or reuse. Starting from scratch is almost impossible to do alone, just Chrome was developed by different teams with different expertises, some of them acquired from other companies. The second option is reusing a web component or browser and modify it for your purposes. Building on top of Chrome or Firefox (or their variants) is a good choice. I think Chrome is better engineered than Firefox and the code is clear. Obviously this is just an opinion. Now the question is what do you want to add to such browser? Is this something you can add via an extension or is something that requires a new direction?
- basch 10y ago
  chrome/blink was built on webkit which was a fork of khtml. none of it was from scratch within the last decade.
- sglane 10y ago
  >Chrome was developed by different teams with different expertises, some of them acquired from other companies. To expand on this, Chrome and Safari use webkit as their rendering/layout engine which is a fork of KDE's KHTML. IE came from NCSA Mosaic-> Spyglass -> Microsoft IE. To be clear on my suggestions below, let's clarify what a browser actually is. The browser manages things like the navigation bar, bookmarks, download/mime handling, etc. This is known as the "chrome" or GUI. The layout engine handles HTML parsing, rendering, possibly is coupled with a Javascript engine like V8. If you are trying to write a browser, the answer is to use webkit2 in whatever language you like. It's fast, light, and portable. If you are trying to write a layout engine (rendering engine), do as another poster suggested and write one to target HTML 3.2.
  - timvdalen 10y ago
    Note that Chrome is actually using Blink now, which itself is a fork of webkit
- fspeech 10y ago
  Servo https://github.com/servo/servo https://github.com/servo/servo is a browser rewrite in Rust. Could be a good place to start.
  - zamalek 10y ago
    Or a good study if the the goal language is different. Studying a clean codebase is where I'd start, there is likely a lot of tribal knowledge in HTML rendering (especially layout) that isn't documented anywhere but in code.
ruraljuror 10y ago
This series of blog posts might be a good place to start: https://limpet.net/mbrubeck/2014/08/08/toy-layout-engine-1.html https://limpet.net/mbrubeck/2014/08/08/toy-layout-engine-1.h...
- magic_beans 10y ago
  This series was really well-written and fun to work through! I second this recommendation.
rer 10y ago
If you wanted to choose how to build the UI, one starting point is here https://news.ycombinator.com/item?id=12376242 https://news.ycombinator.com/item?id=12376242
p333347 10y ago
If you are seeking tutorials, or UI libraries etc, sorry, I can't help you with that as I am unaware of those things. However, if you are asking from which aspect to start building one from scratch, I would say start with layouting. Try implementing a tiny subset of div related things like position and display and see if this is what you want to spend your time working on. Even if you can crack this thing easily, remember, this is just the tip of the iceberg. I am not discouraging you, but know that you would have to digest the entire DOM and HTML spec, not to mention have a JavaScript engine and implement all the security related intricacies, at the least, for your software to become a web browser.
- anatoly 10y ago
  I'm curious: is there a ready-to-use set of tests for a layout engine that checks conformance with the specs? Something that you could, without much effort, run against a toy layout engine you'd write? I know Webkit/Blink have humongous test suites, but these probably aren't easy to adapt to a toy layout engine.
  - chiefofgxbxl 10y ago
    There's something like Acid3, but it doesn't give you information as to what failed when you get a less-than-perfect score. (Test: http://acid3.acidtests.org/ http://acid3.acidtests.org/ | Information: http://www.webstandards.org/action/acid3/ http://www.webstandards.org/action/acid3/) There are also all those other browser benchmarks (e.g. http://peacekeeper.futuremark.com/ http://peacekeeper.futuremark.com/), but those include things other than the layout engine, such as javascript performance, webGL, etc. You could try poking around a few of them and only looking at the CSS and DOM sections though (http://web.basemark.com/ http://web.basemark.com/).
  - zokier 10y ago
    W3C Web Platform Tests probably would be best starting point. http://testthewebforward.org/ http://testthewebforward.org/
knabacks 10y ago
My 2 Cents: Start out with a solid base: (for example) - https://cefbuilds.com/ https://cefbuilds.com/ - https://crosswalk-project.org/ https://crosswalk-project.org/ - http://electron.atom.io/ http://electron.atom.io/ - http://nwjs.io/ http://nwjs.io/ or you try to help with existing projects: (for example) - https://github.com/breach/thrust https://github.com/breach/thrust (started out with https://github.com/breach/breach_core https://github.com/breach/breach_core a "javascript"-Browser (core is not js)) https://github.com/minbrowser/min https://github.com/minbrowser/min
- michaelmior 10y ago
  Breach seems like a poor choice since the project has been inactive for ~2 years.
  - knabacks 10y ago
    Maybe you're right, a project like servo (https://github.com/servo/servo https://github.com/servo/servo) looks like a better fit. easy bugs to hunt: https://github.com/servo/servo/labels/E-easy https://github.com/servo/servo/labels/E-easy
- deleted 10y ago
  [deleted]
_RPM 10y ago
Just a note: The browser as we use it today is a result of 20+ years of human work and innovation. It's what you would call a killer application.
- iamben 10y ago
  Equally - the car we drive today is a result of 130 years of human work and innovation. But after all that time we're seeing a real shake with the electric car. Things change when people try. I say good luck to the girl/guy - perhaps they'll create something incredible :-)
  - richardwhiuk 10y ago
    Even the electric car didn't start from scratch - it'll use the same component manufacturers as the rest of the car industry - e.g. tyres, brakes, suspension, interior, safety systems.
    - reitoei 10y ago
      You and parent poster both make good points.
mixedCase 10y ago
Well, what are your goals?
stellar678 10y ago
"If you wish to make a web browser from scratch, you must first invent the universe."
bigato 10y ago
1. Write a webassembly interpreter/vm; (main step) 2. Using webassembly, write an html interpreter/renderer; 3. Using webassembly, write an css, js ... interpreter/renderer; 4. Keep doing this for all w3c standards for the rest of your life. Chances are that by the time you are somewhere in the middle of step 2, people will have written the other interpreters in webassembly and the result of step 1 will be a functional browser already.
max_ 10y ago
Browser Engine https://limpet.net/mbrubeck/2014/08/08/toy-layout-engine-1.html https://limpet.net/mbrubeck/2014/08/08/toy-layout-engine-1.h... How to Create your Own Internet Browser in 5 Minutes! - https://www.youtube.com/watch?v=Bn22JBRC1UI https://www.youtube.com/watch?v=Bn22JBRC1UI Chrome's Engine http://www.chromium.org/blink http://www.chromium.org/blink
marai2 10y ago
There is a "Building a Web Browser" course on udacity[1]. Maybe someone who has taken the course comment on it. [1] https://www.udacity.com/course/programming-languages--cs262 https://www.udacity.com/course/programming-languages--cs262
pmontra 10y ago
As somebody already wrote "Starting from scratch is almost impossible to do alone". However you could set a first goal of doing some basic rendering of plain HTML, even text based. That should make you familiar with the hurdles of a rendering engine. Then apply some CSS. Even writing a browser for HTML 3.2 from scratch and no JavaScript should be an exercise to be proud of https://www.w3.org/TR/REC-html32 https://www.w3.org/TR/REC-html32 (1997). Then start looking at modern browsers using the links others provided. You should have the experience to understand all the moving parts by then. This is similar to writing the very first web application without using any framework, maybe even interfacing directly with the web server. I remember decoding CGI arguments in C in 1994 before discovering Perl's CGI.pm. Then you understand the magic inside frameworks and don't get burned by surprises.
ayyn0n0n0 10y ago
TempleOS
chvid 10y ago
Back when I did CS 101 this was our final project; the teacher made the HTML-parser, and we students did the renderer and UI. The HTML-parser is probably the part it makes sense to do first. What language and platform do you plan to use? I think a modern OO-language will be helpful.
matthewhall 10y ago
I'm working on one right now. I'm using the electron shell because im not worried about the backend source (chromium) I'm just working on a new ui and a reader feature. Email at matthew349hall@hotmail.com if you want to get a beta
- philippnagel 10y ago
  I sent you an email.
jbb555 10y ago
I'd think about it like this :- 1) Write socket code to retrieve the page from a server 2) Get it to draw the text in a window. 3) Write code to look for tags and start diving up the retrieved file into a proper DOM tree. Perhaps start by creating an object for each paragraph. Then maybe look for <b> tags, then <div> tags 4) Change the rendering code to draw the screen for the data structure you created above. 5) Iterate a lot. Add support for parsing more and more tags into the data model. Start adding annotations for various attributes. 6) Start improving the layout code. Draw things in <b> tags in bold. Draw things in <div> tags under each other and <span> tags separately. 7) Add limited CSS support. Allow just width and height and border sections. make the code read the css file and attach the attributes to the correct sections of the DOM document you are creating. 8) Improve the layout code to look at the width and height tags on each element, and where tags are nested to propogate the width and height information up and down the tree as needed. Draw borders if the css tells you to. Look at font weight etc. 9) Iterate. Add one feature at a time. Repeat for a year until you have an browser that can render basic pages. 10) Iterate some more. Read specs. Rewrite your document to DOM tree parse a number of times perhaps using a more formal grammar. You'll probably be on about year 8 by now. 11) Add javascript support... :P Perhaps my point is though that this seems something where its very easy to start small, just render the text of a page, and incrementally add features and improve the layout. It will take a long time though as there are so many features to add.
- kens 10y ago
  The above list is a good start. I'd add: 2a) make clicking on links work, and 3a) display images. Those two features will make it much more fun. "How to make a browser" is something I've thought about for a while, ever since someone suggested it as an interview question. (I think it would be an awful question.) It really depends on what you're trying to do. If you want to understand how browsers work and build a toy browser, follow jbb555's list. If you want to display web pages, use WebKit. If you want to build a real, commercial-scale browser, I recommend a large team, since there are a lot of components, each of which is insanely complex. (Everything from CSS support to security to plugins to JavaScript to bookmarks to a debugger to all the new network protocols.) One more thing to add to jbb555's list: 0) "telnet news.ycombinator.com 80, GET /" - playing around from the command line can give you a good introduction to what really happens when you access a web page. (Edit: HN returns an error page for a plain GET. google.com returns insane code. apple.com looks like a better place to start.)
IncRnd 10y ago
You can accept entered URLs and fetch & display web pages quite easily nowadays. You should, depending on the vertical, be very concerned about security. What are your goals (to a greater degree than "web browswer")?
newscracker 10y ago
[Disclaimer: I'm not an expert on developing browsers] It depends on what exactly you'd want to implement, since a (modern) browser is really a huge piece of software with several aspects and components in it. Looking at it from one angle, you could split it into chrome (not Google Chrome, but the UI, like tab bar, bookmarks, menus, extensions management, etc.) and content (the rendering engine). You'd most likely get bogged down if you start writing your own rendering engine (processing HTML, CSS and JS, and processing image formats, video and sound formats, even not including any plugins), unless it's a very limited scope toy browser that can handle some simple stuff. In this case, start with base HTML support, including handling incomplete tags and quirks like that have been carried over for decades (the amount of cruft in rendering engines for backward compatibility and handling undocumented but accepted quirks would itself take a long time to develop). Then try a little bit of CSS support. I'd assume JS support could be quite challenging since that moves out of the already challenging "paint the screen" part into an even more challenging part on interpreting code, running it, optimizing it, etc., which is a lot more complex when you consider there would be several JS files loaded on most websites. Something a little more easier (only comparatively) would be developing the chrome around an existing rendering engine and seeing how to handle tabs, background loading, foreground loading, bookmarks, private/incognito browsing, etc., and if you feel adventurous enough, adopt a particular process or threading model for a multi-tab browser. Another alternative here is to create a WebView application and see which parts you could improve. For example, Chrome on iOS and Firefox on iOS are just the chrome for the built-in rendering engine in iOS, but address many things like tabs, networking, incognito mode, bookmarks, sync and other features. You could try creating a similar one on iOS, latching on to the work done by FoxBrowser [1] or Firefox for iOS [2]. On Android, you could try building on top of Firefox for Android [3]. If you'd like to work on a web browser rendering engine that's a bit more closer to "from scratch", I'd recommend working with the Servo [4] team and seeing how you can help. It's a long drawn project that has already made huge strides, but you'd get an inside view of the complexity of developing a browser engine. What's more, you could also learn Rust while doing this. [5] [1]: https://github.com/graetzer/Foxbrowser https://github.com/graetzer/Foxbrowser [2]: https://github.com/mozilla/firefox-ios https://github.com/mozilla/firefox-ios [3]: https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Build_Instructions/Simple_Firefox_for_Android_build https://developer.mozilla.org/en-US/docs/Mozilla/Developer_g... [4]: https://servo.org/ https://servo.org/ [5]: https://www.rust-lang.org/ https://www.rust-lang.org/
WayneBro 10y ago
Do you want to build a web browser or a web browser engine? If you just want to build a web browser, here's my request for a new browser for iOS since web browsing on iOS is a second or third rate experience: - Built-in ad-blocking. - A way to disable JavaScript immediately for any website. - A desktop mode that actually works, so that YouTube, Reddit, Imgur and other sites don't redirect me to their crappy mobile site without asking. - Launches a completely separate app for private browsing. Right now I use Dolphin browser on iOS and despite being in desktop mode, I still have to refresh every YouTube page that I land on in order to get the desktop style native HTML5 video player that allows me to go full-screen. On Imgur, gifv files don't play - I have to change the extension to gif and then wait 60 seconds as it downloads. Lots of other sites have similar types of issues. Meanwhile, gifv files work fine in Brendan Eich's "Brave" browser. It could be as easy to fix as just changing the user-agent. Dolphin has good ad-blocking, but there is no way to quickly turn off JavaScript for a given page or domain (ala Quick JavaScript Switcher for Chrome). The thing that really annoys me though about Dolphin is that Private Browsing does not work at all. That's probably because they want you to buy Dolphin Zero for $2.99.
- speg 10y ago
  I checked out Brave yesterday for the first time and I think it does most of those things.
  - WayneBro 10y ago
    Brave has a desktop mode that mostly works and a built-in adblocker. It does not have a `quick JS disable` button. It also doesn't have a speed dial that I can manually add items to, which is one very important item that I forgot to mention earlier. What I really want to do is make an open source browser that I can compile and put on my iOS devices myself too, without going to the app store. Then I could add all sorts of features and extensions without having to ask Apple.
- softawre 10y ago
  3 dollars? It really annoys you and you can't be bothered to spend 3 dollars?
retro64 10y ago
I did this once back in 2003. It was text only rendered, with mouse support so you could click on the links instead of tabbing around like Links. I built it off of the IE object model (Internet Explorer Browser Extensions? Can’t recall). I think I just used the object model for connectivity settings. If you could get to the net using IE, my browser would work as well. I think I coded to HTML 1.1 specs, and had it working after 2 months (with a few minor rendering issues). It did not support scripting, CSS or really much of anything (although it could POST), but it was surprising how much of the web was accessible. And it was really fast. Not because of my skillz, but when you cut out the garbage, it’s amazing how fast a page will load. Anyway, I would not recommend this approach, but I would offer you encouragement. HTML parsing isn’t really that big of a deal, and you could get something very minimal (text rendering) working in short order. It would at least give you a feeling of what you are up against.
beachstartup 10y ago
https://curl.haxx.se/dev/source.html https://curl.haxx.se/dev/source.html
pif 10y ago
By giving up? ;-)
txutxu 10y ago
Well, this is what I could do in that (or almost any) case: 1) Setting features, key points, points to avoid, and goals of the project. 2) Choosing the toolchain to use, for resolving the points at 1) 3) Coding the features with that toolchain Step 2, may involve: to read a lot (doc, rfc's, specifications, using search engines with keywords, search for "problems", etc) to look at other people's source code of similar programs, to try other people's tools and libraries and evaluate and compare them, to make integration tests, and to make Proofs Of Concept prototypes. The toolchain (programing language, user GUI, etc) comes with an ecosystem (I hope you don't aim at rewriting also your $lang libraries for validate XML or interact with devices), so you have to choose looking at both. To separate steps 2 and 3, use iterations, while you have resources. If you find an opensource product that full-fills all your definitions of point 1), except a few of them, you can evaluate to start by a patch and a pull-request to them, or a fork :P
hugozap 10y ago
I'd say, if you have the opportunity and want to go wild skip the DOM and create an equivalent model using OpenGL. All the other Web platform features like the distribution model, resource streaming etc can still be used. It's just that the DOM is the great bottleneck and most of the hours devoted to improve the Web are being wasted on sorting quirks related to the DOM. Maybe yours will be the starting point towards a DOM-free Web! :)
vmorgulis 10y ago
Instrument an existing browser and synthetize a new layout engine with machine learning and superoptimisation.
OldSchoolJohnny 10y ago
Step one: Build AI or machine learning or genetic code to build a browser by replicating the results of a real world browser. Step two: run it!
VLM 10y ago
Sounds like a pretty good interview question. Better than asking you to invert binary trees or similar WTF. They're testing your judgment and asking you to assess your own skills and possibly they're interested in the actual assessment results assuming you're any good at assessment. Most people would generally break the task chronologically or kinda data flow oriented. Probably. Now should you start at the part you're familiar with or less familiar or unfamiliar with? My philosophy is its a bathtub shaped curve and if you know the topic dead cold or have absolutely no idea how to do it, start there because those will be your constraints, either you are an industry expert on a topic and can define the constraints with the best of them in which case you should definitely start there unless you suspect a trick question (LOL you thought we meant HTML browser, actually we're passing you DB2 ISAM files and COBOL PIC statements), or you have absolutely no idea what you're doing in which case you should start there to begin to discover your constraints and APIs and interfaces and data structures. Never start at a point where you have intermediate level of knowledge, the impact when you smack into the unknown is going to be painful so start in the unknown to begin with. Basically, it boils down to try not to paint yourself into a corner, so either start painting in the most obvious safe area to start or the most mysterious area on the assumption escape will be the easiest. But never, ever start to paint a floor by just dumping the can out in the middle of the floor. If you have a flat level of skill across the entire project (yeah not likely) then if its a "simple" project like a viewer/browser where you've seen all the parts before, start in the middle and work both directions at once if you can handle the inevitable mystifying weirdness at both ends simultaneously, and if its a "complex" project like replace the entire air traffic control system then start at one edge and work to the other to minimize simultaneous confusion. Yeah yeah I know I just said never start painting in the middle of the floor, well if you think there's a perfectly flat skill level you probably haven't analyzed the problem enough so the real answer is not to start at all, and instead to think about it a lot more. Of course half the fun is defining the starting point. Do they give you a working example to clone as a goal, access to all RFCs, no docs and cleanroom reimplement using a protocol analyzers traces, can you use a language you like, can you use language libraries that implement some fraction of the problem, does it have to be a 1995 level of browser or 2015 or 2035 level of browser ... To some extent its a meta interview question where they want to see if you start talking like you're a system architect or systems analyst type like above or if you're a security guy you'd probably start with "review all the latest exploits and avoid all of them simultaneously (good luck lol)" or if you're a code monkey maybe you'd start with a rant about your favorite font to use in Eclipse and how nice your mechanical keyboard is or maybe you're not even a programmer in which case the response will sound like the dude from the office space movie in the scene arguing about his radio "well I was informed this was a programming test and I got permission to list the integers from 1 to 100 and print fizz when they're a multiple of ..." and so on.
deleted 10y ago
[deleted]
c-smile 10y ago
Well, I did exactly that. Sciter ( http://sciter.com http://sciter.com ) is an embeddable HTML/CSS/script engine that has all components of a browser - html, css, svg, script and networking (http,ajax,websockets,etc). Work started in 2002 when I decided to create WYSIWYG HTML editor for my wife. So http://blocknote.net/ http://blocknote.net/ was born. That was pure HTML (no CSS) rendering and editing engine. 9 months for one developer - myself. After its publishing joined initial EverNote team to create WYSIWYG editor for EverNote. Three of us (developers), + Stepan Pachikov that generated the idea of the EverNote, created first published version of the EverNote (v 2.0). That was also done in 9 months (I am pretty sure that 9 months is not a coincidence but rather a law for new projects - optimal period when developers are excited of what they are doing). While working on it, and being a UI-in-all-its-incarnations developer/architect, I've discovered that HTML and CSS are quite convenient format for defining UI layouts and elements. So next step was wrapping it as a reusable component and so HTMLayout engine was born ( http://htmlayout.com http://htmlayout.com ). Almost immediately discovered that plain C++ is not quite convenient for handling complex UI cases - "on click here hide that and expand those". UI objects ownership graph is quite complex and contains loops. Something with GC is definitely required. So decided to add scripting to it. Considered quite many existing languages including JS for the "language behind UI". At the end decided to roll my own tiscript ( http://www.codeproject.com/Articles/33662/TIScript-Language-A-Gentle-Extension-of-JavaScript http://www.codeproject.com/Articles/33662/TIScript-Language-... ) that essentially is JavaScript++. So 7 years ago I had what we will have in ES6 in next couple of years. At the mean time I participated in HTML5 spec. development at W3C as an invited expert. And worked as an architect / team lead in several Web Application projects that required deep understanding of the platform - knowing Web UI stack from inside-out helped that a lot. As Sciter works on all major desktop and mobile platforms I've created core engine wrappers for Windows API, MacOS CoreFoundation, Linux/XWindows/GTK and some others. Sciter core is written in C++ with Objective C fragments in Apple's wrappers. GFX technologies: GDI, GDI+, AGG, Direct2D, CoreGraphics, GTK, OpenGL, DirectX and others. Script in Sciter is not quite a JS so I've created ports of jQuery (zepto feature set) and AngularJS alike data-binding mechanism based on Object.observe() that Sciter has. More of Sciter's history: http://sciter.com/10-years-road-to-sciter/ http://sciter.com/10-years-road-to-sciter/ At the moment Sciter code works on around 350 mln PCs and Mac's worldwide as part of various products. Advice: all this is really possible if you have real motivation to do all this. Estimate that first. Split the task on 6-9 months periods. Start from creating basic DOM and CSS rendering. You can do that without parsers initially. Parser are boring - postpone them for the moment when you will have something breathing. Just a small trick for self-motivation. Other than that it is doable, yes. For the simple reason: as smaller dev team as higher its productivity. Solo developer has highest productivity possible. Some conditions apply of course.

45 points by avindroth 10y ago

deleted 10y ago

[deleted]

wslh 10y ago

You have mainly two options: do everything from scratch or reuse. Starting from scratch is almost impossible to do alone, just Chrome was developed by different teams with different expertises, some of them acquired from other companies. The second option is reusing a web component or browser and modify it for your purposes. Building on top of Chrome or Firefox (or their variants) is a good choice. I think Chrome is better engineered than Firefox and the code is clear. Obviously this is just an opinion. Now the question is what do you want to add to such browser? Is this something you can add via an extension or is something that requires a new direction?

basch 10y ago
chrome/blink was built on webkit which was a fork of khtml. none of it was from scratch within the last decade.
sglane 10y ago
>Chrome was developed by different teams with different expertises, some of them acquired from other companies. To expand on this, Chrome and Safari use webkit as their rendering/layout engine which is a fork of KDE's KHTML. IE came from NCSA Mosaic-> Spyglass -> Microsoft IE. To be clear on my suggestions below, let's clarify what a browser actually is. The browser manages things like the navigation bar, bookmarks, download/mime handling, etc. This is known as the "chrome" or GUI. The layout engine handles HTML parsing, rendering, possibly is coupled with a Javascript engine like V8. If you are trying to write a browser, the answer is to use webkit2 in whatever language you like. It's fast, light, and portable. If you are trying to write a layout engine (rendering engine), do as another poster suggested and write one to target HTML 3.2.
- timvdalen 10y ago
  Note that Chrome is actually using Blink now, which itself is a fork of webkit
fspeech 10y ago
Servo https://github.com/servo/servo https://github.com/servo/servo is a browser rewrite in Rust. Could be a good place to start.
- zamalek 10y ago
  Or a good study if the the goal language is different. Studying a clean codebase is where I'd start, there is likely a lot of tribal knowledge in HTML rendering (especially layout) that isn't documented anywhere but in code.

ruraljuror 10y ago

This series of blog posts might be a good place to start: https://limpet.net/mbrubeck/2014/08/08/toy-layout-engine-1.html https://limpet.net/mbrubeck/2014/08/08/toy-layout-engine-1.h...

magic_beans 10y ago
This series was really well-written and fun to work through! I second this recommendation.

rer 10y ago

If you wanted to choose how to build the UI, one starting point is here https://news.ycombinator.com/item?id=12376242 https://news.ycombinator.com/item?id=12376242

p333347 10y ago

If you are seeking tutorials, or UI libraries etc, sorry, I can't help you with that as I am unaware of those things. However, if you are asking from which aspect to start building one from scratch, I would say start with layouting. Try implementing a tiny subset of div related things like position and display and see if this is what you want to spend your time working on. Even if you can crack this thing easily, remember, this is just the tip of the iceberg. I am not discouraging you, but know that you would have to digest the entire DOM and HTML spec, not to mention have a JavaScript engine and implement all the security related intricacies, at the least, for your software to become a web browser.

anatoly 10y ago
I'm curious: is there a ready-to-use set of tests for a layout engine that checks conformance with the specs? Something that you could, without much effort, run against a toy layout engine you'd write? I know Webkit/Blink have humongous test suites, but these probably aren't easy to adapt to a toy layout engine.
- chiefofgxbxl 10y ago
  There's something like Acid3, but it doesn't give you information as to what failed when you get a less-than-perfect score. (Test: http://acid3.acidtests.org/ http://acid3.acidtests.org/ | Information: http://www.webstandards.org/action/acid3/ http://www.webstandards.org/action/acid3/) There are also all those other browser benchmarks (e.g. http://peacekeeper.futuremark.com/ http://peacekeeper.futuremark.com/), but those include things other than the layout engine, such as javascript performance, webGL, etc. You could try poking around a few of them and only looking at the CSS and DOM sections though (http://web.basemark.com/ http://web.basemark.com/).
- zokier 10y ago
  W3C Web Platform Tests probably would be best starting point. http://testthewebforward.org/ http://testthewebforward.org/

knabacks 10y ago

My 2 Cents: Start out with a solid base: (for example) - https://cefbuilds.com/ https://cefbuilds.com/ - https://crosswalk-project.org/ https://crosswalk-project.org/ - http://electron.atom.io/ http://electron.atom.io/ - http://nwjs.io/ http://nwjs.io/ or you try to help with existing projects: (for example) - https://github.com/breach/thrust https://github.com/breach/thrust (started out with https://github.com/breach/breach_core https://github.com/breach/breach_core a "javascript"-Browser (core is not js)) https://github.com/minbrowser/min https://github.com/minbrowser/min

michaelmior 10y ago
Breach seems like a poor choice since the project has been inactive for ~2 years.
- knabacks 10y ago
  Maybe you're right, a project like servo (https://github.com/servo/servo https://github.com/servo/servo) looks like a better fit. easy bugs to hunt: https://github.com/servo/servo/labels/E-easy https://github.com/servo/servo/labels/E-easy
deleted 10y ago
[deleted]

_RPM 10y ago

Just a note: The browser as we use it today is a result of 20+ years of human work and innovation. It's what you would call a killer application.

iamben 10y ago
Equally - the car we drive today is a result of 130 years of human work and innovation. But after all that time we're seeing a real shake with the electric car. Things change when people try. I say good luck to the girl/guy - perhaps they'll create something incredible :-)
- richardwhiuk 10y ago
  Even the electric car didn't start from scratch - it'll use the same component manufacturers as the rest of the car industry - e.g. tyres, brakes, suspension, interior, safety systems.
  - reitoei 10y ago
    You and parent poster both make good points.

mixedCase 10y ago

Well, what are your goals?

stellar678 10y ago

"If you wish to make a web browser from scratch, you must first invent the universe."

bigato 10y ago

1. Write a webassembly interpreter/vm; (main step) 2. Using webassembly, write an html interpreter/renderer; 3. Using webassembly, write an css, js ... interpreter/renderer; 4. Keep doing this for all w3c standards for the rest of your life. Chances are that by the time you are somewhere in the middle of step 2, people will have written the other interpreters in webassembly and the result of step 1 will be a functional browser already.

max_ 10y ago

Browser Engine https://limpet.net/mbrubeck/2014/08/08/toy-layout-engine-1.html https://limpet.net/mbrubeck/2014/08/08/toy-layout-engine-1.h... How to Create your Own Internet Browser in 5 Minutes! - https://www.youtube.com/watch?v=Bn22JBRC1UI https://www.youtube.com/watch?v=Bn22JBRC1UI Chrome's Engine http://www.chromium.org/blink http://www.chromium.org/blink

marai2 10y ago

There is a "Building a Web Browser" course on udacity[1]. Maybe someone who has taken the course comment on it. [1] https://www.udacity.com/course/programming-languages--cs262 https://www.udacity.com/course/programming-languages--cs262

pmontra 10y ago

As somebody already wrote "Starting from scratch is almost impossible to do alone". However you could set a first goal of doing some basic rendering of plain HTML, even text based. That should make you familiar with the hurdles of a rendering engine. Then apply some CSS. Even writing a browser for HTML 3.2 from scratch and no JavaScript should be an exercise to be proud of https://www.w3.org/TR/REC-html32 https://www.w3.org/TR/REC-html32 (1997). Then start looking at modern browsers using the links others provided. You should have the experience to understand all the moving parts by then. This is similar to writing the very first web application without using any framework, maybe even interfacing directly with the web server. I remember decoding CGI arguments in C in 1994 before discovering Perl's CGI.pm. Then you understand the magic inside frameworks and don't get burned by surprises.

ayyn0n0n0 10y ago

TempleOS

chvid 10y ago

Back when I did CS 101 this was our final project; the teacher made the HTML-parser, and we students did the renderer and UI. The HTML-parser is probably the part it makes sense to do first. What language and platform do you plan to use? I think a modern OO-language will be helpful.

matthewhall 10y ago

I'm working on one right now. I'm using the electron shell because im not worried about the backend source (chromium) I'm just working on a new ui and a reader feature. Email at matthew349hall@hotmail.com if you want to get a beta

philippnagel 10y ago
I sent you an email.

jbb555 10y ago

I'd think about it like this :- 1) Write socket code to retrieve the page from a server 2) Get it to draw the text in a window. 3) Write code to look for tags and start diving up the retrieved file into a proper DOM tree. Perhaps start by creating an object for each paragraph. Then maybe look for <b> tags, then <div> tags 4) Change the rendering code to draw the screen for the data structure you created above. 5) Iterate a lot. Add support for parsing more and more tags into the data model. Start adding annotations for various attributes. 6) Start improving the layout code. Draw things in <b> tags in bold. Draw things in <div> tags under each other and <span> tags separately. 7) Add limited CSS support. Allow just width and height and border sections. make the code read the css file and attach the attributes to the correct sections of the DOM document you are creating. 8) Improve the layout code to look at the width and height tags on each element, and where tags are nested to propogate the width and height information up and down the tree as needed. Draw borders if the css tells you to. Look at font weight etc. 9) Iterate. Add one feature at a time. Repeat for a year until you have an browser that can render basic pages. 10) Iterate some more. Read specs. Rewrite your document to DOM tree parse a number of times perhaps using a more formal grammar. You'll probably be on about year 8 by now. 11) Add javascript support... :P Perhaps my point is though that this seems something where its very easy to start small, just render the text of a page, and incrementally add features and improve the layout. It will take a long time though as there are so many features to add.

kens 10y ago
The above list is a good start. I'd add: 2a) make clicking on links work, and 3a) display images. Those two features will make it much more fun. "How to make a browser" is something I've thought about for a while, ever since someone suggested it as an interview question. (I think it would be an awful question.) It really depends on what you're trying to do. If you want to understand how browsers work and build a toy browser, follow jbb555's list. If you want to display web pages, use WebKit. If you want to build a real, commercial-scale browser, I recommend a large team, since there are a lot of components, each of which is insanely complex. (Everything from CSS support to security to plugins to JavaScript to bookmarks to a debugger to all the new network protocols.) One more thing to add to jbb555's list: 0) "telnet news.ycombinator.com 80, GET /" - playing around from the command line can give you a good introduction to what really happens when you access a web page. (Edit: HN returns an error page for a plain GET. google.com returns insane code. apple.com looks like a better place to start.)

IncRnd 10y ago

You can accept entered URLs and fetch & display web pages quite easily nowadays. You should, depending on the vertical, be very concerned about security. What are your goals (to a greater degree than "web browswer")?

newscracker 10y ago

[Disclaimer: I'm not an expert on developing browsers] It depends on what exactly you'd want to implement, since a (modern) browser is really a huge piece of software with several aspects and components in it. Looking at it from one angle, you could split it into chrome (not Google Chrome, but the UI, like tab bar, bookmarks, menus, extensions management, etc.) and content (the rendering engine). You'd most likely get bogged down if you start writing your own rendering engine (processing HTML, CSS and JS, and processing image formats, video and sound formats, even not including any plugins), unless it's a very limited scope toy browser that can handle some simple stuff. In this case, start with base HTML support, including handling incomplete tags and quirks like that have been carried over for decades (the amount of cruft in rendering engines for backward compatibility and handling undocumented but accepted quirks would itself take a long time to develop). Then try a little bit of CSS support. I'd assume JS support could be quite challenging since that moves out of the already challenging "paint the screen" part into an even more challenging part on interpreting code, running it, optimizing it, etc., which is a lot more complex when you consider there would be several JS files loaded on most websites. Something a little more easier (only comparatively) would be developing the chrome around an existing rendering engine and seeing how to handle tabs, background loading, foreground loading, bookmarks, private/incognito browsing, etc., and if you feel adventurous enough, adopt a particular process or threading model for a multi-tab browser. Another alternative here is to create a WebView application and see which parts you could improve. For example, Chrome on iOS and Firefox on iOS are just the chrome for the built-in rendering engine in iOS, but address many things like tabs, networking, incognito mode, bookmarks, sync and other features. You could try creating a similar one on iOS, latching on to the work done by FoxBrowser [1] or Firefox for iOS [2]. On Android, you could try building on top of Firefox for Android [3]. If you'd like to work on a web browser rendering engine that's a bit more closer to "from scratch", I'd recommend working with the Servo [4] team and seeing how you can help. It's a long drawn project that has already made huge strides, but you'd get an inside view of the complexity of developing a browser engine. What's more, you could also learn Rust while doing this. [5] [1]: https://github.com/graetzer/Foxbrowser https://github.com/graetzer/Foxbrowser [2]: https://github.com/mozilla/firefox-ios https://github.com/mozilla/firefox-ios [3]: https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Build_Instructions/Simple_Firefox_for_Android_build https://developer.mozilla.org/en-US/docs/Mozilla/Developer_g... [4]: https://servo.org/ https://servo.org/ [5]: https://www.rust-lang.org/ https://www.rust-lang.org/

WayneBro 10y ago

Do you want to build a web browser or a web browser engine? If you just want to build a web browser, here's my request for a new browser for iOS since web browsing on iOS is a second or third rate experience: - Built-in ad-blocking. - A way to disable JavaScript immediately for any website. - A desktop mode that actually works, so that YouTube, Reddit, Imgur and other sites don't redirect me to their crappy mobile site without asking. - Launches a completely separate app for private browsing. Right now I use Dolphin browser on iOS and despite being in desktop mode, I still have to refresh every YouTube page that I land on in order to get the desktop style native HTML5 video player that allows me to go full-screen. On Imgur, gifv files don't play - I have to change the extension to gif and then wait 60 seconds as it downloads. Lots of other sites have similar types of issues. Meanwhile, gifv files work fine in Brendan Eich's "Brave" browser. It could be as easy to fix as just changing the user-agent. Dolphin has good ad-blocking, but there is no way to quickly turn off JavaScript for a given page or domain (ala Quick JavaScript Switcher for Chrome). The thing that really annoys me though about Dolphin is that Private Browsing does not work at all. That's probably because they want you to buy Dolphin Zero for $2.99.

speg 10y ago
I checked out Brave yesterday for the first time and I think it does most of those things.
- WayneBro 10y ago
  Brave has a desktop mode that mostly works and a built-in adblocker. It does not have a `quick JS disable` button. It also doesn't have a speed dial that I can manually add items to, which is one very important item that I forgot to mention earlier. What I really want to do is make an open source browser that I can compile and put on my iOS devices myself too, without going to the app store. Then I could add all sorts of features and extensions without having to ask Apple.
softawre 10y ago
3 dollars? It really annoys you and you can't be bothered to spend 3 dollars?

retro64 10y ago

I did this once back in 2003. It was text only rendered, with mouse support so you could click on the links instead of tabbing around like Links. I built it off of the IE object model (Internet Explorer Browser Extensions? Can’t recall). I think I just used the object model for connectivity settings. If you could get to the net using IE, my browser would work as well. I think I coded to HTML 1.1 specs, and had it working after 2 months (with a few minor rendering issues). It did not support scripting, CSS or really much of anything (although it could POST), but it was surprising how much of the web was accessible. And it was really fast. Not because of my skillz, but when you cut out the garbage, it’s amazing how fast a page will load. Anyway, I would not recommend this approach, but I would offer you encouragement. HTML parsing isn’t really that big of a deal, and you could get something very minimal (text rendering) working in short order. It would at least give you a feeling of what you are up against.

beachstartup 10y ago

https://curl.haxx.se/dev/source.html https://curl.haxx.se/dev/source.html

pif 10y ago

By giving up? ;-)

txutxu 10y ago

Well, this is what I could do in that (or almost any) case: 1) Setting features, key points, points to avoid, and goals of the project. 2) Choosing the toolchain to use, for resolving the points at 1) 3) Coding the features with that toolchain Step 2, may involve: to read a lot (doc, rfc's, specifications, using search engines with keywords, search for "problems", etc) to look at other people's source code of similar programs, to try other people's tools and libraries and evaluate and compare them, to make integration tests, and to make Proofs Of Concept prototypes. The toolchain (programing language, user GUI, etc) comes with an ecosystem (I hope you don't aim at rewriting also your $lang libraries for validate XML or interact with devices), so you have to choose looking at both. To separate steps 2 and 3, use iterations, while you have resources. If you find an opensource product that full-fills all your definitions of point 1), except a few of them, you can evaluate to start by a patch and a pull-request to them, or a fork :P

hugozap 10y ago

I'd say, if you have the opportunity and want to go wild skip the DOM and create an equivalent model using OpenGL. All the other Web platform features like the distribution model, resource streaming etc can still be used. It's just that the DOM is the great bottleneck and most of the hours devoted to improve the Web are being wasted on sorting quirks related to the DOM. Maybe yours will be the starting point towards a DOM-free Web! :)

vmorgulis 10y ago

Instrument an existing browser and synthetize a new layout engine with machine learning and superoptimisation.

OldSchoolJohnny 10y ago

Step one: Build AI or machine learning or genetic code to build a browser by replicating the results of a real world browser. Step two: run it!

VLM 10y ago

Sounds like a pretty good interview question. Better than asking you to invert binary trees or similar WTF. They're testing your judgment and asking you to assess your own skills and possibly they're interested in the actual assessment results assuming you're any good at assessment. Most people would generally break the task chronologically or kinda data flow oriented. Probably. Now should you start at the part you're familiar with or less familiar or unfamiliar with? My philosophy is its a bathtub shaped curve and if you know the topic dead cold or have absolutely no idea how to do it, start there because those will be your constraints, either you are an industry expert on a topic and can define the constraints with the best of them in which case you should definitely start there unless you suspect a trick question (LOL you thought we meant HTML browser, actually we're passing you DB2 ISAM files and COBOL PIC statements), or you have absolutely no idea what you're doing in which case you should start there to begin to discover your constraints and APIs and interfaces and data structures. Never start at a point where you have intermediate level of knowledge, the impact when you smack into the unknown is going to be painful so start in the unknown to begin with. Basically, it boils down to try not to paint yourself into a corner, so either start painting in the most obvious safe area to start or the most mysterious area on the assumption escape will be the easiest. But never, ever start to paint a floor by just dumping the can out in the middle of the floor. If you have a flat level of skill across the entire project (yeah not likely) then if its a "simple" project like a viewer/browser where you've seen all the parts before, start in the middle and work both directions at once if you can handle the inevitable mystifying weirdness at both ends simultaneously, and if its a "complex" project like replace the entire air traffic control system then start at one edge and work to the other to minimize simultaneous confusion. Yeah yeah I know I just said never start painting in the middle of the floor, well if you think there's a perfectly flat skill level you probably haven't analyzed the problem enough so the real answer is not to start at all, and instead to think about it a lot more. Of course half the fun is defining the starting point. Do they give you a working example to clone as a goal, access to all RFCs, no docs and cleanroom reimplement using a protocol analyzers traces, can you use a language you like, can you use language libraries that implement some fraction of the problem, does it have to be a 1995 level of browser or 2015 or 2035 level of browser ... To some extent its a meta interview question where they want to see if you start talking like you're a system architect or systems analyst type like above or if you're a security guy you'd probably start with "review all the latest exploits and avoid all of them simultaneously (good luck lol)" or if you're a code monkey maybe you'd start with a rant about your favorite font to use in Eclipse and how nice your mechanical keyboard is or maybe you're not even a programmer in which case the response will sound like the dude from the office space movie in the scene arguing about his radio "well I was informed this was a programming test and I got permission to list the integers from 1 to 100 and print fizz when they're a multiple of ..." and so on.

deleted 10y ago

[deleted]

c-smile 10y ago

Well, I did exactly that. Sciter ( http://sciter.com http://sciter.com ) is an embeddable HTML/CSS/script engine that has all components of a browser - html, css, svg, script and networking (http,ajax,websockets,etc). Work started in 2002 when I decided to create WYSIWYG HTML editor for my wife. So http://blocknote.net/ http://blocknote.net/ was born. That was pure HTML (no CSS) rendering and editing engine. 9 months for one developer - myself. After its publishing joined initial EverNote team to create WYSIWYG editor for EverNote. Three of us (developers), + Stepan Pachikov that generated the idea of the EverNote, created first published version of the EverNote (v 2.0). That was also done in 9 months (I am pretty sure that 9 months is not a coincidence but rather a law for new projects - optimal period when developers are excited of what they are doing). While working on it, and being a UI-in-all-its-incarnations developer/architect, I've discovered that HTML and CSS are quite convenient format for defining UI layouts and elements. So next step was wrapping it as a reusable component and so HTMLayout engine was born ( http://htmlayout.com http://htmlayout.com ). Almost immediately discovered that plain C++ is not quite convenient for handling complex UI cases - "on click here hide that and expand those". UI objects ownership graph is quite complex and contains loops. Something with GC is definitely required. So decided to add scripting to it. Considered quite many existing languages including JS for the "language behind UI". At the end decided to roll my own tiscript ( http://www.codeproject.com/Articles/33662/TIScript-Language-A-Gentle-Extension-of-JavaScript http://www.codeproject.com/Articles/33662/TIScript-Language-... ) that essentially is JavaScript++. So 7 years ago I had what we will have in ES6 in next couple of years. At the mean time I participated in HTML5 spec. development at W3C as an invited expert. And worked as an architect / team lead in several Web Application projects that required deep understanding of the platform - knowing Web UI stack from inside-out helped that a lot. As Sciter works on all major desktop and mobile platforms I've created core engine wrappers for Windows API, MacOS CoreFoundation, Linux/XWindows/GTK and some others. Sciter core is written in C++ with Objective C fragments in Apple's wrappers. GFX technologies: GDI, GDI+, AGG, Direct2D, CoreGraphics, GTK, OpenGL, DirectX and others. Script in Sciter is not quite a JS so I've created ports of jQuery (zepto feature set) and AngularJS alike data-binding mechanism based on Object.observe() that Sciter has. More of Sciter's history: http://sciter.com/10-years-road-to-sciter/ http://sciter.com/10-years-road-to-sciter/ At the moment Sciter code works on around 350 mln PCs and Mac's worldwide as part of various products. Advice: all this is really possible if you have real motivation to do all this. Estimate that first. Split the task on 6-9 months periods. Start from creating basic DOM and CSS rendering. You can do that without parsers initially. Parser are boring - postpone them for the moment when you will have something breathing. Just a small trick for self-motivation. Other than that it is doable, yes. For the simple reason: as smaller dev team as higher its productivity. Solo developer has highest productivity possible. Some conditions apply of course.