23 ms·

The web has been engineered to a complexity level so grossly obscene that it is all but impossible for independent developers to produce fully independent web b

by near 9y ago

The web has been engineered to a complexity level so grossly obscene that it is all but impossible for independent developers to produce fully independent web browsers anymore. You would need hundreds of millions of dollars in capital and strong leadership to produce a browser that would be successful in today's world. I could dedicate the rest of my life working 100 hours a week on a browser, and I would die before I had something that could compete with browsers of 2017, let alone of the browsers far into the future at my demise. The standards grow in complexity faster than a small team of developers could possibly keep up with. The best we have are attempts at forks of major browser engines, which will never gain any serious traction to have any power over the direction of web standards. If people want to do something about it, they need to fork the web itself. HTTP, TLS, HTML, CSS, JS ... the entire stack needs to be scrapped and replaced with something sensible with a focus on simplicity. If a single highly talented developer working full-time can't implement a reasonable browser within a year, then the standard is too complex. This will of course never happen. And so, on the subject of DRM, we've officially lost the war today. And combined with the impending loss of net neutrality, brace yourselves everyone: it's only going to get worse from here on out.

tudorw 9y ago
If we want to win that war, we should focus on content not technology, Mosaic got a lead in the game because, despite the 'techies' seeing no use for images, image support was added, with images came the transition of media, users were au fait with articles containing text and images and could now experience similar on line, this alone was not enough, content creators starting creating content that people wanted to consume, that's what drove people to 'get online', they were missing out. Don't get me wrong, it's still a long shot, but if an organisation could build a raft of talent, get that talent to make content that was only available on an open and progressive platform, then you might get some traction. So what might that look like, well, say I had the funds to secure 100 of the worlds best financial journalists and only delivered that content on a novel protocol, do you think I might be able to persuade some people to run a novel application to access that content, oh hang on, did I just describe Bloomberg...
- cjsuk 9y ago
  I agree. Back in the dark ages, when the battle of www vs Gopher was still on, I sided heavily on the side of Gopher because it didn't get in the way of the content. Gopher was a means to navigate the maze of content and most of the content was textual, postscript and neatly typeset. You could sit down with it and read it focused or print it out. We've regressed to the point where 5KiB of text comes with 2MiB of crap, meaning a noise to signal ratio of 26dB which is insane! Really Gopher is dead though. We could do with a fresh start. A search engine like WAIS, a menu system like Gopher and a standard document format that allows you to get quality content on everything from your e-reader to paper without having to read some fucked up ebook flow layout conversion. And not postscript. Something you can author by hand. I've even had enough of building things on the web. It is a horrible horrible thing to have to do. I envy system programmers who never see the pain of trying to keep a very large web application secure at even a basic level.
  - cabalamat 9y ago
    > We've regressed to the point where 5KiB of text comes with 2MiB of crap, meaning a noise to signal ratio of 26dB which is insane! Sadly the average clickbait article contains a lot less than 5000 characfters of actual text.
    - chris_wot 9y ago
      I don't see why these webpages need to be so huge. If websites used HTML5 properly and used CSS more extensively, a lot of the crap I feel could be removed. Seriously, aside from graphics and video, what is taking up the size of these pages? Mountains of JavaScript?
      - cannonedhamster 9y ago
        As someone who used to work for a web optimization company, yes mountains of Javascript. Javascript for analytics, javascript for transitions, javascript for DRM, javascript for updating pricing in real time, javascript for fonts, javascript for colors, javascript to lazy load the images....six different style sheets and images that are downloaded and resized client side...and this is just the navigation bar.
sexydefinesher 9y ago
Correction: its the loss of _American_ net neutrality, places such as the EU for example already has enforced net neutrality for all members.
- Y_Y 9y ago
  I don't think that's strictly true. For example many mobile ISPs here meter data differently depending on whether it's from their fav websites (Facebook, Snapchat, Netflix etc.) or from the lame weird web where nerds go.
enriquto 9y ago
We should all move back to gopher. Seriously.
- tudorw 9y ago
  I call dibs on the first gopher based social network... [edit] So I've read the RFC (https://tools.ietf.org/html/rfc1436 https://tools.ietf.org/html/rfc1436) looks like it should be fairly simple, we can run gopher servers on both user machine and remote servers, so I can store my personal details locally, then give permission to other remote gopher servers to access that in a granular manner, third parties can operate gopher servers I can interface with to access 'cloud' style computing, storage, search, and manage contacts/calendars, sorted :) [edit 2] Looks like Gopher has a pulse, you can access around 4.9 million unique 'files' using an HTML>Gopher tool run on this site http://gopher.floodgap.com/gopher/ http://gopher.floodgap.com/gopher/ here's a nice Ars write up https://arstechnica.com/tech-policy/2009/11/the-web-may-have-won-but-gopher-tunnels-on/ https://arstechnica.com/tech-policy/2009/11/the-web-may-have... [edit 3] My build of firefox does not seem to have Gopher support, it recognises the link (chrome just does nothing), but asks you for a local application to fulfil the request :( [edit 4] It's now a plugin for firefox, https://addons.mozilla.org/en-US/firefox/addon/overbiteff/ https://addons.mozilla.org/en-US/firefox/addon/overbiteff/
  - tudorw 9y ago
    Cool, I'm searching Gopher, and it works :) Nice and quick, no tracking, no ads, just need something to consume, anyone want to help me whip up an HN > Gopher widget ;)
    - tudorw 9y ago
      there's nothing new right, gopher://hngopher.com/1
- jarman 9y ago
  I am all for text-only, structured web; but gopher transfer protocol is broken mess (which is an achievement considering simplicity of it's purpose).
amelius 9y ago
Agree. Here's how to approach it, imho: 1. Assume we will eventually converge to a virtual machine target like WASM. Thus, eventually, browsers will only have to implement the low-level instructions of WASM, which will make building a browser much simpler. Note that as a side-effect, browsers may also become more secure. 2. Start with a working webbrowser, like Firefox or Chromium. 3. Step by step, move functionality from the browser into the browser's user-space. Eventually, JavaScript and the rendering of CSS and HTML will run in the browser's user-space.
l0b0 9y ago
You wouldn't need to scrap all of it by a long shot. You could take a small subset of the stack and make something enormously simpler to implement. Start with supporting only one version of HTTP, TLS, (X)HTML, CSS and JS and fork them if necessary. Even drop CSS to start with and finally start relying on semantic markup and good browser defaults. Use existing open source libraries where available, such as for TLS and markup/media parsing. Pull out libraries for everything so the browser is easily composable and forkable. You don't want JS support at all? Compile without it.
- ehnto 9y ago
  Not many would be interested in an un-designable web. It bares no comparison to what browsers are capable of delivering for UX currently. What you are suggesting then, a non-comparable browser experience, already exists and has for years, yet to no avail. You would get niche technologists and die hard enthusiasts only and the rest of the world would trudge happily into the future, DRM and net nutrality distant memories. What you and I want, what HN and friends want, is not even a debate most realise is happening let alone that it affects them and will affect them profoundly in the future. The masses are deciding what our internet will look like for they comandeered it years ago.
  - l0b0 9y ago
    Yeah, yeah, the world is going to shit and nobody cares. How about doing something about it instead of telling everybody it can't possibly work? And why do we need the rest of the world on the bandwagon for it to work? The web was built without the masses, and still is. The open web still vastly outweighs the closed parts in amount of content. A bunch of freedom stealing technologies have died silently over the years despite often resulting in "cool" UX wholly different from plain HTML, such as Flash, Silverlight, and Java applets. EME is a fundamentally different wedge inserted between users and their machines by evil corporations, but not one that is in any way unbeatable. So screw 'em, and let's fork the web!
    - ehnto 9y ago
      Forking the technologies that underpin the platform of the web I think tackles the wrong problem. The problem I see facing the open web is the consolidation of users and services into single platforms. Facebook is an easy example. You can build a Facebook alternative with any technology you like, it would take a completely different force to change the social paradigm around it. The great thing about the open web that you rightly mention is still kicking, is that it's full of places and services that are driven by humans and passionate communities. Nothing about EME stops those communities existing, but nothing about those communities is going to stop Netflix being the most popular platform and using EME/DRM. If you will excuse my fluffy analogy, the web is a farmers market that has a shopping mall being built right next door. What I think would help the web, is educating people that it's better to go to the farmers market so that we can support entrepreneurial and local community endeavors rather than stuffing corporate coffers and enabling that consolidation.
thinkloop 9y ago
> If people want to do something about it, they need to fork the web itself. HTTP, TLS, HTML, CSS, JS ... the entire stack needs to be scrapped and replaced with something sensible with a focus on simplicity. That's flash.
- kagamine 9y ago
  > That's flash Comedy aside, CSS isn't the problem, but if an experienced web-dev can't make a site without learning for 5+ years and then feel like they have no knowledge of web development then we have a problem, and that's where we are now. The problem with the long list of frameworks and libraries available is that there is no easy way to make a web2.0 site without years and years of learning and then the resulting site is a mish-mash of what works without it being easy to maintain. Sitting here looking at .NET & angularjs 1.x and wondering where it all went wrong (actually, not the.,NET part, that seems to work).
  - someonewhocar3s 9y ago
    There's a lot of people signing onto stupid web frameworks and other hype. Within .NET there is a single filter pass applied by Microsoft, but honestly I don't touch it. None of that is a proper critique on the Web. You're basically critiquing freedom of choice, which makes sense if you like .NET ;)
  - chris_wot 9y ago
    CSS makes semantic markup work. CSS is not the issue, the issue is getting a decent graphical editor that does it well.
  - bonzini 9y ago
    What do you mean by "make a site"? I made qemu.org last year with 10-years-old knowledge of web development and, apart from the obviously not-done-by-a-graphics-designer theme, it _is_ a decent site, responsive and with a 2010s-ish look. 99% of websites can be done with either a static generator (lean) or WordPress (somewhat bloated, but really mostly standard libraries that can be cached and/or served by a CDN). The remaining 1% are the really bloated ones and sure they are the ones that we visit all the time (medium, BBC, whatever). But it's certainly possible to make a site with not too much knowledge of web development.
dannyw 9y ago
Should a single developer working full time be able to implement an OS within a year? The web is pretty much an operating system.
- byuu 9y ago
  Ideally, yes. It may not've been popular at that point, but Linus did. But what about the next Linus of 2017? Something like that's not going to be possible anymore. Just the basic necessary hardware drivers alone. The hardest challenge for Linux was just avoiding Winmodems. These days the entire nouveau project with years of research can't even manage to change the clock speed of your GPU. VESA's been all but abandoned so there's not even a way to set a widescreen framebuffer anymore for pure software rendering. Nothing is gained from having hopelessly complex technology.
  - nradov 9y ago
    The next Linus of 2017 will be on a completely different platform, not PCs or smartphones. That's the nature of disruptive innovations.
    - fasquoika 9y ago
      You say that like Linus didn't write linux on the same platform everyone else was using
swiley 9y ago
ehhh HTML parsers/DOMs are trivially easy to write. after that you need to go get a JS lib (like ducktape) and write a css and json parser. After that all you need is a layout engine, which GUI tookits like tk and gtk+ already have for text editing.
- nradov 9y ago
  Parsers for valid HTML are trivially easy to write. The hard part is dealing with garbage HTML tag soup and still rendering something that looks good. Browsers have to figure out what the page author intended and work around all the content defects.
  - gsnedders 9y ago
    Parsers for invalid HTML are trivially easy to write: you literally just implement the spec, and that's not hard, and then you implement every major browser.
    - nradov 9y ago
      You're joking right? Sometimes it's hard to tell...
      - gsnedders 9y ago
        I'm not joking. It's not that hard. You literally pick up the spec and implement it, it's mostly just fairly boring menial work.
        nradov 9y ago
        I think you misunderstood. There is no "spec" for handling all of the invalid HTML out there. That's part of what makes writing a browser so hard. It's far from being menial work.
        realityking 9y ago
blitmap 9y ago
I am hoping feature-testing continues to be a good practice - I don't expect to have webgl, the webusb, or webaudio APIs available...
oneeyedpigeon 9y ago
I think you're right, but only for a narrow definition of "browser". Sure, something that looks and works exactly like Chrome, Safari, et. al. would be a gargantuan effort for a sole developer. But an 'HTTP client' can take many forms and still offer value, even advantages over the existing browsers which serve a very specific remit. A basic text browser is the kind of thing that's buildable in a year. Moreover, once you start building that, maybe you have an idea for certain features that current browsers don't do well (bookmarking, navigation) or at all (change monitoring, api integration, editing). I'm optimistic that there is a lot of unlocked potential in the HTTP/HTML platform.
lucaspottersky 9y ago
as the standard gets bloatted, new players could leverage this weakness and build something entirely new. yes, reinvent the wheel. =)
deleted 9y ago
[deleted]
oceanswave 9y ago
Why stop with creating a browser from scratch? You'll need hundreds of millions of dollars in capital just to build your own compiler technology that would be successful enough and incorporates enough instruction set optimizations present in commercial CPUs that your custom browser can be built upon. Maybe you'll need your own hardware as well...

by near 9y ago

tudorw 9y ago

If we want to win that war, we should focus on content not technology, Mosaic got a lead in the game because, despite the 'techies' seeing no use for images, image support was added, with images came the transition of media, users were au fait with articles containing text and images and could now experience similar on line, this alone was not enough, content creators starting creating content that people wanted to consume, that's what drove people to 'get online', they were missing out. Don't get me wrong, it's still a long shot, but if an organisation could build a raft of talent, get that talent to make content that was only available on an open and progressive platform, then you might get some traction. So what might that look like, well, say I had the funds to secure 100 of the worlds best financial journalists and only delivered that content on a novel protocol, do you think I might be able to persuade some people to run a novel application to access that content, oh hang on, did I just describe Bloomberg...

cjsuk 9y ago
I agree. Back in the dark ages, when the battle of www vs Gopher was still on, I sided heavily on the side of Gopher because it didn't get in the way of the content. Gopher was a means to navigate the maze of content and most of the content was textual, postscript and neatly typeset. You could sit down with it and read it focused or print it out. We've regressed to the point where 5KiB of text comes with 2MiB of crap, meaning a noise to signal ratio of 26dB which is insane! Really Gopher is dead though. We could do with a fresh start. A search engine like WAIS, a menu system like Gopher and a standard document format that allows you to get quality content on everything from your e-reader to paper without having to read some fucked up ebook flow layout conversion. And not postscript. Something you can author by hand. I've even had enough of building things on the web. It is a horrible horrible thing to have to do. I envy system programmers who never see the pain of trying to keep a very large web application secure at even a basic level.
- cabalamat 9y ago
  > We've regressed to the point where 5KiB of text comes with 2MiB of crap, meaning a noise to signal ratio of 26dB which is insane! Sadly the average clickbait article contains a lot less than 5000 characfters of actual text.
  - chris_wot 9y ago
    I don't see why these webpages need to be so huge. If websites used HTML5 properly and used CSS more extensively, a lot of the crap I feel could be removed. Seriously, aside from graphics and video, what is taking up the size of these pages? Mountains of JavaScript?
    - cannonedhamster 9y ago
      As someone who used to work for a web optimization company, yes mountains of Javascript. Javascript for analytics, javascript for transitions, javascript for DRM, javascript for updating pricing in real time, javascript for fonts, javascript for colors, javascript to lazy load the images....six different style sheets and images that are downloaded and resized client side...and this is just the navigation bar.

sexydefinesher 9y ago

Correction: its the loss of _American_ net neutrality, places such as the EU for example already has enforced net neutrality for all members.

Y_Y 9y ago
I don't think that's strictly true. For example many mobile ISPs here meter data differently depending on whether it's from their fav websites (Facebook, Snapchat, Netflix etc.) or from the lame weird web where nerds go.

enriquto 9y ago

We should all move back to gopher. Seriously.

tudorw 9y ago
I call dibs on the first gopher based social network... [edit] So I've read the RFC (https://tools.ietf.org/html/rfc1436 https://tools.ietf.org/html/rfc1436) looks like it should be fairly simple, we can run gopher servers on both user machine and remote servers, so I can store my personal details locally, then give permission to other remote gopher servers to access that in a granular manner, third parties can operate gopher servers I can interface with to access 'cloud' style computing, storage, search, and manage contacts/calendars, sorted :) [edit 2] Looks like Gopher has a pulse, you can access around 4.9 million unique 'files' using an HTML>Gopher tool run on this site http://gopher.floodgap.com/gopher/ http://gopher.floodgap.com/gopher/ here's a nice Ars write up https://arstechnica.com/tech-policy/2009/11/the-web-may-have-won-but-gopher-tunnels-on/ https://arstechnica.com/tech-policy/2009/11/the-web-may-have... [edit 3] My build of firefox does not seem to have Gopher support, it recognises the link (chrome just does nothing), but asks you for a local application to fulfil the request :( [edit 4] It's now a plugin for firefox, https://addons.mozilla.org/en-US/firefox/addon/overbiteff/ https://addons.mozilla.org/en-US/firefox/addon/overbiteff/
- tudorw 9y ago
  Cool, I'm searching Gopher, and it works :) Nice and quick, no tracking, no ads, just need something to consume, anyone want to help me whip up an HN > Gopher widget ;)
  - tudorw 9y ago
    there's nothing new right, gopher://hngopher.com/1
jarman 9y ago
I am all for text-only, structured web; but gopher transfer protocol is broken mess (which is an achievement considering simplicity of it's purpose).

amelius 9y ago

Agree. Here's how to approach it, imho: 1. Assume we will eventually converge to a virtual machine target like WASM. Thus, eventually, browsers will only have to implement the low-level instructions of WASM, which will make building a browser much simpler. Note that as a side-effect, browsers may also become more secure. 2. Start with a working webbrowser, like Firefox or Chromium. 3. Step by step, move functionality from the browser into the browser's user-space. Eventually, JavaScript and the rendering of CSS and HTML will run in the browser's user-space.

l0b0 9y ago

You wouldn't need to scrap all of it by a long shot. You could take a small subset of the stack and make something enormously simpler to implement. Start with supporting only one version of HTTP, TLS, (X)HTML, CSS and JS and fork them if necessary. Even drop CSS to start with and finally start relying on semantic markup and good browser defaults. Use existing open source libraries where available, such as for TLS and markup/media parsing. Pull out libraries for everything so the browser is easily composable and forkable. You don't want JS support at all? Compile without it.

ehnto 9y ago
Not many would be interested in an un-designable web. It bares no comparison to what browsers are capable of delivering for UX currently. What you are suggesting then, a non-comparable browser experience, already exists and has for years, yet to no avail. You would get niche technologists and die hard enthusiasts only and the rest of the world would trudge happily into the future, DRM and net nutrality distant memories. What you and I want, what HN and friends want, is not even a debate most realise is happening let alone that it affects them and will affect them profoundly in the future. The masses are deciding what our internet will look like for they comandeered it years ago.
- l0b0 9y ago
  Yeah, yeah, the world is going to shit and nobody cares. How about doing something about it instead of telling everybody it can't possibly work? And why do we need the rest of the world on the bandwagon for it to work? The web was built without the masses, and still is. The open web still vastly outweighs the closed parts in amount of content. A bunch of freedom stealing technologies have died silently over the years despite often resulting in "cool" UX wholly different from plain HTML, such as Flash, Silverlight, and Java applets. EME is a fundamentally different wedge inserted between users and their machines by evil corporations, but not one that is in any way unbeatable. So screw 'em, and let's fork the web!
  - ehnto 9y ago
    Forking the technologies that underpin the platform of the web I think tackles the wrong problem. The problem I see facing the open web is the consolidation of users and services into single platforms. Facebook is an easy example. You can build a Facebook alternative with any technology you like, it would take a completely different force to change the social paradigm around it. The great thing about the open web that you rightly mention is still kicking, is that it's full of places and services that are driven by humans and passionate communities. Nothing about EME stops those communities existing, but nothing about those communities is going to stop Netflix being the most popular platform and using EME/DRM. If you will excuse my fluffy analogy, the web is a farmers market that has a shopping mall being built right next door. What I think would help the web, is educating people that it's better to go to the farmers market so that we can support entrepreneurial and local community endeavors rather than stuffing corporate coffers and enabling that consolidation.

thinkloop 9y ago

> If people want to do something about it, they need to fork the web itself. HTTP, TLS, HTML, CSS, JS ... the entire stack needs to be scrapped and replaced with something sensible with a focus on simplicity. That's flash.

kagamine 9y ago
> That's flash Comedy aside, CSS isn't the problem, but if an experienced web-dev can't make a site without learning for 5+ years and then feel like they have no knowledge of web development then we have a problem, and that's where we are now. The problem with the long list of frameworks and libraries available is that there is no easy way to make a web2.0 site without years and years of learning and then the resulting site is a mish-mash of what works without it being easy to maintain. Sitting here looking at .NET & angularjs 1.x and wondering where it all went wrong (actually, not the.,NET part, that seems to work).
- someonewhocar3s 9y ago
  There's a lot of people signing onto stupid web frameworks and other hype. Within .NET there is a single filter pass applied by Microsoft, but honestly I don't touch it. None of that is a proper critique on the Web. You're basically critiquing freedom of choice, which makes sense if you like .NET ;)
- chris_wot 9y ago
  CSS makes semantic markup work. CSS is not the issue, the issue is getting a decent graphical editor that does it well.
- bonzini 9y ago
  What do you mean by "make a site"? I made qemu.org last year with 10-years-old knowledge of web development and, apart from the obviously not-done-by-a-graphics-designer theme, it _is_ a decent site, responsive and with a 2010s-ish look. 99% of websites can be done with either a static generator (lean) or WordPress (somewhat bloated, but really mostly standard libraries that can be cached and/or served by a CDN). The remaining 1% are the really bloated ones and sure they are the ones that we visit all the time (medium, BBC, whatever). But it's certainly possible to make a site with not too much knowledge of web development.

dannyw 9y ago

Should a single developer working full time be able to implement an OS within a year? The web is pretty much an operating system.

byuu 9y ago
Ideally, yes. It may not've been popular at that point, but Linus did. But what about the next Linus of 2017? Something like that's not going to be possible anymore. Just the basic necessary hardware drivers alone. The hardest challenge for Linux was just avoiding Winmodems. These days the entire nouveau project with years of research can't even manage to change the clock speed of your GPU. VESA's been all but abandoned so there's not even a way to set a widescreen framebuffer anymore for pure software rendering. Nothing is gained from having hopelessly complex technology.
- nradov 9y ago
  The next Linus of 2017 will be on a completely different platform, not PCs or smartphones. That's the nature of disruptive innovations.
  - fasquoika 9y ago
    You say that like Linus didn't write linux on the same platform everyone else was using

swiley 9y ago

ehhh HTML parsers/DOMs are trivially easy to write. after that you need to go get a JS lib (like ducktape) and write a css and json parser. After that all you need is a layout engine, which GUI tookits like tk and gtk+ already have for text editing.

nradov 9y ago
Parsers for valid HTML are trivially easy to write. The hard part is dealing with garbage HTML tag soup and still rendering something that looks good. Browsers have to figure out what the page author intended and work around all the content defects.
- gsnedders 9y ago
  Parsers for invalid HTML are trivially easy to write: you literally just implement the spec, and that's not hard, and then you implement every major browser.
  - nradov 9y ago
    You're joking right? Sometimes it's hard to tell...
    - gsnedders 9y ago
      I'm not joking. It's not that hard. You literally pick up the spec and implement it, it's mostly just fairly boring menial work.
      - nradov 9y ago
        I think you misunderstood. There is no "spec" for handling all of the invalid HTML out there. That's part of what makes writing a browser so hard. It's far from being menial work.
        realityking 9y ago

blitmap 9y ago

I am hoping feature-testing continues to be a good practice - I don't expect to have webgl, the webusb, or webaudio APIs available...

oneeyedpigeon 9y ago

I think you're right, but only for a narrow definition of "browser". Sure, something that looks and works exactly like Chrome, Safari, et. al. would be a gargantuan effort for a sole developer. But an 'HTTP client' can take many forms and still offer value, even advantages over the existing browsers which serve a very specific remit. A basic text browser is the kind of thing that's buildable in a year. Moreover, once you start building that, maybe you have an idea for certain features that current browsers don't do well (bookmarking, navigation) or at all (change monitoring, api integration, editing). I'm optimistic that there is a lot of unlocked potential in the HTTP/HTML platform.

lucaspottersky 9y ago

as the standard gets bloatted, new players could leverage this weakness and build something entirely new. yes, reinvent the wheel. =)

deleted 9y ago

[deleted]

oceanswave 9y ago

Why stop with creating a browser from scratch? You'll need hundreds of millions of dollars in capital just to build your own compiler technology that would be successful enough and incorporates enough instruction set optimizations present in commercial CPUs that your custom browser can be built upon. Maybe you'll need your own hardware as well...