3 ms·

So under the hood it's chromium headless and you are streaming screenshots back using an extension Curious why not run normal chromium and vnc the whole screen

by pgo 5y ago

So under the hood it's chromium headless and you are streaming screenshots back using an extension Curious why not run normal chromium and vnc the whole screen back using a web client such as novnc, would have solved most of your "Normal Browser UI things not yet implemented"

graderjs 5y ago
Not an extension. Just your browser, no download needed. So a "clientless" remote isolated browser. it runs headless Chrome, yes (but eventually I want to repurpose it to support other browsers that can be driven by DevTools/WebDriver protocol). Short answer is because a lot of the work of this was building a custom "head", so having a browser engine without a head made sense. Slightly more detailed answer below... :) ;p xx I might experiment with xvfb chrome and vnc in future. Other people certainly have success with this, e.g Reflect (YC S20)[0]. Maybe I got here accidentally. Original purpose was for mapping web app structure, and recording and replaying of web action sequences (automation) from any device. As most of the time the browsers were going to replay without anyone using them having xvfb seemed unnecessary. And doing the mapping and recording in one platform (real chrome + xvfb) then replaying in another I thought would risk many unnecessary, difficult bugs. Also having my own wire protocol based on DevTools, rather than use what a VNC web client uses, I believed would assist in more reproducible recordings and playback for automation. Also I liked the flexibility and control provided by headless+devtools to change the UI, easily work on mobile, custom handlers for tasks like file upload and other modals, as well as the control over the viewport streaming. On streaming, I experimented a lot to get something working across a range of bandwidths, and settled on: a per-frame ack (to avoid congestion), frames over webrtc or binary werbsocket channels, and periodically switching to whichever of webrtc of websocket is faster. I also want it to be really scalable. And I thought that headless+devtools would be much less resource heavy than chrome+xvfb, but I have no hard data on that and can't find any. Plus I just liked the challenge. Plugging in Chrome+xvfb+VNC just seemed too easy, and I wanted something "meatier" (apologies to vegetarians, I'm on my way there, 'Proteinier'?) I could dig my hands into :) [0]: https://news.ycombinator.com/item?id=23897626 https://news.ycombinator.com/item?id=23897626
- hjek 5y ago
  Yo dawg. I heard you like browsing the web, so I put a web browser in your web browser so you can browse the web while you browse the web.
  - graderjs 5y ago
    Exactly!! :P ;) xx I need to add that meme to the page :p :)xx
    - atatatat 5y ago
      Jesus, please don't. The memey-ness is detracting from your presentation, a bit. Knock it down 25%, unless its 100% genuine. Seems a bit forced from here.
- ktpsns 5y ago
  Are you aware of pure-JS in-browser VNC protocol implementations such as http://guacamole.apache.org/ http://guacamole.apache.org/ ? This works over websockets, so even encryption is covered. On the server side, run a Xvnc, fullscreen chrome without window manager or decorations. As the OP said, this should give superiour performance, responsiveness and features. And no need for a client side browser plugin. Another thing you could look into is https://www.brow.sh/ https://www.brow.sh/ -- they also have a pretty good performance and feature coverage. Maybe there's something your project can learn from them :-)
  - graderjs 5y ago
    Yes, that's a good idea thanks! I'm really impressed by these web VNC projects, like guacamole and novnc. One thing I think I could learn is how do they send the frames?! They seem to achieve some interesting compression. I think their framerates are higher than mine, and also responsiveness. I definitely could learn something from them. For features, they're not really superior because I probably need all that control, and browser internals instrumentation I get with devtools protocol. E.g how do I upload a file over VNC? But it's a great idea, and thanks for sharing it with me! :P ;) xx For brow.sh it's kind of a sad project. I mean because it's so popular, with > 13k stars, but the poor guy's only making USD44 a month from this work. I feel so sad to see that. It's very niche, and too niche and restricted for what I'm doing, but one thing I can learn from them is not to be them. I don't want to be doing this labor of love and putting in all that work, and then at the same time asking for donations and only getting 44 a month. :( I like the text thing but it's just not what I'm looking for with this. I mean there might be but I don't think there's anything more I can learn from it :P ;) xx

by pgo 5y ago

graderjs 5y ago

Not an extension. Just your browser, no download needed. So a "clientless" remote isolated browser. it runs headless Chrome, yes (but eventually I want to repurpose it to support other browsers that can be driven by DevTools/WebDriver protocol). Short answer is because a lot of the work of this was building a custom "head", so having a browser engine without a head made sense. Slightly more detailed answer below... :) ;p xx I might experiment with xvfb chrome and vnc in future. Other people certainly have success with this, e.g Reflect (YC S20)[0]. Maybe I got here accidentally. Original purpose was for mapping web app structure, and recording and replaying of web action sequences (automation) from any device. As most of the time the browsers were going to replay without anyone using them having xvfb seemed unnecessary. And doing the mapping and recording in one platform (real chrome + xvfb) then replaying in another I thought would risk many unnecessary, difficult bugs. Also having my own wire protocol based on DevTools, rather than use what a VNC web client uses, I believed would assist in more reproducible recordings and playback for automation. Also I liked the flexibility and control provided by headless+devtools to change the UI, easily work on mobile, custom handlers for tasks like file upload and other modals, as well as the control over the viewport streaming. On streaming, I experimented a lot to get something working across a range of bandwidths, and settled on: a per-frame ack (to avoid congestion), frames over webrtc or binary werbsocket channels, and periodically switching to whichever of webrtc of websocket is faster. I also want it to be really scalable. And I thought that headless+devtools would be much less resource heavy than chrome+xvfb, but I have no hard data on that and can't find any. Plus I just liked the challenge. Plugging in Chrome+xvfb+VNC just seemed too easy, and I wanted something "meatier" (apologies to vegetarians, I'm on my way there, 'Proteinier'?) I could dig my hands into :) [0]: https://news.ycombinator.com/item?id=23897626 https://news.ycombinator.com/item?id=23897626

hjek 5y ago
Yo dawg. I heard you like browsing the web, so I put a web browser in your web browser so you can browse the web while you browse the web.
- graderjs 5y ago
  Exactly!! :P ;) xx I need to add that meme to the page :p :)xx
  - atatatat 5y ago
    Jesus, please don't. The memey-ness is detracting from your presentation, a bit. Knock it down 25%, unless its 100% genuine. Seems a bit forced from here.
ktpsns 5y ago
Are you aware of pure-JS in-browser VNC protocol implementations such as http://guacamole.apache.org/ http://guacamole.apache.org/ ? This works over websockets, so even encryption is covered. On the server side, run a Xvnc, fullscreen chrome without window manager or decorations. As the OP said, this should give superiour performance, responsiveness and features. And no need for a client side browser plugin. Another thing you could look into is https://www.brow.sh/ https://www.brow.sh/ -- they also have a pretty good performance and feature coverage. Maybe there's something your project can learn from them :-)
- graderjs 5y ago
  Yes, that's a good idea thanks! I'm really impressed by these web VNC projects, like guacamole and novnc. One thing I think I could learn is how do they send the frames?! They seem to achieve some interesting compression. I think their framerates are higher than mine, and also responsiveness. I definitely could learn something from them. For features, they're not really superior because I probably need all that control, and browser internals instrumentation I get with devtools protocol. E.g how do I upload a file over VNC? But it's a great idea, and thanks for sharing it with me! :P ;) xx For brow.sh it's kind of a sad project. I mean because it's so popular, with > 13k stars, but the poor guy's only making USD44 a month from this work. I feel so sad to see that. It's very niche, and too niche and restricted for what I'm doing, but one thing I can learn from them is not to be them. I don't want to be doing this labor of love and putting in all that work, and then at the same time asking for donations and only getting 44 a month. :( I like the text thing but it's just not what I'm looking for with this. I mean there might be but I don't think there's anything more I can learn from it :P ;) xx