10 ms·

Next browser – web browser in Common LISP

182 points by lelf 7y ago

mark_l_watson 7y ago
Good to see the recent github activity - I will give it another try when I get to a laptop. The keyboard driven navigation using search to flip between tabs and the tree structured history is interesting. Also, as always, I am happy to see people using Common Lisp for implementing creative ideas! Off topic, but I have been using Common Lisp professionally since 1983 and the first book I ever wrote was a Common Lisp book for Springer Verlag. It has been a fun ride. Thanks for this project!
- jmercouris 7y ago
  Thank you for your kind words! It's what keeps us going :)
mruts 7y ago
I used to use Conkeror, but it hasn't seen any development in a longtime. Hopefully this is or something else will be able to take it's place and eventually implement basic features that for some reason or another no current web browsers have: tiling frames, emacs keybindings, and a powerful extension language that allows for the modification of core structures (analogous to elisp for emacs).
- jmercouris 7y ago
  We have emacs bindings, and everything can be extended in CL, live. You can even connect to emacs while it is running in both directions. We wrote an interesting article about that here: https://next.atlas.engineer/article/emacs-hacks.org https://next.atlas.engineer/article/emacs-hacks.org
  - mark_l_watson 7y ago
    I didn't spend time trying this but when I played with the Next Browser about a month ago it occurred to me that one could mix in application code with a local web server component and a web interface to combine browsing and a custom application in one app. One possible app idea might be a research tool that allowed annotating material on the web - really, anything app that used web content.
  - mruts 7y ago
    That’s awesome. Does it support any ad blockers? Or would I have to go with a dns solution?
    - jmercouris 7y ago
      DNS solution for the time being, but all provisions are in place for ad-blocking support, just have to tweak this here: https://github.com/atlas-engineer/next/blob/master/source/remote.lisp#L404 https://github.com/atlas-engineer/next/blob/master/source/re...
      - mruts 7y ago
        Just tried it out! Pretty great. Reminds me a conkeror a lot. One nitpick though: I feel like you should be able to open a new buffer with C-x C-f, and maybe open in the same buffer with C-x f? C-l in emacs centers the text on where your cursor is, so using that and M-l to open links/buffers feels a little odd. Edit: Also, I don't know if you maintain the AUR package. But I had to install libfixposix manually as it wasn't listed in the dependencies and had to restart the compilation from the sbcl debugger.
        jmercouris 7y ago
        We do not maintain the AUR package. None of us use Arch. Libfixposix is a new dependency released a few days ago, we just updated the documentation. Feel free to add new keybindings in your init.lisp file :) Its just as hackable as Emacs :)
dmix 7y ago
Looks interesting. I presume it'd be easy to add a Vim key mode instead of the current Emacs-y one?
- rgrau 7y ago
  Can't wait for the next evil-mode happening
- jmercouris 7y ago
  Hi, one of the developers here, yes, we spent a lot of time to make sure that writing a VI style keybinding system (and properly) without all of the EVIL compromises would be possible
userbinator 7y ago
Note that this appears to be yet another Webkit shell, and not an implementation of parsing/rendering pages in Lisp.
- Koshkin 7y ago
  Which is good, otherwise web pages might look like lisp.
  - _emacsomancer_ 7y ago
    > Which is good, otherwise web pages might look like lisp. (what-is-p wrong (with 'that)) More seriously, there's also the security angle with browser, that using an independently developed engine has certain advantages.
- mepian 7y ago
  The latter is a hell of of a task, given the complexity of the current web standards. I'm not sure if it's possible to do this decently without some kind of serious financial backing.
  - userbinator 7y ago
    Targeting a subset, i.e. the "document-centric web" (no JS, only basic HTML and CSS), is not that difficult in comparison and has been done by a few others; the most prominent of these "second tier" browsers being https://en.wikipedia.org/wiki/Dillo https://en.wikipedia.org/wiki/Dillo and https://en.wikipedia.org/wiki/NetSurf https://en.wikipedia.org/wiki/NetSurf . Of course, writing a text-based browser in Lisp or any other language is far simpler but also less practical.
    - jmercouris 7y ago
      This is certainly possible with our current architecture. Maybe we will do it at some point down the road.
    - RodgerTheGreat 7y ago
tgibbster 7y ago
At least in regard to easy navigation, I don’t see much that trumps Chrome’s Vimium extension.
- minieggs 7y ago
  Chromium could treat extensions better. “First class” keybindings provide a much better experience (think of opening a new tab that contains solely an image — you can’t close this tab with Vimium key bindings while the image is loading). Edit: one example but I’ve found Vimium — while great — falls short due to “the platform.”
eadmund 7y ago
You know what'd be extra-awesome? If it had a text editor and an Emacs Lisp emulation layer … Seriously, a web browser which is capable of running the decades of elisp out there would be basically the only tool I need on my desktop: text editor, web browser, mail reader, git client, console, shell, calculator, diary, organiser. Boot directly into that and life would be perfect.
- ssivark 7y ago
  Next is the next best thing. Instead of making your browser the environment, think of it as a fantastic browser extension for Emacs. You can use the SLIME Emacs extension to connect to a running instance of the Next browser. Then, you have complete programmatic access to the full browser data structure through a REPL in Emacs (eg: switching tabs by using shortcut/command in Emacs). And Lisp REPLs are awesomer than something like ipython! Keyboard driven browsing is now the least imaginative thing possible. Take the high-quality tools you have for processing text, and apply that to your browsing workflow! Imagine being able to integrate your browsing workflow with your information management system, eg: org-mode. Switch to a project, and with a couple of keystrokes, open the relevant links on a page in a set of browser tabs/buffers. Filter all the HN pages in your browser and bookmark them to a text file (to be resumed later). Save all arXiv pages to your reference management system (eg: bibtex), etc. When browsing webpages, evaluate code blocks (via Emacs). Download/cache videos for all all Youtube tabs you have open. Emacs becomes the IFTTT-like glue for all your browsing activity! For some chatter/ideas along these lines, take a look at this Github discussion: https://github.com/atlas-engineer/next/issues/195 https://github.com/atlas-engineer/next/issues/195
  - mark_l_watson 7y ago
    +1 nice ideas! The last time I played with the Next Browser, I loaded it from source and in principle all your ideas are doable. Please post a link if you implement something and writer it up.
    - ssivark 7y ago
      Definitely! I became excited by Next when I stumbled on it about a month ago. I have several ideas/itches, but a high activation barrier to implementing and experimenting as I'm a total lisp & emacs newbie. I'm also given to understand that the Next codebase is currently undergoing significant refactoring, so I hope to delve into this more once the internals have stabilized. If somebody has a complementary outlook/background, I'm happy to be baited into discussion and activity. As they say, two people together may be thrice as effective.
eloisant 7y ago
We'll see, but from my experience when the tagline is "<product> in <language>", we no distinctive feature for end-users, the project never goes very far.
- jmercouris 7y ago
  Yes, only a mere two years of continuous development with a lot of features. Definitely not far at all :)
  - harperlee 7y ago
    I don’t like the tone of your parent post, but he does have a point: during communication/marketing, highlight and focus on unique features, not on a unique programming process, or only those interested with browser programming / architecture will be interested, not the majority of end users. Nonetheless, congratulations on your project! It is already a big accomplishment!
    - klez 7y ago
      I see what you mean, but on certain projects the language used can be a differentiator (not in marketing, but at least in concept). See for example Redox. The whole point is to build an operating system in Rust because of the language safety, which allegedly trickles down to more safety in the user space. Or PyPy, whose whole point is that it's written in Python.
      - harperlee 7y ago
        Yes. Because of the language safety. This is a feature, and everyone that has written “malloc” and read about rust understands where it comes from. “Lisp --> better browser” is not so immediate. Re. Pypy, the advantage of “the language you like written in the language you like” is also pretty immediate. My point being, dont leave the target of the message do the work of researching and understanding. Just selling the language is normally not enough.
inflatableDodo 7y ago
>Security warning for macOS users: The Cocoa webkit platform port has been dropped in 1.2.2 and it's currently not possible to build it without exposing Next to a remote-execution vulnerability. We recommend macOS users to wait for the next release which will feature a new platform port. Anyone out there with a good idea of how serious a risk this is? Edit - I mean, it sounds very serious, so I don't really want to run it on OSX. That said, I woudn't mind knowing if I am being overly careful here.
- jmercouris 7y ago
  Basically the issue is, someone can craft a URL that when you visit it it will allow the execution of arbitrary Lisp code on your machine- a huge problem. The likelihood of someone setting up such a website/link is low, but full disclosure is the policy we've decided to adopt.
  - inflatableDodo 7y ago
    Ahh, that is a problem. Though I sort of like the idea that you are accidentally making macs into remotely operated lisp machines.
  - masukomi 7y ago
    When the malicious hackers take up lisp... THEN we'll really have something serious to worry about. ;)
inops 7y ago
I wonder if there's any effort to make this work on Windows? I tried Next on my Linux box and loved it. Best thing emacs-y browser since Conkeror stopped development. I would love to switch to it, but I'm forced to use Windows most of the time.
- jmercouris 7y ago
  There is not a current effort targeting Windows, but the development of the PyQt port will add Windows support, so keep watching :D
samirsd 7y ago
meta key isn't working for me on my 2017 macbook pro... am i doing something wrong?
- jmercouris 7y ago
  Probably not, the latest Mac version is a bit flaky, new release coming soon ~1-2 weeks
igreulich 7y ago
I am genuinly curious; I promise I am not just trolling here... Why is it important to call out the fact that is in [lang]?
- jmercouris 7y ago
  There is something quite special about Common LISP. I've written about it a few times, you can find a short article here: https://atlas.engineer/technical-article/why-lisp.org https://atlas.engineer/technical-article/why-lisp.org it also contains some nice links at the bottom to further articles. I'd be curious to know your opinion, thanks!

182 points by lelf 7y ago

Good to see the recent github activity - I will give it another try when I get to a laptop. The keyboard driven navigation using search to flip between tabs and the tree structured history is interesting. Also, as always, I am happy to see people using Common Lisp for implementing creative ideas! Off topic, but I have been using Common Lisp professionally since 1983 and the first book I ever wrote was a Common Lisp book for Springer Verlag. It has been a fun ride. Thanks for this project!

jmercouris 7y ago
Thank you for your kind words! It's what keeps us going :)

mruts 7y ago

I used to use Conkeror, but it hasn't seen any development in a longtime. Hopefully this is or something else will be able to take it's place and eventually implement basic features that for some reason or another no current web browsers have: tiling frames, emacs keybindings, and a powerful extension language that allows for the modification of core structures (analogous to elisp for emacs).

jmercouris 7y ago
We have emacs bindings, and everything can be extended in CL, live. You can even connect to emacs while it is running in both directions. We wrote an interesting article about that here: https://next.atlas.engineer/article/emacs-hacks.org https://next.atlas.engineer/article/emacs-hacks.org
- mark_l_watson 7y ago
  I didn't spend time trying this but when I played with the Next Browser about a month ago it occurred to me that one could mix in application code with a local web server component and a web interface to combine browsing and a custom application in one app. One possible app idea might be a research tool that allowed annotating material on the web - really, anything app that used web content.
- mruts 7y ago
  That’s awesome. Does it support any ad blockers? Or would I have to go with a dns solution?
  - jmercouris 7y ago
    DNS solution for the time being, but all provisions are in place for ad-blocking support, just have to tweak this here: https://github.com/atlas-engineer/next/blob/master/source/remote.lisp#L404 https://github.com/atlas-engineer/next/blob/master/source/re...
    - mruts 7y ago
      Just tried it out! Pretty great. Reminds me a conkeror a lot. One nitpick though: I feel like you should be able to open a new buffer with C-x C-f, and maybe open in the same buffer with C-x f? C-l in emacs centers the text on where your cursor is, so using that and M-l to open links/buffers feels a little odd. Edit: Also, I don't know if you maintain the AUR package. But I had to install libfixposix manually as it wasn't listed in the dependencies and had to restart the compilation from the sbcl debugger.
      - jmercouris 7y ago
        We do not maintain the AUR package. None of us use Arch. Libfixposix is a new dependency released a few days ago, we just updated the documentation. Feel free to add new keybindings in your init.lisp file :) Its just as hackable as Emacs :)

dmix 7y ago

Looks interesting. I presume it'd be easy to add a Vim key mode instead of the current Emacs-y one?

rgrau 7y ago
Can't wait for the next evil-mode happening
jmercouris 7y ago
Hi, one of the developers here, yes, we spent a lot of time to make sure that writing a VI style keybinding system (and properly) without all of the EVIL compromises would be possible

userbinator 7y ago

Note that this appears to be yet another Webkit shell, and not an implementation of parsing/rendering pages in Lisp.

Koshkin 7y ago
Which is good, otherwise web pages might look like lisp.
- _emacsomancer_ 7y ago
  > Which is good, otherwise web pages might look like lisp. (what-is-p wrong (with 'that)) More seriously, there's also the security angle with browser, that using an independently developed engine has certain advantages.
mepian 7y ago
The latter is a hell of of a task, given the complexity of the current web standards. I'm not sure if it's possible to do this decently without some kind of serious financial backing.
- userbinator 7y ago
  Targeting a subset, i.e. the "document-centric web" (no JS, only basic HTML and CSS), is not that difficult in comparison and has been done by a few others; the most prominent of these "second tier" browsers being https://en.wikipedia.org/wiki/Dillo https://en.wikipedia.org/wiki/Dillo and https://en.wikipedia.org/wiki/NetSurf https://en.wikipedia.org/wiki/NetSurf . Of course, writing a text-based browser in Lisp or any other language is far simpler but also less practical.
  - jmercouris 7y ago
    This is certainly possible with our current architecture. Maybe we will do it at some point down the road.
  - RodgerTheGreat 7y ago

tgibbster 7y ago

At least in regard to easy navigation, I don’t see much that trumps Chrome’s Vimium extension.

minieggs 7y ago
Chromium could treat extensions better. “First class” keybindings provide a much better experience (think of opening a new tab that contains solely an image — you can’t close this tab with Vimium key bindings while the image is loading). Edit: one example but I’ve found Vimium — while great — falls short due to “the platform.”

eadmund 7y ago

You know what'd be extra-awesome? If it had a text editor and an Emacs Lisp emulation layer … Seriously, a web browser which is capable of running the decades of elisp out there would be basically the only tool I need on my desktop: text editor, web browser, mail reader, git client, console, shell, calculator, diary, organiser. Boot directly into that and life would be perfect.

ssivark 7y ago
Next is the next best thing. Instead of making your browser the environment, think of it as a fantastic browser extension for Emacs. You can use the SLIME Emacs extension to connect to a running instance of the Next browser. Then, you have complete programmatic access to the full browser data structure through a REPL in Emacs (eg: switching tabs by using shortcut/command in Emacs). And Lisp REPLs are awesomer than something like ipython! Keyboard driven browsing is now the least imaginative thing possible. Take the high-quality tools you have for processing text, and apply that to your browsing workflow! Imagine being able to integrate your browsing workflow with your information management system, eg: org-mode. Switch to a project, and with a couple of keystrokes, open the relevant links on a page in a set of browser tabs/buffers. Filter all the HN pages in your browser and bookmark them to a text file (to be resumed later). Save all arXiv pages to your reference management system (eg: bibtex), etc. When browsing webpages, evaluate code blocks (via Emacs). Download/cache videos for all all Youtube tabs you have open. Emacs becomes the IFTTT-like glue for all your browsing activity! For some chatter/ideas along these lines, take a look at this Github discussion: https://github.com/atlas-engineer/next/issues/195 https://github.com/atlas-engineer/next/issues/195
- mark_l_watson 7y ago
  +1 nice ideas! The last time I played with the Next Browser, I loaded it from source and in principle all your ideas are doable. Please post a link if you implement something and writer it up.
  - ssivark 7y ago
    Definitely! I became excited by Next when I stumbled on it about a month ago. I have several ideas/itches, but a high activation barrier to implementing and experimenting as I'm a total lisp & emacs newbie. I'm also given to understand that the Next codebase is currently undergoing significant refactoring, so I hope to delve into this more once the internals have stabilized. If somebody has a complementary outlook/background, I'm happy to be baited into discussion and activity. As they say, two people together may be thrice as effective.

eloisant 7y ago

We'll see, but from my experience when the tagline is "<product> in <language>", we no distinctive feature for end-users, the project never goes very far.

jmercouris 7y ago
Yes, only a mere two years of continuous development with a lot of features. Definitely not far at all :)
- harperlee 7y ago
  I don’t like the tone of your parent post, but he does have a point: during communication/marketing, highlight and focus on unique features, not on a unique programming process, or only those interested with browser programming / architecture will be interested, not the majority of end users. Nonetheless, congratulations on your project! It is already a big accomplishment!
  - klez 7y ago
    I see what you mean, but on certain projects the language used can be a differentiator (not in marketing, but at least in concept). See for example Redox. The whole point is to build an operating system in Rust because of the language safety, which allegedly trickles down to more safety in the user space. Or PyPy, whose whole point is that it's written in Python.
    - harperlee 7y ago
      Yes. Because of the language safety. This is a feature, and everyone that has written “malloc” and read about rust understands where it comes from. “Lisp --> better browser” is not so immediate. Re. Pypy, the advantage of “the language you like written in the language you like” is also pretty immediate. My point being, dont leave the target of the message do the work of researching and understanding. Just selling the language is normally not enough.

inflatableDodo 7y ago

>Security warning for macOS users: The Cocoa webkit platform port has been dropped in 1.2.2 and it's currently not possible to build it without exposing Next to a remote-execution vulnerability. We recommend macOS users to wait for the next release which will feature a new platform port. Anyone out there with a good idea of how serious a risk this is? Edit - I mean, it sounds very serious, so I don't really want to run it on OSX. That said, I woudn't mind knowing if I am being overly careful here.

jmercouris 7y ago
Basically the issue is, someone can craft a URL that when you visit it it will allow the execution of arbitrary Lisp code on your machine- a huge problem. The likelihood of someone setting up such a website/link is low, but full disclosure is the policy we've decided to adopt.
- inflatableDodo 7y ago
  Ahh, that is a problem. Though I sort of like the idea that you are accidentally making macs into remotely operated lisp machines.
- masukomi 7y ago
  When the malicious hackers take up lisp... THEN we'll really have something serious to worry about. ;)

inops 7y ago

I wonder if there's any effort to make this work on Windows? I tried Next on my Linux box and loved it. Best thing emacs-y browser since Conkeror stopped development. I would love to switch to it, but I'm forced to use Windows most of the time.

jmercouris 7y ago
There is not a current effort targeting Windows, but the development of the PyQt port will add Windows support, so keep watching :D

samirsd 7y ago

meta key isn't working for me on my 2017 macbook pro... am i doing something wrong?

jmercouris 7y ago
Probably not, the latest Mac version is a bit flaky, new release coming soon ~1-2 weeks

igreulich 7y ago

I am genuinly curious; I promise I am not just trolling here... Why is it important to call out the fact that is in [lang]?

jmercouris 7y ago
There is something quite special about Common LISP. I've written about it a few times, you can find a short article here: https://atlas.engineer/technical-article/why-lisp.org https://atlas.engineer/technical-article/why-lisp.org it also contains some nice links at the bottom to further articles. I'd be curious to know your opinion, thanks!