16 ms·
Marcel the Shell
- mousetree 3y agoMy name is Marcel and I’m partially a shell, as you can see from my body.
- deleted 3y ago[deleted]
- jfax 3y agoWhen I watched that in cinema I couldn't help divert my attention to a family of two adults and two children in the rows ahead of me who left a third the way through. Wonder what they were expecting.
- cryptoz 3y agoI saw Marcel's movie in theaters probably 7 times at least. Top, top movie. The YouTube shorts a great of course, but the movie is next-level.
- mgaunard 3y agoNice.
- mgaunard 3y agowhy is this modded -1?
- defrost 3y agoAt least two other HN users chose to downvote a comment that effectively had no content (beyond "yep", "nice", "upvoted", "w00t!", etc). No HN mods were involved.
- mgaunard 3y agoMost of the comments are similar in nature, but I see the point. Not seeing why my follow-up question itself was also downmodded.
- abnry 3y agoHow does this compare to nushell? I am in the market to learn something other than bash and right now nushell seems like the best candidate.
- hprotagonist 3y agoweird to say, but powershell should also be on your list.
- chaps 3y agoYep. It's a very clean, powerful and well thought out API. And I'm saying that coming from a place of love towards unix and its pipes. I'd almost argue it's better than the unix model. Almost. Been able to give poweshell commands in FOIA litigation and its power is on full display with one liners where it's hard for a gov agency to say no to something so simple.
- bee_rider 3y agoThat’s really interesting; how do you use powershell in that context? I’m assume it isn’t just Here’s FOIA request for the output of the command find /home/jbiden -name “*secret*doc” haha.
- chaps 3y agoIt was most notably helpful in litigation against the White House Office of Management and Budget. Was suing for one week of email metadata records (to, from, CC, bcc, time, date) in the last week of Jan 2017. They said that they had no way to complete the request. So we sent them multiple one liners that extracts the info from outlook 365, which pipes it to a csv export command. They told us that they didn't run that sort of command as a routine practice, but we found a separate lawsuit that found emails of the exact command we were asking them to run.
- galaxyLogic 3y ago
- besnn00 3y agoMessage to the person responsible for the site: they should use code blocks instead of images for their code. I say this because I can't read the text inside the images from mobile. It's way too small.
- deleted 3y ago[deleted]
- geophile 3y agoI'm responsible. I need to revise those, and images seemed like the way to get colorization right. I'll try to figure out how to do it with code blocks, it will hopefully be more maintainable. Cutting/pasting images is ridiculous.
- beeburrt 3y agoyou could try Carbon: https://carbon.now.sh/ https://carbon.now.sh/
- rahimnathwani 3y agoWould highlight.js do what you need?
- petepete 3y agoI suspect Pygments will be to your liking. https://pygments.org/ https://pygments.org/
- geophile 3y agoThank you for all the suggestions for code formatting/colorizing. The problem is that I'm hosting on wix. Posting images sucks, as already pointed out. How do I get formatted, console-like text on wix? I'm not seeing how these tools solve that problem.
- chaps 3y agoHate to be "that guy", but it's kinda sad that they pretty much stole the name... especially when the (fantastic!) movie with the original character came out recently in 2021. https://youtu.be/k98Afd7Nf3Y https://youtu.be/k98Afd7Nf3Y
- deleted 3y ago[deleted]
- geophile 3y agoI wrote marcel. I picked the name because of the short that preceded the movie!
- ubermonkey 3y ago[flagged]
- chaps 3y agoOut of curiosity, did you reach out to them? I think the project is neat, just feels a bit like you're leaning on an established (and extremely creative) piece of work instead of creating your own brand. Hope I'm wrong.
- geophile 3y agoActually, I did. My daughter loves marcel the shell, I was writing a shell, so it's an obvious choice. It is far enough afield that I can't believe they would care. But I reached out to them, and explained what I was going. They were amused, said they might want to object later, but they never have.
- chaps 3y agoThanks! That's a relief.
- ubermonkey 3y agoIt's cute, but I suspect the rightsholders for the character won't be thrilled about the name appropriation.
- rbanffy 3y agoThey are two very different products in completely different markets. In many jurisdictions any such lawsuit would have no standing.
- chaps 3y agoBeing legally right and being morally right are often two very different things though. Like, they even have the full domain name of the original thing. If I were the owner of the original video and the subsequent film, I know I'd be, at least, annoyed.
- beeburrt 3y agoWhat's the original thing? Sorry, I don't know the reference.
- 98codes 3y agohttps://www.google.com/search?hl=en&q=marcel%20the%20shell https://www.google.com/search?hl=en&q=marcel%20the%20shell
- rbanffy 3y agoIf I were the author of the original, I'd be thrilled. And would offer the character as the project's mascot tied to a set of conditions based on the success of the product so that, in case it fails to gain traction, it should be renamed and allow the name to be used by another product. It's more or less the same spirit as the Roddenberry estate allows someone to use the names - you are allowed to build a warp drive, as long as it warps spacetime and allows FTL travel.
- 3y ago
- rbanffy 3y agoI’m not sure this would be the way to go, but it’d be awesome if most Unix command-line tools had a way to accept and output JSON objects.
- geophile 3y agoThis is a place where marcel's python grounding is really useful. Reading json, marcel generates a stream of nested dicts and lists. That stream can be turned back to json for output. Similar for CSV.
- all2 3y agoI will note that Powershell has typed inputs and outputs and that can be very powerful. Now if it weren't so verbose...
- jen20 3y agolibxo has gone some way towards making that a reality in FreeBSD.
- mbakke 3y agoThe "tree" command will output JSON on fd3 (which it dubs 'stddata') when available. If more programs adopt that pattern we could use stdout (fd1) only for interactive programs.
- rbanffy 3y agoThis is a pretty brilliant idea. Ideally, someone from BSD-land and GNU-land should meet and come up with some sane JSON schemas.
- rahimnathwani 3y agoHowever, instead of passing strings from one command to the next, marcel passes Python values: builtin types such as lists, tuples, strings, and numbers; but also objects representing files and processes. The bit before the semicolon sounds like what you'd do in a functional language like Elixir. It would be cool to be able to do this in regular Python programs. I know some libraries (like pandas) fake it by having a method return an object of the same class, allowing method chaining, but that doesn't work universally.
- geophile 3y agoThat works. There is a Marcel API for Python that allows shell-like usage within Python programs. See this for an example: https://www.marceltheshell.org/scripting-1 https://www.marceltheshell.org/scripting-1
- geophile 3y agoHi, I’m the author of Marcel. If you like it and want to work on it, I can use volunteers, for developing features, debugging, doc, porting, improving the website, you name it.
- greenreptar 3y agoSurely there is some copyright infringement here?
- metabagel 3y agoYou’re thinking trademark infringement. IANAL, but I wouldn’t worry about it. Worse comes to worst, you could rename it. That’s what Firefox did.
- benatkin 3y agoThe likelihood of confusion is low. https://en.wikipedia.org/wiki/United_States_trademark_law#Infringement:_likelihood_of_confusion_test https://en.wikipedia.org/wiki/United_States_trademark_law#In...
- vetinari 3y ago"Marcel" is a common male name in some parts of the world. That would make a trademark infringement claim somewhat difficult.
- sebstefan 3y agoWhat's the other thing named Marcel?
- fdgjgbdfhgb 3y agoMarcel the Shell with Shoes On [1], a recent Oscar-nominated film [1] https://en.wikipedia.org/wiki/Marcel_the_Shell_with_Shoes_On_(2021_film) https://en.wikipedia.org/wiki/Marcel_the_Shell_with_Shoes_On...
- 1attice 3y agoI haven't kept up on my Python-flavoured shells. How is this different with xonsh? What similarities does it have with xonsh? Looking for a compare/contrast, not a ranking. Diversity is usually good Kudos to author, this looks really good
- geophile 3y agoThank you! Integration with Python: Xonsh defines a language that is a superset of Python and shell, as I understand it. Marcel takes a different approach, defining only a bash-like shell language. Any customization is done in Python, delimited by parens. The separation between shell and Python is much stricter. Also, Marcel provides a Python API so that you can write shell-like marcel commands inside of a Python program. Shelling out from Python is notoriously ugly; the marcel API fixes that. Sublanguages: In bash, there are lots of sublanguages, e.g. the arguments to 'date', awk, find, sed, and so on. Marcel's idea is to use Python as the sublanguage, because so many people already know it. I guess xonsh has a similar approach here. Pipes: I think that xonsh, like more familiar shells, pipes strings. Marcel pipes python values in streams. So if you run ls, you don't get a stream of filenames, you get a stream of File objects, and you can operate on them downstream. Database access: A stream of Python tuples is very similar to database query output. So database access is simple. There is an sql command which produces a stream of Python tuples. And a stream of tuples can be piped into the sql command, e.g. to populated a database. Remote access: If you have a cluster, you can use marcel to upload a file to all nodes of the cluster, download from the nodes, or to execute the same command on each, streaming results back as streams of python tuples, each with an element identifying the node from which the data originated. I don't think xonsh does this. https://marceltheshell.org https://marceltheshell.org has lots of information and examples of all this.
- eviks 3y ago> Marcel pipes python values in streams That's indeed much better, all those untyped strings in shells in a bad old design Though hopefully xonsh will implement this as well https://github.com/xonsh/xonsh/issues/3967 https://github.com/xonsh/xonsh/issues/3967
- 1attice 3y agoThis is very cool but it needs a nix package. :) I may try my hand at packaging it for Nix(OS) later this week
- geophile 3y agoIm the Marcel author. Get in touch, I’d love to do what I can to assist.
- canucker2016 3y agofor some reason I was singing the title to the Beatles' tune, "Michelle"... surprisingly works well. now i can't stop doing it
- dopidopHN 3y agoMichelle, the sheeel. Yeah. It works.
- jw_cook 3y ago> A more powerful form of scripting can be done from Python, using the `marcel.api` module. With this module, you have access to the operators of marcel, neatly integrated into Python. This looks like an appealing feature! Having some more coreutils-like functions available in python (i.e., stuff that I wish was in the `os` module) would simplify some cases where I otherwise might have to resort to `subprocess` calls. The GPL license definitely narrows the number of situations where I would realistically be able to add this as a dependency, though.
- geophile 3y agoI'm not a license connoiseur. Why does GPL present a problem? (Not trying to be conentious, it's a genuine question.)
- jw_cook 3y agoGPLv3 is a "strong copyleft" license. It comes with some significant pros and cons compared to more permissive licenses like MIT and BSD. You can find more about the philosophy behind it here: https://www.gnu.org/philosophy https://www.gnu.org/philosophy As the author, a benefit it offers you is protection against someone selling your work or a derivative of it, or redistributing it with added restrictions. From a user's perspective, the main downside is that using GPL software as a dependency makes the user's project a derivative work, which requires them to license their own project with the GPL (or from a selection of other compatible copyleft licenses). In that sense it's considered a "viral license," i.e. everything downstream of the GPL software needs to adopt GPL. This is less of a problem for an application, but is more troublesome for a library (with many dependents of its own, with dependents of their own, and so on). It also means that if you use it in commercial software, you essentially need to open-source all of it. So in your case, just using Marcel as a shell with GPL doesn't put any licensing obligations on the user, but using it as a library does. I hope that helps! Someone else feel free to chime in if I misrepresented any of that.
- qwertyzxcvmnbvw 3y agoAnyone is welcome to sell GPL'd work, also someone else's GPL'd work, as long as the license remains GPL and source code is available to the buyers of the work, under the GPL, if you distribute the software to them. If you don't (i.e. SaaS, you run the software, it is never distributed further) then you don't have to give the source to anybody! This is what the AGPL was created to address.
- dark-star 3y agoFor those who are wondering: The name is a reference to the 2021 movie "Marcel the Shell with Shoes on" https://en.wikipedia.org/wiki/Marcel_the_Shell_with_Shoes_On_(2021_film) https://en.wikipedia.org/wiki/Marcel_the_Shell_with_Shoes_On...
- deleted 3y ago[deleted]
- eli 3y agoNo it was a bunch of cute little web videos by Jenny Slate at least a decade before there was a real movie.
- irjustin 3y agoBook too! I read it to my kids all the time. Wish I could do the voice though.
- throwawaaarrgh 3y ago[flagged]
- benatkin 3y agoIt is a useful correction. This project predates the release of the movie: https://github.com/geophile/marcel/commit/bb6adacbb6b3a683cec9a907329057ae58df1759 https://github.com/geophile/marcel/commit/bb6adacbb6b3a683ce...
- throwawaaarrgh 3y agoAnd the online video series (that is described in the movie) predates the name of this project by a decade. Who gives a shit? Why do some people feel the need to correct every little thing?
- galaxyLogic 3y agoThis seems like obviously inspired by PowerShell. No?
- geophile 3y agoNo. Marcel is the successor to osh (http://geophile.com/osh http://geophile.com/osh, https://github.com/geophile/osh https://github.com/geophile/osh) which actually precedes powershell. The fact is that many many people have implemented the idea of a shell piping objects instead of strings.
- deleted 3y ago[deleted]
- sriram_malhar 3y agoI like this a LOT. Not only don't I have to get used to another language, I can lean on Python's rich ecosystem.
- larodi 3y agoFirst of all - Linuxes and alike desperately need something like this shell becoming standard. And second - I always wonder how all these great minds who where throwing rubbish at Microsoft ever since... somehow forgot to mention that same company created important product called PowerShell like 20 years ago. And you know PowerShell is so good already you can easily drop it in any OS/X, Linux and it actually works. No one, except for Jeffrey Snover figured it out on time, there is great interview here - https://evrone.com/blog/jeffrey-snover-interview https://evrone.com/blog/jeffrey-snover-interview. But for PowerShell to become Python on Perl steroids on top of all the .NET and COM+ there is, well, of course it took years, and top engineers to work on it, but also was needed an unified interface (API) to the underlying OS magic. Which is not there in Linux the same sense .NET became "available". OS/X has something similar to COM+ called CORBA which was left half-baked at best. Python is not .NET, sorry. Not anywhere near. Someone may say the UNIX ideology is about your programs doing one thing at time, and having this STDIN->munch->STDOUT approach. But these streams never got any structure, and ppl out there been creating great tools to fill this gap, tools such as - jq, mlr, xidel to name a few. perhaps we should include ack, ag and rg in the list. You know, guys, you hard-loving bash die-hards and sed+awk magicians, you should give this PowerShell thing a try before inventing the wheel again. And before legacies you wrote 20 years ago become as difficult to support as COBOL. Oh yeah, and PoSH is open source, based on open-source runtime, and available in brew/apt/yum... Disclaimer: I do not work, and have never worked for MS (Guido does now!). Have used DOS since 6.22, FreeBSD since 8th version, Debian since v5, and Windows since v3.1. I did Perl for 20 years, actually I gave Perl classes in some university, still doing fair amount of Python, and accidentally discovered PowerShell 3 years ago. I'm not a .NET advocate, although I can tell the good parts.
- riedel 3y agoI was immediately thinking powershell. I have been doing some Powershell scripting for building chocolatey packages: it weirdly did not feel like shell though and my muscle memory of POSIX commands fails me on the terminal I guess. What I was wondering though: can you easily access Powershell data types from python?
- Tabular-Iceberg 3y agoI hear you, and I have tried it, but the commands feel gratuitously different to me. Also they squandered the opportunity to make the output interactive like with the Lisp machines. That would also have helped smooth the learning curve.
- t43562 3y agoTo me the interest in this would be as a replacement for bash in makefiles with the hope that it could be cross platform and not require most of the utilities like sed/awk etc. It's an enormous pain to write x-platform builds with make because of shell issues - even with cygwin. For this to work one would probably want to embed python and have a pool of interpreters so as not to pay the startup cost of python on every job invocation. "That Said" I think passing around structures is not that useful to me - one is often trying to automate programs that are not written that way. Python's pipe handling is complicated and horrible compared to bash and that's the thing I'd really want to "buy" from this - together with more shell-like ways to copy, search, touch files that are platform independent.
- omgwtfusb 3y agofor this purpose Just[1] with a python shebang line works great [1] https://github.com/casey/just https://github.com/casey/just
- shanti2001 3y agoNice
- mpweiher 3y agoMarcel the author here. Didn't know I was also a shell, though I do also have one that is somewhat similar in concept. While taking an existing programming language like Python is an obvious choice (and one that's been made a bunch of times), I think that ultimately you need a language where something like a pipe (of objects or text) is not bolted on, but a natural consequence of a more general view of programming based on architectural interconnection. "Pipes and filters" is an architectural style. And it is itself a polymorphic style, meaning there can be substyles that pass objects and substyles that pass bytes, substyles where the filters are processes and substyles where filters are components within a process. So: good direction, more of this!
- jedisct1 3y agoVery cool. Love the fact that it uses Python!
- zelphirkalt 3y agoThis looks a lot like an idea I am having for a shell written in GNU Guile. MI am still at the very beginning and POC phase. My ideas are: (1) making it possible to have any procedure that follows a certain interface to be usable as a "command" in the shell (2) have a fallback function, that gives you a procedure for a standard shell command like 'ls' (or reimplement your own!) (3) have piped commands automatically use lightweight processes (guile-fibers) to automatically parallelize work, if possible (4) pass structured data instead of strings, unless the structured data is actually a string or a fallback command of standard shell is used (although perhaps one could write functions parsing that string output into structured data and have structured data at the next step ...) (5) provide composition concepts, that make it easy to compose commands or pipe their outputs into other commands (6) have readable arguments for things like input or output redirection (7) no need to be POSIX compatible, I just want a useful tool for myself and anyone, who might want to try it at some point. Don't want to get mired in arcane stuff. Well, many things to consider, probably tons of work hiding somewhere under the hood of all this. I also noticed there are issues with parallelizing things, if I want to keep the shell state local to a command. Previous commands in a pipeline would have to communicate shell state updates down the pipeline. So one needs to distinguish between communication of a shell state update and more input for the next command. Concurrency issues might ensue. I think I will be glad, if I can get some subset of my goals to work.
- celeritascelery 3y agoThis seems like it has some really cool ideas! This reminds me a little of eshell, which has some of the same features around allowing you to define shell functions in lisp. However it does run into issues moving between the lisp (sexp) and shell (string) boundaries.
- geophile 3y agoA lot of this is already in marcel. A command run from the command line is a pipeline, e.g. list files recursively and output (path, filesize): ls -fr | (f: (f, f.size)) But now limit the command to .py files: ls -fr | select (f: f.suffix == '.py') | (f: (f, f.size)) Also, just look at files changed in the past day: ls -fr | select (f: f.suffix == '.py') | select (f: now() - f.mtime < days(1)) | (f: (f, f.size)) Those selects can be abstracted. They can be turned into their own parameterized pipelines, and assigned to variables. Here is a pipeline to filter files by a given extension: ext = (| e: select (f: f.suffix == '.' + e) |) And for files changed in the last d days: recent = (| d: select (f: now() - f.mtime < days(int(d))) |) So the original pipeline becomes: ls -fr | ext py | recent 1 | (f: (f, f.size))
- t43562 3y agoTo do the things I wanted to do I needed reduce and I think there's no documentation for it. It took me a while to understand the code: ls | map (f: f.size) | red + # this adds all the file sizes together one can also use lamdas: ls | map (f: f.size) | red (lambda acc,y: (y if not acc else acc+y)) (this does the same thing - adds up the file sizes) BUT: how would I then divide the result by 1024? .... or assign the result to a variable? There are a number of other functions one can use with "red" like concat, * (times), max, min, count, and, or and a few more. At the moment I feel like I'm playing with a very fun toy that might give up on me at any moment and turn out to be limited. It obviously has limits but I'm really enjoying it.
- geophile 3y agoJust continue the pipeline, e.g. ls | map (f: f.size) | red + | map (size: size / 1024) You can also leave off "map", so: ls | (f: f.size) | red + | (size: size / 1024) A pipeline produces a stream, so there's no clean way to get the single int result in a variable, but you can store the stream (containing the int) into a variable: ls | ... | (size: size / 1024) >$ x Now x has the stream. To read back the stream and print it: x <$ To do something else with the stream: x <$ (n: n * 1000) etc.