10 ms·

scrapscript.py

316 points by surprisetalk 3y ago

muzamil2002 3y ago
[dead]
rahimnathwani 3y ago
This is about how scrapscript is implemented. There was a popular Show HN about 9 months ago, about scrapscript itself: https://news.ycombinator.com/item?id=35712163 https://news.ycombinator.com/item?id=35712163
aardvarkr 3y ago
Interesting tidbit: the book series that this website is named for is actually spelled berEnstAin bears, emphasis on the letters that everyone (including myself) remembers being spelled the other way. I literally learned this yesterday
- NoZebra120vClip 3y ago
  > everyone The books were a very minor part of my childhood, but I noticed immediately, and my family always pronounced it correctly.
- juliusgeo 3y ago
  I think the website name is a pun based on the last name of its author.
  - tekknolagi 3y ago
    Correct!
- LeonB 3y ago
  I concur! This is considered part of her Mandela effect^1 isn’t it? The Wikipedia article on the aforementioned bears even has a section on it — https://en.wikipedia.org/wiki/Berenstain_Bears#Name_confusion https://en.wikipedia.org/wiki/Berenstain_Bears#Name_confusio... ^1 - which was the Mandala effect, in my original universe, I’m sure.
  - lkirkwood 3y ago
    Why would it be the Mandala effect? It's named for the mass false memory of Nelson Mandela dying in prison in the 80s.
    - amenhotep 3y ago
      One high profile archetype of the mandala is the sand mandala, where practitioners painstakingly construct an intricate mandala out of sand over the course of days and then ritually sweep it away once it's complete, leaving no trace, as a meditation on impermanence or something like that. Much like how in the Mandela effect the original universe is wiped at least partially away, leaving no trace of what was a complex and fully featured aspect of the timeline, other than what remains in your memory. Other people say "no, that's always been a table" while you remember the sand that was on top of it. Or something along those lines! For some people the resonance is strong enough for the mandala imagery to potentially overwrite the Mandela etymology. Especially if you're a person who's never experienced the effect about Mandela himself.
hitekker 3y ago
I enjoyed the link to the language checklist https://www.mcmillen.dev/language_checklist.html https://www.mcmillen.dev/language_checklist.html
- CodeCompost 3y ago
  Same here. It's the first time I've ever seen it.
- deleted 3y ago
  [deleted]
- popcorncowboy 3y ago
  > Programming in this language is an adequate punishment for inventing it I was already laughing hard by this final punchline. Bravo.
deepnet 3y ago
Elegant and pure. I also like the Javascript lambda calculus this is a fork of. Like early Haskell when it was just for fun before Haskell's Meta-monadic library sprawl that upped the learning curve
throwaway290 3y ago
[flagged]
- tekknolagi 3y ago
  The website does lean a little more marketing-heavy than my blog but I think that is just how Taylor writes and presents on the internet. A lot of the fancy-sounding claims are just very boring details. The dependency fixing thing is mostly a claim based on something like static linking, where you can do the moral equivalent of require(x) and have x downloaded and checksum'd and loaded into memory---and then you keep using that version. Like a Merkle DAG. I don't know that this approach fixes all problems, but the solution is a lot less interesting than it sounds in that description IMO. "scrap" is just an expression but you can reference any expression by its hash since it's all serializable. That's also what he means by the next bullet. "Optimized for AI" is, IMO, a bad misrepresentation because what he means (I think) is since you put the use before the definition (f x . f = ...), it gives more room for sufficiently advanced autocomplete to fill in the definition. If you can picture a language where every expression is a Git object and you can pull in objects from other repos at will and you can take arbitrary snapshots of your program (a tree-equivalent), you kind of get an idea of what's going on.
- hombre_fatal 3y ago
  You're going to resist this, but your cynicism here is a lot closer to “ugh why didn’t he create this with me in mind?” than you think. Just consider for a second that you’re commenting on a blog post about two people who were compelled by the project to contribute some good work to it. Yet your reaction is to come here and tell us about how another website doesn't live up to your standards or something, and I bet you thought the quote by quote takedown was quite scathing. ;)
  - throwaway290 3y ago
    > ugh why didn’t he create this with me in mind? Who is "he" and created what? I only said that the poster ("creator"?) of this blog article should explain what from his view the thing is, because the website for the thing (whoever made that seems to be another person) that he links to is superbly malinformative, which I illustrated with vivid examples. Whoever implemented this library did some maybe good work but he does it a disservice by offloading the explanation to poorly written marketing copy
cdchn 3y ago
Making python scripts into an 'Actually Portable Executable' is what really interested me here.
- fermigier 3y ago
  Is this the reason why there is only one 5+ KLOC module (which includes the tests)? I personally prefer short / shorter modules with clear responsibilities.
  - tekknolagi 3y ago
    No. This is not a limitation of APE/Cosmopolitan. This is just my personal preference because imports get tricky in Python land unless you either have a single file or go Full Package Mode. There's probably a world where we split the tests out, though.
    - fermigier 3y ago
      Well, just to better understand how the code is organized (and experiment a bit with it), I have forked it: https://github.com/sfermigier/scrapscript/tree/trunk/src/scrapscript https://github.com/sfermigier/scrapscript/tree/trunk/src/scr...
- semi-extrinsic 3y ago
  Not just python scripts, you can package any C code as well. We've used it to compile up a "python.com" APE file with a python 3.11 interpreter that has lots of packages (including C extensions) that we can just drop straight into old airgapped lab instrument computers and get a modern python data analysis suite up and running.
  - actionfromafar 3y ago
    Now integrate Shedskin the Python compiler. :)
    - 3y ago
Karupan 3y ago
I have to admit this broke my brain. This is the first time I'm hearing about content addressable languages, and once you get over that barrier, a distributed language doesn't seem far fetched. As a big fan of functional programming, is this something that is going to just end up being an esoteric language? Don't get me wrong, I absolutely love the vision of the authors, but after being bitten by the Elm bug and that crashing and burning, I'm just cautious of getting invested in new languages and tools.
- adius 3y ago
  Nothing crashed and burned. Elm is in a state where it’s fully usable and has all the futures you need. I use it every day! Since it’s a DSL to create HTML + JS + CSS websites, you still get all the new features of browsers!
  - Karupan 3y ago
    Except for the compiler bugs, lack of self hosted package management, improvements to tooling, etc. I use it regularly too, but it is frustrating to see it in a state of decay. They can definitely claim the above points and more are not goals, and Evan absolutely has every right to do so. But don’t be surprised when devs see it as dead.
  - mst 3y ago
    I think, even though I can absolutely see the arguments for doing so, locking down the compiler to no longer accept external native extensions was a huge mistake community wise, since a lot of the people advocating for Elm were the sort of early adopter who really really wants an escape hatch because they're in the habit of getting into situations where they need one no matter what tools they're using. Certainly that describes me, and when all the people who seemed to be like me got told to go sit on a cactus by the Elm core developers and bailed out to work with something else, my experiments got pretty much immediately shelved and Elm moved into the "interesting place to steal ideas from, actively hostile to my actually using it" category. This may be unfair, but I'm pretty sure it's a reasonable description of what -did- happen, fair or not.
- 3y ago
Hackbraten 3y ago
[flagged]
- tekknolagi 3y ago
  Yeah. We decided pretty early on to use .scrap
  - Hackbraten 3y ago
    Thank you!
- dtx1 3y ago
  German here. That is the stupidest thing I have heard in a while. Just because a language might be abbreviated with SS will not stop any reasonable person from using it.
- deleted 3y ago
  [deleted]
fermigier 3y ago
This reminds me of a talk Tim Berners-Lee did in 2002 (at the 10th Python conference): https://www.w3.org/2002/Talks/0206-python/ https://www.w3.org/2002/Talks/0206-python/ ("Webizing Python") I wasn't there but I remember hearing that this wasn't well received by the participants. Also, TBL references a post by Aaron Swartz at the end of his slides: https://web.archive.org/web/20050208021219/logicerror.com/webizingPython https://web.archive.org/web/20050208021219/logicerror.com/we... (also titled "Webizing Python")
account-5 3y ago
Can someone explain what this is? Why it's a good thing? What's it's for? I have to admit based on reading the post I have absolutely no idea.
- KTibow 3y ago
  From reading https://scrapscript.org/ https://scrapscript.org/ it sounds like its main feature is that things can be split up, put on platforms like IPFS, and distributed allowing you to access them from wherever.
- surprisetalk 3y ago
  I should probably write a longer post about this, but scrapscript is an attempt to fix a lot of the "in-between" problems in software engineering. Instead of working on "real" problems, I find myself battling untyped/undocumented YAML/JSON configurations, syncing JSON encoders/decoders, massaging incompatible dependencies, writing unholy SQL, etc. I obviously don't have all the answers, but a system with the following properties seems like a worthwhile pursuit: (1) small enough to be used like JSON yet powerful enough to used like Javascript, (2) cryptographic guarantees that code is compatible over time, (3) a compiler that checks live servers for compatibility before deploying, (4) simple but expressive type system, (5) a package manager that facilitates all of this at a granular level... and so on. On top of all that, I think these properties lend themselves to some grand ambitions like "a new internet" and a "google-docs live coding editor experience". Maybe I'm just full of myself though haha scrapscript.py is the first real attempt at making scrapscript a reality, so some folks who feel these pains are getting excited to see some movement on the project. EDIT: Here’s my recent scrapyard demo, if you want to see it in action: https://www.youtube.com/watch?v=SngOLU5G1Eg https://www.youtube.com/watch?v=SngOLU5G1Eg
  - account-5 3y ago
    Still not sure I fully understand, but that is more than likely down to my ignorance. I really appreciate your effort in explaining here. I should mention I'm not a full time developer and certainly not a webdev so this might be why I'm not grokking this. Thanks.
ingenieroariel 3y ago
Since this uses cosmopolitan and the build script already downloads portable binaries from https://cosmo.zip https://cosmo.zip has there any thought been given to wrap other portable binaries in scrapscript / download them? Small, pure, functional, content-addressable and network-first sounds a lot like a mini Nix+ca-derivations [1] [1] https://www.tweag.io/blog/2021-12-02-nix-cas-4/ https://www.tweag.io/blog/2021-12-02-nix-cas-4/
0xf20x00 3y ago
[dead]

316 points by surprisetalk 3y ago

muzamil2002 3y ago

[dead]

This is about how scrapscript is implemented. There was a popular Show HN about 9 months ago, about scrapscript itself: https://news.ycombinator.com/item?id=35712163 https://news.ycombinator.com/item?id=35712163

aardvarkr 3y ago

Interesting tidbit: the book series that this website is named for is actually spelled berEnstAin bears, emphasis on the letters that everyone (including myself) remembers being spelled the other way. I literally learned this yesterday

NoZebra120vClip 3y ago
> everyone The books were a very minor part of my childhood, but I noticed immediately, and my family always pronounced it correctly.
juliusgeo 3y ago
I think the website name is a pun based on the last name of its author.
- tekknolagi 3y ago
  Correct!
LeonB 3y ago
I concur! This is considered part of her Mandela effect^1 isn’t it? The Wikipedia article on the aforementioned bears even has a section on it — https://en.wikipedia.org/wiki/Berenstain_Bears#Name_confusion https://en.wikipedia.org/wiki/Berenstain_Bears#Name_confusio... ^1 - which was the Mandala effect, in my original universe, I’m sure.
- lkirkwood 3y ago
  Why would it be the Mandala effect? It's named for the mass false memory of Nelson Mandela dying in prison in the 80s.
  - amenhotep 3y ago
    One high profile archetype of the mandala is the sand mandala, where practitioners painstakingly construct an intricate mandala out of sand over the course of days and then ritually sweep it away once it's complete, leaving no trace, as a meditation on impermanence or something like that. Much like how in the Mandela effect the original universe is wiped at least partially away, leaving no trace of what was a complex and fully featured aspect of the timeline, other than what remains in your memory. Other people say "no, that's always been a table" while you remember the sand that was on top of it. Or something along those lines! For some people the resonance is strong enough for the mandala imagery to potentially overwrite the Mandela etymology. Especially if you're a person who's never experienced the effect about Mandela himself.

hitekker 3y ago

I enjoyed the link to the language checklist https://www.mcmillen.dev/language_checklist.html https://www.mcmillen.dev/language_checklist.html

CodeCompost 3y ago
Same here. It's the first time I've ever seen it.
deleted 3y ago
[deleted]
popcorncowboy 3y ago
> Programming in this language is an adequate punishment for inventing it I was already laughing hard by this final punchline. Bravo.

deepnet 3y ago

Elegant and pure. I also like the Javascript lambda calculus this is a fork of. Like early Haskell when it was just for fun before Haskell's Meta-monadic library sprawl that upped the learning curve

throwaway290 3y ago

[flagged]

tekknolagi 3y ago
The website does lean a little more marketing-heavy than my blog but I think that is just how Taylor writes and presents on the internet. A lot of the fancy-sounding claims are just very boring details. The dependency fixing thing is mostly a claim based on something like static linking, where you can do the moral equivalent of require(x) and have x downloaded and checksum'd and loaded into memory---and then you keep using that version. Like a Merkle DAG. I don't know that this approach fixes all problems, but the solution is a lot less interesting than it sounds in that description IMO. "scrap" is just an expression but you can reference any expression by its hash since it's all serializable. That's also what he means by the next bullet. "Optimized for AI" is, IMO, a bad misrepresentation because what he means (I think) is since you put the use before the definition (f x . f = ...), it gives more room for sufficiently advanced autocomplete to fill in the definition. If you can picture a language where every expression is a Git object and you can pull in objects from other repos at will and you can take arbitrary snapshots of your program (a tree-equivalent), you kind of get an idea of what's going on.
hombre_fatal 3y ago
You're going to resist this, but your cynicism here is a lot closer to “ugh why didn’t he create this with me in mind?” than you think. Just consider for a second that you’re commenting on a blog post about two people who were compelled by the project to contribute some good work to it. Yet your reaction is to come here and tell us about how another website doesn't live up to your standards or something, and I bet you thought the quote by quote takedown was quite scathing. ;)
- throwaway290 3y ago
  > ugh why didn’t he create this with me in mind? Who is "he" and created what? I only said that the poster ("creator"?) of this blog article should explain what from his view the thing is, because the website for the thing (whoever made that seems to be another person) that he links to is superbly malinformative, which I illustrated with vivid examples. Whoever implemented this library did some maybe good work but he does it a disservice by offloading the explanation to poorly written marketing copy

cdchn 3y ago

Making python scripts into an 'Actually Portable Executable' is what really interested me here.

fermigier 3y ago
Is this the reason why there is only one 5+ KLOC module (which includes the tests)? I personally prefer short / shorter modules with clear responsibilities.
- tekknolagi 3y ago
  No. This is not a limitation of APE/Cosmopolitan. This is just my personal preference because imports get tricky in Python land unless you either have a single file or go Full Package Mode. There's probably a world where we split the tests out, though.
  - fermigier 3y ago
    Well, just to better understand how the code is organized (and experiment a bit with it), I have forked it: https://github.com/sfermigier/scrapscript/tree/trunk/src/scrapscript https://github.com/sfermigier/scrapscript/tree/trunk/src/scr...
semi-extrinsic 3y ago
Not just python scripts, you can package any C code as well. We've used it to compile up a "python.com" APE file with a python 3.11 interpreter that has lots of packages (including C extensions) that we can just drop straight into old airgapped lab instrument computers and get a modern python data analysis suite up and running.
- actionfromafar 3y ago
  Now integrate Shedskin the Python compiler. :)
  - 3y ago

Karupan 3y ago

I have to admit this broke my brain. This is the first time I'm hearing about content addressable languages, and once you get over that barrier, a distributed language doesn't seem far fetched. As a big fan of functional programming, is this something that is going to just end up being an esoteric language? Don't get me wrong, I absolutely love the vision of the authors, but after being bitten by the Elm bug and that crashing and burning, I'm just cautious of getting invested in new languages and tools.

adius 3y ago
Nothing crashed and burned. Elm is in a state where it’s fully usable and has all the futures you need. I use it every day! Since it’s a DSL to create HTML + JS + CSS websites, you still get all the new features of browsers!
- Karupan 3y ago
  Except for the compiler bugs, lack of self hosted package management, improvements to tooling, etc. I use it regularly too, but it is frustrating to see it in a state of decay. They can definitely claim the above points and more are not goals, and Evan absolutely has every right to do so. But don’t be surprised when devs see it as dead.
- mst 3y ago
  I think, even though I can absolutely see the arguments for doing so, locking down the compiler to no longer accept external native extensions was a huge mistake community wise, since a lot of the people advocating for Elm were the sort of early adopter who really really wants an escape hatch because they're in the habit of getting into situations where they need one no matter what tools they're using. Certainly that describes me, and when all the people who seemed to be like me got told to go sit on a cactus by the Elm core developers and bailed out to work with something else, my experiments got pretty much immediately shelved and Elm moved into the "interesting place to steal ideas from, actively hostile to my actually using it" category. This may be unfair, but I'm pretty sure it's a reasonable description of what -did- happen, fair or not.
3y ago

Hackbraten 3y ago

[flagged]

tekknolagi 3y ago
Yeah. We decided pretty early on to use .scrap
- Hackbraten 3y ago
  Thank you!
dtx1 3y ago
German here. That is the stupidest thing I have heard in a while. Just because a language might be abbreviated with SS will not stop any reasonable person from using it.
deleted 3y ago
[deleted]

fermigier 3y ago

This reminds me of a talk Tim Berners-Lee did in 2002 (at the 10th Python conference): https://www.w3.org/2002/Talks/0206-python/ https://www.w3.org/2002/Talks/0206-python/ ("Webizing Python") I wasn't there but I remember hearing that this wasn't well received by the participants. Also, TBL references a post by Aaron Swartz at the end of his slides: https://web.archive.org/web/20050208021219/logicerror.com/webizingPython https://web.archive.org/web/20050208021219/logicerror.com/we... (also titled "Webizing Python")

account-5 3y ago

Can someone explain what this is? Why it's a good thing? What's it's for? I have to admit based on reading the post I have absolutely no idea.

KTibow 3y ago
From reading https://scrapscript.org/ https://scrapscript.org/ it sounds like its main feature is that things can be split up, put on platforms like IPFS, and distributed allowing you to access them from wherever.
surprisetalk 3y ago
I should probably write a longer post about this, but scrapscript is an attempt to fix a lot of the "in-between" problems in software engineering. Instead of working on "real" problems, I find myself battling untyped/undocumented YAML/JSON configurations, syncing JSON encoders/decoders, massaging incompatible dependencies, writing unholy SQL, etc. I obviously don't have all the answers, but a system with the following properties seems like a worthwhile pursuit: (1) small enough to be used like JSON yet powerful enough to used like Javascript, (2) cryptographic guarantees that code is compatible over time, (3) a compiler that checks live servers for compatibility before deploying, (4) simple but expressive type system, (5) a package manager that facilitates all of this at a granular level... and so on. On top of all that, I think these properties lend themselves to some grand ambitions like "a new internet" and a "google-docs live coding editor experience". Maybe I'm just full of myself though haha scrapscript.py is the first real attempt at making scrapscript a reality, so some folks who feel these pains are getting excited to see some movement on the project. EDIT: Here’s my recent scrapyard demo, if you want to see it in action: https://www.youtube.com/watch?v=SngOLU5G1Eg https://www.youtube.com/watch?v=SngOLU5G1Eg
- account-5 3y ago
  Still not sure I fully understand, but that is more than likely down to my ignorance. I really appreciate your effort in explaining here. I should mention I'm not a full time developer and certainly not a webdev so this might be why I'm not grokking this. Thanks.

ingenieroariel 3y ago

Since this uses cosmopolitan and the build script already downloads portable binaries from https://cosmo.zip https://cosmo.zip has there any thought been given to wrap other portable binaries in scrapscript / download them? Small, pure, functional, content-addressable and network-first sounds a lot like a mini Nix+ca-derivations [1] [1] https://www.tweag.io/blog/2021-12-02-nix-cas-4/ https://www.tweag.io/blog/2021-12-02-nix-cas-4/

0xf20x00 3y ago

[dead]