4 ms·

Can someone explain what this is? Why it's a good thing? What's it's for? I have to admit based on reading the post I have absolutely no idea.

KTibow 3y ago
From reading https://scrapscript.org/ https://scrapscript.org/ it sounds like its main feature is that things can be split up, put on platforms like IPFS, and distributed allowing you to access them from wherever.
surprisetalk 3y ago
I should probably write a longer post about this, but scrapscript is an attempt to fix a lot of the "in-between" problems in software engineering. Instead of working on "real" problems, I find myself battling untyped/undocumented YAML/JSON configurations, syncing JSON encoders/decoders, massaging incompatible dependencies, writing unholy SQL, etc. I obviously don't have all the answers, but a system with the following properties seems like a worthwhile pursuit: (1) small enough to be used like JSON yet powerful enough to used like Javascript, (2) cryptographic guarantees that code is compatible over time, (3) a compiler that checks live servers for compatibility before deploying, (4) simple but expressive type system, (5) a package manager that facilitates all of this at a granular level... and so on. On top of all that, I think these properties lend themselves to some grand ambitions like "a new internet" and a "google-docs live coding editor experience". Maybe I'm just full of myself though haha scrapscript.py is the first real attempt at making scrapscript a reality, so some folks who feel these pains are getting excited to see some movement on the project. EDIT: Here’s my recent scrapyard demo, if you want to see it in action: https://www.youtube.com/watch?v=SngOLU5G1Eg https://www.youtube.com/watch?v=SngOLU5G1Eg
- account-5 3y ago
  Still not sure I fully understand, but that is more than likely down to my ignorance. I really appreciate your effort in explaining here. I should mention I'm not a full time developer and certainly not a webdev so this might be why I'm not grokking this. Thanks.
- mst 3y ago
  I'd been kind of interested by https://yglu.io/ https://yglu.io/ and now ingy's new piece of insanity https://yamlscript.org/ https://yamlscript.org/ - helm appears to let you inject your own script to template charts and I was wondering about trying a wrapper around one of those (because text templating an indentation sensitive language like YAML makes me itch). I think scrapscript is a really interesting idea, mind, this isn't a "here's an alternative" type comment, it's a "here's things that I think are neat in a similar way to how I think scrapscript is neat" :) Edit: I forgot something! https://trout.me.uk/lisp/termite-r7rs.pdf https://trout.me.uk/lisp/termite-r7rs.pdf is a paper on adding library support to the cross-network (kinda erlangish) termite scheme extensions - and leans heavily on content addressable-ness. Termite itself has gone the way of small lisp projects but I kept this around specifically for the content addressable stuff having been solidly worked out in a language I understood; maybe that'll come in handy for ideas for you as well.
- dflock 3y ago
  I assume you're well aware of: https://www.unison-lang.org/ https://www.unison-lang.org/ - as well as 9p and union mounting from plan9.
  - surprisetalk 3y ago
    Yes, I'm aware :) I actually built the first scrapscript demo in 2018, drawing on inspiration from Ethereum's Solidity. Somebody pointed me toward Unison when I attended Strange Loop in 2019, and I chatted with Paul Chiusano, and it seemed like Unison and Scrapscript had incompatible design goals. Even now, I don't see much overlap outside of content-addressability. Unison is super cool though, and I wish their team the best!
- nerdponx 3y ago
  It might be interesting to include a comparison with Dhall and Jsonnet while you're writing docs.
- asveikau 3y ago
  > (2) cryptographic guarantees that code is compatible over time, What does this mean? You hash dependencies? > (3) a compiler that checks live servers for compatibility before deploying, Why does a compiler need to talk to a server? Why should it? Seems like a huge step backwards in what a compiler is and expecting it to work later on.
  - surprisetalk 3y ago
    > What does this mean? You hash dependencies? Yes, but everything is hashed at the expression-level rather than at the file-level, which prevents a few classes of errors. > Why does a compiler need to talk to a server? Why should it? Seems like a huge step backwards in what a compiler is and expecting it to work later on. Imagine if Javascript tooling could throw an error when a client implementation diverges from the server's expected input/output types: > const res = await fetch("https://example.com/api", [1, 2, 3]); ERROR: You're sending this REST endpoint a list of integers, but it expects a string! Wouldn't that be nice in some applications?
- manifoldgeo 3y ago
  > I find myself battling untyped/undocumented YAML/JSON configurations, syncing JSON encoders/decoders, massaging incompatible dependencies I feel your pain on having to manage so many dependencies. I write primarily in Python, and the various pip / Pipenv / pipx / PDM / Poetry dependency managers drive me pretty crazy. That's not even accounting for the multiple Python versions I need! That said, I'm surprised that you're trying to _alleviate_ this by implementing your FP language in Python. The Python ecosystem is full of half-documented config files, incompatible dependency trees, etc. Have you considered implementing it in any other languages after the Python one proves its worth? For example, if the language becomes strong enough, would you consider writing a scrapscript compiler in scrapscript, itself?
  - surprisetalk 3y ago
    Yeah, I'm not a huge fan of Python, but Max and Chris are world-class in that domain, so that's what we're doing for now. Max has already started working on a meta scrapscript compiler: https://github.com/tekknolagi/scrapscript/pull/100 https://github.com/tekknolagi/scrapscript/pull/100 One thing I think we all agree on is that the implementations should be simple enough to easily port themselves to other languages. For example, one could probably port the existing scrapscript.py to Rust or Javascript using GPT in a single weekend. You can see echoes of what I'm talking about in my tiny JS POC: https://github.com/tekknolagi/scrapscript/blob/trunk/scrapscript.js https://github.com/tekknolagi/scrapscript/blob/trunk/scrapsc... Some languages like Rust and Go put a lot of weight on the "official" implementation. I think scrapscript can be more like Lisp/Json where the spec guides parallel implementations. There are obvious downsides to this in general, but I think that content-addressability makes some of those problems moot.
  - tekknolagi 3y ago
    None of these config/dependency problems are present in scrapscript.py because it has no external dependencies and is written in one file. This is intentional!

KTibow 3y ago

From reading https://scrapscript.org/ https://scrapscript.org/ it sounds like its main feature is that things can be split up, put on platforms like IPFS, and distributed allowing you to access them from wherever.

surprisetalk 3y ago

I should probably write a longer post about this, but scrapscript is an attempt to fix a lot of the "in-between" problems in software engineering. Instead of working on "real" problems, I find myself battling untyped/undocumented YAML/JSON configurations, syncing JSON encoders/decoders, massaging incompatible dependencies, writing unholy SQL, etc. I obviously don't have all the answers, but a system with the following properties seems like a worthwhile pursuit: (1) small enough to be used like JSON yet powerful enough to used like Javascript, (2) cryptographic guarantees that code is compatible over time, (3) a compiler that checks live servers for compatibility before deploying, (4) simple but expressive type system, (5) a package manager that facilitates all of this at a granular level... and so on. On top of all that, I think these properties lend themselves to some grand ambitions like "a new internet" and a "google-docs live coding editor experience". Maybe I'm just full of myself though haha scrapscript.py is the first real attempt at making scrapscript a reality, so some folks who feel these pains are getting excited to see some movement on the project. EDIT: Here’s my recent scrapyard demo, if you want to see it in action: https://www.youtube.com/watch?v=SngOLU5G1Eg https://www.youtube.com/watch?v=SngOLU5G1Eg

account-5 3y ago
Still not sure I fully understand, but that is more than likely down to my ignorance. I really appreciate your effort in explaining here. I should mention I'm not a full time developer and certainly not a webdev so this might be why I'm not grokking this. Thanks.
mst 3y ago
I'd been kind of interested by https://yglu.io/ https://yglu.io/ and now ingy's new piece of insanity https://yamlscript.org/ https://yamlscript.org/ - helm appears to let you inject your own script to template charts and I was wondering about trying a wrapper around one of those (because text templating an indentation sensitive language like YAML makes me itch). I think scrapscript is a really interesting idea, mind, this isn't a "here's an alternative" type comment, it's a "here's things that I think are neat in a similar way to how I think scrapscript is neat" :) Edit: I forgot something! https://trout.me.uk/lisp/termite-r7rs.pdf https://trout.me.uk/lisp/termite-r7rs.pdf is a paper on adding library support to the cross-network (kinda erlangish) termite scheme extensions - and leans heavily on content addressable-ness. Termite itself has gone the way of small lisp projects but I kept this around specifically for the content addressable stuff having been solidly worked out in a language I understood; maybe that'll come in handy for ideas for you as well.
dflock 3y ago
I assume you're well aware of: https://www.unison-lang.org/ https://www.unison-lang.org/ - as well as 9p and union mounting from plan9.
- surprisetalk 3y ago
  Yes, I'm aware :) I actually built the first scrapscript demo in 2018, drawing on inspiration from Ethereum's Solidity. Somebody pointed me toward Unison when I attended Strange Loop in 2019, and I chatted with Paul Chiusano, and it seemed like Unison and Scrapscript had incompatible design goals. Even now, I don't see much overlap outside of content-addressability. Unison is super cool though, and I wish their team the best!
nerdponx 3y ago
It might be interesting to include a comparison with Dhall and Jsonnet while you're writing docs.
asveikau 3y ago
> (2) cryptographic guarantees that code is compatible over time, What does this mean? You hash dependencies? > (3) a compiler that checks live servers for compatibility before deploying, Why does a compiler need to talk to a server? Why should it? Seems like a huge step backwards in what a compiler is and expecting it to work later on.
- surprisetalk 3y ago
  > What does this mean? You hash dependencies? Yes, but everything is hashed at the expression-level rather than at the file-level, which prevents a few classes of errors. > Why does a compiler need to talk to a server? Why should it? Seems like a huge step backwards in what a compiler is and expecting it to work later on. Imagine if Javascript tooling could throw an error when a client implementation diverges from the server's expected input/output types: > const res = await fetch("https://example.com/api", [1, 2, 3]); ERROR: You're sending this REST endpoint a list of integers, but it expects a string! Wouldn't that be nice in some applications?
manifoldgeo 3y ago
> I find myself battling untyped/undocumented YAML/JSON configurations, syncing JSON encoders/decoders, massaging incompatible dependencies I feel your pain on having to manage so many dependencies. I write primarily in Python, and the various pip / Pipenv / pipx / PDM / Poetry dependency managers drive me pretty crazy. That's not even accounting for the multiple Python versions I need! That said, I'm surprised that you're trying to _alleviate_ this by implementing your FP language in Python. The Python ecosystem is full of half-documented config files, incompatible dependency trees, etc. Have you considered implementing it in any other languages after the Python one proves its worth? For example, if the language becomes strong enough, would you consider writing a scrapscript compiler in scrapscript, itself?
- surprisetalk 3y ago
  Yeah, I'm not a huge fan of Python, but Max and Chris are world-class in that domain, so that's what we're doing for now. Max has already started working on a meta scrapscript compiler: https://github.com/tekknolagi/scrapscript/pull/100 https://github.com/tekknolagi/scrapscript/pull/100 One thing I think we all agree on is that the implementations should be simple enough to easily port themselves to other languages. For example, one could probably port the existing scrapscript.py to Rust or Javascript using GPT in a single weekend. You can see echoes of what I'm talking about in my tiny JS POC: https://github.com/tekknolagi/scrapscript/blob/trunk/scrapscript.js https://github.com/tekknolagi/scrapscript/blob/trunk/scrapsc... Some languages like Rust and Go put a lot of weight on the "official" implementation. I think scrapscript can be more like Lisp/Json where the spec guides parallel implementations. There are obvious downsides to this in general, but I think that content-addressability makes some of those problems moot.
- tekknolagi 3y ago
  None of these config/dependency problems are present in scrapscript.py because it has no external dependencies and is written in one file. This is intentional!