5 ms·

Hey HN! I’m a Repl.it engineer who’s been focused on bringing Database to life. Happy to read your comments and answer any questions you might have.

prionassembly 6y ago
This is the first I'm hearing of repl.it and it looks like an amazing resource. How does this work business-wise? How does it make money? Edit: Will Julia be supported in the future? This is not good... https://repl.it/@asemic/ConsiderateBelatedAccounting https://repl.it/@asemic/ConsiderateBelatedAccounting
- jtanderson 6y ago
  It's subscription-based to get advanced features: https://repl.it/site/pricing https://repl.it/site/pricing
- deleted 6y ago
  [deleted]
- kasperset 6y ago
  Repl.it cannot add install packages and has similar issues with R as well. I hope they can add that feature.
- kwertzzz 6y ago
  I tried Julia as well to test the database feature. Unfortunately, when I `import HTTP`, I get "repl process died unexpectedly: signal killed". (julia 1.3.1 and HTTP v0.8.17)
nexuist 6y ago
This is really cool! Can I ask what you're using under the hood to power this? Redis, CouchDB or some in-house system? For filtering/search queries, is the idea to use the prefix system to group related items together, and then run matches using that?
- simonw 6y ago
  +1 I'd love to know about the underlying storage system.
- kochman 6y ago
  It's Postgres under the hood. The key-value service itself is Go and we run it on GKE alongside some of our other services. Yep, that's the idea. If you had a user with ID 1, one scheme to get all her info could be storing attributes in keys such as: user:1:name user:1:email user:1:city Then a prefix search for "user:1:" would get you the relevant keys for that user. But you could also go another way and drop a JSON-encoded object in a single key and work with it inside your app, since that might be the fastest way to get something off the ground!
  - jedberg 6y ago
    I didn't look super closely, but it looks like your database works similarly to DynamoDB, in which case it could use similar data modeling. Here is a great post on single table design for DynamoDB that would probably be mostly applicable here: https://www.alexdebrie.com/posts/dynamodb-single-table/ https://www.alexdebrie.com/posts/dynamodb-single-table/ The post author actually wrote an entire book on data modeling with a single key/value table.
simonw 6y ago
Did you consider SQLite for this?
- kochman 6y ago
  We did! Our infrastructure has some unique constraints around filesystem persistence. We use btrfs and regularly take snapshots of the repl's filesystem while someone is editing the repl. However, we don't take snapshots when a repl is only acting as a web server without anyone editing it, meaning that we would need a solution that lives outside of the repl's filesystem. Keeping each database outside of the filesystem also gives us the flexibility to let people access the same database across multiple repls in the future. I have lots of love for SQLite; early prototypes of Database were backed by it!
  - simonw 6y ago
    Totally makes sense - SQLite is very dependent on a traditional filesystem. I've been figuring out how to run backups recently and grabbing a copy of the file isn't enough - you need to be sure there are no transactions going on, so you need to use the SQLite backup API or run VACUUM INTO a separate copy, which then doubles the amount of disk space you need.
noworriesnate 6y ago
This sounds like a database-per-tenant architecture, which is something I'm interested in. Are you having to build a lot of custom tooling to manage that many databases? Does the existing repl.it code make it easy to just spin up an extra docker container for each session? Or are there off-the-shelf tools that make this easy?
gregsadetsky 6y ago
Congrats on the launch! Very small thing, but I didn't initially see the database icon in the sidebar. Trying to `from replit import db` in Python and using the `db` resulted in an unclear error message about the REPLIT_DB_URL missing ("are we running on repl.it?") Turns out I had to sign in to repl.it for the sidebar icon to appear and the code to work. The documentation (and that error message) might need to reflect this? And/or should databases be available to all users (i.e., those who aren't signed in as well)? --- Another issue: when going to https://repl.it/repls/HandyNiceBug https://repl.it/repls/HandyNiceBug (providential URL...) ;-) even when signed in, running the code shows the same `REPLIT_DB_URL does not exist` error. If you click the database sidebar icon and any of the 'Insert ->' buttons, then the env seems to update and the code runs successfully. --- Hopefully useful nitpick: after clicking the 'Insert ->' button once in the db sidebar UI (in the url above), all of the 'Insert ->' buttons stop working. Aaaand, if I try manually copying any of the code snippets to insert one myself, what's copied into the clipboard is not-the-best. Instead of copying `db["key"] = "value"` for instance, it copies it but with many extra new lines and some bits missing. Might be useful to have a copy-to-clipboard icon? Or do some DOM trickery to allow selection/copy&paste? Cheers :-)
- kochman 6y ago
  Good points, you're right: you have to be signed in for Database to work. We'll make this clearer. We'll also fix the code snippet copying. Thank you for giving it a try!
- vhbit 6y ago
  on Golang REPL there is no error message displayed which makes it even more puzzling, thanks for hint with logging in!
grugagag 6y ago
I love your product, it's definitely a game changer! I will continue to use my local setup but if I had so many environments at my fingertips when I was a student I would have gotten further in my career. I'm more or less stuck with my stack nowadays. I have a tangent question though, is the .it in repl.it connected in any way to Italy?

prionassembly 6y ago

This is the first I'm hearing of repl.it and it looks like an amazing resource. How does this work business-wise? How does it make money? Edit: Will Julia be supported in the future? This is not good... https://repl.it/@asemic/ConsiderateBelatedAccounting https://repl.it/@asemic/ConsiderateBelatedAccounting

jtanderson 6y ago
It's subscription-based to get advanced features: https://repl.it/site/pricing https://repl.it/site/pricing
deleted 6y ago
[deleted]
kasperset 6y ago
Repl.it cannot add install packages and has similar issues with R as well. I hope they can add that feature.
kwertzzz 6y ago
I tried Julia as well to test the database feature. Unfortunately, when I `import HTTP`, I get "repl process died unexpectedly: signal killed". (julia 1.3.1 and HTTP v0.8.17)

nexuist 6y ago

This is really cool! Can I ask what you're using under the hood to power this? Redis, CouchDB or some in-house system? For filtering/search queries, is the idea to use the prefix system to group related items together, and then run matches using that?

simonw 6y ago
+1 I'd love to know about the underlying storage system.
kochman 6y ago
It's Postgres under the hood. The key-value service itself is Go and we run it on GKE alongside some of our other services. Yep, that's the idea. If you had a user with ID 1, one scheme to get all her info could be storing attributes in keys such as: user:1:name user:1:email user:1:city Then a prefix search for "user:1:" would get you the relevant keys for that user. But you could also go another way and drop a JSON-encoded object in a single key and work with it inside your app, since that might be the fastest way to get something off the ground!
- jedberg 6y ago
  I didn't look super closely, but it looks like your database works similarly to DynamoDB, in which case it could use similar data modeling. Here is a great post on single table design for DynamoDB that would probably be mostly applicable here: https://www.alexdebrie.com/posts/dynamodb-single-table/ https://www.alexdebrie.com/posts/dynamodb-single-table/ The post author actually wrote an entire book on data modeling with a single key/value table.

simonw 6y ago

Did you consider SQLite for this?

kochman 6y ago
We did! Our infrastructure has some unique constraints around filesystem persistence. We use btrfs and regularly take snapshots of the repl's filesystem while someone is editing the repl. However, we don't take snapshots when a repl is only acting as a web server without anyone editing it, meaning that we would need a solution that lives outside of the repl's filesystem. Keeping each database outside of the filesystem also gives us the flexibility to let people access the same database across multiple repls in the future. I have lots of love for SQLite; early prototypes of Database were backed by it!
- simonw 6y ago
  Totally makes sense - SQLite is very dependent on a traditional filesystem. I've been figuring out how to run backups recently and grabbing a copy of the file isn't enough - you need to be sure there are no transactions going on, so you need to use the SQLite backup API or run VACUUM INTO a separate copy, which then doubles the amount of disk space you need.

noworriesnate 6y ago

This sounds like a database-per-tenant architecture, which is something I'm interested in. Are you having to build a lot of custom tooling to manage that many databases? Does the existing repl.it code make it easy to just spin up an extra docker container for each session? Or are there off-the-shelf tools that make this easy?

gregsadetsky 6y ago

Congrats on the launch! Very small thing, but I didn't initially see the database icon in the sidebar. Trying to `from replit import db` in Python and using the `db` resulted in an unclear error message about the REPLIT_DB_URL missing ("are we running on repl.it?") Turns out I had to sign in to repl.it for the sidebar icon to appear and the code to work. The documentation (and that error message) might need to reflect this? And/or should databases be available to all users (i.e., those who aren't signed in as well)? --- Another issue: when going to https://repl.it/repls/HandyNiceBug https://repl.it/repls/HandyNiceBug (providential URL...) ;-) even when signed in, running the code shows the same `REPLIT_DB_URL does not exist` error. If you click the database sidebar icon and any of the 'Insert ->' buttons, then the env seems to update and the code runs successfully. --- Hopefully useful nitpick: after clicking the 'Insert ->' button once in the db sidebar UI (in the url above), all of the 'Insert ->' buttons stop working. Aaaand, if I try manually copying any of the code snippets to insert one myself, what's copied into the clipboard is not-the-best. Instead of copying `db["key"] = "value"` for instance, it copies it but with many extra new lines and some bits missing. Might be useful to have a copy-to-clipboard icon? Or do some DOM trickery to allow selection/copy&paste? Cheers :-)

kochman 6y ago
Good points, you're right: you have to be signed in for Database to work. We'll make this clearer. We'll also fix the code snippet copying. Thank you for giving it a try!
vhbit 6y ago
on Golang REPL there is no error message displayed which makes it even more puzzling, thanks for hint with logging in!

grugagag 6y ago

I love your product, it's definitely a game changer! I will continue to use my local setup but if I had so many environments at my fingertips when I was a student I would have gotten further in my career. I'm more or less stuck with my stack nowadays. I have a tangent question though, is the .it in repl.it connected in any way to Italy?