3 ms·

Google Sheets is banned in our households, we planned our neighborhood block party in Row Zero! Tech stack: Everything runs on AWS Frontend: TypeScript + som

Google Sheets is banned in our households, we planned our neighborhood block party in Row Zero! Tech stack: Everything runs on AWS Frontend: TypeScript + some Rust compiled to WASM, our own virtualized custom canvas magic Backend: Rust, Apache Arrow The biggest tech challenges... where to start. 1. As with any editor application, Undo/Redo is devilishly hard 2. A spreadsheet is modeled as a Directed Acyclic Graph where cells depend on other cells down to some root cells. There are a TON of fascinating graph theory algorithms you can bring to bear here. Some of the hardest problems I have ever worked on. My whiteboard looks like an insane person's with crazy arrows between cells and stuff. 3. Sometimes there really is no magic bullet and you just have to run benchmark, get flamegraph, chip away 3%, repeat, do that 100 times and you are very fast. Gotta grind.

sidcool 3y ago
How is the data modelled and persisted? What database and how is conflict handled? CRDT?
- gamegoblin 3y ago
  We use Apache Arrow data format, so it's fairly columnar. We have built some custom layers on top of Arrow as well to handle some fancier data types. For storage and orchestration we use S3 and Dynamo. Yes exactly, we use CRDTs for multiplayer stuff.
  - sidcool 3y ago
    Thanks! A digressing question. How does one go about learning to build an amazing platform line Row Zero?
    - gamegoblin 3y ago
      We were fortunate to work in AWS which has some of the gnarliest datastructures and algorithms problems in the world, so we got really good there. Spreadsheets are really just more datastructures and algorithms on the backend. Our frontend guy is brilliant and built a lot of awesome stuff at Airtable, but I can't speak to the craziness that is high-complexity high-performance frontend. But it's absolutely necessary to a good product too!
      - sidcool 3y ago
        Excuse me for probing further. Can you share some of the books that can help with this? Which algorithms were used, how etc. Thank in advance, I am using Row Zero since yesterday and loving it thoroughly.
        gamegoblin 3y ago
        I don't have any great book recommendations because I learned on the job mostly. For a project that will teach you literally everything there is to know about backend development: 1. Write a simple PUT/GET/DELETE REST API network layer, this should be a few hundred lines of code max depending on how much you choose to lean on libraries vs write it yourself 2. Then, write a simple in-memory blob store behind the PUT/GET/DELETE REST API. Just use a very simple hashmap of keys to data. You now have an in-memory S3 mock! 3. Now, rather than in-memory hashmap, start saving the files to disk. Write code that allows you to start and stop the process and recover all the data from disk. Now you have a persistent S3 mock. 4. Now, start finding the limits of this application. Write a program that calls your API with different usage patterns to load test it. Find the limits. 5. Now start optimizing. You can go as far as you want here. This optimization process was my whole career at AWS S3, and most of it at Row Zero. 6. If you want to really learn some advanced data structures and algorithms, stop using the filesystem directly, and write your own storage mechanism. Allocate 1 giant 16GB (or however large) file and store all of your blob data inside that 16GB "partition". You have to do all the serialization and retrieval etc yourself. Your filesystem is already doing this for you when you store as files (look up how the ext4 filesystem works, inodes, pages, etc). But your filesystem is probably optimized for consumer use, not this blob-storage system use. So you can implement your own much faster version if you want. If you want to really learn datastructures and algorithms, try to code a simple Log Structured Merge Tree. This will teach hashes, trees, bloom filters, serialization, deserialization, etc, all with high performance in mind.
        sidcool 3y ago
        Thanks! Appreciate your detailed response!!
ramoneguru 3y ago
Nice work, did you start off by looking at any previously built canvas-like spreadsheets or was it straight to "virtualized custom canvas magic"? I interviewed at a spreadsheet company (for a frontend role) and they asked, "how would you go about determining what cells need a border when a user clicks an individual cell, clicks a cell and selects multiple cells, clicks a cell next to an already selected cell." Fascinating problem and we talked about solutions for a little bit. Noticed that you can't unselect a cell once it's selected? I'm on a Mac with Chrome (latest, no updates available). Repo steps: 1. Select a few cells (⌘ + click) or an individual cell 2. Try unselecting (⌘ + click) those same cells clicked in #1 3. Cell is not unselected
- billylitt 3y ago
  Row Zero frontend dev here -- when architecting, we looked at some off-the-shelf canvas-based table tools, but ultimately rolled our engine for more control & flexibility with our growing feature set. We elected for canvas over DOM for perf among other reasons (eg DOM scrollbar virtualization is hard when MAX_ROW * ROW_HEIGHT exceeds the maximum allowed browser element height). Great interview question. Tons of nuance to drawing borders on adjacent cells, how to handle varying thickness, etc. Once you start looking closely, you notice the pixel differences between how this gets handled by various spreadsheeting tools. Thanks for the report! This one's already on my list actually (selection negation & unique selection deduping) -- look for a fix soon.
  - alooPotato 3y ago
    Did you look at https://grid.glideapps.com/ https://grid.glideapps.com/ by chance for rendering? Curious what it didn't support if you did.
    - billylitt 3y ago
      I don't remember looking at Glide, although it looks really nice & full-featured. I'll have to play around with it sometime. I do remember trying out https://www.npmjs.com/package/@deephaven/grid https://www.npmjs.com/package/@deephaven/grid. One pivotal feature that is difficult to map onto 3P tools is our data table UI, which is a separate scrollable grid that floats on top of the main sheet. That, combined with the complexity of formula selection, inserting buttons into cells (header dropdowns, filter, sort), led us to decide that rolling our own solution for full control was the right choice.
edanm 3y ago
Coming to this late, but... What made you choose this tech stack? I'm especially wondering about Rust, was that something you had familiarity with before so just went with that? Did you evaluate a few different options and decided that would best fit this use-case? (I have very minimal familiarity with Rust, I'm just wondering here.)

sidcool 3y ago

How is the data modelled and persisted? What database and how is conflict handled? CRDT?

gamegoblin 3y ago
We use Apache Arrow data format, so it's fairly columnar. We have built some custom layers on top of Arrow as well to handle some fancier data types. For storage and orchestration we use S3 and Dynamo. Yes exactly, we use CRDTs for multiplayer stuff.
- sidcool 3y ago
  Thanks! A digressing question. How does one go about learning to build an amazing platform line Row Zero?
  - gamegoblin 3y ago
    We were fortunate to work in AWS which has some of the gnarliest datastructures and algorithms problems in the world, so we got really good there. Spreadsheets are really just more datastructures and algorithms on the backend. Our frontend guy is brilliant and built a lot of awesome stuff at Airtable, but I can't speak to the craziness that is high-complexity high-performance frontend. But it's absolutely necessary to a good product too!
    - sidcool 3y ago
      Excuse me for probing further. Can you share some of the books that can help with this? Which algorithms were used, how etc. Thank in advance, I am using Row Zero since yesterday and loving it thoroughly.
      - gamegoblin 3y ago
        I don't have any great book recommendations because I learned on the job mostly. For a project that will teach you literally everything there is to know about backend development: 1. Write a simple PUT/GET/DELETE REST API network layer, this should be a few hundred lines of code max depending on how much you choose to lean on libraries vs write it yourself 2. Then, write a simple in-memory blob store behind the PUT/GET/DELETE REST API. Just use a very simple hashmap of keys to data. You now have an in-memory S3 mock! 3. Now, rather than in-memory hashmap, start saving the files to disk. Write code that allows you to start and stop the process and recover all the data from disk. Now you have a persistent S3 mock. 4. Now, start finding the limits of this application. Write a program that calls your API with different usage patterns to load test it. Find the limits. 5. Now start optimizing. You can go as far as you want here. This optimization process was my whole career at AWS S3, and most of it at Row Zero. 6. If you want to really learn some advanced data structures and algorithms, stop using the filesystem directly, and write your own storage mechanism. Allocate 1 giant 16GB (or however large) file and store all of your blob data inside that 16GB "partition". You have to do all the serialization and retrieval etc yourself. Your filesystem is already doing this for you when you store as files (look up how the ext4 filesystem works, inodes, pages, etc). But your filesystem is probably optimized for consumer use, not this blob-storage system use. So you can implement your own much faster version if you want. If you want to really learn datastructures and algorithms, try to code a simple Log Structured Merge Tree. This will teach hashes, trees, bloom filters, serialization, deserialization, etc, all with high performance in mind.
        sidcool 3y ago
        Thanks! Appreciate your detailed response!!

ramoneguru 3y ago

Nice work, did you start off by looking at any previously built canvas-like spreadsheets or was it straight to "virtualized custom canvas magic"? I interviewed at a spreadsheet company (for a frontend role) and they asked, "how would you go about determining what cells need a border when a user clicks an individual cell, clicks a cell and selects multiple cells, clicks a cell next to an already selected cell." Fascinating problem and we talked about solutions for a little bit. Noticed that you can't unselect a cell once it's selected? I'm on a Mac with Chrome (latest, no updates available). Repo steps: 1. Select a few cells (⌘ + click) or an individual cell 2. Try unselecting (⌘ + click) those same cells clicked in #1 3. Cell is not unselected

billylitt 3y ago
Row Zero frontend dev here -- when architecting, we looked at some off-the-shelf canvas-based table tools, but ultimately rolled our engine for more control & flexibility with our growing feature set. We elected for canvas over DOM for perf among other reasons (eg DOM scrollbar virtualization is hard when MAX_ROW * ROW_HEIGHT exceeds the maximum allowed browser element height). Great interview question. Tons of nuance to drawing borders on adjacent cells, how to handle varying thickness, etc. Once you start looking closely, you notice the pixel differences between how this gets handled by various spreadsheeting tools. Thanks for the report! This one's already on my list actually (selection negation & unique selection deduping) -- look for a fix soon.
- alooPotato 3y ago
  Did you look at https://grid.glideapps.com/ https://grid.glideapps.com/ by chance for rendering? Curious what it didn't support if you did.
  - billylitt 3y ago
    I don't remember looking at Glide, although it looks really nice & full-featured. I'll have to play around with it sometime. I do remember trying out https://www.npmjs.com/package/@deephaven/grid https://www.npmjs.com/package/@deephaven/grid. One pivotal feature that is difficult to map onto 3P tools is our data table UI, which is a separate scrollable grid that floats on top of the main sheet. That, combined with the complexity of formula selection, inserting buttons into cells (header dropdowns, filter, sort), led us to decide that rolling our own solution for full control was the right choice.

edanm 3y ago

Coming to this late, but... What made you choose this tech stack? I'm especially wondering about Rust, was that something you had familiarity with before so just went with that? Did you evaluate a few different options and decided that would best fit this use-case? (I have very minimal familiarity with Rust, I'm just wondering here.)