5 ms·

We looked at building a two-way sync tool in Google sheets to test out an MVP instead of building some custom tool and it was a shitshow from a data synchroniza

We looked at building a two-way sync tool in Google sheets to test out an MVP instead of building some custom tool and it was a shitshow from a data synchronization standpoint figuring out how to keep the state of each in sync is very hard. We abandoned the idea pretty quickly. For problem (2), I'm not sure if you're directly returning data from lambda or if you're using lambda to use the sheets API to update the sheet after the data is returned. If it's the first one, there's no way around it since lambda functions called through API Gateway have the 30 second timeout that you're running into. If it's the second you should be able to use two lambdas one as one that ingests the requests and adds it to a queue, and the second one (which can have up to a 15 minute timeout) that consumes the queue.

ayazhan 7y ago
Yes, data is returned directly from lambda. We’re working on the pro version and enterprise versions that will run queries in the backend and update sheets as you described. For that, we’d need to request for additional scopes, to be able to update sheets remotely. Currently, updates are made by the addon and only when it is in use.
neeleshs 7y ago
Two-way sync without maintaining state is indeed a complex problem. It is loosely a multi-master replication problem. Shameless plug : We built a multi-directional sync engine and a company called Syncari around it to solve exactly this problem. And added data pipelines around it. Mostly meant for sales operations/marketing operations folks, self-service style.