3 ms·
At Airbnb we are implementing translation serving on top of Sparkey; part of our Ruby system open sourced here: https://github.com/airbnb/hammerspace https://gi
by jitl 3y ago
At Airbnb we are implementing translation serving on top of Sparkey; part of our Ruby system open sourced here: https://github.com/airbnb/hammerspace https://github.com/airbnb/hammerspace
The way the system worked (iirc it’s been a while): we’d periodically dump all the translation strings from our translation editor app (MySQL backed) into a sparkey S3 path, and use a Kubernetes daemonset to fetch the sparkey file to each host in the cluster. Finally application pods written in any language use a host volume to read translation strings from the local sparkey file. Certainly overkill for smaller apps but with a bunch of services and hundreds of thousands of translation strings, the out of band updates are well worth it and keep the app container small.
- igor47 3y agoAh yes! For the Rails monolith the sparkey integration was accomplished through https://github.com/airbnb/hammerspace https://github.com/airbnb/hammerspace
- esprehn 3y agoIIRC many of Airbnb's systems were implemented on top of sparkey: translations, feature flags, experiments, dynamic config. It stood the test of time really well. Probably all built by Jon Tai.
- nurgasemetey 3y agoVery ingenious indeed. How would you deal with data corruptions? How it was detected? Is repair automatic?
- esprehn 3y agoI don't believe there was any special handling for corruption, but there was probably hash checking when syncing new data every few minutes. The data stored in there was not user data, so the failure modes are not as severe. Sparkey is also never the source of truth, it's a local cache for each service instance.
- mooreds 3y agoDo you know if that system is still in use? The gem looks abandoned.