3 ms·

I have a related question. How is instacart so slow? I am usually pretty unbothered by slow load times, but searching for and selecting groceries is a full 10 t

by ouid 6y ago

I have a related question. How is instacart so slow? I am usually pretty unbothered by slow load times, but searching for and selecting groceries is a full 10 times slower than any other experience on the internet. Is this a deliberate push to get me to use the mobile app? Some dark pattern thing?

nattaylor 6y ago
Early on they built a Rails app backed by Postgres [0] to store, these days, 500 million items. Then in this post [1] they say they're rearchitecting the database, then mention Snowflake. It sounds to me like they need a document store like Lucene in order to get fast search performance--but they may be optimizing for other use cases like tracking orders which is probably easier in a RDMBS. It also looks like they might first do all the work to retrieve all the results for a query. The first request is to an endpoint `search_v3/{term}?cache_key={key}` then subsequent requests are to `asyncresultset_{n}?cache_key={key}` so seems like they might have the result set cached from the first query. The response to the autocomplete endpoint, which is fast, contains an `elastic_id` so perhaps they do partially use a document store. [0] - https://stackshare.io/posts/the-tech-behind-instacarts-grocery-delivery-service https://stackshare.io/posts/the-tech-behind-instacarts-groce... [1] - https://tech.instacart.com/the-story-behind-an-instacart-order-part-1-building-a-digital-catalog-46df5a8ff705 https://tech.instacart.com/the-story-behind-an-instacart-ord...
adventured 6y ago
It is remarkably, painfully slow. Every letter entered into their search box (eg when searching a particular grocery store for a product) appears to be doing a full reload of results. You can feeeeel the horrible lag as you try to type and the site can't keep up with either your typed text or the results. They should be doing a timed release on that. If you stop entering text for N ms, then go for a result. Otherwise they need to have cached results for a huge number of common combinations of letters, very frequently updated, for every store. It's a resource intensive thing to do well at their scale, for such a seemingly simple feature. If they are in fact caching all of those drop-down search results properly, something is very wrong on serving up the cached content. And for the full results pages, when you try to load them for a given grocery store - the only answer I can guess for that is again mediocre caching. There usually isn't any other culprit other than that for such simple pages. In an effort to match current inventory, my guess is their caching isn't very good (constantly invalidated or rarely put into cache, so they're doing something that isn't very performant; I'd be astounded if they weren't doing some amount of caching on the results).
- Traubenfuchs 6y ago
  > They should be doing a timed release on that. If you stop entering text for N ms, then go for a result. I believe that is called debouncing and it's something any non-junior frontend developer should have in their toolkit.
- ouid 6y ago
  This makes sense technically, but it still doesn't answer the question of why it isn't fixed. Is it really that hard to experiment with caching policies? Is there secretly only one developer at instacart?

by ouid 6y ago

nattaylor 6y ago

Early on they built a Rails app backed by Postgres [0] to store, these days, 500 million items. Then in this post [1] they say they're rearchitecting the database, then mention Snowflake. It sounds to me like they need a document store like Lucene in order to get fast search performance--but they may be optimizing for other use cases like tracking orders which is probably easier in a RDMBS. It also looks like they might first do all the work to retrieve all the results for a query. The first request is to an endpoint `search_v3/{term}?cache_key={key}` then subsequent requests are to `asyncresultset_{n}?cache_key={key}` so seems like they might have the result set cached from the first query. The response to the autocomplete endpoint, which is fast, contains an `elastic_id` so perhaps they do partially use a document store. [0] - https://stackshare.io/posts/the-tech-behind-instacarts-grocery-delivery-service https://stackshare.io/posts/the-tech-behind-instacarts-groce... [1] - https://tech.instacart.com/the-story-behind-an-instacart-order-part-1-building-a-digital-catalog-46df5a8ff705 https://tech.instacart.com/the-story-behind-an-instacart-ord...

adventured 6y ago

It is remarkably, painfully slow. Every letter entered into their search box (eg when searching a particular grocery store for a product) appears to be doing a full reload of results. You can feeeeel the horrible lag as you try to type and the site can't keep up with either your typed text or the results. They should be doing a timed release on that. If you stop entering text for N ms, then go for a result. Otherwise they need to have cached results for a huge number of common combinations of letters, very frequently updated, for every store. It's a resource intensive thing to do well at their scale, for such a seemingly simple feature. If they are in fact caching all of those drop-down search results properly, something is very wrong on serving up the cached content. And for the full results pages, when you try to load them for a given grocery store - the only answer I can guess for that is again mediocre caching. There usually isn't any other culprit other than that for such simple pages. In an effort to match current inventory, my guess is their caching isn't very good (constantly invalidated or rarely put into cache, so they're doing something that isn't very performant; I'd be astounded if they weren't doing some amount of caching on the results).

Traubenfuchs 6y ago
> They should be doing a timed release on that. If you stop entering text for N ms, then go for a result. I believe that is called debouncing and it's something any non-junior frontend developer should have in their toolkit.
ouid 6y ago
This makes sense technically, but it still doesn't answer the question of why it isn't fixed. Is it really that hard to experiment with caching policies? Is there secretly only one developer at instacart?