3 ms·
Ask HN: How Is Google Analytics So Damn Fast?
It updates in real-time, it certainly can't be using bigquery as I see every query in bigquery takes atleast 2 seconds. So, how is it so realtime?
- tixocloud 9y agoThis is where architecture is important and there are several techniques you can use to make things faster. You could look at pre-calculation, subsetting the data, in-memory databases, dedicated servers, non-blocking requests, etc.
- timmm 9y agoIt's sampled.
- borplk 9y agoAre you talking about a specific feature of it or the whole thing in general?
- youeseh 9y agoI'm going to guess: - If you are logged into the Analytics dashboard, then a web-socket has been activated between your browser and GA. For fast lookup, there's probably an in-memory hash of GA IDs, web-socket IDs. SO: - User visits your webpage / performs some action worth analyzing. - GA script gets executed; sends request to GA server. - GA server acknowledges reception and schedules jobs for: -- storing the hit for associated GA-ID. -- sending info of hit to websocket associated with GA-ID.
- antidaily 9y agoDHTML
- trakout 9y agoCan you clarify?
- jamesjamesm 9y agoGoogle Bigtable
- usgroup 9y agoLikely by processing different data at different times. E.g. big aggregates and time series are likely updated on a lower interval whilst near-time event data is probably stored in fast recall storage so that you can satisfy near-time queries immediately. Otherwise, their long view queries are always sampled. They control the fields available so they can make sure samples are properly stratified in order to be representative. IMO, it's just good old fashioned good engineering. It could be done with no special tech whatsoever. E.g. sharded Postgres DBs and network storage would do just fine.
- deleted 9y ago[deleted]
- ddorian43 9y agoPre aggregated reports. Custom reports have `using x out of y sessions`. Custom reports using bigquery or something similar and more compact/specialized.