That's good information but I hope others have more complete answers to the OP's questions. There are a few problems with BigQuery being the only solution (you

That's good information but I hope others have more complete answers to the OP's questions. There are a few problems with BigQuery being the only solution (you didn't say that's the case, but one has to wonder). The problems include cost, vulnerability to shutdown, potential rate limiting, to name a few.