3 ms·
Wow what a cool writeup. Two things stuck out for me (I'm only halfway so forgive me if you address these issues). I have two comments on processing the 511.org
by javajosh 2y ago
Wow what a cool writeup. Two things stuck out for me (I'm only halfway so forgive me if you address these issues). I have two comments on processing the 511.org data. First, you generally want to use a streaming parser rather than one that allocates memory to the entire response. Second, you should filter out the most data first (in this case, dropping the stops you don't care about) and then filter the least data second (dropping the fields you don't care about). This idea is similar to how, in SQL, you want to order your WHERE clauses such that the most impactful comes first.
This may not be strictly required by your use-case at this scale - 27MB of data is not a lot, after all. And the filter ordering performance is probably trivial given its all in memory (I'd be curious to see a benchmark!) However, a) efficiency is always good, especially on a Pi, and b) if you make the code more efficient it makes it easier to scale later if you want to.
Also, regarding the BOM problem you had, wouldn't it be nice if all APIs had a "developer feedback" mode built into it? That is, you can send feedback to the ppl who own the API endpoint by...posting to the endpoint. In this case you could send "Please remove the BOM. k thanks."