4 ms·
ApiRoad is an interesting project, are you running this all by yourself? In any case, do you have a write-up somewhere of how you use a columnar DBMS to slurp
by ayewo 6y ago
ApiRoad is an interesting project, are you running this all by yourself?
In any case, do you have a write-up somewhere of how you use a columnar DBMS to slurp your server logs?
- jetter 6y agoThank you! Yes this is a big solo project now. Didn't have time to do a write up yet. I am a big fan of Clickhouse, so I use it for log storage. To collect server logs, it is not exactly a simple file parsing setup - I have a gateway daemon that stays in front of the API server. The daemon receives all the requests from subscribers, authorizes that this specific connection has access to specific API, rate limits, and proxies all the connections to upstream API, dumping HTTP logs to Clickhouse once in a while. Then Vue.js & Laravel powered dashboard queries Clickhouse to generate various stats, which is later used for analytics and usage-based billing.