4 ms·

This is super important work, and is kind of why I built https://civic.band https://civic.band and https://civic.observer https://civic.observer, which are gene

by phildini 10mo ago

This is super important work, and is kind of why I built https://civic.band https://civic.band and https://civic.observer https://civic.observer, which are generalized tools for monitoring civic govts. (You can search for anything, not just ALPR)

ZeWaka 10mo ago
Seem to be getting 405s from https://civic.observer/auth/login https://civic.observer/auth/login And 404 from https://civic.band/why.html https://civic.band/why.html
- phildini 10mo ago
  Fixed the 404s on civic.band, thanks
kpw94 10mo ago
Very cool! And important for sure, thank you. Few questions: - is the stack to index those open source? - is there some standardized APIs each municipality provides, or do you go through the tedious task of building a per-municipality crawling tool? - how often do you refresh the data? Checked a city, it has meeting minutes until 6/17, but the official website has more recent minutes (up to 12/2 at least)
- phildini 10mo ago
  Thanks for asking! - The framework for crawling is open-source. https://github.com/civicband https://github.com/civicband - There is absolutely not a standardized API for nearly any of this. I build generalized crawlers when I can, and then build custom crawlers when I need. - Can you let me know which city? The crawlers run for every municipality at least once every day, so that's probably a bug
tayari- 10mo ago
This is incredible, great work and will definitely be using and sharing this! Where in the repos can we find the plugin/scraper for given municipalities to help contribute when they seem to be broken? As looks like the last meetings and agendas scraped for Cook County are from March/April of this year
- mdnahas 10mo ago
  Also missing: Austin, TX
  - phildini 10mo ago
    Fixed Austin TX!
- phildini 10mo ago
  Hello! The crawlers are not currently public, but I'm happy to take volunteers behind the curtain. I also fixed Cook County
tonymet 10mo ago
very cool can you check the login page i'm getting error 405 on https://civic.observer/auth/login https://civic.observer/auth/login I am interested in monitoring local legislation in Clark County, WA

ZeWaka 10mo ago

Seem to be getting 405s from https://civic.observer/auth/login https://civic.observer/auth/login And 404 from https://civic.band/why.html https://civic.band/why.html

phildini 10mo ago
Fixed the 404s on civic.band, thanks

kpw94 10mo ago

Very cool! And important for sure, thank you. Few questions: - is the stack to index those open source? - is there some standardized APIs each municipality provides, or do you go through the tedious task of building a per-municipality crawling tool? - how often do you refresh the data? Checked a city, it has meeting minutes until 6/17, but the official website has more recent minutes (up to 12/2 at least)

phildini 10mo ago
Thanks for asking! - The framework for crawling is open-source. https://github.com/civicband https://github.com/civicband - There is absolutely not a standardized API for nearly any of this. I build generalized crawlers when I can, and then build custom crawlers when I need. - Can you let me know which city? The crawlers run for every municipality at least once every day, so that's probably a bug

tayari- 10mo ago

This is incredible, great work and will definitely be using and sharing this! Where in the repos can we find the plugin/scraper for given municipalities to help contribute when they seem to be broken? As looks like the last meetings and agendas scraped for Cook County are from March/April of this year

mdnahas 10mo ago
Also missing: Austin, TX
- phildini 10mo ago
  Fixed Austin TX!
phildini 10mo ago
Hello! The crawlers are not currently public, but I'm happy to take volunteers behind the curtain. I also fixed Cook County

tonymet 10mo ago

very cool can you check the login page i'm getting error 405 on https://civic.observer/auth/login https://civic.observer/auth/login I am interested in monitoring local legislation in Clark County, WA