3 ms·

I just woke up and wow. Well, I'd be happy to answer any questions you might have of course.

by steeve 12y ago

nyandaber 12y ago
How does it feed the data to the repo? Is it parsing reports from the parliament? If yes, how smooth did the process go?
- steeve 12y ago
  It goes on each article on Legifrance, creates a JSON file with all the articles, which book/section they belong to, and for each article, tracks it's versions (they are dates). The crawler is in Go. Then there is a python script that takes that JSON file, creates the .md files and runs the git commands in the shell. Ultimately the sad thing is that I had to scrape this information. There were lots of pitfalls due to bad formatting and so on... Well, scraping.
  - rouxrc 12y ago
    It might not be necessary to scrap anything since this data was finally released as OpenData last year: https://www.data.gouv.fr/fr/datasets/legi-codes-lois-et-reglements-consolides/ https://www.data.gouv.fr/fr/datasets/legi-codes-lois-et-regl...
    - steeve 12y ago
      i tried using this, but it seems it seems to be just a snapshot, not the full dataset with history (i wish i'm wrong, though)
antimagic 12y ago
So how do you go about creating the commits like the example given? Even though I live in France I've not much looked at French law - the only exception being a few articles relating to immigration. In those examples, there are modifications made to preceding laws, and it seems to me that it would require strong AI to correctly update the original laws as required by the new law. The example in the README gives the impression that we can just pull from Git and get the updated version of the whole law, is that correct? If so, how?!?!
- steeve 12y ago
  It's because legifrance carries snapshots of the articles. Git is very good at tracking the changes between them. So for earch article, the codes tracks "when" the article changed, stores it in a hashtable, and replays the "time" (date) for each snapshot, and creates a commit each time.
ponytech 12y ago
Wonderful! When will you do the same for the "Code Général des Impots" ? :)
- steeve 12y ago
  Really soon, working on penal code right now (and open sourcing the scraper)

I just woke up and wow. Well, I'd be happy to answer any questions you might have of course.

by steeve 12y ago

I just woke up and wow. Well, I'd be happy to answer any questions you might have of course.

How does it feed the data to the repo? Is it parsing reports from the parliament? If yes, how smooth did the process go?

steeve 12y ago
It goes on each article on Legifrance, creates a JSON file with all the articles, which book/section they belong to, and for each article, tracks it's versions (they are dates). The crawler is in Go. Then there is a python script that takes that JSON file, creates the .md files and runs the git commands in the shell. Ultimately the sad thing is that I had to scrape this information. There were lots of pitfalls due to bad formatting and so on... Well, scraping.
- rouxrc 12y ago
  It might not be necessary to scrap anything since this data was finally released as OpenData last year: https://www.data.gouv.fr/fr/datasets/legi-codes-lois-et-reglements-consolides/ https://www.data.gouv.fr/fr/datasets/legi-codes-lois-et-regl...
  - steeve 12y ago
    i tried using this, but it seems it seems to be just a snapshot, not the full dataset with history (i wish i'm wrong, though)

antimagic 12y ago

So how do you go about creating the commits like the example given? Even though I live in France I've not much looked at French law - the only exception being a few articles relating to immigration. In those examples, there are modifications made to preceding laws, and it seems to me that it would require strong AI to correctly update the original laws as required by the new law. The example in the README gives the impression that we can just pull from Git and get the updated version of the whole law, is that correct? If so, how?!?!

steeve 12y ago
It's because legifrance carries snapshots of the articles. Git is very good at tracking the changes between them. So for earch article, the codes tracks "when" the article changed, stores it in a hashtable, and replays the "time" (date) for each snapshot, and creates a commit each time.

ponytech 12y ago

Wonderful! When will you do the same for the "Code Général des Impots" ? :)

steeve 12y ago
Really soon, working on penal code right now (and open sourcing the scraper)