4 ms·

GDPR made searchable by Algolia

158 points by vvoyer 8y ago

amelius 8y ago
This makes me wonder how Algolia deals with GDPR. For instance, if a user requests their data to be removed, how fast can this be reflected in Algolia's indices, and in the instant-search?
- deleted 8y ago
  [deleted]
- redox_ 8y ago
  Indexing is an asynchronous process so while the data won't be deleted as soon as the deletion API call returns, the data will be deleted shortly after. Usually we're talking about hundreds of milliseconds or a few seconds. This is pretty much the same for all search-engines. Algolia also exposes a "task" API endpoint to check the state of an indexing operation and to be able to "wait" until the operation has actually been processed. This is for instance what is used in unit tests where we need to wait until the indexing operation has been processed to continue testing. Then for GDPR; if one of Algolia's customers receives a "personal data deletion" request on their side, it's of course their responsibility to forward it to Algolia using the deletion API (and a few seconds later it will be removed from all indices).
- jacquesm 8y ago
  And for an encore: Does Algolia have a data processing agreement in place with HN?
  - the_mitsuhiko 8y ago
    You would need to ask YC to disclose this. They have a dpa ready to sign.
jacquesm 8y ago
This: https://ico.org.uk/for-organisations/guide-to-the-general-data-protection-regulation-gdpr/ https://ico.org.uk/for-organisations/guide-to-the-general-da... Also has a search function and is a bit closer to the horses mouth.
nrjames 8y ago
This is great and would be even better if it included the official adopted guidance from the working groups. For example, this is a very useful document: http://ec.europa.eu/newsroom/article29/item-detail.cfm?item_id=622227 http://ec.europa.eu/newsroom/article29/item-detail.cfm?item_...
maxyme 8y ago
Not sure if Algolia doesn't generally work on mobile but this page is unusable on my Galaxy S8 in Chrome. See https://imgur.com/a/SEP6qdo https://imgur.com/a/SEP6qdo
- iampims 8y ago
  Same on iOS. It’s unusable.
- acobster 8y ago
  Yeah that's what it looks like on Android Firefox too. Algolia powers HN search and it's always worked fine for me.
- redox_ 8y ago
  Thanks for sharing! Responsive design fixes are on their ways.
ThePhysicist 8y ago
If you want the law in a machine-readable format (YAML / JSON), we open-sourced our GDPR portal which includes parsers and raw data for the entire law as well as all recitals and footnotes in German as well as English: https://github.com/DPKit/gdpr-portal https://github.com/DPKit/gdpr-portal The website itself is available here: https://gdpr.dpkit.com https://gdpr.dpkit.com (EN) https://dsgvo.dpkit.com https://dsgvo.dpkit.com (DE) (works well on mobile too) This is a work-in-progress, please let us know if you have any questions.
AlphaWeaver 8y ago
This is great! I love how it automatically pulls full definitions for terms from the relevant "Article 4. Definitions" section and surfaces them throughout the text. I would love to see more legislation made available in this easy to understand format!
jahewson 8y ago
This is nice. Though it’s worth pointing out to UX folks that flags are not languages. http://www.flagsarenotlanguages.com/blog/why-flags-do-not-represent-language/ http://www.flagsarenotlanguages.com/blog/why-flags-do-not-re...
- fixermark 8y ago
  Oh, darn. I read the domain name and thought it'd be a discussion about how forcing users to memorize two-hundred command-line switches is an extremely poor substitute for an actual domain-specific language. ;)
- hopeless 8y ago
  Particularly when the wrong flag is used. In this case, Gaelige is given the Scottish flag when it's actually the official language of Ireland in the EU. Scots and Irish Gaelige are not the same language. And Scotland is not an EU member (it's membership, sadly, is by virtue of the UK's membership)
- bryanrasmussen 8y ago
  Although I agree flags are not languages in the case of English different variants are recognized, so if you used an American Flag as your language indicator I would expect I would see a site written in EN-US and would only be confused if color were written as colour.
shevu 8y ago
Is the code for this website opensourced?
ropeadopepope 8y ago
This is awesome. Any chance algolia can make the whole of American Jurisprudence searchable? :D
james_in_the_uk 8y ago
Great work! UK charity The Society for Computers and Law is running a hackathon with the ICO in June in case anyone is interested in helping bring more tech to GDPR. Details at https://www.scl.org https://www.scl.org
- james_in_the_uk 8y ago
  Sorry full link : https://www.scl.org/events/525-scl-data-protection-hackathon-saturday-16-june-2018-london https://www.scl.org/events/525-scl-data-protection-hackathon...
daguire 8y ago
The presentation here is the most user friendly I have encountered so far: https://gdpr-info.eu/ https://gdpr-info.eu/
bryanrasmussen 8y ago
I went in and searched for children, so I can see there is not even English language stemming going on - which I mean that is the most common language to find a stemmer from in my experience. I used to work for Thomson Reuters on their legal services products and while it is nice seeing something like this outside of a paid for service it's not that impressive. Maybe I'm jaded but from the comments here I was expecting to say 'wow, that is cool' and not immediately find that stemming didn't work. on edit: fixed misspelling, formatting

158 points by vvoyer 8y ago

This makes me wonder how Algolia deals with GDPR. For instance, if a user requests their data to be removed, how fast can this be reflected in Algolia's indices, and in the instant-search?

deleted 8y ago
[deleted]
redox_ 8y ago
Indexing is an asynchronous process so while the data won't be deleted as soon as the deletion API call returns, the data will be deleted shortly after. Usually we're talking about hundreds of milliseconds or a few seconds. This is pretty much the same for all search-engines. Algolia also exposes a "task" API endpoint to check the state of an indexing operation and to be able to "wait" until the operation has actually been processed. This is for instance what is used in unit tests where we need to wait until the indexing operation has been processed to continue testing. Then for GDPR; if one of Algolia's customers receives a "personal data deletion" request on their side, it's of course their responsibility to forward it to Algolia using the deletion API (and a few seconds later it will be removed from all indices).
jacquesm 8y ago
And for an encore: Does Algolia have a data processing agreement in place with HN?
- the_mitsuhiko 8y ago
  You would need to ask YC to disclose this. They have a dpa ready to sign.

jacquesm 8y ago

This: https://ico.org.uk/for-organisations/guide-to-the-general-data-protection-regulation-gdpr/ https://ico.org.uk/for-organisations/guide-to-the-general-da... Also has a search function and is a bit closer to the horses mouth.

nrjames 8y ago

This is great and would be even better if it included the official adopted guidance from the working groups. For example, this is a very useful document: http://ec.europa.eu/newsroom/article29/item-detail.cfm?item_id=622227 http://ec.europa.eu/newsroom/article29/item-detail.cfm?item_...

maxyme 8y ago

Not sure if Algolia doesn't generally work on mobile but this page is unusable on my Galaxy S8 in Chrome. See https://imgur.com/a/SEP6qdo https://imgur.com/a/SEP6qdo

iampims 8y ago
Same on iOS. It’s unusable.
acobster 8y ago
Yeah that's what it looks like on Android Firefox too. Algolia powers HN search and it's always worked fine for me.
redox_ 8y ago
Thanks for sharing! Responsive design fixes are on their ways.

ThePhysicist 8y ago

If you want the law in a machine-readable format (YAML / JSON), we open-sourced our GDPR portal which includes parsers and raw data for the entire law as well as all recitals and footnotes in German as well as English: https://github.com/DPKit/gdpr-portal https://github.com/DPKit/gdpr-portal The website itself is available here: https://gdpr.dpkit.com https://gdpr.dpkit.com (EN) https://dsgvo.dpkit.com https://dsgvo.dpkit.com (DE) (works well on mobile too) This is a work-in-progress, please let us know if you have any questions.

AlphaWeaver 8y ago

This is great! I love how it automatically pulls full definitions for terms from the relevant "Article 4. Definitions" section and surfaces them throughout the text. I would love to see more legislation made available in this easy to understand format!

jahewson 8y ago

This is nice. Though it’s worth pointing out to UX folks that flags are not languages. http://www.flagsarenotlanguages.com/blog/why-flags-do-not-represent-language/ http://www.flagsarenotlanguages.com/blog/why-flags-do-not-re...

fixermark 8y ago
Oh, darn. I read the domain name and thought it'd be a discussion about how forcing users to memorize two-hundred command-line switches is an extremely poor substitute for an actual domain-specific language. ;)
hopeless 8y ago
Particularly when the wrong flag is used. In this case, Gaelige is given the Scottish flag when it's actually the official language of Ireland in the EU. Scots and Irish Gaelige are not the same language. And Scotland is not an EU member (it's membership, sadly, is by virtue of the UK's membership)
bryanrasmussen 8y ago
Although I agree flags are not languages in the case of English different variants are recognized, so if you used an American Flag as your language indicator I would expect I would see a site written in EN-US and would only be confused if color were written as colour.

shevu 8y ago

Is the code for this website opensourced?

ropeadopepope 8y ago

This is awesome. Any chance algolia can make the whole of American Jurisprudence searchable? :D

james_in_the_uk 8y ago

Great work! UK charity The Society for Computers and Law is running a hackathon with the ICO in June in case anyone is interested in helping bring more tech to GDPR. Details at https://www.scl.org https://www.scl.org

james_in_the_uk 8y ago
Sorry full link : https://www.scl.org/events/525-scl-data-protection-hackathon-saturday-16-june-2018-london https://www.scl.org/events/525-scl-data-protection-hackathon...

daguire 8y ago

The presentation here is the most user friendly I have encountered so far: https://gdpr-info.eu/ https://gdpr-info.eu/

bryanrasmussen 8y ago

I went in and searched for children, so I can see there is not even English language stemming going on - which I mean that is the most common language to find a stemmer from in my experience. I used to work for Thomson Reuters on their legal services products and while it is nice seeing something like this outside of a paid for service it's not that impressive. Maybe I'm jaded but from the comments here I was expecting to say 'wow, that is cool' and not immediately find that stemming didn't work. on edit: fixed misspelling, formatting