39 ms·

I buy data from LexisNexis. AMA.

1996 8y ago
What do you have from the healthcare part now? Drugs, diagnosis, lab test (from the like of labcorps), or just point of sale addresses because hipaa? How is the data linked? ssn, phone numbers? (just a best guess, in case you spotted some linking anomalies) Is the primary key still a number internal to lexis nexis? How detailed is the social network now? (just people living together and neighbours, or does it includes relatives and family members)
- 394549 8y ago
  > Drugs, diagnosis, lab test (from the like of labcorps), or just point of sale addresses? If they had access to that, wouldn't it be a HIPAA violation?
  - 1996 8y ago
    Yes, there are laws now. I edited my comment to mention that. Outside the US, labs are still a goldmine of data: time indexed numerical value that are used by providers to decide medical interventions. Except medical record or insurance history, you can't get any better than labs! Still I noticed in the US that labs selling directly to clients (like private md labs, only in some states) insist that the patient provide an accurate phone number. Given how phone numbers are used in the industry, I think they may want to keep the database accuracy for "future uses" when the law will change.
- staticautomatic 8y ago
  >What do you have from the healthcare part now? Drugs, diagnosis, lab test (from the like of labcorps), or just point of sale addresses because hipaa? Nothing. The data at issue in this article belongs to a category called "Non-FCRA" (Fair Credit Reporting Act), which means they cannot (really, are not supposed to) be used for things like credit and hiring decisions. Most of the data are from the public domain, although there are some data that are only quasi-public (meaning they're nominally public but you must have special privileges to see them). I haven't seen any truly non-public data. >How is the data linked? ssn, phone numbers? (just a best guess, in case you spotted some linking anomalies) ?Is the primary key still a number internal to lexis nexis? The primary key is an internal UID, where each person has his/her own. There are edge cases where people in their DB do not have UID's at all or where a single person has multiple UID's. They obviously have some linking but I've seen little indication that they have a robust graph under the hood. For example, Lexis may "know" that you've cohabitated with a member of the opposite sex about the same age as you for 10 years starting when you were 30, and also "know" that you're married to someone with the same name as the cohabitant, but fail to identify that person as your spouse because the marriage records are siloed in their back end. >How detailed is the social network now? (just people living together and neighbours, or does it includes relatives and family members) There are lots of data available on your relatives and family members, but again, the linkages are piss-poor. For example, your record may say that your name is "John Smith Jr." and you lived with a man who is 30 years older than you named "John Smith Sr." until you were 18, but fail to identify that person as your father.
  - 1996 8y ago
    Very interesting, thank you! I thought they would have improved the linkage since I last saw - especially the marriage records and name changes. They are still weak on that, so basic opsec will work.
394549 8y ago
What kind of bulk data products do they sell? I'm almost certain they are one of the companies that does the legwork to directly ingest public records information from government sources, but I'm unsure if they sell that bulk data on to other companies.
- staticautomatic 8y ago
  They have separate divisions for FCRA and non-FCRA data, so which products and data points you have access to depend upon who you are and what you're doing with the data. I believe the data at issue in this article are non-FCRA, which puts it in a category I'd call "enterprise-grade public records search" (as opposed to consumer-facing public records companies like Intelius and Instant Checkmate). Yes, they are one of the few companies that does the legwork of directly ingesting public records. For example, they buy and resell a lot of data from DMV agencies in states where the DMV is authorized to sell it (Google DPPA for more info). It is otherwise nearly impossible to get these data. I have tried, with the help of the right lawyers, and concluded that the only way to do it would be to sue the DMV.
  - 394549 8y ago
    > They have separate divisions for FCRA and non-FCRA data, so which products and data points you have access to depend upon who you are and what you're doing with the data. Do they still any of that data in bulk, or only in a productized form for a particular requested person or group of people (e.g. a company's customer list)? Just trying to get a sense of how extensively their data might be getting repackaged and sold by others.
    - staticautomatic 8y ago
      It depends on what sort of data broker you're talking about. For companies like Lexis, I'm pretty sure the answer is no. You can't just blindly buy a giant batch of data (this may have to do with regulatory requirements governing access to certain data points). However, you could totally do that with a marketing/"lifestyle" data broker like Acxiom, BlueKai, Epsilon, etc. I once directly asked a sales rep if they would sell me, say, all of the data they have on everyone in San Francisco County. Their answer was yes, with the caveat that it would be really expensive. Edit to answer: >Just trying to get a sense of how extensively their data might be getting repackaged and sold by others. Reselling is certainly not prohibited, provided the broker understands and approves of the purpose of the resale. I would guess that most consumer-facing background check tools are really just reselling data from one of the big public records brokers (Lexis, WestLaw, TLO, etc.). Some will just charge you higher monthly minimums for the privilege of reselling. The marketing/"lifestyle" data brokers are much stricter about prohibiting resale.
mindslight 8y ago
What is the pricing model? (eg per request?) Roughly what is that cost? Is there an "all-you-can-eat" access level that other organizations have? How granular are the data fields per record (/person) ? Do requesters generally cache the retrieved data on their own stores, continually re-retrieve it, do analysis in large batches, or what? Would we expect to see big caches of this data sitting around outside of Lexis's vaults?
- staticautomatic 8y ago
  > What is the pricing model? (eg per request?) Roughly what is that cost? Across all the products I've used (web, batch, API), the price is per-request (where a batch is multiple requests). The actual price can vary by several orders of magnitude depending on what you're requesting. If we're talking about list pricing, it ranges from about 10 cents to tens of dollars per request. There are significant volume discounts available, of course. >Is there an "all-you-can-eat" access level that other organizations have? I have not seen one myself and I doubt they offer one even to their largest enterprise customers. However, I am fairly certain that they offer it to certain government agencies. For example, I have seen federal government RFP's that require it. >How granular are the data fields per record (/person) ? It depends on the kind of response/report you're looking at, but as a general matter they are highly granular. For example, provided the data are available, I could itemize a list of every car you've ever owned by make/model/year, infer whether you're in a same-sex relationship, or see what the grounds were for granting your divorce. >Do requesters generally cache the retrieved data on their own stores, continually re-retrieve it, do analysis in large batches, or what? Aggressive caching is an absolute requirement for API users. That is proximately because Lexis's server does not "remember" your requests. They are logged and have unique transaction ID's but if you run the same search twice they will not check the request against a log cache to see if it's the same as one you ran recently. Instead they'll just charge you a second time (a very easy way to accidentally run up a big bill when running tests against a production endpoint). In combination with the high price per request, you'd be stupid not to cache the whole response. >Would we expect to see big caches of this data sitting around outside of their vaults? Yeah certainly. You can't arbitrarily run requests and store the data (e.g. in order to resell it), but you can store the results of requests you've made.

1996 8y ago

What do you have from the healthcare part now? Drugs, diagnosis, lab test (from the like of labcorps), or just point of sale addresses because hipaa? How is the data linked? ssn, phone numbers? (just a best guess, in case you spotted some linking anomalies) Is the primary key still a number internal to lexis nexis? How detailed is the social network now? (just people living together and neighbours, or does it includes relatives and family members)

394549 8y ago
> Drugs, diagnosis, lab test (from the like of labcorps), or just point of sale addresses? If they had access to that, wouldn't it be a HIPAA violation?
- 1996 8y ago
  Yes, there are laws now. I edited my comment to mention that. Outside the US, labs are still a goldmine of data: time indexed numerical value that are used by providers to decide medical interventions. Except medical record or insurance history, you can't get any better than labs! Still I noticed in the US that labs selling directly to clients (like private md labs, only in some states) insist that the patient provide an accurate phone number. Given how phone numbers are used in the industry, I think they may want to keep the database accuracy for "future uses" when the law will change.
staticautomatic 8y ago
>What do you have from the healthcare part now? Drugs, diagnosis, lab test (from the like of labcorps), or just point of sale addresses because hipaa? Nothing. The data at issue in this article belongs to a category called "Non-FCRA" (Fair Credit Reporting Act), which means they cannot (really, are not supposed to) be used for things like credit and hiring decisions. Most of the data are from the public domain, although there are some data that are only quasi-public (meaning they're nominally public but you must have special privileges to see them). I haven't seen any truly non-public data. >How is the data linked? ssn, phone numbers? (just a best guess, in case you spotted some linking anomalies) ?Is the primary key still a number internal to lexis nexis? The primary key is an internal UID, where each person has his/her own. There are edge cases where people in their DB do not have UID's at all or where a single person has multiple UID's. They obviously have some linking but I've seen little indication that they have a robust graph under the hood. For example, Lexis may "know" that you've cohabitated with a member of the opposite sex about the same age as you for 10 years starting when you were 30, and also "know" that you're married to someone with the same name as the cohabitant, but fail to identify that person as your spouse because the marriage records are siloed in their back end. >How detailed is the social network now? (just people living together and neighbours, or does it includes relatives and family members) There are lots of data available on your relatives and family members, but again, the linkages are piss-poor. For example, your record may say that your name is "John Smith Jr." and you lived with a man who is 30 years older than you named "John Smith Sr." until you were 18, but fail to identify that person as your father.
- 1996 8y ago
  Very interesting, thank you! I thought they would have improved the linkage since I last saw - especially the marriage records and name changes. They are still weak on that, so basic opsec will work.

394549 8y ago

What kind of bulk data products do they sell? I'm almost certain they are one of the companies that does the legwork to directly ingest public records information from government sources, but I'm unsure if they sell that bulk data on to other companies.

staticautomatic 8y ago
They have separate divisions for FCRA and non-FCRA data, so which products and data points you have access to depend upon who you are and what you're doing with the data. I believe the data at issue in this article are non-FCRA, which puts it in a category I'd call "enterprise-grade public records search" (as opposed to consumer-facing public records companies like Intelius and Instant Checkmate). Yes, they are one of the few companies that does the legwork of directly ingesting public records. For example, they buy and resell a lot of data from DMV agencies in states where the DMV is authorized to sell it (Google DPPA for more info). It is otherwise nearly impossible to get these data. I have tried, with the help of the right lawyers, and concluded that the only way to do it would be to sue the DMV.
- 394549 8y ago
  > They have separate divisions for FCRA and non-FCRA data, so which products and data points you have access to depend upon who you are and what you're doing with the data. Do they still any of that data in bulk, or only in a productized form for a particular requested person or group of people (e.g. a company's customer list)? Just trying to get a sense of how extensively their data might be getting repackaged and sold by others.
  - staticautomatic 8y ago
    It depends on what sort of data broker you're talking about. For companies like Lexis, I'm pretty sure the answer is no. You can't just blindly buy a giant batch of data (this may have to do with regulatory requirements governing access to certain data points). However, you could totally do that with a marketing/"lifestyle" data broker like Acxiom, BlueKai, Epsilon, etc. I once directly asked a sales rep if they would sell me, say, all of the data they have on everyone in San Francisco County. Their answer was yes, with the caveat that it would be really expensive. Edit to answer: >Just trying to get a sense of how extensively their data might be getting repackaged and sold by others. Reselling is certainly not prohibited, provided the broker understands and approves of the purpose of the resale. I would guess that most consumer-facing background check tools are really just reselling data from one of the big public records brokers (Lexis, WestLaw, TLO, etc.). Some will just charge you higher monthly minimums for the privilege of reselling. The marketing/"lifestyle" data brokers are much stricter about prohibiting resale.

mindslight 8y ago

What is the pricing model? (eg per request?) Roughly what is that cost? Is there an "all-you-can-eat" access level that other organizations have? How granular are the data fields per record (/person) ? Do requesters generally cache the retrieved data on their own stores, continually re-retrieve it, do analysis in large batches, or what? Would we expect to see big caches of this data sitting around outside of Lexis's vaults?

staticautomatic 8y ago
> What is the pricing model? (eg per request?) Roughly what is that cost? Across all the products I've used (web, batch, API), the price is per-request (where a batch is multiple requests). The actual price can vary by several orders of magnitude depending on what you're requesting. If we're talking about list pricing, it ranges from about 10 cents to tens of dollars per request. There are significant volume discounts available, of course. >Is there an "all-you-can-eat" access level that other organizations have? I have not seen one myself and I doubt they offer one even to their largest enterprise customers. However, I am fairly certain that they offer it to certain government agencies. For example, I have seen federal government RFP's that require it. >How granular are the data fields per record (/person) ? It depends on the kind of response/report you're looking at, but as a general matter they are highly granular. For example, provided the data are available, I could itemize a list of every car you've ever owned by make/model/year, infer whether you're in a same-sex relationship, or see what the grounds were for granting your divorce. >Do requesters generally cache the retrieved data on their own stores, continually re-retrieve it, do analysis in large batches, or what? Aggressive caching is an absolute requirement for API users. That is proximately because Lexis's server does not "remember" your requests. They are logged and have unique transaction ID's but if you run the same search twice they will not check the request against a log cache to see if it's the same as one you ran recently. Instead they'll just charge you a second time (a very easy way to accidentally run up a big bill when running tests against a production endpoint). In combination with the high price per request, you'd be stupid not to cache the whole response. >Would we expect to see big caches of this data sitting around outside of their vaults? Yeah certainly. You can't arbitrarily run requests and store the data (e.g. in order to resell it), but you can store the results of requests you've made.