8 ms·

I'd like to think I'm one of the good guys here. I was one of the inventors of RSS and Atom and worked to push open content and social media. I started a socia

I'd like to think I'm one of the good guys here. I was one of the inventors of RSS and Atom and worked to push open content and social media. I started a social data search platform named Datastreamer (http://www.datastreamer.io/ http://www.datastreamer.io/) which is basically a petabyte-scale content indexing engine. We provide API feeds to search engines and social media analytics companies needing bulk data but don't want to have to build a crawler. For the last 5 years we've had major problems with customers coming to us asking for data which we felt was unethical (at best). We actually had Saudi Arabia approach us... It was clear that they were intending to something pretty evil with the data. Their RFP questions were a bit frightening: - can you track people by religion? - can you give us their email address? - can you provide their address? - can your provide their ethnicity? - can you provide their social connections? We're actually losing business to other companies that are performing highly unethical and probably illegal techniques. We just can't compete with data at that type of fidelity. If you're a researcher and you want to access bulk data for combating this type of non-sense WE WILL PROVIDE DATA AT COST. We can provide up to 1PB of data but for now we have to charge for the shipping and handling of that data. We're reaching out to some other companies like Google and also the Internet Archive to see if we can provide more cost effective solutions. I'm working on more tools to give the power back to the users. Polar (https://getpolarized.io/ https://getpolarized.io/) is a web browser which allows people to control their own data. The idea is that I can keep a local repository of data and eventually build our own cloud platform based on open systems like IPFS and encrypt the data using group encryption.

microcolonel 8y ago
Ultimately, I think there are far too many actors involved to rely on the ethics of third parties to protect your privacy. If Saudi Arabia wanted to track down gays and apostates by mining the same data you did, they could do it themselves. The first line of defense against this is, and always will be, not publishing so much information about yourself. They can not mine what you do not provide. We can argue all day about the ethics of the conduct of companies and states looking at these data, but it's a non-issue if the data simply don't exist.
- Viliam1234 8y ago
  Even if you publish nothing about yourself, information will leak. Some of your friends will post things about you... for example they have a photo together with you, will upload it to a social network, enter your name, provide a date and place for the photo... with enough photos with your name, the social network can make a guess about which other unnamed photos contain the same person... and gradually a profile on you is being built. And it will only get worse, as more data will be shared by other people. Soon the streets may be full of people using cameras all day long, just because it allows them to make cooler diaries or blogs, and then the data companies will get universal surveillance. The only way to keep privacy will be to have your face covered in public (but hey, that will be made illegal, because terrorism or something), or avoid public places completely.
wpdev_63 8y ago
What other companies?
3pt14159 8y ago
First, thank you so much for the work you've done with RSS / Atom. I implemented it (and my own static site compiler) myself from scratch on my own personal site and it took me very little time. The only little trick was figuring out absolute vs relative paths. Second, I've been fighting for regulations for over three years and I'm getting somewhere, but I'm also starting to think that we need technical solutions to many of these problems. One of the things I see as a problem is that people always want more, but privacy and security often require less. For example, old charsets only supported Latin characters. With the introduction of Cyrillic characters many assumptions started breaking. Each time I try to think through how to make the web / internet simpler I realize that it either requires pushing the complexity onto people unprepared to deal with it—an English speaking daughter may want to copy and paste her Russian mother's Cyrillic name, say—or it fails to handle the use cases we need it to handle. I know this seems kinda abstract, but do you ever think about that interplay? Are there any insights or anecdotes you find useful?
burtonator 8y ago
By other companies I mean companies like Cambridge Analytica. There are tons of other shady companies out there doing nefarious things with your data. We're going to need platforms that support group encryption and better security for apps.
- deleted 8y ago
  [deleted]
- simonebrunozzi 8y ago
  As a person living in the current world, I want to thank you for both standing up against these kind of requests, and for leaving money on the table, which other companies like Cambridge Analitica easily grabbed. I don't know if there's any way for people/companies like you to defend against unscrupulous companies. I can only say thank you.
  - vfulco2 8y ago
    I second the hat tip and gratitude
- ericlnu 8y ago
  Thank you very much for being awesome
- indigochill 8y ago
  >We're going to need platforms that support group encryption and better security for apps. Definitely. Though I also feel we need to give a shock to public awareness of just how evil people can be with this data. My perception is that people are trending towards "vaguely uncomfortable" with the news of foreign interference with targeted ads, election hacking, and so on, but we've a ways to go yet before most will give up supremely-engineered convenience in exchange for security. As long as the modern world is democratic and the voting populace is subjecting itself to targeted manipulation by data-armed bad actors, we have a problem of not just national security, but international security. Thanks for being one of the good guys.
kodablah 8y ago
> We're actually losing business to other companies that are performing highly unethical and probably illegal techniques. And you always will. Principles and money are often at odds. This is not an easily fixable problem as the well-intended solutions often cause more problems. Enforcement of existing statutes and accepting legal-yet-unethical practices is unfortunately the most rational approach. > We just can't compete with data at that type of fidelity. Only in a situation where interminable growth is required in a race to the top. Otherwise, there's room for everyone and that's why there are thousands of software products that "compete" just fine. The key is just making sure there are platforms that allow everyone to build everything.
- burtonator 8y ago
  > Only in a situation where interminable growth is required in a race to the top. The key is that they choke you out... They have more engineers, more R&D and a better product because they have more revenue.
  - kodablah 8y ago
    If they have a better product, so be it. But I'm not convinced that more engineers and more R&D necessarily lead to that, nor am I convinced that smaller, more niche products are always choked out. If that were the case, we'd only have huge companies and a bunch of choked out entrepreneurs. Rather, a bunch of us out here with small companies are doing just fine. Definitely not a give up situation like "we just can't compete". That mentality gives rise to nuclear options with unintended consequences.
    - islandowl 8y ago
      This is very true. I have been a paying customer of Fastmail for many years, and in my opinion, they are the best in the industry and very small to boot. OpenBSD is yet another example of a small(ish) team of people making some truly great software. On the Windows side, Fookes Software comes to mind, again small operation, great software.
  - 8y ago
da_chicken 8y ago
And people wonder why I've argued that, at some point, IT needs to be a certified and regulated profession like lawyers, doctors, engineers, nurses, pilots, accountants, electricians, plumbers, teachers, etc. If the future is going to be built using software and data, it's probably going to be essential that the general public have some agency -- public or private -- to safeguard the public interest. It takes years of dedication to be a professional electrician or plumber, but anybody off the street can build an application that aggregates personal information and isn't subject to any sort of regulation or oversight or system of professional ethics. That doesn't just safeguard the public. It also safeguards the employed individuals when they say, "No, that is not allowed under the ethics of my profession." This isn't about "let's make the government save us," it's about creating a legal framework to protect the interests of the public and give ethical considerations in data management and software design some legal backing.
- 52-6F-62 8y ago
  I generally like the idea— it's probably a sign of maturity. I'd add that it would be nice to be able to operate outside of certain fields and certain types of operations without that level of certification. For instance, freelance web devs, small business software employees, etc who aren't dealing in things like personal or trivial data could continue operation. For example: you don't need to be a doctor to be certified in first aid, or even administer first aid—but you likely wouldn't attempt an invasive, life-threatening surgery. I'd also like to see—if that kind of regulation were to pass—the inclusion of some kind of grandfather clause that would include the ability to test without formal education. The reason being there are very many highly capable developers/engineers in the field who don't possess the exact formal background—and in many cases came from other formal backgrounds. I'd definitely hear arguments for not requiring education at any time, but to keep it on par with the other professions you listed I'll leave it as is. This might not be the thread for a larger discussion on this—because it seems like it would be a larger discussion. An interesting one, though... Implementation seems like it would be a challenge, but then again I don't know the stories behind who more modern professions like electricians were regulated. I imagine that field grew much more slowly.
  - 8y ago
blihp 8y ago
Despite your intentions, I don't think it's possible for you to be one of the good guys. The mass collection and aggregation of the data in the first place is the problem, not merely a few bad actors downstream.
- sangnoir 8y ago
  It's very easy to assign blame in hindsight. "Web 2.0" (i.e. data aggregation) was a fantastic tool and most of us did not have the imagination to see how it would be abused (just as any tool can be abused).
  - blihp 8y ago
    It was very easy to see where this was going from the very beginning and many did raise red flags but were largely disregarded because... FREE! There was plenty of user generated content for decades before Facebook/Google/etc came along (i.e. mailing lists/Usenet/forums/wikis/etc.) and one of the ways some chose to mitigate the risk was behind pseudonyms etc... which the big data collectors/aggregators have a problem with. Those who started requiring 'real names' and other personal details who then integrated disparate data sources to build profiles on users knew exactly what they were doing from day one... it's always been their business model. That said, I have no issue with anyone who voluntarily trades their personal information for access to a service. That's their choice to make. But it also seems reasonable that there be full disclosure as to the scope and scale of the deal they're making so they can make an informed choice. This isn't even remotely the case today.
  - rhizome 8y ago
    You don't need an imagination, there is at least one negative example in living memory of putting data aggregation to work against a mass populace.
spirodonfl 8y ago
If I wanted to offer whatever programming chops I have and support one of the initiatives you're working on, what's the best way for me to do that? I appreciate all your work and contributions to the community in general. Please keep your chin up and keep moving forward.
bfuller 8y ago
Thank you.
chickenfries 8y ago
Thanks for doing the right thing.
distortednet 8y ago
IPFS is dope as shit, and aims to be much more than what it advertises on the surface.

microcolonel 8y ago

Ultimately, I think there are far too many actors involved to rely on the ethics of third parties to protect your privacy. If Saudi Arabia wanted to track down gays and apostates by mining the same data you did, they could do it themselves. The first line of defense against this is, and always will be, not publishing so much information about yourself. They can not mine what you do not provide. We can argue all day about the ethics of the conduct of companies and states looking at these data, but it's a non-issue if the data simply don't exist.

Viliam1234 8y ago
Even if you publish nothing about yourself, information will leak. Some of your friends will post things about you... for example they have a photo together with you, will upload it to a social network, enter your name, provide a date and place for the photo... with enough photos with your name, the social network can make a guess about which other unnamed photos contain the same person... and gradually a profile on you is being built. And it will only get worse, as more data will be shared by other people. Soon the streets may be full of people using cameras all day long, just because it allows them to make cooler diaries or blogs, and then the data companies will get universal surveillance. The only way to keep privacy will be to have your face covered in public (but hey, that will be made illegal, because terrorism or something), or avoid public places completely.

wpdev_63 8y ago

What other companies?

3pt14159 8y ago

First, thank you so much for the work you've done with RSS / Atom. I implemented it (and my own static site compiler) myself from scratch on my own personal site and it took me very little time. The only little trick was figuring out absolute vs relative paths. Second, I've been fighting for regulations for over three years and I'm getting somewhere, but I'm also starting to think that we need technical solutions to many of these problems. One of the things I see as a problem is that people always want more, but privacy and security often require less. For example, old charsets only supported Latin characters. With the introduction of Cyrillic characters many assumptions started breaking. Each time I try to think through how to make the web / internet simpler I realize that it either requires pushing the complexity onto people unprepared to deal with it—an English speaking daughter may want to copy and paste her Russian mother's Cyrillic name, say—or it fails to handle the use cases we need it to handle. I know this seems kinda abstract, but do you ever think about that interplay? Are there any insights or anecdotes you find useful?

burtonator 8y ago

By other companies I mean companies like Cambridge Analytica. There are tons of other shady companies out there doing nefarious things with your data. We're going to need platforms that support group encryption and better security for apps.

deleted 8y ago
[deleted]
simonebrunozzi 8y ago
As a person living in the current world, I want to thank you for both standing up against these kind of requests, and for leaving money on the table, which other companies like Cambridge Analitica easily grabbed. I don't know if there's any way for people/companies like you to defend against unscrupulous companies. I can only say thank you.
- vfulco2 8y ago
  I second the hat tip and gratitude
ericlnu 8y ago
Thank you very much for being awesome
indigochill 8y ago
>We're going to need platforms that support group encryption and better security for apps. Definitely. Though I also feel we need to give a shock to public awareness of just how evil people can be with this data. My perception is that people are trending towards "vaguely uncomfortable" with the news of foreign interference with targeted ads, election hacking, and so on, but we've a ways to go yet before most will give up supremely-engineered convenience in exchange for security. As long as the modern world is democratic and the voting populace is subjecting itself to targeted manipulation by data-armed bad actors, we have a problem of not just national security, but international security. Thanks for being one of the good guys.

kodablah 8y ago

> We're actually losing business to other companies that are performing highly unethical and probably illegal techniques. And you always will. Principles and money are often at odds. This is not an easily fixable problem as the well-intended solutions often cause more problems. Enforcement of existing statutes and accepting legal-yet-unethical practices is unfortunately the most rational approach. > We just can't compete with data at that type of fidelity. Only in a situation where interminable growth is required in a race to the top. Otherwise, there's room for everyone and that's why there are thousands of software products that "compete" just fine. The key is just making sure there are platforms that allow everyone to build everything.

burtonator 8y ago
> Only in a situation where interminable growth is required in a race to the top. The key is that they choke you out... They have more engineers, more R&D and a better product because they have more revenue.
- kodablah 8y ago
  If they have a better product, so be it. But I'm not convinced that more engineers and more R&D necessarily lead to that, nor am I convinced that smaller, more niche products are always choked out. If that were the case, we'd only have huge companies and a bunch of choked out entrepreneurs. Rather, a bunch of us out here with small companies are doing just fine. Definitely not a give up situation like "we just can't compete". That mentality gives rise to nuclear options with unintended consequences.
  - islandowl 8y ago
    This is very true. I have been a paying customer of Fastmail for many years, and in my opinion, they are the best in the industry and very small to boot. OpenBSD is yet another example of a small(ish) team of people making some truly great software. On the Windows side, Fookes Software comes to mind, again small operation, great software.
- 8y ago

da_chicken 8y ago

And people wonder why I've argued that, at some point, IT needs to be a certified and regulated profession like lawyers, doctors, engineers, nurses, pilots, accountants, electricians, plumbers, teachers, etc. If the future is going to be built using software and data, it's probably going to be essential that the general public have some agency -- public or private -- to safeguard the public interest. It takes years of dedication to be a professional electrician or plumber, but anybody off the street can build an application that aggregates personal information and isn't subject to any sort of regulation or oversight or system of professional ethics. That doesn't just safeguard the public. It also safeguards the employed individuals when they say, "No, that is not allowed under the ethics of my profession." This isn't about "let's make the government save us," it's about creating a legal framework to protect the interests of the public and give ethical considerations in data management and software design some legal backing.

52-6F-62 8y ago
I generally like the idea— it's probably a sign of maturity. I'd add that it would be nice to be able to operate outside of certain fields and certain types of operations without that level of certification. For instance, freelance web devs, small business software employees, etc who aren't dealing in things like personal or trivial data could continue operation. For example: you don't need to be a doctor to be certified in first aid, or even administer first aid—but you likely wouldn't attempt an invasive, life-threatening surgery. I'd also like to see—if that kind of regulation were to pass—the inclusion of some kind of grandfather clause that would include the ability to test without formal education. The reason being there are very many highly capable developers/engineers in the field who don't possess the exact formal background—and in many cases came from other formal backgrounds. I'd definitely hear arguments for not requiring education at any time, but to keep it on par with the other professions you listed I'll leave it as is. This might not be the thread for a larger discussion on this—because it seems like it would be a larger discussion. An interesting one, though... Implementation seems like it would be a challenge, but then again I don't know the stories behind who more modern professions like electricians were regulated. I imagine that field grew much more slowly.
- 8y ago

blihp 8y ago

Despite your intentions, I don't think it's possible for you to be one of the good guys. The mass collection and aggregation of the data in the first place is the problem, not merely a few bad actors downstream.

sangnoir 8y ago
It's very easy to assign blame in hindsight. "Web 2.0" (i.e. data aggregation) was a fantastic tool and most of us did not have the imagination to see how it would be abused (just as any tool can be abused).
- blihp 8y ago
  It was very easy to see where this was going from the very beginning and many did raise red flags but were largely disregarded because... FREE! There was plenty of user generated content for decades before Facebook/Google/etc came along (i.e. mailing lists/Usenet/forums/wikis/etc.) and one of the ways some chose to mitigate the risk was behind pseudonyms etc... which the big data collectors/aggregators have a problem with. Those who started requiring 'real names' and other personal details who then integrated disparate data sources to build profiles on users knew exactly what they were doing from day one... it's always been their business model. That said, I have no issue with anyone who voluntarily trades their personal information for access to a service. That's their choice to make. But it also seems reasonable that there be full disclosure as to the scope and scale of the deal they're making so they can make an informed choice. This isn't even remotely the case today.
- rhizome 8y ago
  You don't need an imagination, there is at least one negative example in living memory of putting data aggregation to work against a mass populace.

spirodonfl 8y ago

If I wanted to offer whatever programming chops I have and support one of the initiatives you're working on, what's the best way for me to do that? I appreciate all your work and contributions to the community in general. Please keep your chin up and keep moving forward.

bfuller 8y ago

Thank you.

chickenfries 8y ago

Thanks for doing the right thing.

distortednet 8y ago

IPFS is dope as shit, and aims to be much more than what it advertises on the surface.