16 ms·

Oh hi! That’s me! (And a bunch of other folks). Happy to answer any questions. I also really like this founders note that we wrote. It’s in our own words, etc:

by sayhar 5y ago

Oh hi! That’s me! (And a bunch of other folks). Happy to answer any questions. I also really like this founders note that we wrote. It’s in our own words, etc: https://integrityinstitute.org/founders-letter https://integrityinstitute.org/founders-letter We are recruiting new members! If you work on integrity / trust and safety / antispam / content quality / etc, let’s talk.

piva00 5y ago
Not working on integrity but really, really interested in seeing this coming alive. I've had a very deep curiosity about this topic for a couple of years, allegedly Facebook's products might have influenced the elections on my home country (Brazil), which has very directly impacted the quality of life of my family still living there. Looking forward to see what comes out of this and wishing you and the team all the best luck, thank you.
- ChrisMarshallNY 5y ago
  I am not working on integrity (I have a feeling that the integrity offices of many companies have brooms and buckets in them), but I do write software that Serves a constituency that has a very vested interest in the matter, and wish you well. I also have a personal code of ethics, and hold myself to a very high standard of Personal Integrity. In my experience, talking about that in the tech community does not end well. Ethics and Integrity do not seem to be popular topics for discussion in SV.
  - kwertyoowiyop 5y ago
    Corporate departments you do NOT want to be in: Innovation, Ethics, Trust… unless of course it’s corporate doublespeak and the department is doing the exact opposite of what its name implies.
    - Nevermark 5y ago
      Exactly. If "Innovation, Ethics, Trust" are being addressed by a department: 1. They have no profit and loss, so are going to get sidelined by the higher impact (in any short or medium term) concerns of every department that does. 2. They have no direct control over serious corporate activities, so are going to be considered a distraction at best, interference at worst, by others under their own pressures. If they are noticed at all. It isn't an accident when universal values are silo'd into departments from the rest of the company. It means leadership doesn't want the rest of the company to waste time or focus on them. But feel like the values need token recognition. -- It gets even worse if these were governmental departments. Department of Innovation: enforces acceptable methods and subjects of innovation, i.e. anti-innovation. Department of Ethics: Reinforces the governments views by justifying them as ethical, and demands lip service to those fictional high ethics. Department of Trust: You must trust us. Worst one of all: Department of Truth. Here it is. Don't look elsewhere.
- sayhar 5y ago
  Thank you! I helped set up the Brazil Election War Room -- the first election war room inside of FB. It was intense! There is a special place in my heart for your country <3
  - malandrew 5y ago
    This sounds like election interference by a foreign power. What business do employees in a US company have influencing the elections of a foreign country?
  - blitzar 5y ago
    More or less intense than the Myanmar Genocide War Room? Was it the first genocide war room at facebook or were there ones before it?
gwittel 5y ago
(1) Thank you so much for doing this. Facebook has been after me for years (> 15 years in anti-spam/email abuse). I keep putting it off, but its time for a clear "no thanks". As an outsider saying "no" - is there anything I can convey to them to drive the message home? (2) Minor - your "Join Us" link from the Founders Letter page is 404 -- https://integrityinstitute.org/join-us https://integrityinstitute.org/join-us
- sayhar 5y ago
  Thank you! (1) - I'm not sure. I don't think I'm an expert here. But if you've got 15 years of anti-spam experience, we'd love to have you join us as a member :-) (2) - Thank you. Fixed!
rajin444 5y ago
Are there any examples of sticking to your principles by standing up for socially and morally reprehensible groups? My initial impression is this is just another group pushing left leaning elite American values as "integrity". Things like this: https://www.aclu.org/other/aclu-history-taking-stand-free-speech-skokie https://www.aclu.org/other/aclu-history-taking-stand-free-sp... go a long way.
_bfhp 5y ago
Have you spoken or do you plan to speak out about the most normalized form of state-sponsored propaganda in the U.S., namely hasbara [1]? For example, Hasbara Fellowships and Israel on Campus Coalition [2] [1] https://en.wikipedia.org/wiki/Hasbara https://en.wikipedia.org/wiki/Hasbara [2] https://en.wikipedia.org/wiki/Israel_on_Campus_Coalition#Mission_and_activities https://en.wikipedia.org/wiki/Israel_on_Campus_Coalition#Mis...
zestyping 5y ago
I have a hypothesis. It goes like this. Today all the major text-based social platforms work in about the same way: we type text into an empty little box that's threaded under another box. There are variations in ranking and flagging, but the basic mechanism is unchanged. It is striking to me that, in all these years, we have explored only a tiny little corner of the design space. There are no mechanisms to lower the temperature when arguments get intense, for example. Nothing to help keep us from misconstruing comments out of context. Nothing to assist us in feeling compassion for the people we're talking to, or understanding their intent as they mean it to be understood. And so on. It's almost as though we sold millions of cars without brakes, everyone is crashing into things and hurting each other, and our response as a society is to throw up our hands and say "Welp, guess humans are just too stupid to drive cars safely." Many people have suggested eliminating engagement as a metric and going back to purely chronological feeds. That sounds pretty reasonable given where "engagement" has gotten us so far. But what if there were such a thing as "healthy engagement"? The hypothesis is that healthy engagement is achievable. I don't know whether it is, but there's a huge range of design possibilities that we have yet to explore. Do you know of anyone working on healthy engagement? Is it something you want to work on, or do you have any recommendations on starting an effort in this direction?
- sayhar 5y ago
  Hey this is a really good question. And I agree with you 100%! We have only explored a tiny corner of the design space. Since you asked for links, I think you'd like this talk I gave at Berkman a little while ago: https://cyber.harvard.edu/events/governing-social-media-city https://cyber.harvard.edu/events/governing-social-media-city It's about: if we think of social media as a new city, what are the alternatives to hiring tons of cops/censors? What about urban planning, bike lanes, etc? As for healthy engagement: I think there are a few people in this space. I honestly don't know as many as I'd like. This will be a learning experience for everyone. I think New Public might be doing good work, but I'm not sure! https://newpublic.org/ https://newpublic.org/ Hope that helps!
- rhizome 5y ago
  >Do you know of anyone working on healthy engagement? Wikipedia is one, by not "working on" it. >It's almost as though we sold millions of cars without brakes, everyone is crashing into things and hurting each other, and our response as a society is to throw up our hands and say "Welp, guess humans are just too stupid to drive cars safely." No, they have brakes, it's just that half of the car owners are punching people who work at Jiffy Lube and screaming that brakers should be killed. >But what if there were such a thing as "healthy engagement"? "Engagement" can not ever be healthy because "working on" it relies on manipulation. Curiosity implies an absence of manipulation and a maximum of arbitrary connections. You want healthy engagement? Create something well-loved that doesn't depend on or require any additional action from the appreciator for them to experience its complete impact. Oh shit, that's hard! "Well then how about I scare them into clicking on a link that pays me money when they do so, then use an image and/or words to call them inadequate in some aspect of their lives so that they send me more money?" Tomato-tomahto?
dogman144 5y ago
Many PMs and engineers and laymen knew FB was a rotten product for a long time. I also assume a lot of people at this institute didn’t take the Alex Stamos (ex, short term FB CISO) U-turn either. So… How is this effort not a virtue signaling by people that made their fortune on the back of this generation’s cigarettes, and now hoping to get traction on being listened to for fixing the mess they created?
- deleted 5y ago
  [deleted]
- shug2k 5y ago
  Integrity folks across the industry work on solving these problems, not creating them. It is more of a balance of whether to tackle the problems inside the organization, or outside - depending on which is more effective. What is not in question is that we should be spending time on tackling these problems.
  - SrslyJosh 5y ago
    > It is more of a balance of whether to tackle the problems inside the organization, or outside - depending on which is more effective. I think you answered your own question already.
  - dogman144 5y ago
    Of course. Regarding internal solutions: it’s impossible to overlook the significantly misaligned incentives facing integrity teams. They try to solve problems inherently created by the profit model of the company and what also pays their salary. The parallels to the internal scientist teams at Phillip Morris are uncanny. As I said, this paradox, and the company kicking the can down the road, has been obvious for years externally and also seemed internally per the Slack (or w/e) leaks post 1/6. So, for these teams, if they’re staying longer than a quick in and out once realities trumped ones idealism at “going where the problem is” (look, I really do get that impulse), continuing to claim the moral high ground in the way this Institute is so tone deaf. Sorry buddy - I’m on levels.FYI as well, we all know what you got paid to (increasingly ineffectually) support this product. Like there’s been leaks for years of PMs discussing Myanmar genocide inflammation via FB/WA for a few years now. Was that not enough to leave? This is inescapably how a lot of tech is going to judge this period/company. It went on for too long to claim ignorance otherwise. Edit: You know what, no it's not at all about an external vs. internal balance, that very much misses the point. It's likely that a meaningful change which preserves what FB (and related) intrinsically is will have to come from an internally-driven fix. The technical abstractions are just too much of a problem for outsiders to grok, then suggest a fix for, and etc. etc. ("Senator, we sell ads..."). It's about understanding where the moral and character capital that leadership, especially transformational leadership, comes from, and understanding how the founders of this have none of it. People that have stayed at FB and profited immensely from the experience, integrity team or not, stayed silent about it, and then start coming out with this publicly now vs. years ago (see: Alex Stamos' example), don't have that capital. And it's so distasteful to see them think that they do via efforts like this. That's the problem.
spywaregorilla 5y ago
How do you define integrity in this context. (you, personally)
- sayhar 5y ago
  This is a great question! I'm still trying to find a top-down definition instead of a "I know it when I see it" one. To me it's probably something like this: we can think about an information ecosystem or social platform as a system. "Normal" hacking of the system happens through finding loopholes code. (That's cybersecurity). "Integrity-related" hacking of the system happens through finding loopholes in design and rules. For some easy examples, that covers things like realizing you can post to 1000 groups in an hour. Or using sockpuppets to give artificial boosts to posts. The attackers aren't hacking code, but are hacking a system of rules, norms, and defaults on that system. (And often, finding the holes between where one part of the system was soldered onto the other). That's the technical part. Integrity also has a sort of ethical component. I think that's meaningful too.
  - spywaregorilla 5y ago
    What you describe sounds like you're there to counter user fraud. It's definitely not the first thing people imagine when they hear you're there to support integrity imo. Doesn't that feel misleading to you?
    - sayhar 5y ago
      We spent a good amount of space on the website laying out different types of work that are covered in the term "integrity". I also do very strongly think there is a moral/ethical dimension to the work. That's one reason I shy away from "Trust and Safety" label. Integrity work may have started with connotations of "structural integrity", but it also must be done carefully and ethically. It's not just engineering, it's also about doing the right thing. Hence the Hippocratic oath we all take. [1] [1] https://integrityinstitute.org/our-values https://integrityinstitute.org/our-values
  - dogman144 5y ago
tmule 5y ago
What’s the agenda here? (Fit the description, but am very skeptical of your motives :))
- sayhar 5y ago
  I think this should answer your question: https://integrityinstitute.org/founders-letter https://integrityinstitute.org/founders-letter But also, our values: https://integrityinstitute.org/our-values https://integrityinstitute.org/our-values Also, succinctly -- this is a real, grassroots thing. We gathered a bunch of friends and coworkers for this big idea of "what if we had best practices and a professional association for integrity work, just like we do for cybersecurity" and then worked for 10 months to do it. We've tried hard to put all kinds of pieces into place. We're making it a place that is both independent of companies and also safe for current employees of those companies to join. It's also cool that we can draw on this community to give expert advice to stakeholders (policymakers, journalists, companies, academics, etc). Mad that congress doesn't understand how Instagram works, or whatever? We can explain things -- as people whose training was in looking at the total information ecosystem of a platform. A goal is to have integrity work be at least as prestigious, valued, and essential as cybersecurity, or software engineering is. A thing where quality matters, and if you do shoddy work you will be called out on it. Does that help?
  - tmule 5y ago
    Thanks! The values resonate strongly (rarely practiced in industry though - power, the pursuit of glory, and partisanship can and do form a toxic combination).
  - malandrew 5y ago
    a statement of values is meaningless. What organizational procedures do you have in place to prevent some faction from hijacking the organization and morphing it into an ideological echo chamber. The values on that page are as mutable as the html they are written in. For example, in the past 10 to 20 years or so we've seen both the ADL and ACLU morph into institutions that would be unrecognizable to ADL or ACLU staffers from 10 to 20 years ago. No one ever imagined that they would abandoned their classically liberal values and replace those values with the illiberal "liberal" values they practice today.
btown 5y ago
What are the institute's thoughts, if you've formed them, on end-to-end encryption, especially as it applies to social media where the line between group and group-text blurs? I feel it's an incredibly nuanced topic that's become incredibly polarizing in recent days with some of Haugen's comments. On the one hand, in favor of E2EE, companies can and will use the content of messages, if they have access to them, to micro-target suggested content to users, and this can lead to increased levels of misinformation being promoted to people who have engaged with misinformation. And of course there's the government surveillance angle, which is an entirely separate story! But if you remove the signals in that content by encrypting in a way that is opaque to the platform, do you substantially reduce the ability to microtarget? Very possibly not, given the amount of graph data the social media company has anyways about group members independent from the content itself. And encryption gives the social media company the ability to wash its hands of any responsibility or awareness of content. Assuming it were easy to technically achieve (which is a huge leap, to be fair!) do you think it better serves the definition of integrity you've adopted, that a social media platform have the majority of its content end-to-end encrypted, or not?
- sayhar 5y ago
  Hi! Thanks for the constructive question. I agree with you that it's nuanced, and also I'm sad that it's getting polarized/simplified in some venues. I don't think we have gotten an institute stance on very much relating to E2EE. Our community advisory board (composed of integrity workers) is the moral core of the organization. So far, when we say "the institute has a stance on X", that has meant "the advisory board signs off on X, and that X represents a good faith consensus view of workers in the industry". We don't yet has a doc that lays out why we think integrity and privacy can coexist nicely. Speaking only for myself, I think the answer lies in careful design. As an example, you could see, via research and experimentation on FB Messenger, that messages that are forwarded in chains of > N are just empirically overwhelmingly likely to be bad faith, spammy, etc. You could then take that finding to WhatsApp, Signal, etc, and then bake in changes to the UX that make it slightly more annoying to forward messages if they've been on a reshare chain of N/M. That kind of stuff. There's also some consideration to group size -- if a group is 5000 people large on, say, Telegram, it might be encrypted, but it's no longer really private. Maybe it should be treated differently? Unclear, let's think and research about it. I think a rough consensus we might move towards is treating messaging differently than broadcast, and also treating broadcast features inside of messaging apps differently than straight up messaging themselves. But again, those are just some of my more idle thoughts. There are members and fellows who are better experts on this particular subject than I am. Does that make sense? Is that helpful?
smsm42 5y ago
How do you ensure that "integrity" does not morph into "ban everything our groupthink says is wrong"? How do you ensure you're not coopted into an ideological or partisan propaganda/speech control efforts - a trap that so many "fact checkers" gladly fallen into?
- shug2k 5y ago
  Transparency goes a long way towards addressing this concern. This is one area we are focused on - asking companies to be more transparent, both in their overall metrics and samples of public content, so we can have those debates as a society rather than behind closed doors.
  - smsm42 5y ago
    Transparency is not enough. If the decision is "we ban everything that the Troika we appointed finds not to its liking" - it's transparent, but not helpful and has nothing to do with "integrity". Transparency is good, but not enough.
- wolfram74 5y ago
  One content agnostic approach to improving signal to noise is the tendency for deep chain sharing to be mostly false/misleading content. Introducing a forced copy-paste hurdle after the chain gets, say, 4 deep, would be content agnostic, but probably improve the quality of what get's spread.
  - smsm42 5y ago
    > the tendency for deep chain sharing to be mostly false/misleading content Does it rely on some specific research? I mean I'm sure there's a lot of viral stories that are false. But there's also a lot of viral stories that are true - or at least no less true than what you'd commonly see on network TV or read in major newspapers, like NYT. I am not sure why it'd be obvious that something that is shared a lot is necessarily false.
  - sayhar 5y ago
neural_thing 5y ago
It's understandable that people go to work for Sauron to get paid big money. But starting a "human-orc ethics think tank" because you have relevant experience in the field is a bit rich.
- labster 5y ago
  We need to be able to forgive those who have done evil, if they truly repent. If they’re trying to do good now, we shouldn’t hold it against them that they worked for a monopoly spreading lies that incited violence, created online addicts, and took the pieces of silver happily. No, it only matters do good now.
  - hn_throwaway_99 5y ago
    I wholeheartedly agree. But, to be frank, I really don't see any amount of contrition in the Founders' Letter here, https://integrityinstitute.org/founders-letter https://integrityinstitute.org/founders-letter. Primarily, I don't see anything here that says "You know what, we're sorry, because we were part of the problem, despite working on 'the inside' trying to improve things." I'm not asking for the founders to fall on a proverbial sword, but I don't see any acknowledgement of how the entire Facebook "machine", which causes all the problems they point out in their letter, is only possible because legions of really smart people choose to work there.
  - dogman144 5y ago
    We can forgive them but not listen to their thought leadership on solving the post-Sauron world.
  - smsm42 5y ago
    Repenting is indeed a major step on the way from evil to not evil, but I do not see any sign of repentance in their materials. Repenting means recognizing you did wrong, recognizing and admitting, sincerely and honestly, what was wrong in what you did, and then building a corrective action on top of that. I do not see any of that. I only see a bunch of platitides, which I could find in any corporate "our values" binder, we're all for everything good and against everything bad. "Strive to be a good, honorable person" - ok, sure, who doesn't? Are there many people striving to be bad and dishonorable? That's not doing anything.
deleted 5y ago
[deleted]

by sayhar 5y ago

piva00 5y ago

Not working on integrity but really, really interested in seeing this coming alive. I've had a very deep curiosity about this topic for a couple of years, allegedly Facebook's products might have influenced the elections on my home country (Brazil), which has very directly impacted the quality of life of my family still living there. Looking forward to see what comes out of this and wishing you and the team all the best luck, thank you.

ChrisMarshallNY 5y ago
I am not working on integrity (I have a feeling that the integrity offices of many companies have brooms and buckets in them), but I do write software that Serves a constituency that has a very vested interest in the matter, and wish you well. I also have a personal code of ethics, and hold myself to a very high standard of Personal Integrity. In my experience, talking about that in the tech community does not end well. Ethics and Integrity do not seem to be popular topics for discussion in SV.
- kwertyoowiyop 5y ago
  Corporate departments you do NOT want to be in: Innovation, Ethics, Trust… unless of course it’s corporate doublespeak and the department is doing the exact opposite of what its name implies.
  - Nevermark 5y ago
    Exactly. If "Innovation, Ethics, Trust" are being addressed by a department: 1. They have no profit and loss, so are going to get sidelined by the higher impact (in any short or medium term) concerns of every department that does. 2. They have no direct control over serious corporate activities, so are going to be considered a distraction at best, interference at worst, by others under their own pressures. If they are noticed at all. It isn't an accident when universal values are silo'd into departments from the rest of the company. It means leadership doesn't want the rest of the company to waste time or focus on them. But feel like the values need token recognition. -- It gets even worse if these were governmental departments. Department of Innovation: enforces acceptable methods and subjects of innovation, i.e. anti-innovation. Department of Ethics: Reinforces the governments views by justifying them as ethical, and demands lip service to those fictional high ethics. Department of Trust: You must trust us. Worst one of all: Department of Truth. Here it is. Don't look elsewhere.
sayhar 5y ago
Thank you! I helped set up the Brazil Election War Room -- the first election war room inside of FB. It was intense! There is a special place in my heart for your country <3
- malandrew 5y ago
  This sounds like election interference by a foreign power. What business do employees in a US company have influencing the elections of a foreign country?
- blitzar 5y ago
  More or less intense than the Myanmar Genocide War Room? Was it the first genocide war room at facebook or were there ones before it?

gwittel 5y ago

(1) Thank you so much for doing this. Facebook has been after me for years (> 15 years in anti-spam/email abuse). I keep putting it off, but its time for a clear "no thanks". As an outsider saying "no" - is there anything I can convey to them to drive the message home? (2) Minor - your "Join Us" link from the Founders Letter page is 404 -- https://integrityinstitute.org/join-us https://integrityinstitute.org/join-us

sayhar 5y ago
Thank you! (1) - I'm not sure. I don't think I'm an expert here. But if you've got 15 years of anti-spam experience, we'd love to have you join us as a member :-) (2) - Thank you. Fixed!

rajin444 5y ago

Are there any examples of sticking to your principles by standing up for socially and morally reprehensible groups? My initial impression is this is just another group pushing left leaning elite American values as "integrity". Things like this: https://www.aclu.org/other/aclu-history-taking-stand-free-speech-skokie https://www.aclu.org/other/aclu-history-taking-stand-free-sp... go a long way.

_bfhp 5y ago

Have you spoken or do you plan to speak out about the most normalized form of state-sponsored propaganda in the U.S., namely hasbara [1]? For example, Hasbara Fellowships and Israel on Campus Coalition [2] [1] https://en.wikipedia.org/wiki/Hasbara https://en.wikipedia.org/wiki/Hasbara [2] https://en.wikipedia.org/wiki/Israel_on_Campus_Coalition#Mission_and_activities https://en.wikipedia.org/wiki/Israel_on_Campus_Coalition#Mis...

zestyping 5y ago

I have a hypothesis. It goes like this. Today all the major text-based social platforms work in about the same way: we type text into an empty little box that's threaded under another box. There are variations in ranking and flagging, but the basic mechanism is unchanged. It is striking to me that, in all these years, we have explored only a tiny little corner of the design space. There are no mechanisms to lower the temperature when arguments get intense, for example. Nothing to help keep us from misconstruing comments out of context. Nothing to assist us in feeling compassion for the people we're talking to, or understanding their intent as they mean it to be understood. And so on. It's almost as though we sold millions of cars without brakes, everyone is crashing into things and hurting each other, and our response as a society is to throw up our hands and say "Welp, guess humans are just too stupid to drive cars safely." Many people have suggested eliminating engagement as a metric and going back to purely chronological feeds. That sounds pretty reasonable given where "engagement" has gotten us so far. But what if there were such a thing as "healthy engagement"? The hypothesis is that healthy engagement is achievable. I don't know whether it is, but there's a huge range of design possibilities that we have yet to explore. Do you know of anyone working on healthy engagement? Is it something you want to work on, or do you have any recommendations on starting an effort in this direction?

sayhar 5y ago
Hey this is a really good question. And I agree with you 100%! We have only explored a tiny corner of the design space. Since you asked for links, I think you'd like this talk I gave at Berkman a little while ago: https://cyber.harvard.edu/events/governing-social-media-city https://cyber.harvard.edu/events/governing-social-media-city It's about: if we think of social media as a new city, what are the alternatives to hiring tons of cops/censors? What about urban planning, bike lanes, etc? As for healthy engagement: I think there are a few people in this space. I honestly don't know as many as I'd like. This will be a learning experience for everyone. I think New Public might be doing good work, but I'm not sure! https://newpublic.org/ https://newpublic.org/ Hope that helps!
rhizome 5y ago
>Do you know of anyone working on healthy engagement? Wikipedia is one, by not "working on" it. >It's almost as though we sold millions of cars without brakes, everyone is crashing into things and hurting each other, and our response as a society is to throw up our hands and say "Welp, guess humans are just too stupid to drive cars safely." No, they have brakes, it's just that half of the car owners are punching people who work at Jiffy Lube and screaming that brakers should be killed. >But what if there were such a thing as "healthy engagement"? "Engagement" can not ever be healthy because "working on" it relies on manipulation. Curiosity implies an absence of manipulation and a maximum of arbitrary connections. You want healthy engagement? Create something well-loved that doesn't depend on or require any additional action from the appreciator for them to experience its complete impact. Oh shit, that's hard! "Well then how about I scare them into clicking on a link that pays me money when they do so, then use an image and/or words to call them inadequate in some aspect of their lives so that they send me more money?" Tomato-tomahto?

dogman144 5y ago

Many PMs and engineers and laymen knew FB was a rotten product for a long time. I also assume a lot of people at this institute didn’t take the Alex Stamos (ex, short term FB CISO) U-turn either. So… How is this effort not a virtue signaling by people that made their fortune on the back of this generation’s cigarettes, and now hoping to get traction on being listened to for fixing the mess they created?

deleted 5y ago
[deleted]
shug2k 5y ago
Integrity folks across the industry work on solving these problems, not creating them. It is more of a balance of whether to tackle the problems inside the organization, or outside - depending on which is more effective. What is not in question is that we should be spending time on tackling these problems.
- SrslyJosh 5y ago
  > It is more of a balance of whether to tackle the problems inside the organization, or outside - depending on which is more effective. I think you answered your own question already.
- dogman144 5y ago
  Of course. Regarding internal solutions: it’s impossible to overlook the significantly misaligned incentives facing integrity teams. They try to solve problems inherently created by the profit model of the company and what also pays their salary. The parallels to the internal scientist teams at Phillip Morris are uncanny. As I said, this paradox, and the company kicking the can down the road, has been obvious for years externally and also seemed internally per the Slack (or w/e) leaks post 1/6. So, for these teams, if they’re staying longer than a quick in and out once realities trumped ones idealism at “going where the problem is” (look, I really do get that impulse), continuing to claim the moral high ground in the way this Institute is so tone deaf. Sorry buddy - I’m on levels.FYI as well, we all know what you got paid to (increasingly ineffectually) support this product. Like there’s been leaks for years of PMs discussing Myanmar genocide inflammation via FB/WA for a few years now. Was that not enough to leave? This is inescapably how a lot of tech is going to judge this period/company. It went on for too long to claim ignorance otherwise. Edit: You know what, no it's not at all about an external vs. internal balance, that very much misses the point. It's likely that a meaningful change which preserves what FB (and related) intrinsically is will have to come from an internally-driven fix. The technical abstractions are just too much of a problem for outsiders to grok, then suggest a fix for, and etc. etc. ("Senator, we sell ads..."). It's about understanding where the moral and character capital that leadership, especially transformational leadership, comes from, and understanding how the founders of this have none of it. People that have stayed at FB and profited immensely from the experience, integrity team or not, stayed silent about it, and then start coming out with this publicly now vs. years ago (see: Alex Stamos' example), don't have that capital. And it's so distasteful to see them think that they do via efforts like this. That's the problem.

spywaregorilla 5y ago

How do you define integrity in this context. (you, personally)

sayhar 5y ago
This is a great question! I'm still trying to find a top-down definition instead of a "I know it when I see it" one. To me it's probably something like this: we can think about an information ecosystem or social platform as a system. "Normal" hacking of the system happens through finding loopholes code. (That's cybersecurity). "Integrity-related" hacking of the system happens through finding loopholes in design and rules. For some easy examples, that covers things like realizing you can post to 1000 groups in an hour. Or using sockpuppets to give artificial boosts to posts. The attackers aren't hacking code, but are hacking a system of rules, norms, and defaults on that system. (And often, finding the holes between where one part of the system was soldered onto the other). That's the technical part. Integrity also has a sort of ethical component. I think that's meaningful too.
- spywaregorilla 5y ago
  What you describe sounds like you're there to counter user fraud. It's definitely not the first thing people imagine when they hear you're there to support integrity imo. Doesn't that feel misleading to you?
  - sayhar 5y ago
    We spent a good amount of space on the website laying out different types of work that are covered in the term "integrity". I also do very strongly think there is a moral/ethical dimension to the work. That's one reason I shy away from "Trust and Safety" label. Integrity work may have started with connotations of "structural integrity", but it also must be done carefully and ethically. It's not just engineering, it's also about doing the right thing. Hence the Hippocratic oath we all take. [1] [1] https://integrityinstitute.org/our-values https://integrityinstitute.org/our-values
- dogman144 5y ago

tmule 5y ago

What’s the agenda here? (Fit the description, but am very skeptical of your motives :))

sayhar 5y ago
I think this should answer your question: https://integrityinstitute.org/founders-letter https://integrityinstitute.org/founders-letter But also, our values: https://integrityinstitute.org/our-values https://integrityinstitute.org/our-values Also, succinctly -- this is a real, grassroots thing. We gathered a bunch of friends and coworkers for this big idea of "what if we had best practices and a professional association for integrity work, just like we do for cybersecurity" and then worked for 10 months to do it. We've tried hard to put all kinds of pieces into place. We're making it a place that is both independent of companies and also safe for current employees of those companies to join. It's also cool that we can draw on this community to give expert advice to stakeholders (policymakers, journalists, companies, academics, etc). Mad that congress doesn't understand how Instagram works, or whatever? We can explain things -- as people whose training was in looking at the total information ecosystem of a platform. A goal is to have integrity work be at least as prestigious, valued, and essential as cybersecurity, or software engineering is. A thing where quality matters, and if you do shoddy work you will be called out on it. Does that help?
- tmule 5y ago
  Thanks! The values resonate strongly (rarely practiced in industry though - power, the pursuit of glory, and partisanship can and do form a toxic combination).
- malandrew 5y ago
  a statement of values is meaningless. What organizational procedures do you have in place to prevent some faction from hijacking the organization and morphing it into an ideological echo chamber. The values on that page are as mutable as the html they are written in. For example, in the past 10 to 20 years or so we've seen both the ADL and ACLU morph into institutions that would be unrecognizable to ADL or ACLU staffers from 10 to 20 years ago. No one ever imagined that they would abandoned their classically liberal values and replace those values with the illiberal "liberal" values they practice today.

btown 5y ago

What are the institute's thoughts, if you've formed them, on end-to-end encryption, especially as it applies to social media where the line between group and group-text blurs? I feel it's an incredibly nuanced topic that's become incredibly polarizing in recent days with some of Haugen's comments. On the one hand, in favor of E2EE, companies can and will use the content of messages, if they have access to them, to micro-target suggested content to users, and this can lead to increased levels of misinformation being promoted to people who have engaged with misinformation. And of course there's the government surveillance angle, which is an entirely separate story! But if you remove the signals in that content by encrypting in a way that is opaque to the platform, do you substantially reduce the ability to microtarget? Very possibly not, given the amount of graph data the social media company has anyways about group members independent from the content itself. And encryption gives the social media company the ability to wash its hands of any responsibility or awareness of content. Assuming it were easy to technically achieve (which is a huge leap, to be fair!) do you think it better serves the definition of integrity you've adopted, that a social media platform have the majority of its content end-to-end encrypted, or not?

sayhar 5y ago
Hi! Thanks for the constructive question. I agree with you that it's nuanced, and also I'm sad that it's getting polarized/simplified in some venues. I don't think we have gotten an institute stance on very much relating to E2EE. Our community advisory board (composed of integrity workers) is the moral core of the organization. So far, when we say "the institute has a stance on X", that has meant "the advisory board signs off on X, and that X represents a good faith consensus view of workers in the industry". We don't yet has a doc that lays out why we think integrity and privacy can coexist nicely. Speaking only for myself, I think the answer lies in careful design. As an example, you could see, via research and experimentation on FB Messenger, that messages that are forwarded in chains of > N are just empirically overwhelmingly likely to be bad faith, spammy, etc. You could then take that finding to WhatsApp, Signal, etc, and then bake in changes to the UX that make it slightly more annoying to forward messages if they've been on a reshare chain of N/M. That kind of stuff. There's also some consideration to group size -- if a group is 5000 people large on, say, Telegram, it might be encrypted, but it's no longer really private. Maybe it should be treated differently? Unclear, let's think and research about it. I think a rough consensus we might move towards is treating messaging differently than broadcast, and also treating broadcast features inside of messaging apps differently than straight up messaging themselves. But again, those are just some of my more idle thoughts. There are members and fellows who are better experts on this particular subject than I am. Does that make sense? Is that helpful?

smsm42 5y ago

How do you ensure that "integrity" does not morph into "ban everything our groupthink says is wrong"? How do you ensure you're not coopted into an ideological or partisan propaganda/speech control efforts - a trap that so many "fact checkers" gladly fallen into?

shug2k 5y ago
Transparency goes a long way towards addressing this concern. This is one area we are focused on - asking companies to be more transparent, both in their overall metrics and samples of public content, so we can have those debates as a society rather than behind closed doors.
- smsm42 5y ago
  Transparency is not enough. If the decision is "we ban everything that the Troika we appointed finds not to its liking" - it's transparent, but not helpful and has nothing to do with "integrity". Transparency is good, but not enough.
wolfram74 5y ago
One content agnostic approach to improving signal to noise is the tendency for deep chain sharing to be mostly false/misleading content. Introducing a forced copy-paste hurdle after the chain gets, say, 4 deep, would be content agnostic, but probably improve the quality of what get's spread.
- smsm42 5y ago
  > the tendency for deep chain sharing to be mostly false/misleading content Does it rely on some specific research? I mean I'm sure there's a lot of viral stories that are false. But there's also a lot of viral stories that are true - or at least no less true than what you'd commonly see on network TV or read in major newspapers, like NYT. I am not sure why it'd be obvious that something that is shared a lot is necessarily false.
- sayhar 5y ago

neural_thing 5y ago

It's understandable that people go to work for Sauron to get paid big money. But starting a "human-orc ethics think tank" because you have relevant experience in the field is a bit rich.

labster 5y ago
We need to be able to forgive those who have done evil, if they truly repent. If they’re trying to do good now, we shouldn’t hold it against them that they worked for a monopoly spreading lies that incited violence, created online addicts, and took the pieces of silver happily. No, it only matters do good now.
- hn_throwaway_99 5y ago
  I wholeheartedly agree. But, to be frank, I really don't see any amount of contrition in the Founders' Letter here, https://integrityinstitute.org/founders-letter https://integrityinstitute.org/founders-letter. Primarily, I don't see anything here that says "You know what, we're sorry, because we were part of the problem, despite working on 'the inside' trying to improve things." I'm not asking for the founders to fall on a proverbial sword, but I don't see any acknowledgement of how the entire Facebook "machine", which causes all the problems they point out in their letter, is only possible because legions of really smart people choose to work there.
- dogman144 5y ago
  We can forgive them but not listen to their thought leadership on solving the post-Sauron world.
- smsm42 5y ago
  Repenting is indeed a major step on the way from evil to not evil, but I do not see any sign of repentance in their materials. Repenting means recognizing you did wrong, recognizing and admitting, sincerely and honestly, what was wrong in what you did, and then building a corrective action on top of that. I do not see any of that. I only see a bunch of platitides, which I could find in any corporate "our values" binder, we're all for everything good and against everything bad. "Strive to be a good, honorable person" - ok, sure, who doesn't? Are there many people striving to be bad and dishonorable? That's not doing anything.

deleted 5y ago

[deleted]