8 ms·

I, an American, have a friend in France, where is our friendship stored? I send a message to my friend in France they send me one back, where are these two mess

I, an American, have a friend in France, where is our friendship stored? I send a message to my friend in France they send me one back, where are these two messages stored? We have a photo taken while traveling together in Belgium with 2 other Americans, what server? I could see this being a problem only effectively solved by a separate EU and US Facebook. It is not clear what data they are worried about transmission of, but each type seems to need special consideration. You mention ad-targeting data, but most data collected by Facebook useful to the surveillance agencies Ireland is worried about are more personal than that.

trasz 5y ago
I suspect cases like this are a tiny minority. You could handle them by falling back to the current way of doing this, ie US servers, and still protect the vast majority of data that's not related to US citizens.
- pjc50 5y ago
  A tiny minority of Facebook users is millions of people. Lots of people have international family and friends. A fallback approach is infeasible for legal requirements, unless it's there for appealing the fine to a lower value. The question is potentially very difficult, and could only be resolved by constructive engagement with the party making the rules. Don't forget what happens when people travel.
  - peoplefromibiza 5y ago
    many more people have friends and family in the US, but are not allowed to enter the country freely. Borders are heavily controlled by USA, why shouldn't other countries to the same?
  - trasz 5y ago
    The question can be solved trivially easily by just using a big-hammer approach of banning any transfer of personal information to US, period. There are no reasons for allowing the transfer to happen, other than making it easier for Facebook to make money. FB would implement it by only storing interactions involving any EU nationals in EU. We do want to make it easier for Facebook for political reasons, though, and it's still not particularly hard: just declare that only the data/conversations involving US citizens can be stored on US servers.
    - bryan_w 5y ago
      Are you saying that an American shouldn't be able to pull up the profile of an EU user? Seems a bit extreme.
      - trasz 5y ago
        Of course not; the data would come from FB's EU servers. The "transfer" above applies to where the data is stored, not where it can be accessed from. From the users point of view the only change would be... nothing; the latency is already at the point where it wouldn't incur any observable penalty.
  - ConceptJunkie 5y ago
    I don't have many Facebook friends, but they include people on at least 3 other continents. I'm sure a lot of folks have at least some friends in other countries.
- messe 5y ago
  > I suspect cases like this are a tiny minority. Knowing people in other countries is a tiny minority? Maybe in the US, given its size, but it's pretty widespread and normal in Europe.
  - stingraycharles 5y ago
    I think you’d be surprised that HN is not representative for the general population, and that talking with people from another continent over WhatsApp is, in fact, an exception and not the rule.
  - peoplefromibiza 5y ago
    > Knowing people in other countries is a tiny minority? Absolutely yes. Especially across two continents. Europeans who have friends from Europe would all be in Europe anyway. Europe is larger than US btw, it has two times the population. Russia+Turkey+Germany alone account for 95% of the population of the United States.
    - deleted 5y ago
      [deleted]
    - jltsiren 5y ago
      I would assume that most Europeans know people living on another continent, or at least people who have lived there in the recent past. There are plenty of careers and hobbies where you make international connections, plenty of jobs that require you to live somewhere else for a while, and plenty of people looking for opportunities elsewhere. Globally, around 1 person in 30 lives outside their country of origin, so knowing people in other countries should be common.
      - peoplefromibiza 5y ago
        > I would assume that most Europeans know people living on another continent Not most, just some. 1 out of 30 is a bit more than 3% and many of those have family connections, they are not strangers living abroad, they are - for example - Italians living in Canada. EU citizens whose data should be kept in EU.
amelius 5y ago
> I, an American, have a friend in France, where is our friendship stored? Sounds almost philosophical. I have a friend. We both have brains. In which brain is our friendship stored?
- MaxBarraclough 5y ago
  If you're both cryonically frozen, is the friendship suspended? Does it spring back into being when you're both defrosted, arising from the process of living? By this account, friendships aren't purely a matter of data storage. If we're feeling especially poetic, we could make the case that friendships can live on despite the death of one of the participants. Also, do we always assume a binary friendship of exactly two participants?
- ta988 5y ago
  Both, unless your "friend" does not consider you as a friend, in that case it is only in yours.
  - adolph 5y ago
    People who live in society have learnt how to see themselves, in mirrors, as they appear to their friends. I have no friends: is that why my flesh is so naked? https://www.goodreads.com/quotes/548471-people-who-live-in-society-have-learnt-how-to-see https://www.goodreads.com/quotes/548471-people-who-live-in-s... Let us consider your hypothetical glitched friend-state within Antoine’s concept of distributed self awareness. Does the friend delusion lead to self delusion or vice versa?
    - ta988 5y ago
      I may not consider someone a friend, but still know they do. Imagine the case of people that make "friends" to abuse them or steal information. Note that it doesnt stop them from feeling the friendship is fake and faking that themselves. You see patterns like that in spy stories with double or triple agents. Not sure how many recursions a human brain is able to handle on an everyday basis, I would guess between 2 and 4?
teachingassist 5y ago
> We have a photo taken while traveling together in Belgium with 2 other Americans, what server? EU GDPR is pretty clear that all of the cases which involve transferring data from the EU to the US fall under EU GDPR rules. It refers to the location of the data, not the citizenship or residency status of the individuals who are involved. So, the photo that you took and uploaded within EU borders is theoretically in scope for EU GDPR - even if your French friend is not in the photo.
- teachingassist 5y ago
  I'm not sure why I get downvoted for observing this - people don't like the truth?
  - Isinlor 5y ago
    Could you provide specific article that says so?
rich_sasha 5y ago
Good point. Is the ban more about where content is stored or how it is accessed? I guess there is nothing bad about you seeing a page served from Europe. Otherwise, a lot of this data presumably isn’t covered by GDPR. You and your friend are ultimately user_ids with a relationship_id or sth, and these surely aren’t GDPR-controlled (but they point at details that are)
- corty 5y ago
  Data that can be traced back to a person is covered by GDPR. If the user_ids have attached user datasets somewhere they are covered. If they don't have that, but the graph formed by user_ids and relationship_ids is isomorphic to publically known facts or facts that people with access to the graph have, that make people identifiable in the graph, it is covered by GDPR. One example would be a is-neighbour-of relation for peoples' addresses together with a public phone/address-book.
NicoJuicy 5y ago
What I think: Your message will be stored in the US. Your friends message in the EU. The metadata ( conversation table of both) would be stored in the US. Your picture from Europe as a US user with US residency will be stored in the US. As Facebook wouldn't want your data under GDPR and since it's also not required.
pm24601 5y ago
Oh gee, you're right. URLs don't work across national boundaries.</snark> Seriously?
evanelias 5y ago
Yes, this is exactly the challenge. It extends far beyond just ad-targeting. On the database side, social network sharding, replication, and caching schemes aren't typically designed based on national origin of users. If you have a full datacenter (i.e. containing databases, not just a frontend or CDN / PoP footprint) in a country, then typically the entire logical data set -- all data for all users worldwide -- is presumably replicated there. Other systems and services will then make assumptions that any object can be looked up with low sub-ms latency. Social networks often contain activity streams and other pages that include content from many users at once. Consider algorithmic ranking of feed content, comments on popular page content, etc: how do you even implement this if even some small subset of the data needs to be fetched from halfway around the world on every page view? Anyway, to answer your original question, friendships are bidirectional associations and would typically be stored in two places: one entry in your db shard, and one entry in your friend's shard. Photos are objects and presumably would be "owned" by a single user or page and located there (at least in terms of the metadata about the photo); however tags may be associations which have entries on multiple shards just like friendships. If some of these shards can only be accessed across a trans-Atlantic link, the entire scheme falls apart due to the latency.
- mackman 5y ago
  This guy shards. This is the correct answer. "The social graph is tightly interconnected; it is not possible to group users so that cross-partition requests are rare. This means that each TAO follower must be local to a tier of databases holding a complete multi-petabyte copy of the social graph. It would be prohibitively expensive to provide full replicas in every data center. Our solution to this problem is to choose data center locations that are clustered into only a few regions, where the intra-region latency is small (typically less than 1 millisecond). It is then sufficient to store one complete copy of the social graph per region. Figure 2 shows the overall architecture of the master/slave TAO system." https://www.usenix.org/system/files/conference/atc13/atc13-bronson.pdf https://www.usenix.org/system/files/conference/atc13/atc13-b... edit: Oh hah, didn't realize parent was former DB person at FB. I was too, just a few years before :-)
- corobo 5y ago
  Sounds like the challenge is using peoples' data in a way that isn't scumbaggerific to be honest
oauea 5y ago
Facebook hires a lot of very smart people. I'm sure they can figure it out, or just, you know, stop their illegal behavior.
gumby 5y ago
Still, solving that is hardly “devastating”.
foolfoolz 5y ago
i don’t think it’s that complicated. your account is in a US database. your friends in a french database. the us db can have a list of friends by user id and those friends might be in other countries db. however your identity and data live in us db only. if you create a message the message can either be stored in the us and referenced by id in france (or duplicated since you intentionally sent it there). same for photos this makes each transmission of data across regional boundaries more intentional and easy to add governance checks
mrighele 5y ago
An "easy" solution would be for FB to lobby for the USA to adopt privacy laws similar to those in Europe. Then maybe the data transfer ban could be lifted (somehow I suspect that will not be the case)
- deleted 5y ago
  [deleted]
- martin8412 5y ago
  The US would have to agree that data on EU citizens is not theirs to snoop in. They would also have to trust that the US would actually honor that, and I think most will agree that the EU can't trust the US on that.
  - galgalesh 5y ago
    In case it wasn't clear; this was the exact issue. The EU decided that current US legislation makes it impossible for Facebook to ensure the US government will respect the privacy of EU residents. Search "privacy shield invalidated" for more info.
endymi0n 5y ago
This is spot on and a big problem. And although nobody here including me would shed a tear about Facebook in this case, I am worried about the precedent this sets. Facebook has the deep pockets to either build this through or fight it legally to the bitter end. But for any startup trying to build a global X, this can put a serious blocker in the way. Global sharding of entities and working with that isn‘t for the faint of heart. I know we‘d stand still for a year or two trying to implement something similar as a small to mid sized startup. Philosophically and as a user, I actually like the idea, but wearing the systems architect here, this requirement scares me to no end and could throw a literal wrench into the operations of any global effort.
- 6510 5y ago
  Implementing it from scratch does seem less of a clusterfuck.
- trasz 5y ago
  For any startup parasitizing on its users, you mean.
deleted 5y ago
[deleted]
trasz 5y ago
Your friendship can be stored in US or France. The problem is, everything else about that French national is still being stored in US, even if it's in no way related to US nationals.
duxup 5y ago
That's an interesting challenge. Social connections span legal jurisdictions.
6510 5y ago
You 4356324 have a friend 6434535. Easier than CSS.

trasz 5y ago

I suspect cases like this are a tiny minority. You could handle them by falling back to the current way of doing this, ie US servers, and still protect the vast majority of data that's not related to US citizens.

pjc50 5y ago
A tiny minority of Facebook users is millions of people. Lots of people have international family and friends. A fallback approach is infeasible for legal requirements, unless it's there for appealing the fine to a lower value. The question is potentially very difficult, and could only be resolved by constructive engagement with the party making the rules. Don't forget what happens when people travel.
- peoplefromibiza 5y ago
  many more people have friends and family in the US, but are not allowed to enter the country freely. Borders are heavily controlled by USA, why shouldn't other countries to the same?
- trasz 5y ago
  The question can be solved trivially easily by just using a big-hammer approach of banning any transfer of personal information to US, period. There are no reasons for allowing the transfer to happen, other than making it easier for Facebook to make money. FB would implement it by only storing interactions involving any EU nationals in EU. We do want to make it easier for Facebook for political reasons, though, and it's still not particularly hard: just declare that only the data/conversations involving US citizens can be stored on US servers.
  - bryan_w 5y ago
    Are you saying that an American shouldn't be able to pull up the profile of an EU user? Seems a bit extreme.
    - trasz 5y ago
      Of course not; the data would come from FB's EU servers. The "transfer" above applies to where the data is stored, not where it can be accessed from. From the users point of view the only change would be... nothing; the latency is already at the point where it wouldn't incur any observable penalty.
- ConceptJunkie 5y ago
  I don't have many Facebook friends, but they include people on at least 3 other continents. I'm sure a lot of folks have at least some friends in other countries.
messe 5y ago
> I suspect cases like this are a tiny minority. Knowing people in other countries is a tiny minority? Maybe in the US, given its size, but it's pretty widespread and normal in Europe.
- stingraycharles 5y ago
  I think you’d be surprised that HN is not representative for the general population, and that talking with people from another continent over WhatsApp is, in fact, an exception and not the rule.
- peoplefromibiza 5y ago
  > Knowing people in other countries is a tiny minority? Absolutely yes. Especially across two continents. Europeans who have friends from Europe would all be in Europe anyway. Europe is larger than US btw, it has two times the population. Russia+Turkey+Germany alone account for 95% of the population of the United States.
  - deleted 5y ago
    [deleted]
  - jltsiren 5y ago
    I would assume that most Europeans know people living on another continent, or at least people who have lived there in the recent past. There are plenty of careers and hobbies where you make international connections, plenty of jobs that require you to live somewhere else for a while, and plenty of people looking for opportunities elsewhere. Globally, around 1 person in 30 lives outside their country of origin, so knowing people in other countries should be common.
    - peoplefromibiza 5y ago
      > I would assume that most Europeans know people living on another continent Not most, just some. 1 out of 30 is a bit more than 3% and many of those have family connections, they are not strangers living abroad, they are - for example - Italians living in Canada. EU citizens whose data should be kept in EU.

amelius 5y ago

> I, an American, have a friend in France, where is our friendship stored? Sounds almost philosophical. I have a friend. We both have brains. In which brain is our friendship stored?

MaxBarraclough 5y ago
If you're both cryonically frozen, is the friendship suspended? Does it spring back into being when you're both defrosted, arising from the process of living? By this account, friendships aren't purely a matter of data storage. If we're feeling especially poetic, we could make the case that friendships can live on despite the death of one of the participants. Also, do we always assume a binary friendship of exactly two participants?
ta988 5y ago
Both, unless your "friend" does not consider you as a friend, in that case it is only in yours.
- adolph 5y ago
  People who live in society have learnt how to see themselves, in mirrors, as they appear to their friends. I have no friends: is that why my flesh is so naked? https://www.goodreads.com/quotes/548471-people-who-live-in-society-have-learnt-how-to-see https://www.goodreads.com/quotes/548471-people-who-live-in-s... Let us consider your hypothetical glitched friend-state within Antoine’s concept of distributed self awareness. Does the friend delusion lead to self delusion or vice versa?
  - ta988 5y ago
    I may not consider someone a friend, but still know they do. Imagine the case of people that make "friends" to abuse them or steal information. Note that it doesnt stop them from feeling the friendship is fake and faking that themselves. You see patterns like that in spy stories with double or triple agents. Not sure how many recursions a human brain is able to handle on an everyday basis, I would guess between 2 and 4?

teachingassist 5y ago

> We have a photo taken while traveling together in Belgium with 2 other Americans, what server? EU GDPR is pretty clear that all of the cases which involve transferring data from the EU to the US fall under EU GDPR rules. It refers to the location of the data, not the citizenship or residency status of the individuals who are involved. So, the photo that you took and uploaded within EU borders is theoretically in scope for EU GDPR - even if your French friend is not in the photo.

teachingassist 5y ago
I'm not sure why I get downvoted for observing this - people don't like the truth?
- Isinlor 5y ago
  Could you provide specific article that says so?

rich_sasha 5y ago

Good point. Is the ban more about where content is stored or how it is accessed? I guess there is nothing bad about you seeing a page served from Europe. Otherwise, a lot of this data presumably isn’t covered by GDPR. You and your friend are ultimately user_ids with a relationship_id or sth, and these surely aren’t GDPR-controlled (but they point at details that are)

corty 5y ago
Data that can be traced back to a person is covered by GDPR. If the user_ids have attached user datasets somewhere they are covered. If they don't have that, but the graph formed by user_ids and relationship_ids is isomorphic to publically known facts or facts that people with access to the graph have, that make people identifiable in the graph, it is covered by GDPR. One example would be a is-neighbour-of relation for peoples' addresses together with a public phone/address-book.

NicoJuicy 5y ago

What I think: Your message will be stored in the US. Your friends message in the EU. The metadata ( conversation table of both) would be stored in the US. Your picture from Europe as a US user with US residency will be stored in the US. As Facebook wouldn't want your data under GDPR and since it's also not required.

pm24601 5y ago

Oh gee, you're right. URLs don't work across national boundaries.</snark> Seriously?

evanelias 5y ago

Yes, this is exactly the challenge. It extends far beyond just ad-targeting. On the database side, social network sharding, replication, and caching schemes aren't typically designed based on national origin of users. If you have a full datacenter (i.e. containing databases, not just a frontend or CDN / PoP footprint) in a country, then typically the entire logical data set -- all data for all users worldwide -- is presumably replicated there. Other systems and services will then make assumptions that any object can be looked up with low sub-ms latency. Social networks often contain activity streams and other pages that include content from many users at once. Consider algorithmic ranking of feed content, comments on popular page content, etc: how do you even implement this if even some small subset of the data needs to be fetched from halfway around the world on every page view? Anyway, to answer your original question, friendships are bidirectional associations and would typically be stored in two places: one entry in your db shard, and one entry in your friend's shard. Photos are objects and presumably would be "owned" by a single user or page and located there (at least in terms of the metadata about the photo); however tags may be associations which have entries on multiple shards just like friendships. If some of these shards can only be accessed across a trans-Atlantic link, the entire scheme falls apart due to the latency.

mackman 5y ago
This guy shards. This is the correct answer. "The social graph is tightly interconnected; it is not possible to group users so that cross-partition requests are rare. This means that each TAO follower must be local to a tier of databases holding a complete multi-petabyte copy of the social graph. It would be prohibitively expensive to provide full replicas in every data center. Our solution to this problem is to choose data center locations that are clustered into only a few regions, where the intra-region latency is small (typically less than 1 millisecond). It is then sufficient to store one complete copy of the social graph per region. Figure 2 shows the overall architecture of the master/slave TAO system." https://www.usenix.org/system/files/conference/atc13/atc13-bronson.pdf https://www.usenix.org/system/files/conference/atc13/atc13-b... edit: Oh hah, didn't realize parent was former DB person at FB. I was too, just a few years before :-)
corobo 5y ago
Sounds like the challenge is using peoples' data in a way that isn't scumbaggerific to be honest

oauea 5y ago

Facebook hires a lot of very smart people. I'm sure they can figure it out, or just, you know, stop their illegal behavior.

gumby 5y ago

Still, solving that is hardly “devastating”.

foolfoolz 5y ago

i don’t think it’s that complicated. your account is in a US database. your friends in a french database. the us db can have a list of friends by user id and those friends might be in other countries db. however your identity and data live in us db only. if you create a message the message can either be stored in the us and referenced by id in france (or duplicated since you intentionally sent it there). same for photos this makes each transmission of data across regional boundaries more intentional and easy to add governance checks

mrighele 5y ago

An "easy" solution would be for FB to lobby for the USA to adopt privacy laws similar to those in Europe. Then maybe the data transfer ban could be lifted (somehow I suspect that will not be the case)

deleted 5y ago
[deleted]
martin8412 5y ago
The US would have to agree that data on EU citizens is not theirs to snoop in. They would also have to trust that the US would actually honor that, and I think most will agree that the EU can't trust the US on that.
- galgalesh 5y ago
  In case it wasn't clear; this was the exact issue. The EU decided that current US legislation makes it impossible for Facebook to ensure the US government will respect the privacy of EU residents. Search "privacy shield invalidated" for more info.

endymi0n 5y ago

This is spot on and a big problem. And although nobody here including me would shed a tear about Facebook in this case, I am worried about the precedent this sets. Facebook has the deep pockets to either build this through or fight it legally to the bitter end. But for any startup trying to build a global X, this can put a serious blocker in the way. Global sharding of entities and working with that isn‘t for the faint of heart. I know we‘d stand still for a year or two trying to implement something similar as a small to mid sized startup. Philosophically and as a user, I actually like the idea, but wearing the systems architect here, this requirement scares me to no end and could throw a literal wrench into the operations of any global effort.

6510 5y ago
Implementing it from scratch does seem less of a clusterfuck.
trasz 5y ago
For any startup parasitizing on its users, you mean.

deleted 5y ago

[deleted]

trasz 5y ago

Your friendship can be stored in US or France. The problem is, everything else about that French national is still being stored in US, even if it's in no way related to US nationals.

duxup 5y ago

That's an interesting challenge. Social connections span legal jurisdictions.

6510 5y ago

You 4356324 have a friend 6434535. Easier than CSS.