6 ms·

> I'm always more impressed that sites like twitter and facebook need so many engineers for what are fundamentally fairly simple platforms that haven't hugely c

> I'm always more impressed that sites like twitter and facebook need so many engineers for what are fundamentally fairly simple platforms that haven't hugely changed in years The bit that you get wrong is assuming that Facebook or Twitter are simple platforms, and that their main and most complex features is allowing users to post, search and read messages. This is entirely wrong as these platforms are actually ad-selling platforms and thus include region-specific services, localized ranking, all sorts of data harvesting, etc etc. Also, as companies grow they start to feel the need to implement efficient development processes, and thus start to allocate resources to stuff like putting together their own build farms, CI/CD infrastructure, deployment auditing, etc etc etc. They also develop things like front-end and back-end frameworks. It's not like a random team at Facebook just signs up for gitlab and does all it's business from there.

beebmam 4y ago
Likewise, I get annoyed when I hear people scream "publish the algorithm!" at social media companies. There isn't an algorithm. There's an enormous number of applications, some of which are data science/ML-based, that rely on stream and batch processing of incoming and pre-existing data to render recommendations and timelines. This is far more than an algorithm, this is an entire ecosystem, more complicated than a single individual can grok without an enormous amount of time on their hands (and the experience to understand it all).
- BoorishBears 4y ago
  You're missing the point. They don't need to release the actual code, but releasing "these are the inputs we use, these are the high level functions of the pipeline, these are the outputs" would be enough. And the fact that pipelines that drive how people consume an increasingly large majority of their content have become this convoluted, should get you annoyed at social media companies: not people who don't understand how bad its gotten
  - korla 4y ago
    Isn't the point that companies should be judged by the cost of their actions, and not the details of their implementations?
    - BoorishBears 4y ago
      Why the false dichotomy? The cost of their actions is easier to frame if you know the details of their implementation. If people just naturally like using FB that's fine. If FB is hiring experts for behavioral studies in order hypertune a pipeline that exploits blind spots in the way the human brain works... that's a lot less fine. And if the inputs are wider in scope than the general public realizes, it becomes even less ok. People will always have free will and most would rather we don't try and override that by just making all social media illegal or something... but there's an inherent asymmetry in the resources social media companies have in dissecting the psyche vs individuals have countering that. Forcing them to show their cards is one way to thrust the results of that asymmetry into the spotlight and better equip individuals who often don't understand the extent to which they're being manipulated.
  - threeseed 4y ago
    > these are the inputs we use, these are the high level functions of the pipeline, these are the outputs" would be enough You do understand that these are ML models where there is no way to explain their behaviour in a layman way other than what is bleedingly obvious. That it will try and show you more content that you and the people you follow might like.
    - beezlewax 4y ago
      It isn't entirely about what you like, it is about what will drive interaction. This is why hateful posts or controversial ones rise to the top so frequently. It encourages bad and often anti social behaviour. They've created a monster.
      - pixl97 4y ago
- ezst 4y ago
  It's a good question to ask nonetheless, if the answer is "we can't tell you exactly why we exposed you to this information preferentially, oh and by the way, we have no way to prove that pushing divisive and sometimes assumed fake content isn't an emerging property of this insanely complex system meant to generate revenue for our bottom line with close to no checks in place", then the follow-up argument should be a much needed (IMO) "simplify this crap so users and regulators come to know how much neutrality and facts they get served". I understand enough ML to know how and why such systems can't be introspected and why their output is inscrutable to humans, but that's where I draw the line personally: a key component of your business, or one that affects many users, shouldn't be permitted to be so complex that its outcome can't be explained based on a finite number of documented variables and logical branches. We can even think about having a standard to describe and document ML systems (and I would totally look that up because I have the right to know how my keyboard suggests words, why this search result that seems unusual is in fact a sponsored placement, or why this autonomous vehicle chose to run over the pedestrian so not to kill the baby on board...). And this is not meant to hinder research and development in AI, but to protect people and businesses from potentially dangerous blackboxes, and bring back accountability and ethics in a field that knows very little of it.
  - yunwal 4y ago
    > shouldn't be permitted to be so complex that its outcome can't be explained based on a finite number of documented variables and logical branches There are plenty of effective pharmaceuticals for which we don’t entirely understand the mechanism (e.g. lithium). I think it’s a good goal to be able to understand mechanisms, but if you can effectively show that a solution works to solve a problem through methodical trials, we shouldn’t disallow it’s use. The issue with Facebook is not complexity or black-box-ness. It’s that the goals of the company and how it makes a profit are fundamentally misaligned with a healthy society.
  - lanza 4y ago
    > I understand enough ML to know how hmmmm > We can even think about having a standard to describe and document ML systems (and I would totally look that up because I have the right to know how my keyboard suggests words, why this search result that seems unusual is in fact a sponsored placement, or why this autonomous vehicle chose to run over the pedestrian so not to kill the baby on board...). Yea you know nothing about ML...
saurik 4y ago
> Also, as companies grow they start to feel the need to implement efficient development processes... You have managed to make these all sound incredibly inefficient ;P.

beebmam 4y ago

Likewise, I get annoyed when I hear people scream "publish the algorithm!" at social media companies. There isn't an algorithm. There's an enormous number of applications, some of which are data science/ML-based, that rely on stream and batch processing of incoming and pre-existing data to render recommendations and timelines. This is far more than an algorithm, this is an entire ecosystem, more complicated than a single individual can grok without an enormous amount of time on their hands (and the experience to understand it all).

BoorishBears 4y ago
You're missing the point. They don't need to release the actual code, but releasing "these are the inputs we use, these are the high level functions of the pipeline, these are the outputs" would be enough. And the fact that pipelines that drive how people consume an increasingly large majority of their content have become this convoluted, should get you annoyed at social media companies: not people who don't understand how bad its gotten
- korla 4y ago
  Isn't the point that companies should be judged by the cost of their actions, and not the details of their implementations?
  - BoorishBears 4y ago
    Why the false dichotomy? The cost of their actions is easier to frame if you know the details of their implementation. If people just naturally like using FB that's fine. If FB is hiring experts for behavioral studies in order hypertune a pipeline that exploits blind spots in the way the human brain works... that's a lot less fine. And if the inputs are wider in scope than the general public realizes, it becomes even less ok. People will always have free will and most would rather we don't try and override that by just making all social media illegal or something... but there's an inherent asymmetry in the resources social media companies have in dissecting the psyche vs individuals have countering that. Forcing them to show their cards is one way to thrust the results of that asymmetry into the spotlight and better equip individuals who often don't understand the extent to which they're being manipulated.
- threeseed 4y ago
  > these are the inputs we use, these are the high level functions of the pipeline, these are the outputs" would be enough You do understand that these are ML models where there is no way to explain their behaviour in a layman way other than what is bleedingly obvious. That it will try and show you more content that you and the people you follow might like.
  - beezlewax 4y ago
    It isn't entirely about what you like, it is about what will drive interaction. This is why hateful posts or controversial ones rise to the top so frequently. It encourages bad and often anti social behaviour. They've created a monster.
    - pixl97 4y ago
ezst 4y ago
It's a good question to ask nonetheless, if the answer is "we can't tell you exactly why we exposed you to this information preferentially, oh and by the way, we have no way to prove that pushing divisive and sometimes assumed fake content isn't an emerging property of this insanely complex system meant to generate revenue for our bottom line with close to no checks in place", then the follow-up argument should be a much needed (IMO) "simplify this crap so users and regulators come to know how much neutrality and facts they get served". I understand enough ML to know how and why such systems can't be introspected and why their output is inscrutable to humans, but that's where I draw the line personally: a key component of your business, or one that affects many users, shouldn't be permitted to be so complex that its outcome can't be explained based on a finite number of documented variables and logical branches. We can even think about having a standard to describe and document ML systems (and I would totally look that up because I have the right to know how my keyboard suggests words, why this search result that seems unusual is in fact a sponsored placement, or why this autonomous vehicle chose to run over the pedestrian so not to kill the baby on board...). And this is not meant to hinder research and development in AI, but to protect people and businesses from potentially dangerous blackboxes, and bring back accountability and ethics in a field that knows very little of it.
- yunwal 4y ago
  > shouldn't be permitted to be so complex that its outcome can't be explained based on a finite number of documented variables and logical branches There are plenty of effective pharmaceuticals for which we don’t entirely understand the mechanism (e.g. lithium). I think it’s a good goal to be able to understand mechanisms, but if you can effectively show that a solution works to solve a problem through methodical trials, we shouldn’t disallow it’s use. The issue with Facebook is not complexity or black-box-ness. It’s that the goals of the company and how it makes a profit are fundamentally misaligned with a healthy society.
- lanza 4y ago
  > I understand enough ML to know how hmmmm > We can even think about having a standard to describe and document ML systems (and I would totally look that up because I have the right to know how my keyboard suggests words, why this search result that seems unusual is in fact a sponsored placement, or why this autonomous vehicle chose to run over the pedestrian so not to kill the baby on board...). Yea you know nothing about ML...

saurik 4y ago

> Also, as companies grow they start to feel the need to implement efficient development processes... You have managed to make these all sound incredibly inefficient ;P.