5 ms·

They claim machine learning, but I'd guess their workhorse is a "grep -f badwords tweets.json | makepdf | send boss@employer" kind of thingy.

allovernow 7y ago
They're definitely using some form of sentiment analysis, but IMO these are exactly the kind of results you'd get after setting a programmer, with little to no actual data science background, loose to train his nets on data with a limited or absent intuitive understanding of bias from training data. And what's worse is that in their business false flags probably aren't even considered. It's an extra wide supercharged net. E.G the "BIG DICK ENERGY" post being flagged as bigotry and sexism, no doubt they trained on limited data of hand curated "questionable posts" and I wouldn't be surprised if they used a source like 4chan and just automatically assigned negative labels to the vast majority of posts.
- BlueTemplar 7y ago
  The problem here is rather likely to be to not have someone with human science background... but then that person would likely just tell them their whole premise is flawed?
  - allovernow 7y ago
    I don't think so. There are valid use cases for sentiment analysis, but you need to understand the limitations of your training data and probably still want humans to QC at least some representative proportion of flagged posts, if you're going to do this legitimately. Of course a company like this just wants to sell any garbage they can dig up.
    - BlueTemplar 7y ago
      Well, I will admit that I am quite ignorant of what exactly a background check is, but I just don't see how such a morally fraught question can be legally allowed to be decided by anyone else than a psychologist. In fact, considering the moral hazard, I don't even see how even using an AI, even as simple as "grep", helping that psychologist in the cost-minimization contexts of a private company would not result in an unacceptable slippery slope where the psychologist would end up just rubber-stamping the decisions of the AI and its creator ? Maybe someone with a dual data "science" / psychology degree would be acceptable, but I'm guessing he/she wouldn't be able to use any "black box" AI...
      - isoskeles 7y ago
        That’s not what moral hazard means. https://en.wikipedia.org/wiki/Moral_hazard https://en.wikipedia.org/wiki/Moral_hazard
        BlueTemplar 7y ago
        Right, it's probably inappropriate to use this term at this specific place in my argument. However, in what is probably not just a coincidence, the global issue this is only one facet of is about the information asymmetry between citizens and corporations...
- vntok 7y ago
  "BIG DICK ENERGY" is arguably a childish and immature post. Someone liking that post is arguably being childish and immature in their minds at that time . Of course, a single like of a childish or immature post or meme is a meaningless data point (one could like it to laugh at it). So are 5 likes. So are 10 likes. However, say that twitter account regulary likes memes that are only shared in a particular incels-community. Or say that account likes immature posts by the dozens, at least way more than serious ones. Now we're no longer dealing with mere data points, we're dealing with trends. And those trends give a hint of the type of character the company is dealing with. Maybe a serious B2B company doesn't want to have childish, immature, incels-type people amongst their ranks?
  - allovernow 7y ago
    This is exactly the kind of hysteria that will tear society apart. The types of jokes a person shares and enjoys outside of the office by and large are not a reflection of character, even when they are racist, sexist, or some other -ist. In fact that's the very reason that they are funny, because they are counter to norms. There's nothing immature or childish about this if you're not oversocialized. Believe it or not, it's possible to be a competent programmer and decent human being while occasionally indulging in an off color joke. Humor is a form of release. You've lived a sheltered life if you're unaware of how perfectly normal and nearly ubiquitous so called "locker room talk" is. Your post is a perfect example of a dangerous overreaction. Suddenly liking the wrong post makes me an "incel" and keeps me from contributing to society. Isn't that a little crazy?
    - vntok 7y ago
      > The types of jokes a person shares and enjoys outside of the office by and large are not a reflection of character, even when they are racist, sexist, or some other -ist. If most of your social media posts are you liking racist, sexist or some other ist content, I will avoid you. Who wouldn't?
      - lainga 7y ago
        I wouldn't, because I don't use social media, and it wouldn't make sense to avoid someone based on something I don't know about.
- Balgair 7y ago
  People who live in Scunthorpe or Apeniston would have a great time with these things, I imagine. https://en.wikipedia.org/wiki/Scunthorpe_problem https://en.wikipedia.org/wiki/Scunthorpe_problem
- tor291674 7y ago
  AWS and Azure both have services for this now. Tweet sentiment analysis is like the first getting started tutorial you land on when you hit both of their docs.
0xff00ffee 7y ago
ML? No. But regular expressions are a type of artificial intelligence the same way an A* search is.
dylan604 7y ago
That's some pretty high level stuff. I imagined their machine learning is a large cubicle filled room of humanoid machines manually sifting through people's profiles.
thecleaner 7y ago
And I guarantee that this will be faster than using Hadoop with distributed NoSQL database. Which I presume is what the company is probably doing.

allovernow 7y ago

They're definitely using some form of sentiment analysis, but IMO these are exactly the kind of results you'd get after setting a programmer, with little to no actual data science background, loose to train his nets on data with a limited or absent intuitive understanding of bias from training data. And what's worse is that in their business false flags probably aren't even considered. It's an extra wide supercharged net. E.G the "BIG DICK ENERGY" post being flagged as bigotry and sexism, no doubt they trained on limited data of hand curated "questionable posts" and I wouldn't be surprised if they used a source like 4chan and just automatically assigned negative labels to the vast majority of posts.

BlueTemplar 7y ago
The problem here is rather likely to be to not have someone with human science background... but then that person would likely just tell them their whole premise is flawed?
- allovernow 7y ago
  I don't think so. There are valid use cases for sentiment analysis, but you need to understand the limitations of your training data and probably still want humans to QC at least some representative proportion of flagged posts, if you're going to do this legitimately. Of course a company like this just wants to sell any garbage they can dig up.
  - BlueTemplar 7y ago
    Well, I will admit that I am quite ignorant of what exactly a background check is, but I just don't see how such a morally fraught question can be legally allowed to be decided by anyone else than a psychologist. In fact, considering the moral hazard, I don't even see how even using an AI, even as simple as "grep", helping that psychologist in the cost-minimization contexts of a private company would not result in an unacceptable slippery slope where the psychologist would end up just rubber-stamping the decisions of the AI and its creator ? Maybe someone with a dual data "science" / psychology degree would be acceptable, but I'm guessing he/she wouldn't be able to use any "black box" AI...
    - isoskeles 7y ago
      That’s not what moral hazard means. https://en.wikipedia.org/wiki/Moral_hazard https://en.wikipedia.org/wiki/Moral_hazard
      - BlueTemplar 7y ago
        Right, it's probably inappropriate to use this term at this specific place in my argument. However, in what is probably not just a coincidence, the global issue this is only one facet of is about the information asymmetry between citizens and corporations...
vntok 7y ago
"BIG DICK ENERGY" is arguably a childish and immature post. Someone liking that post is arguably being childish and immature in their minds at that time . Of course, a single like of a childish or immature post or meme is a meaningless data point (one could like it to laugh at it). So are 5 likes. So are 10 likes. However, say that twitter account regulary likes memes that are only shared in a particular incels-community. Or say that account likes immature posts by the dozens, at least way more than serious ones. Now we're no longer dealing with mere data points, we're dealing with trends. And those trends give a hint of the type of character the company is dealing with. Maybe a serious B2B company doesn't want to have childish, immature, incels-type people amongst their ranks?
- allovernow 7y ago
  This is exactly the kind of hysteria that will tear society apart. The types of jokes a person shares and enjoys outside of the office by and large are not a reflection of character, even when they are racist, sexist, or some other -ist. In fact that's the very reason that they are funny, because they are counter to norms. There's nothing immature or childish about this if you're not oversocialized. Believe it or not, it's possible to be a competent programmer and decent human being while occasionally indulging in an off color joke. Humor is a form of release. You've lived a sheltered life if you're unaware of how perfectly normal and nearly ubiquitous so called "locker room talk" is. Your post is a perfect example of a dangerous overreaction. Suddenly liking the wrong post makes me an "incel" and keeps me from contributing to society. Isn't that a little crazy?
  - vntok 7y ago
    > The types of jokes a person shares and enjoys outside of the office by and large are not a reflection of character, even when they are racist, sexist, or some other -ist. If most of your social media posts are you liking racist, sexist or some other ist content, I will avoid you. Who wouldn't?
    - lainga 7y ago
      I wouldn't, because I don't use social media, and it wouldn't make sense to avoid someone based on something I don't know about.
Balgair 7y ago
People who live in Scunthorpe or Apeniston would have a great time with these things, I imagine. https://en.wikipedia.org/wiki/Scunthorpe_problem https://en.wikipedia.org/wiki/Scunthorpe_problem
tor291674 7y ago
AWS and Azure both have services for this now. Tweet sentiment analysis is like the first getting started tutorial you land on when you hit both of their docs.

0xff00ffee 7y ago

ML? No. But regular expressions are a type of artificial intelligence the same way an A* search is.

dylan604 7y ago

That's some pretty high level stuff. I imagined their machine learning is a large cubicle filled room of humanoid machines manually sifting through people's profiles.

thecleaner 7y ago

And I guarantee that this will be faster than using Hadoop with distributed NoSQL database. Which I presume is what the company is probably doing.