3 ms·

It appeared to do even less than that, because they don’t have a ground truth of throwaway and main account associations. It sounds like they where just able t

by lotu 8y ago

It appeared to do even less than that, because they don’t have a ground truth of throwaway and main account associations. It sounds like they where just able to re-associate an account with it’s origional name after removing the name by looking at metadata. This is throughly unimpressive.

arif_sohaib 8y ago
If the reassociation returns multiple accounts, then it should get both the main and the throwaway. Unless it is over fitting to the main somehow.
- Bartweiss 8y ago
  The basic result is horribly overfitted, because "account creation minute" and "account creation hour" are two of the parameters. (They split those up because "account creation time" was effectively a unique feature all on its own, and they wanted a 'harder' problem.) This is basically an exercise in overfitting - they learned to recognize account data using that exact same data, and called it deanonymization. The data fuzzing was a little interesting, but it's still overfitting from top to bottom.
  - arif_sohaib 8y ago
    I just read quickly through on my way to work so I did not get to that part. I was thinking more like using tweet content, language useage(words used, sentence length etc) and ip data into a generative kind of network similar to what is used to fix images with missing content. Then the missing content can be the username and it might be able to at least find very similar users.
mc32 8y ago
And there I thought it might be possible to discover all the puppet accounts celebrities use to rouse interest in themselves and create “engagement”... sigh
taurine 8y ago
Glad I am not the only one. This seems like a task of memory, not author identification. How could this be used at test time? 10.000 users is meaningless on a social network with millions of accounts. What about the static features (like account creation dates)? Aren't those overfitting with cross-validation? Would not learning curves be required when classifying on unseen future data (the reason d'etre of ML)?
- Bartweiss 8y ago
  > What about the static features (like account creation dates)? Aren't those overfitting with cross-validation? Yes, absolutely. The paper admits that using only account creation time with KNN was enough to get 98% accuracy all on its own. The authors then broke creation time into hour and minute to increase difficulty (slightly), and introduced heavy data fuzzing to see how far they could go and still get results. But in practice, that's not an ML problem. It's just asking "how little data is needed to perform recall on this dataset?", and finding that it's relatively easy to associate Twitter accounts with, um, themselves. I think this could have been made interesting by not just fuzzing some data, but actively stripping everything account-level instead of tweet-level; if metadata like posting time was enough to tie tweets together, that could have interesting consequences for identifying sockpuppets or even deanonymizing the human users. But as far as I can tell, including account-level features makes this a non-story.

by lotu 8y ago

arif_sohaib 8y ago

If the reassociation returns multiple accounts, then it should get both the main and the throwaway. Unless it is over fitting to the main somehow.

Bartweiss 8y ago
The basic result is horribly overfitted, because "account creation minute" and "account creation hour" are two of the parameters. (They split those up because "account creation time" was effectively a unique feature all on its own, and they wanted a 'harder' problem.) This is basically an exercise in overfitting - they learned to recognize account data using that exact same data, and called it deanonymization. The data fuzzing was a little interesting, but it's still overfitting from top to bottom.
- arif_sohaib 8y ago
  I just read quickly through on my way to work so I did not get to that part. I was thinking more like using tweet content, language useage(words used, sentence length etc) and ip data into a generative kind of network similar to what is used to fix images with missing content. Then the missing content can be the username and it might be able to at least find very similar users.

mc32 8y ago

And there I thought it might be possible to discover all the puppet accounts celebrities use to rouse interest in themselves and create “engagement”... sigh

taurine 8y ago

Glad I am not the only one. This seems like a task of memory, not author identification. How could this be used at test time? 10.000 users is meaningless on a social network with millions of accounts. What about the static features (like account creation dates)? Aren't those overfitting with cross-validation? Would not learning curves be required when classifying on unseen future data (the reason d'etre of ML)?

Bartweiss 8y ago
> What about the static features (like account creation dates)? Aren't those overfitting with cross-validation? Yes, absolutely. The paper admits that using only account creation time with KNN was enough to get 98% accuracy all on its own. The authors then broke creation time into hour and minute to increase difficulty (slightly), and introduced heavy data fuzzing to see how far they could go and still get results. But in practice, that's not an ML problem. It's just asking "how little data is needed to perform recall on this dataset?", and finding that it's relatively easy to associate Twitter accounts with, um, themselves. I think this could have been made interesting by not just fuzzing some data, but actively stripping everything account-level instead of tweet-level; if metadata like posting time was enough to tie tweets together, that could have interesting consequences for identifying sockpuppets or even deanonymizing the human users. But as far as I can tell, including account-level features makes this a non-story.