4 ms·

Lol. Predicting the survival of passengers on the Titanic is meaningless and misleading - there is literally no connection to reality, despite the framing of th

Lol. Predicting the survival of passengers on the Titanic is meaningless and misleading - there is literally no connection to reality, despite the framing of the task which suggests a certain connection. There is absolutely nothing that could be predicted. It is just a simulation of oversimplified model which describes nothing, but an oversimplified view of a historical event. It is as meaningless as the ant simulator written by Rich Hickey to demonstrate features of Clojure - it has that much connection to real ants.

edgyswingset 10y ago
Huh? Why would a connection to reality be required to get started with machine learning?
- dschiptsov 10y ago
  Because otherwise it should be called machine hallucinations? The process of learning could be defined as a task of extraction of relevant information (knowledge) about reality (shared environment) not mere accumulation of a fancy nonsense or false beliefs.
  - BickNowstrom 10y ago
    So knowledge like: Did the passenger have kids on board? Was the passenger nobility? Was the passenger travelling first class? Where was the passenger located on the ship after boarding? And how do these factors influence survivability? And reality like: The actual sinking of the Titanic? If your model concludes that nobility, traveling first class, close to the exits, without family, has a higher chance of surviving, then this is fancy nonsense or a false belief? You make a really strange case for your view.
    - dschiptsov 10y ago
      Correlations does not imply causation. There were many more relevant but "invisible" variables, which, probably, related to some genetic factors, like ability to sustain exposure to the cold water, ability to calm oneself down to avoid panic and self-control in general, strong survival instinct to literally fight the others, etc. The variables you have described, except the age of a passenger, are visible but irrelevant. And pure luck must have a way bigger weight and it is, obviously, related to the genetic favorable factors, age, health and fitness.
      - BickNowstrom 10y ago
        This challenge is not about causal inference. I do agree it is more of a toy dataset, to get started with the basics, and that there are a lot of other variables that go into survivability. But to say these variables, except for age, are irrelevant is mathematically unsound: You can show with cross-validation and test set performance that your model using these variables generalizes (around 0.80 ROC AUC). You can do statistical/information theoretical tests that show the majority of these variables is a significant signal for predicting the target. In real life it is also very rare to have free pickings of the variables you want. Some variables have to substituted with available ones. The Titanic story is to make things interesting for beginners. One could leave out all the semantics of this challenge, anonymize the variables and the target, and still use this dataset to learn about going from a table with variables to a target. In fact, doing so teaches you to leave your human bias at the door. Domain experts get beaten on Kaggle, because they think they need other variables, or that some variables (and their interactions) can't possibly work. Let the data and evaluation metric do the talking.
        dschiptsov 10y ago
        > You can show with cross-validation and test set performance that your model using these variables generalizes (around 0.80 ROC AUC). It shows only that given set of variables (observable and inferred) could be used to build a model. The given data set is not descriptive, because it does not contain more relevant hidden variables, so any predictions or inferences based on this data set are nothing but a story, a myth made from statistics and data.
        taeric 10y ago
        How does this not violate [1]? That is, this seems specifically anti-statistical. The best you can come up with on this is a predictive model that you then have to test on new events. In this case, that would likely mean new crashes. [1] https://en.wikipedia.org/wiki/Testing_hypotheses_suggested_by_the_data https://en.wikipedia.org/wiki/Testing_hypotheses_suggested_b...
        10y ago
BickNowstrom 10y ago
You seem to be saying a whole lot without backing up your argument. If anything, your view is meaningless. https://en.wikipedia.org/wiki/Ant_colony_optimization_algorithms https://en.wikipedia.org/wiki/Ant_colony_optimization_algori... https://en.wikipedia.org/wiki/Artificial_ants https://en.wikipedia.org/wiki/Artificial_ants
- dschiptsov 10y ago
  There is a very important notion from The Sciences of the Artificial book by Herbert A. Simon, that the visible (to an external observer) behavior of an ant (its tracks, if you wish) is not due to its supposed intelligence, but mostly due to the obstacles in the environment. Most of the models mimic and simulate (very naively) that observable behavior, not its origin. When people cite "the map is not the territory" they mean this. Simulation is not even an experiment. It is mere an animation of a model - a cartoon.
  - BickNowstrom 10y ago
    It is swarm intelligence: How does the system keep finding successful paths in a changing environment? Can we take inspiration from this behavior to create better optimization algorithms? Simulation can be a very beneficial experiment. See for instance: https://papers.nips.cc/paper/5351-searching-for-higgs-boson-decay-modes-with-deep-learning.pdf https://papers.nips.cc/paper/5351-searching-for-higgs-boson-...
    - dschiptsov 10y ago
      Why not. I remember a paper which compares the behavior of foraging ants (they send more or less ants according to the rate of returned with food) to adjustment of the window size based on the data rate in TCP. Simulations are not experiments. It is an animation of a formalized imagination, if you wish.
nl 10y ago
It's very closely correlated to reality. If you work through the data, you'll find things like women, children and first class passengers had a higher survival rate than men with lower class tickets[1]. This matches exactly the stories of what happened: Staff took first class passengers to the lifeboats first, then women and children. Then they ran out of lifeboats. So the data shows correlation, and eye-witness accounts shows causation. That's close to the ideal combination: eyewitness accounts can be unreliable because we can't know how widespread they are, and correlation doesn't show causation. But the combination of them both is pretty much the best case for studying something which can't be replicated. [1] See examples like https://www.kaggle.com/omarelgabry/titanic/a-journey-through-titanic https://www.kaggle.com/omarelgabry/titanic/a-journey-through...
- dschiptsov 10y ago
  This is only one of many aspects of that event. The data reflects that the efforts of organized evacuation in the beginning were efficient. But any attempt to frame it as a "prediction", an accurate model of the event or adequate description of reality is just nonsense. To call things by its proper names (precise use of the language) is the foundation of the scientific method. This is mere oversimplified, non-descriptive toy model of one aspect of historical event, made from of statistics of partially observable environment. A few inferred correlations reflects that there was not a total chaos, but some systematic activity. No doubt about it. But this is absolutely unscientific to say anything else about the toy model, let alone claim that any predictions based on it have any connection to reality.
  - nl 10y ago
    That is absolute nonsense. There is clear correlation between gender and survival rates. Given the data, a decent prior would absolutely take that into account. Yes, there are other factors. But the foundation of statistical models is simplification, and descriptive statistics are an important foundation of that. In any case, it isn't exactly clear that there are magical hidden factors which predicted survival. It appears you maybe unfamiliar with the event, because basically those who got into a lifeboat survived, and those who didn't, didn't survive. To quote Wikipedia: Almost all those who jumped or fell into the water drowned within minutes due to the effects of hypothermia.... The disaster caused widespread outrage over the lack of lifeboats, lax regulations, and the unequal treatment of the three passenger classes during the evacuation..... The thoroughness of the muster was heavily dependent on the class of the passengers; the first-class stewards were in charge of only a few cabins, while those responsible for the second- and third-class passengers had to manage large numbers of people. The first-class stewards provided hands-on assistance, helping their charges to get dressed and bringing them out onto the deck. With far more people to deal with, the second- and third-class stewards mostly confined their efforts to throwing open doors and telling passengers to put on lifebelts and come up top. In third class, passengers were largely left to their own devices after being informed of the need to come on deck. Even more tellingly: The two officers interpreted the "women and children" evacuation order differently; Murdoch took it to mean women and children first, while Lightoller took it to mean women and children only. Lightoller lowered lifeboats with empty seats if there were no women and children waiting to board, while Murdoch allowed a limited number of men to board if all the nearby women and children had embarked All this behavior matches exactly what the model tells us about the event. I'd be very interested if you can point to something specific that is wrong about it. All models are wrong, but some are useful.

edgyswingset 10y ago

Huh? Why would a connection to reality be required to get started with machine learning?

dschiptsov 10y ago
Because otherwise it should be called machine hallucinations? The process of learning could be defined as a task of extraction of relevant information (knowledge) about reality (shared environment) not mere accumulation of a fancy nonsense or false beliefs.
- BickNowstrom 10y ago
  So knowledge like: Did the passenger have kids on board? Was the passenger nobility? Was the passenger travelling first class? Where was the passenger located on the ship after boarding? And how do these factors influence survivability? And reality like: The actual sinking of the Titanic? If your model concludes that nobility, traveling first class, close to the exits, without family, has a higher chance of surviving, then this is fancy nonsense or a false belief? You make a really strange case for your view.
  - dschiptsov 10y ago
    Correlations does not imply causation. There were many more relevant but "invisible" variables, which, probably, related to some genetic factors, like ability to sustain exposure to the cold water, ability to calm oneself down to avoid panic and self-control in general, strong survival instinct to literally fight the others, etc. The variables you have described, except the age of a passenger, are visible but irrelevant. And pure luck must have a way bigger weight and it is, obviously, related to the genetic favorable factors, age, health and fitness.
    - BickNowstrom 10y ago
      This challenge is not about causal inference. I do agree it is more of a toy dataset, to get started with the basics, and that there are a lot of other variables that go into survivability. But to say these variables, except for age, are irrelevant is mathematically unsound: You can show with cross-validation and test set performance that your model using these variables generalizes (around 0.80 ROC AUC). You can do statistical/information theoretical tests that show the majority of these variables is a significant signal for predicting the target. In real life it is also very rare to have free pickings of the variables you want. Some variables have to substituted with available ones. The Titanic story is to make things interesting for beginners. One could leave out all the semantics of this challenge, anonymize the variables and the target, and still use this dataset to learn about going from a table with variables to a target. In fact, doing so teaches you to leave your human bias at the door. Domain experts get beaten on Kaggle, because they think they need other variables, or that some variables (and their interactions) can't possibly work. Let the data and evaluation metric do the talking.
      - dschiptsov 10y ago
        > You can show with cross-validation and test set performance that your model using these variables generalizes (around 0.80 ROC AUC). It shows only that given set of variables (observable and inferred) could be used to build a model. The given data set is not descriptive, because it does not contain more relevant hidden variables, so any predictions or inferences based on this data set are nothing but a story, a myth made from statistics and data.
      - taeric 10y ago
        How does this not violate [1]? That is, this seems specifically anti-statistical. The best you can come up with on this is a predictive model that you then have to test on new events. In this case, that would likely mean new crashes. [1] https://en.wikipedia.org/wiki/Testing_hypotheses_suggested_by_the_data https://en.wikipedia.org/wiki/Testing_hypotheses_suggested_b...
        10y ago

BickNowstrom 10y ago

You seem to be saying a whole lot without backing up your argument. If anything, your view is meaningless. https://en.wikipedia.org/wiki/Ant_colony_optimization_algorithms https://en.wikipedia.org/wiki/Ant_colony_optimization_algori... https://en.wikipedia.org/wiki/Artificial_ants https://en.wikipedia.org/wiki/Artificial_ants

dschiptsov 10y ago
There is a very important notion from The Sciences of the Artificial book by Herbert A. Simon, that the visible (to an external observer) behavior of an ant (its tracks, if you wish) is not due to its supposed intelligence, but mostly due to the obstacles in the environment. Most of the models mimic and simulate (very naively) that observable behavior, not its origin. When people cite "the map is not the territory" they mean this. Simulation is not even an experiment. It is mere an animation of a model - a cartoon.
- BickNowstrom 10y ago
  It is swarm intelligence: How does the system keep finding successful paths in a changing environment? Can we take inspiration from this behavior to create better optimization algorithms? Simulation can be a very beneficial experiment. See for instance: https://papers.nips.cc/paper/5351-searching-for-higgs-boson-decay-modes-with-deep-learning.pdf https://papers.nips.cc/paper/5351-searching-for-higgs-boson-...
  - dschiptsov 10y ago
    Why not. I remember a paper which compares the behavior of foraging ants (they send more or less ants according to the rate of returned with food) to adjustment of the window size based on the data rate in TCP. Simulations are not experiments. It is an animation of a formalized imagination, if you wish.

nl 10y ago

It's very closely correlated to reality. If you work through the data, you'll find things like women, children and first class passengers had a higher survival rate than men with lower class tickets[1]. This matches exactly the stories of what happened: Staff took first class passengers to the lifeboats first, then women and children. Then they ran out of lifeboats. So the data shows correlation, and eye-witness accounts shows causation. That's close to the ideal combination: eyewitness accounts can be unreliable because we can't know how widespread they are, and correlation doesn't show causation. But the combination of them both is pretty much the best case for studying something which can't be replicated. [1] See examples like https://www.kaggle.com/omarelgabry/titanic/a-journey-through-titanic https://www.kaggle.com/omarelgabry/titanic/a-journey-through...

dschiptsov 10y ago
This is only one of many aspects of that event. The data reflects that the efforts of organized evacuation in the beginning were efficient. But any attempt to frame it as a "prediction", an accurate model of the event or adequate description of reality is just nonsense. To call things by its proper names (precise use of the language) is the foundation of the scientific method. This is mere oversimplified, non-descriptive toy model of one aspect of historical event, made from of statistics of partially observable environment. A few inferred correlations reflects that there was not a total chaos, but some systematic activity. No doubt about it. But this is absolutely unscientific to say anything else about the toy model, let alone claim that any predictions based on it have any connection to reality.
- nl 10y ago
  That is absolute nonsense. There is clear correlation between gender and survival rates. Given the data, a decent prior would absolutely take that into account. Yes, there are other factors. But the foundation of statistical models is simplification, and descriptive statistics are an important foundation of that. In any case, it isn't exactly clear that there are magical hidden factors which predicted survival. It appears you maybe unfamiliar with the event, because basically those who got into a lifeboat survived, and those who didn't, didn't survive. To quote Wikipedia: Almost all those who jumped or fell into the water drowned within minutes due to the effects of hypothermia.... The disaster caused widespread outrage over the lack of lifeboats, lax regulations, and the unequal treatment of the three passenger classes during the evacuation..... The thoroughness of the muster was heavily dependent on the class of the passengers; the first-class stewards were in charge of only a few cabins, while those responsible for the second- and third-class passengers had to manage large numbers of people. The first-class stewards provided hands-on assistance, helping their charges to get dressed and bringing them out onto the deck. With far more people to deal with, the second- and third-class stewards mostly confined their efforts to throwing open doors and telling passengers to put on lifebelts and come up top. In third class, passengers were largely left to their own devices after being informed of the need to come on deck. Even more tellingly: The two officers interpreted the "women and children" evacuation order differently; Murdoch took it to mean women and children first, while Lightoller took it to mean women and children only. Lightoller lowered lifeboats with empty seats if there were no women and children waiting to board, while Murdoch allowed a limited number of men to board if all the nearby women and children had embarked All this behavior matches exactly what the model tells us about the event. I'd be very interested if you can point to something specific that is wrong about it. All models are wrong, but some are useful.