3 ms·
I think this is a bit of a false dilemma. It seems like if you do not have enough skill to evaluate a model, then you have to trust someone who does have the s
by mikekchar 6y ago
I think this is a bit of a false dilemma. It seems like if you do not have enough skill to evaluate a model, then you have to trust someone who does have the skill. At that point you are now back to trusting someone rather than making your own informed decision.
However, the validity of a model is not a binary state. Models in science are not truth. They are models that are built using observations. Predictions are made with the models and efforts are made to find circumstances in which the predictions do not come true. If we fail to find such circumstances, then it's usually a pretty good model.
The important point is that all models are flawed. A model is not the actual thing. The equations for how gravity work are not perfect. There are things we don't understand and simplifications in our models for things that we don't care about. The model is only there to help us make predictions.
Some models are very good -- no matter how you try it's virtually impossible to find a way where the model does not work exactly as we observe. Some models are very bad -- it's easy to find failing cases. But even bad models can be useful if you understand their limitations.
When you are looking at competing models, it is not usually important to understand the models completely. What you need to be able to do is to "sanity check" them. Are there easy ways that you can say that the model will not work well for your situation? Quite frequently there are.
Often you don't even need to to evaluate how good a model is. You just need to choose the better of a few proposed models. Again, doing some sanity checking can often find ways where you can easily determine that one model is better than another.
The mistake I see people making is that they evaluate models based on trust. I trust the person selling model A, so I will reject model B. This is a very poor way of evaluating models because it does not sanity check the model itself. Instead it's important to look at the predictions of the models and then look at as much data as you can find to see if the observations match the predictions.
This is where people often fall short -- they do not do the leg work. It's one of the real failings of our education system IMHO. Teachers are given the status of the "source of truth". The students trust the teachers and when they leave school the lose that source of truth. They ask "Who should I trust?" Even people who dig into issues look at two papers with different results and they ask "Which result should I trust?" It's a real problem.
If people could be taught to respond with, "I don't know what the real answer is, but for now A seems to have the most consistent data that I've seen", things would be quite a bit better. However, it is very possible that I ask too much from the general person in society...