4 ms·

So how would a better dataset look like? Does bigger always equal harder? What are the criteria to measure that? And wouldn't video datasets be somewhat easier

So how would a better dataset look like? Does bigger always equal harder? What are the criteria to measure that? And wouldn't video datasets be somewhat easier to analyse given the fact that you have multiple frames of the same object?

HappyTypist 11y ago
The point of a dataset is to serve as a representative sample of the full spectrum of possible input data. Datasets need to be properly classified, and with different elements / 'intricacies' well distributed. Think of it as using Hacker News posts to illustrate what posts are on topic. You could give someone a better impression if you showed them 10 posts instead of just 3, but if the 10 posts were all on the same subject then it wouldn't be of any additional utility. And if you accidentally include an off-topic post, then that user is going to have a mistaken impression of the post. As for video... that's a long story.
ninjin 11y ago
It is very difficult to measure data set quality. The goal of a data set to give you insight into how well an approach performs for any given data point (in this case an image and associated label). The problem is that we can not collect all the possible data points, so we have to settle for what we hope to be a fair sample of all data points. While bigger usually means better, if for example there was a bias in how the data points were selected the data set can actually be worse than a smaller one where there was no such bias. Building a good data set really depends on the current understanding of the nature and difficulty of the task, both on the part of the scientific field and the researchers working on the data set. The very same problem exists in the medical domain. How do you select the right patients to evaluate for a treatment? How do you know that there is not a specific genetic, racial, gender, etc. trait that leads to side effects? The answer is of course that you can not know with absolute certainty unless you include every human on the planet (and then there is the question as-of-yet unborn humans), but you can use your experience and medical knowledge to try to select patients (your data set) for the medical trial to minimise the possible impact of as-of-yet unknown side effects.

HappyTypist 11y ago

The point of a dataset is to serve as a representative sample of the full spectrum of possible input data. Datasets need to be properly classified, and with different elements / 'intricacies' well distributed. Think of it as using Hacker News posts to illustrate what posts are on topic. You could give someone a better impression if you showed them 10 posts instead of just 3, but if the 10 posts were all on the same subject then it wouldn't be of any additional utility. And if you accidentally include an off-topic post, then that user is going to have a mistaken impression of the post. As for video... that's a long story.

ninjin 11y ago

It is very difficult to measure data set quality. The goal of a data set to give you insight into how well an approach performs for any given data point (in this case an image and associated label). The problem is that we can not collect all the possible data points, so we have to settle for what we hope to be a fair sample of all data points. While bigger usually means better, if for example there was a bias in how the data points were selected the data set can actually be worse than a smaller one where there was no such bias. Building a good data set really depends on the current understanding of the nature and difficulty of the task, both on the part of the scientific field and the researchers working on the data set. The very same problem exists in the medical domain. How do you select the right patients to evaluate for a treatment? How do you know that there is not a specific genetic, racial, gender, etc. trait that leads to side effects? The answer is of course that you can not know with absolute certainty unless you include every human on the planet (and then there is the question as-of-yet unborn humans), but you can use your experience and medical knowledge to try to select patients (your data set) for the medical trial to minimise the possible impact of as-of-yet unknown side effects.