4 ms·

It's arbitrary if you make it arbitrary. Depending on the data you might target various time intervals to find the optimal way to trade on that information just

by jjn2009 8y ago

It's arbitrary if you make it arbitrary. Depending on the data you might target various time intervals to find the optimal way to trade on that information just like any hyperparameter and the bins are a function of that time, on average in a 30 minute period bitcoin moves by 0.2% on high liquidity exchanges.

dx034 8y ago
But as I understand they used the whole sample to determine bins, including validation sets. That alone is a big no-go. I didn't mean that you can't find a proper way to choose bins but there is a lot of potential for errors. And in the end you want to invest based on the model. Arbitrary bins aren't that helpful if some of them combine desirable and undesirable (e.g. positive and negative) results.
- jjn2009 8y ago
  >whole sample to determine bins, including validation sets. well thats not random or arbitrary just wrong. >And in the end you want to invest based on the model. I'm not quite sure what you mean here. Either you determine a model and hyperparameters (which includes the bins) is correct enough of the time via testing on out of sample data or even synthetic data or you are talking about a determining what to do given a single observation, which I would assume you give it to the model and ask it what it thinks and the bins are part of the determination (as done here in the article with softmax output) of what to do and given you've done the testing you should have a level of confidence about the outcome of acting upon the models output. The bins aren't a post processing step, the post processing you might do to trade recall for accuracy might be to expect the bin to have a stronger signal (class > 0.5 or something maybe and otherwise ignore), all of this is "the model". >Arbitrary bins aren't that helpful if some of them combine desirable and undesirable results Correct me if I'm wrong but it sounds like you are making the case that there are models and bins which would have only good outcomes (and those are the only useful ones)? Am I misunderstanding something here?

dx034 8y ago

But as I understand they used the whole sample to determine bins, including validation sets. That alone is a big no-go. I didn't mean that you can't find a proper way to choose bins but there is a lot of potential for errors. And in the end you want to invest based on the model. Arbitrary bins aren't that helpful if some of them combine desirable and undesirable (e.g. positive and negative) results.

jjn2009 8y ago
>whole sample to determine bins, including validation sets. well thats not random or arbitrary just wrong. >And in the end you want to invest based on the model. I'm not quite sure what you mean here. Either you determine a model and hyperparameters (which includes the bins) is correct enough of the time via testing on out of sample data or even synthetic data or you are talking about a determining what to do given a single observation, which I would assume you give it to the model and ask it what it thinks and the bins are part of the determination (as done here in the article with softmax output) of what to do and given you've done the testing you should have a level of confidence about the outcome of acting upon the models output. The bins aren't a post processing step, the post processing you might do to trade recall for accuracy might be to expect the bin to have a stronger signal (class > 0.5 or something maybe and otherwise ignore), all of this is "the model". >Arbitrary bins aren't that helpful if some of them combine desirable and undesirable results Correct me if I'm wrong but it sounds like you are making the case that there are models and bins which would have only good outcomes (and those are the only useful ones)? Am I misunderstanding something here?