4 ms·
I've been participating in Numerai for a few months now. (I've only made some beer money from it, nothing serious.) When you get the data, you have no idea wh
by antognini 10y ago
I've been participating in Numerai for a few months now. (I've only made some beer money from it, nothing serious.) When you get the data, you have no idea what it is. It's just a file with ~70,000 data points, each of which has 21 features. Each feature is uniformly distributed between 0 and 1. All you have to do is make a binary classification of 0 or 1 (or, more accurately, the probability that the data point is in class 0 or 1). They don't tell you what the 21 features represent.
As far as you know, these predictions could be used to make currency trades, or stock predictions, or real estate purchase, or something more exotic. You really have no idea. And since you don't know what these data points represent you can't use any insider knowledge about anything to help you.
- acchow 10y agoThe grandparent's point wasn't that the people doing the analysis are injecting insider information....it's that the data dump you've been presented could be some fancy encoding of insider information.
- usefulcat 10y agoHow would that work? If Numerai already has insider information, why not just use it directly? Why go to the bother (and risk) of anonymizing and distributing it?
- sanswork 10y agoPlausible deniability.
- mrich 10y agoThere would be none if they were aware the information they hand out contains insider info.
- ForHackernews 10y agoHow would you prove it? It's just an anonymous list of features to train a model.
- ubernostrum 10y agoAs someone else suggested, think of it like parallel construction in law enforcement: an agency already knows someone is guilty, but they know it from illegal surveillance they can't use in court. So they give a tip to an unsuspecting officer to watch out for a car with a broken tail light at a particular location/time, and in the course of the traffic stop the officer finds drugs in the car, then they claim that's how they found out (despite the officer only being there because they already knew).
- antognini 10y agoI think it's unlikely that insider information comes in some nicely formatted dataset with 21 features week in and week out. I think of insider information as usually being a one-off tip that a company is going to take a particular action before they make that information public. This feels more like prosaic financial data. (But I'm not in finance so maybe my perceptions are way off!)
- acchow 10y agoHence why I phrased it "encodes insider information" So you get your tip, and you generate a data set that will result in some data analyst getting the conclusion you want.
- gohrt 10y agoNo, grandparent's point was the the AI data dump is misdirection from the actual trading logic.
- iamthepieman 10y agoEDIT: This is total conjecture. only 1 or at most a few of those 21 features represent real data. The real data represents similar information to the insider information which they wish to act upon. Example data prep: 1. Insider source says that a contract is falling through, a patent is being filed for, quarterly numbers have been missed/surpassed etc. 2. Similar information is gathered from historic performance data of the company, similar companies or market segments. 3. The information is correlated with whatever metric they wish to move along and encoded in one of the 21 feature classes. 4. Repeat for whatever relevant information that can be linked to the insider source - i.e. competing companies, re-encoding separately for long and short positions etc. 5. Fill in remainder of 21 feature classes with noise. 6. Profit.
- victor9000 10y agoaka parallel construction
- gohrt 10y agoWhat would be the point of all this? Step 1 is the illegal part, and the other steps don't erase that.
- iamthepieman 10y agoit's like if law enforcement uses a legally questionable tactic to get information about someone. Then, with the help of that illegally obtained knowledge, they can go and re-create it (along with a legal paper trail) via other perfectly legal means.
- foota 10y agoSeems that it would make detecting it more difficult
- poikniok 10y agoWell of course it is illegal, you just don't want to get caught, and this allows you to have plausible deniability.
- 10y ago
- billconan 10y agoCan I participate as an investor putting my money into the fund? Or I can only participate as a data scientist?
- dilemma 10y ago>They don't tell you what the 21 features represent. Half of 42.