4 ms·

This is an existing problem with physical world sensing. Some data models are already heavily polluted by people manipulating observations for various purposes.

This is an existing problem with physical world sensing. Some data models are already heavily polluted by people manipulating observations for various purposes. Most people who rely on those measurements are just collateral damage. Prediction markets provide yet another incentive. It is far more difficult to reliably get at "ground truth" than I think people imagine. Unlike computing systems, the physical world is a shared mutable environment. It is effectively impossible to lock people out so that you can get a reliable, clean measurement even when people are not intentionally sabotaging the data model. There are ways to robustly clean this up analytically but it is largely beyond the capabilities of current tech stacks.

pavl- 6mo ago
"Some data models are already heavily polluted by people manipulating observations for various purposes" Example of this?
dymk 6mo ago
Manipulation of datasets is something that already existed, yes. But what betting markets do is create more incentives to screw with the real world. You're right that this isn't a technical problem, it's a social one. This is why most sports leagues simply banned betting... until recently, now that gambling companies have bought their way in. The players always end up manipulating games.
functionmouse 6mo ago
that argument is insane
AgentMatt 6mo ago
> There are ways to robustly clean this up analytically but it is largely beyond the capabilities of current tech stacks. Can you expand on that? Even just conceptually that sounds really hard, how would you know whether you're measuring genuine (unexpected) changes in the environment rather than the result of (possibly sophisticated and coordinated) deliberate manipulation?
- jandrewrogers 6mo ago
  If you don't trust your measurements, and you shouldn't because all physical world measurements are proxies, the alternative is to find several unrelated and redundant proxies for "ground truth" and have them corroborate and correct each other. There is a lot of errors, bugs, noise, idiosyncratic behavior, etc degrading the data even ignoring intentional manipulation so you really should be doing this anyway. Stitching unrelated proxies and sensing modalities into a coherent data model is a spatiotemporal graph reconstruction problem. The join predicates require non-trivial inference algorithms if you want to avoid being buried in false positives. From this you can derive an estimate of ground truth and a model of uncertainty at a point in space and time. The model of uncertainty is dynamic and unpredictable. It is difficult to manipulate the measurement without producing data that falls outside the uncertainty model across every proxy by which someone might construct that uncertainty model. This is similar to how e.g. GPS spoofing is detected in military systems. All GPS updates must fit within a (classified) dynamic uncertainty model relative to INS; if an update falls outside the model then the GPS signal is presumed compromised and updates ignored. At the limit, this restricts manipulation to values within the uncertainty model. If you have a lot of unrelated proxies, you can make the window of uncertainty tight enough that manipulation becomes effectively impossible. At a minimum, the adversary would need to be able to manipulate every proxy and modality feeding your uncertainty model simultaneously. These graph, spatial, and spatiotemporal algorithms scale very poorly on traditional data infrastructure and these data models easily run into petabytes if you are stacking multiple independent data sources.
  - uoaei 6mo ago
    I recommend for you to read Feyerabend's Science in a Free Society.
    - jandrewrogers 6mo ago
      I'm not sure I see the relevance?
      - uoaei 6mo ago
ambicapter 6mo ago
> for various purposes. What kinds of purposes?
- jandrewrogers 6mo ago
  Typical ones in no particular order: fraud; obscuring crime, espionage, or covert activity; manipulating your competitors or markets; skirting regulations. Quite a lot of low-level organized crime. A lot of it is transnational and many countries are complicit so there is relatively little you can do to stop it. If you use data at risk of this then systems have to be designed under the assumption that it is going to happen. It has spread and become worse over time. It is a bit like FinCEN data in that if you work with it long enough you'll see a lot more going on in it than it is your job to notice.

pavl- 6mo ago

"Some data models are already heavily polluted by people manipulating observations for various purposes" Example of this?

dymk 6mo ago

Manipulation of datasets is something that already existed, yes. But what betting markets do is create more incentives to screw with the real world. You're right that this isn't a technical problem, it's a social one. This is why most sports leagues simply banned betting... until recently, now that gambling companies have bought their way in. The players always end up manipulating games.

functionmouse 6mo ago

that argument is insane

AgentMatt 6mo ago

> There are ways to robustly clean this up analytically but it is largely beyond the capabilities of current tech stacks. Can you expand on that? Even just conceptually that sounds really hard, how would you know whether you're measuring genuine (unexpected) changes in the environment rather than the result of (possibly sophisticated and coordinated) deliberate manipulation?

jandrewrogers 6mo ago
If you don't trust your measurements, and you shouldn't because all physical world measurements are proxies, the alternative is to find several unrelated and redundant proxies for "ground truth" and have them corroborate and correct each other. There is a lot of errors, bugs, noise, idiosyncratic behavior, etc degrading the data even ignoring intentional manipulation so you really should be doing this anyway. Stitching unrelated proxies and sensing modalities into a coherent data model is a spatiotemporal graph reconstruction problem. The join predicates require non-trivial inference algorithms if you want to avoid being buried in false positives. From this you can derive an estimate of ground truth and a model of uncertainty at a point in space and time. The model of uncertainty is dynamic and unpredictable. It is difficult to manipulate the measurement without producing data that falls outside the uncertainty model across every proxy by which someone might construct that uncertainty model. This is similar to how e.g. GPS spoofing is detected in military systems. All GPS updates must fit within a (classified) dynamic uncertainty model relative to INS; if an update falls outside the model then the GPS signal is presumed compromised and updates ignored. At the limit, this restricts manipulation to values within the uncertainty model. If you have a lot of unrelated proxies, you can make the window of uncertainty tight enough that manipulation becomes effectively impossible. At a minimum, the adversary would need to be able to manipulate every proxy and modality feeding your uncertainty model simultaneously. These graph, spatial, and spatiotemporal algorithms scale very poorly on traditional data infrastructure and these data models easily run into petabytes if you are stacking multiple independent data sources.
- uoaei 6mo ago
  I recommend for you to read Feyerabend's Science in a Free Society.
  - jandrewrogers 6mo ago
    I'm not sure I see the relevance?
    - uoaei 6mo ago

ambicapter 6mo ago

> for various purposes. What kinds of purposes?

jandrewrogers 6mo ago
Typical ones in no particular order: fraud; obscuring crime, espionage, or covert activity; manipulating your competitors or markets; skirting regulations. Quite a lot of low-level organized crime. A lot of it is transnational and many countries are complicit so there is relatively little you can do to stop it. If you use data at risk of this then systems have to be designed under the assumption that it is going to happen. It has spread and become worse over time. It is a bit like FinCEN data in that if you work with it long enough you'll see a lot more going on in it than it is your job to notice.