4 ms·

Optimisation algorithms have no hidden agenda and are predictable. Given we agree on the variable they optimise for and can check the source code, what's not to

by rixed 9y ago

Optimisation algorithms have no hidden agenda and are predictable. Given we agree on the variable they optimise for and can check the source code, what's not to like?

nils-m-holm 9y ago
The people setting the parameters may have hidden agendas. Or even not so hidden ones, like "make me rich at anybody's cost".
- BoorishBears 9y ago
  I think a much more likely cause would be biased data. If you trained against a dataset of all police stops in my state (CT) to predict which vehicles should be stopped to maximize drug finds you'd include multiple large departments that have been under federal investigation for racial discrimination (and were found guilty). I don't think the hidden agenda would be intentional, but there would be a hidden agenda.
acbabis 9y ago
Those givens are far from trivial, and getting them wrong is dangerous. You may as well say "Given we can perfectly predict the market, it's possible to make a lot of money"
TeMPOraL 9y ago
First of all, for any non-trivial real-world problem (like e.g. doing a simplest of tasks while respecting human moral values), we do not, generally, agree on the variable (function, actually) being optimized. Nor can we even define it with sufficient amount of precision. This is the core of what's known as "Friendly AI" problem (also known as "AI control problem"). Secondly, optimization algorithms "have no hidden agenta and are predictable" only insofar as you're able to comprehend their implementation. Which, for anything more trivial than blinking an LED, you're most likely not. This is the core of what's known as "programs have bugs" problem in software engineering, and also of the "bugs are exploitable" and "programs can have hidden backdoors" problems in software security engineering.
imron 9y ago
> Given we ... can check the source code That's not a given.
CPLX 9y ago
A cancerous tumor or virus is essentially an organically programmed optimization algorithm. I can think of a few things not to like about them.
contravariant 9y ago
This is probably the most inaccurate statement I've ever read. Here are just some of the assumptions it is making that are flawed beyond repair: - Algorithms don't have unexpected behaviour. - Computer programs operating in real time are predictable - We can agree on what to optimise for - Checking the source code is always possible - Checking the source code allows you to understand a program
- rixed 9y ago
  All those objections I agree with. Yet, this still sounds better than electing people from time to time on the basis of unverifiable promises, with no control on what is actually done unless you can throw a lot of money in lobbying.
pjmorris 9y ago
The algorithm, the problem it is applied to, the variables to consider and to optimize, the variables excluded from consideration, and the dats sources used are all choices made by someone with some agenda. And they don't have to agree with you on their choices.

by rixed 9y ago

Optimisation algorithms have no hidden agenda and are predictable. Given we agree on the variable they optimise for and can check the source code, what's not to like?

nils-m-holm 9y ago

The people setting the parameters may have hidden agendas. Or even not so hidden ones, like "make me rich at anybody's cost".

BoorishBears 9y ago
I think a much more likely cause would be biased data. If you trained against a dataset of all police stops in my state (CT) to predict which vehicles should be stopped to maximize drug finds you'd include multiple large departments that have been under federal investigation for racial discrimination (and were found guilty). I don't think the hidden agenda would be intentional, but there would be a hidden agenda.

acbabis 9y ago

Those givens are far from trivial, and getting them wrong is dangerous. You may as well say "Given we can perfectly predict the market, it's possible to make a lot of money"

TeMPOraL 9y ago

First of all, for any non-trivial real-world problem (like e.g. doing a simplest of tasks while respecting human moral values), we do not, generally, agree on the variable (function, actually) being optimized. Nor can we even define it with sufficient amount of precision. This is the core of what's known as "Friendly AI" problem (also known as "AI control problem"). Secondly, optimization algorithms "have no hidden agenta and are predictable" only insofar as you're able to comprehend their implementation. Which, for anything more trivial than blinking an LED, you're most likely not. This is the core of what's known as "programs have bugs" problem in software engineering, and also of the "bugs are exploitable" and "programs can have hidden backdoors" problems in software security engineering.

imron 9y ago

> Given we ... can check the source code That's not a given.

CPLX 9y ago

A cancerous tumor or virus is essentially an organically programmed optimization algorithm. I can think of a few things not to like about them.

contravariant 9y ago

This is probably the most inaccurate statement I've ever read. Here are just some of the assumptions it is making that are flawed beyond repair: - Algorithms don't have unexpected behaviour. - Computer programs operating in real time are predictable - We can agree on what to optimise for - Checking the source code is always possible - Checking the source code allows you to understand a program

rixed 9y ago
All those objections I agree with. Yet, this still sounds better than electing people from time to time on the basis of unverifiable promises, with no control on what is actually done unless you can throw a lot of money in lobbying.

pjmorris 9y ago

The algorithm, the problem it is applied to, the variables to consider and to optimize, the variables excluded from consideration, and the dats sources used are all choices made by someone with some agenda. And they don't have to agree with you on their choices.