4 ms·
Note that MIRI's current position no longer suggests the development of actual artificial consciousness, just the development of human-equivalent optimization p
by JoshTriplett 11y ago
Note that MIRI's current position no longer suggests the development of actual artificial consciousness, just the development of human-equivalent optimization processes. In other words, they argue that you can develop a process capable of solving human-level and harder problems without giving it self-awareness. And that seems like a feature: if you avoid building self-aware machine intelligences, you don't have to worry about what they want; you can build them to only care about what existing sapient beings want.
- TeMPOraL 11y agoKeep in mind that this does not sidestep the biggest practical concern with AIs, namely misalignment of values. You don't need a self-aware, conscious being to have a system with wants and values. In context of AIs, it's good to understand intelligence (including that of ourselves) as a very strong, multi-domain optimization process.
- JoshTriplett 11y agoI absolutely agree that the problem remains hard. However, it's not so much that you can't avoid building a system with wants and values of its own; it's that you have to implement a system for how exactly to value what we value, especially when there are a lot of us and we don't all share identical values.