I honestly read this article as taking more of a philosophical stance than a technical one. The fact is that we as humans will require proof and verification of human control over a system is an inescapable one. Basically, on some level we need guarantees and proof that the machine will never come up with a rule that says "kill all humans".
This means that we'll need human-supplied rules, and any mature ML platform will need to be able to incorporate these rules. IMO it's a decent problem-based hypothesis as to what the future evolution of ML platforms will look like, but as with any hypothesis, it will need to be tested, proven wrong in some way, and adjusted accordingly.
The word "need" in this comment seems a bit exaggerated. Yes, we'd want to have some proof and verification, but that is not an inescapable requirement.
I'm fairly sure that we as a society in general are willing to launch systems where we have only a reasonable expectation and trust that it will most likely work properly instead of total proof and verification that it definitely will do so. Just take a look at pretty much any life-critical system in use today; some verification and testing is required, but formal proof of correctness is a very high bar that's never required for practical systems. It's highly desirable, but it will only be an absolute requirement if it's reasonably easy to achieve.
I'm fairly sure that if it turns out that we can't provide guarantees and proof about what the machine might do, then we'll just do our best even if the result is not provable and verifiable and launch such systems anyway.
And I think that even if we have a choice between two systems where one is verified and proved that it never can do anything bad but is otherwise inferior in what it can do, and the second system has no such proof but seems safe in general testing and simply performs better than the first one.... then it's quite likely that we'll choose the second one anyway.
That is my opinion as well. And indeed the post talks mainly about dynamic verification and achieving some "good enough" verification quality (as determined by coverage and other metrics).
And it is in that context that "soft" techniques like ML can help a lot, and thus the question of how to connect them to "hard" rules (which are also part of dynamic verification) becomes interesting.
Is there a hard boundary between "the philosophical" and "the technical"? Or rather, I don't think "the philosophical" is necessarily that useful a tool for understanding how to resolve the thorny question of "human control" of AI.
Human beings are very good at communicating our "intentions" through natural language. Such communication, however involves "good faith". So communication with an AI would seem to involve more than just following logical directive, the AI would need a model of a person which would allow it to deduce and follow the wants/desires/intentions behind a given group's humans' communication with it.
Of course, if we go down deeper, we can notice that these wants/desires/intentions themselves "don't really exist" in the sense that a single person generally has impulses that go in different directions in the general case and it is the way that they live in a stable society that keeps those contradiction from appearing (which brings to mind many "sausage and sausage factory" analogies - lots of people want the results of a process but would object if they knew everything that goes in - while vaguely knowing/hoping there is a bit of regulation keeping things not too far out of bounds).
Traditionally, philosophy is where conceptions of language, society and logic meet and supposedly get sorted-out. But unfortunately our traditional philosophy seems a woefully inadequate tool for the problems of language interactions coming out of social processes and we most likely should look at some more contemporary theories of language and society for this (standard philosophy especially arose strengthening the false intuition that an intention could exist as a "free-standing" rather than getting that interactive fabric that human society and just as much the idea that a phisolophical would not be "technical" illustrates the problems). My favorite for understanding this stuff instead, is evolutionary game theory.
I indeed meant it in the philosophical sense you describe. But I am very interested in the possible technical solutions. I tried to describe (in the chapter "Connecting ML and rules") the approaches I know of, none of which are very exciting.
I'd love to hear if anybody knows of good approaches.
As part of my graduate work with George Konidaris we've been exploring the creation of symbols and operators with ML. The goal being symbolic planning for continuous systems, however I see similarities in our approach, and the goals of rule based systems.
maybe your test cases are your rules. as long as you're recording things over time you have data to feed back to
and learn from. also, each level of abstraction you could store less data to potential learn from. instead of storing every pixel just store edges and other low level features from the first layer.
don't kill humans as a rule won't work just don't recognize that "meat-bag" over there as a human goal accomplished rule circumvented. or maybe it's lower which colors are skin colors? how do you define valid colors? to define human properties?
At the end of the day I think there is a reason for the old saying the exception that proves the rule.
This means that we'll need human-supplied rules, and any mature ML platform will need to be able to incorporate these rules. IMO it's a decent problem-based hypothesis as to what the future evolution of ML platforms will look like, but as with any hypothesis, it will need to be tested, proven wrong in some way, and adjusted accordingly.