Our current approaches to the regulation of system risk management and prevention of deadly accidents remains very much deterministic. In the most critical applications such as in nuclear power plants or aircraft controls, regulatory authorities require a deterministic demonstration of the links between input and outputs. Superfluous code that is not used needs to be removed, just in case. Older processors are used which reactions are fully known.
With the advances of Artificial Intelligence, this won’t be possible any more. In particular because the devices become black boxes that have learned to behave in a certain manner most of the time when exposed to certain stimulus. However deterministic proof of the relationship between input and output is impossible and we don’t quite know how it really works inside. It can only be a statistical measure.
This situation is an extensive challenge for the regulatory authorities that will have to regulate safety-critical applications based on AI such as automatic driving. Most current regulatory approaches will become obsolete.
Some regulatory authorities have identified this challenge but most have not, although this will constitute a real revolution in regulation.