Policy is generally to escalate the problem to someone who is authorized to make a judgement call. Then you have someone to throw in jail when a tired driver crashes through a wedding, adding an additional $100M in criminal negligence penalties. You probably don't want your AI to be making judgement calls.
I admit to not reading most of the paper, but afaict the setup here is that the authorized person *has" made the judgement call and is asking the AI to implement that judgement and we're looking at whether the AI pushes back.
