I was watching the AI Stop Button Problem from Computerphile, and was very confused on the difference between two approaches.
Background: there is a barebones general intelligence running in the form of some robotic device. The device acts in accordance to optimize some utility function. The GI is asked to make tea, and this is the dilemma of how it will choose to press its stop button rather than make tea if the two are given the same reward, as pressing its stop button is easier and yields the same reward as making tea for the programmer.
One approach to try and get the GI to not care about the stop button is to set the reward function of pressing the button equal to what it's trying to achieve (let's say it gets 100 reward to make tea and 100 reward if the button is pressed). Then, he explains how this will cause the robot to hit its own button immediately, because it's the same reward but easier than making tea (explained 5 minutes in). If we tried making the button something only us humans can hit, then it would try to slap us immediately, threatening us into hitting the button, as slapping us to hit the button would also be easier and give us the same reward as getting the tea (explained 9 minutes in).
Then, at 10:00, it says that another, better approach is to define the utility function is set with an adjustment term to make the value of utility of the button being pressed and not being pressed would be, theoretically, exactly equal, so that it would be indifferent to the button being pressed.
I don't see how this second approach is different than the first approach. If, in the second approach, the utility function is defined so that the button being pressed and not pressed is equal, isn't that the same thing as giving 100 reward to make tea and 100 reward if the button is pressed? Wouldn't this supposedly better solution still cause the robot to either press its own button if possible, or slap you to threaten you to press its button? Because they both yield the same utility, but slapping or hitting its own button is easier than what it wants to achieve in life.