An angle I don’t see people looking at is to reframe the problem with amounts that are much more understandable, there is one thousand times more money in the mystery box, so let’s do the following:
The Open box has 1 cent in it, and the mystery box might have $10, what do you do?
Y’all are telling me you’d rather take a penny and have a tiny Chance at $10, rather than taking $10 with a tiny Chance of getting zero?
You can flip the problem around and have it be mathematically the same. The predictor has some knowable accuracy, you can run the experiment many times to determine what it is. Let’s also replace the predictor with an Oracle, guaranteed 100% always correct, and we’ll manually impose some error by doing the opposite of its prediction with some probability. This is fully indistinguishable from our original predictor.
Now, instead of the predictor making a prediction, let’s choose our box first, then decide what to put in the mystery box afterwards, with some probability of being “wrong” (not putting the money in for the 1 box taker, or putting the money in for the 2 box taker). This is identical to having an Oracle, we know exactly what boxes will be taken, but there is some error in the system.
Now we ask, should you take one box or two? Obviously it depends on what the probability is. There’s no more “fooling” the predictor. So, you do the EV calculation and find that if the probability is more than 50% accurate (in other words, if the probability of error is less than 50%), you should always take 1 box