I definitely have the bias you mention. In my case I don't think it's mainly due to not wanting to push the same button many times in a row, but because I compare with the previous color, so if previously I was already somewhat unsure but I chose green and now it became slightly bluer, it "must" be blue, right?
I think I can get over it, but it requires conscious effort and even then, who knows. Bias is often unconscious.
Another possible improvement would be to alternate the binary search colors with some randomly-generated hues. Even if those answers are outright ignored, and the process becomes longer, I think they would help to alleviate that bias. At least you wouldn't be directly comparing to the previous color.
I think I can get over it, but it requires conscious effort and even then, who knows. Bias is often unconscious.
Another possible improvement would be to alternate the binary search colors with some randomly-generated hues. Even if those answers are outright ignored, and the process becomes longer, I think they would help to alleviate that bias. At least you wouldn't be directly comparing to the previous color.