A bunch of armchair devil's advocating here, but is it really the NN that's fooled or the humans? The adversarial turtle isn't a real turtle, so the human is wrong in judging it as that. The NN is presumably seeing features of a rifle camouflaged in the surface of the object - which are really there but our human brain decides the turtle-ness is more important and is very confident that it's only a turtle despite having a rifle stock on it. Since a real turtle would never have those markings, it's not obvious to me that this object should be called a turtle. The NN could be doing a super-human job of detecting that it's not a turtle, but fails in identifying what it really is. Maybe this weakness of the NN would actually make it perform better than a human at picking out camouflaged objects where humans are distracted by the shape of the outline but the NN looks more at the texture.
To make your experiment more interesting you could theoretically (to avoid torture) overpaint real living turtle. Who would be more wrong then? But this work gives bunch of hope for fooling future security check machines - these will see your riffles as turtles.
Please don't talk about murdering the person you're talking to. It's intented to provoke painful emotion.
I'll clarify my comment though. The object is really a model that's shaped like a turtle but with pictures of rifle parts on it. It's neither a rifle nor a turtle. Both human and computer are too confident in their classification and both are just as wrong.
Using your analogy. You could actually hide gun inside a lunchbox and fool humans.
That doesn't follow because the pistol painted to classify as a lunch box is still a pistol.
The "Turtle" is a plastic replica of a turtle not a real turtle. It's the treachery of images idea - "Ceci n'est pas une pipe."
Humans see its form and recognize the plastic replica to be a representation of a turtle because we prioritize its shape over its textured image which seems more correct to us, but I'm not sure that it really is more correct in some objective way. In this case I suppose you could say it is because a turtle is what we mean for it to represent, but the test seems rigged in favor of human visual classification.
I think an interesting question is what adversarial attacks exist on human vision that may not affect a machine (certain optical illusions?). If we're also vulnerable to this kind of manipulation then it may not be something unique to computer vision we may just be picking test cases that we're better at. Then it's just a matter of tradeoffs and deciding when we want human style classification.
I agree, but it's an unfair test - it was designed to confuse the computer and not the human.
For a counter example - imagine that you make a toaster that looks exactly like a pistol, but it actually just toasts bread.
A human would think it's a pistol when looking at it (so would the machine in this case). There may be adversarial examples where the human classification is worse than the machine if you specifically try and make examples that are bad for the human visual system.
