If you don't want people to see that you have declined an FB event, you should 'remove yourself from it' (I forget the exact terminology) rather than declining it.
Perhaps the fact that these are distinct actions is a problem with the FB event RSVP interface.
