Click the down button: it'll knock them down another karma point and fade out their comment more, but it won't display lower than -4 to prevent pile-on downvoting to -50.