In an at-will employment environment, sure, maybe. Anywhere else it's definitely not enough to be let go for sexual harassment. One remark is nothing.

But these are the kinds of people you don't want to hire.

Maybe a self-employed shop?

This isn't necessarily "firing offense." It is "fireable offense" and if you aren't sweating through your talk with HR you are messed up.

And if your company doesn't have an HR department: congratulations, now they need one.