I’ve seen a lot of systems, including corporate systems for internal use, reject apostrophes in email addresses (and sometimes even in other fields). Apparently the developers are too lazy to deal with strings properly and fear SQL injection attacks, and perhaps they don’t trust all the other systems they may interface with. So their escape hatch is to prevent these from being allowed.
(“Little Bobby Tables” from xkcd comes to my mind whenever I see these restrictions)
I wonder if this is a good indicator of a bad product/company to be a user of. If they're so uncertain about their tech stack that they have to prevent certain characters from being used in passwords/emails/etc, maybe it's not something you should trust?
Honestly, I would actually consider this best practice. There is absolutely no reason to go the trouble of allowing special characters in emails and fight every system you encounter.
Note that allowing Unicode Letter characters is a whole different topic, and in fact much less risky than allowing random punctuation. At least for the vast majority of people, this is much more important to personal identity than having your name spelled without a quote mark that will anyway confuse numerous systems where you may want access.
(“Little Bobby Tables” from xkcd comes to my mind whenever I see these restrictions)