In this type of scenario, it's sometimes because there are two or more implementors that are trying to develop the standard in a way that makes their existing implementation compliant. Likely there were existing scripts that relied on the behavior being specific to that implementation, so it was explicitly made optional, which is viewed as better than not adding the behavior to the standard at all. "May" is just a polite standards way of saying "Warning: check yourself before you wreck yourself."
