The boost comes from when the attacker does not have the key; and when the attacker knows the plaintext, key recovery is still difficult. AES is designed to be fast.