In theory, any impact from longer addresses would be outweighed by the benefit of the shorter non-CIDR routing table (and in turn that should be outweighed by avoiding NAT, and that should be outweighed by avoiding CGNAT). (Plus with most systems being natively 64-bit these days, that impact should be 0 - the routable part of an IPv6 address is 64 bits, and comparing a 64-bit value is no harder than comparing a 32-bit value).
In practice IPv6 is newer, which has good and bad sides; IPv6 routing paths are more likely to be using newer (and therefore faster) equipment, but there's also a bigger risk of someone making a mistake that messes up your routing/latency, particularly if your ISP hasn't been doing IPv6 for very long.
In practice IPv6 is newer, which has good and bad sides; IPv6 routing paths are more likely to be using newer (and therefore faster) equipment, but there's also a bigger risk of someone making a mistake that messes up your routing/latency, particularly if your ISP hasn't been doing IPv6 for very long.