I've in the camp of never noticed my terminal emulator being slow. Used iTerm2 for 10 years, switched to Alacritty recently for reasons unrelated to performance. Before iTerm2 on Linux I don't really remember what I used but I don't remember it being slow. Haven't noticed scrolling speed being slow, and I don't get this "insert latency" obsession.