It absolutely does. Improving your page speed (or google's idea of your page speed) is a critical step in optimizing a site's organic google search ranking.
Why do people feel the need to "fight back against Google"? Should their actual fighting energy be going to fighting dictatorships around the world and torture by the CIA? Priorities are really messed up.