The US denounced Japan's invasion of Manchuria in 1931 and refused to recognize that as new Japanese territory. A military response was not then considered, but to say that nobody in the US cared is false.
Edit: More so, the fact that America's initial response to Japan's imperialism was diplomatic rather than military undermines the narrative that America was just using Japan as a pretext to cover for America's own imperial ambition. America, far from leaping at an opportunity to invade Asia, was committed to diplomatic resolution of the matter until Japan's imperial ambitions drove them to directly attack the American military, forcing America to respond in kind.
Edit: More so, the fact that America's initial response to Japan's imperialism was diplomatic rather than military undermines the narrative that America was just using Japan as a pretext to cover for America's own imperial ambition. America, far from leaping at an opportunity to invade Asia, was committed to diplomatic resolution of the matter until Japan's imperial ambitions drove them to directly attack the American military, forcing America to respond in kind.