Literally no one is arguing to the contrary. The problems people have with immigration into the US typically involve the sheer scale of it, the fact that many people who come to the United States do so illegally, difficulty with integrating, and difference in values from native citizens. Nobody is arguing against small-scale immigration of highly skilled scientists, there aren't even enough of them to make a demographic difference.
I'd say the current stories about people enforcing poorly written rules at the border prove you wrong.
People say yes to scientists, but not if they're from certain countries. On top of that we have scientists from Europe being stopped at the border, making it unlikely that they or their compatriots may try to make the trip in the future. I know of at least one computer conference that has discussed moving the venue outside the US to try and mitigate the issues at the border.
While you may be right that no one debates elite people in their respective fields from moving or coming to the US, the enforcement is causing exactly the opposite to occur.
So, I respectfully disagree with you that what you suggest would be acceptable.
A few days ago some Indian engineer was shot dead because he "looked Iranian". There is a very strong political reason why people like this killer do what they do.
Perhaps I should have clarified: by "literally nobody" I meant "virtually nobody", as in "the number of people who believe that we should accept no immigrants whatsoever is very small and statistically negligible".
