The fact remains that you would not expect to see black Africans speaking any Afroasiatic language without some kind of significant cultural shock. The reason those languages are in Ethiopia ( / Nigeria / etc. ) today is not that Ethiopia is populated by the descendants of an originally afroasiatic group. Something must have happened.
The Semitic languages are Afroasiatic languages. I agree with your general point but ironically you yourself now try to link language to genetics (skin color).
In the horn of Africa they've always spoken Semitic languages as far as we know, and actually it's possible the entire language family originated in Africa, not in West Asia as I guess you assume. This hasn't been proven one way or another, both theories exist.
Ethopians also just don't really look very much like west African (Niger-Congo speakers, but it is nonsense to presume Ethopians area "hybrid" population while west African Niger-Congo speakers and west Asian Semitic speakers are are "basal".
Niger-Congo does not belong to the Afroasiatic language group. I mentioned Nigeria because of Hausa. Obviously I would no more assume that the Hausa are a basal Afroasiatic group than that any Ethiopian group is.
Nigeria is all the way on the other side of the African content while the Arabian peninsula is right across the very narrow Red Sea. So it would make less sense for Ethiopians to speak a language that’s related to languages spoken thousands of kms away in West Africa than for them to speak a language that’s related to the languages spoken on the other side of the Red Sea - which is exactly what they do.
Edit: also, if you look at pictures of Ethiopians and Eritreans and of Yemenites you’ll see they’re way more similar in appearance than Ethiopians and, say, Nigerians.
Yeah, my wife is Ethiopian and she was shocked to see in TV news coverage how "Ethiopian" a lot of the Houthi rebels in Yemen looked, but this is actually not surprising once you know the histories of the Horn of Africa and Red Sea at a greater detail.