While genes must play a part in this (if they didn't, all non-humans would also share our IQ*), genetics shift on a much slower timescale than the entire history of IQ tests.
* This pattern matches to the Motte-and-Bailey rhetorical technique, ergo I am suspicious of people who try to tie genetics and IQ until they're clear they're not making a racially charged claim. Last I checked, there is no real evidence that human races are a meaningful genetic category, let alone that anything usually described as "race" correlates to any genes connected to IQ scores.
Because where you draw the lines for "race" is down to your own culture, not a constant dividing line that all cultures agree upon.
To an American, race may be e.g.: {White, Black, Hispanic, Native American, Arabic, Asian}.
To most Europeans, everyone who an an American would call Hispanic, we'd probably either call "Caucasian" (i.e. white) if their heritage is more from the Spanish side or Native American if their heritage is more from the pre-Columbian (Aztec?) side.
If you're Chinese, they may say the ethnicities are "Han, Zhuang, Hui, Manchu, Uyghur, …" where those are all ethnic groups within China.
Rwanda, infamously, would get you a distinction between Hutu and Tutsi; if you show me a picture of two people and ask me which was which, I wouldn't be able to answer, or even know if I was being pranked with any of the other ethnic groups in Africa.
But more broadly, while skin colour is easy to spot from the outside, it's about as useful as hair or eye colour when it comes to correspondence with the huge range of invisible genetic variation.
While genes must play a part in this (if they didn't, all non-humans would also share our IQ*), genetics shift on a much slower timescale than the entire history of IQ tests.
* This pattern matches to the Motte-and-Bailey rhetorical technique, ergo I am suspicious of people who try to tie genetics and IQ until they're clear they're not making a racially charged claim. Last I checked, there is no real evidence that human races are a meaningful genetic category, let alone that anything usually described as "race" correlates to any genes connected to IQ scores.