The permanent resident permit concerns immigration, not the tax office.

The tax office cares if you have a domicile ("living base") in Japan or possess a residence there for over a year.

Not a tax professional.