A better comparison might be moving to a different country and getting a job there when you don't completely understand the culture and possibly have a language barrier (which I've gone through). You don't feel unsafe, but you don't quite fit in, you feel homesick, you don't mesh well with your coworkers, and you feel a constant background radiation of insecurity because you aren't used to the legal system, the social support system, etc. Your coworkers might make slight jokes about you. They mean well, but it sure as heck doesn't help your situation.

That feeling really (really) sucks. It must suck ten times as much when it's happening in your own country. I really feel for people who are forced to experience this.