The worst part of it is that it's easy to feel like it's your own fault when something like this happens. The real problem was the hiring manager misleading the applicant about what the job would be, but the OP clearly felt responsible for it.

I had a similar experience (working for family, at that), where I joined their R&D group and it went well for a while, but eventually they decided they wanted to get into selling solar power and didn't need a software/electrical engineer, so I was lectured about this or that bullshit and eventually brought into a meeting with the IT director and offered a position on his team. I realize now that they were trying to make me feel unwelcome, but I felt entirely responsible at the time. I never felt like I had a choice in taking the IT position (or lose my employment there entirely), and I hated it. The new boss was, for the most part, a nice guy, but he turned into a major nitpicking asshole over little things.

I'm at a new place now and it's still hard to kick the habit of trying to avoid coworkers because you know they don't like you and want you gone.

Culture has a huge impact on hiring and working. I wish I had been taught about culture when I was in school, I could have avoided the problems I faced had I realized the warning signs earlier.