Jim was over dealing with people who asked his opinion and they wouldn't listen. He was over all the crazy that's why he was disconnected.
He cared he got the cold shoulder. The family would include him bring in on the drama. Jim refused to let them bring him down, so he mind his mouth and business.
After a few bad relationships; he chose to be single. Burned by a few friends it was easier to be alone. He didn't want people in his way, or slowing down.