He never said anything to them because all they did was talk about everyone. He never talked about anyone, if he has something to say he would said it to their face. They want everyone to chang, but never want to change themselves. He knew he wasn't perfect, so he never said anything about anyone.
He loved them but kept a distance since they wanted him to alter himself. He was over the fact they would pick and choose when to accept him. He went through change they dk his history or back story. He just wanted to be accepted for him out respect for his elders he kept his mouth shut.