He was tired of everyone coming at him. once he stood up his ground they played the victim. He was never considered unless they needed or wanted something from him. He was always excluded on the outside looking in.
They made up rules to keep him out use to being excluded. He didn't care for these made up rules that weren't in his favor. He was thankful for what he had; while they had it all still talking about everyone. No one owes them anything.
Let people live their life, owning a house doesn't give you the right to treat others bad look down on them. He worked for his helped family not expecting anything in return.
He didn't care to answer deal with those people. He wasn't perfect no way near it and the last person that should be talking about anyone. It angered him, he walked away but they kept provoking him robbing him of his peace.