People assume everything about you. Just because they do not see your good deeds does not make you a bad person. Every good deeds you do is between you and God and humans do not necessarily need to see every single thing you do. Humans are not judges. God is. I usually will close the door when I pray and people will come up to me and say "why you haven't pray? We are Islam!"  I really take that personally. So please, words can be poisonous if you do not think before you speak.