If you are bothered when people do not understand you, you still have too much pride. When they are the ones who are closest to you, it becomes even more difficult to remain calm. Know that what others think about you doesn't matter. They don't understand you because they are not all-knowing like God. Only what God thinks about you matters. Since God always understands us, we only need to be concerned about not sinning. Therefore, never mind the others and focus upon being holy yourself. Then no one, not even God, can bother you anymore.