That's a nice gesture and sentiment. But this kind of thinking would be a ridiculous way to live. A smile towards someone can be interpreted as you laughing at them, which wouldn't help someone who felt bad about themselves. Yes, you've got to consider this possibility if you accept that train of thought. There are simply too many variables and the state of people's mind is too complex to ever hope to save people from going over the edge by such small gestures and actions, or to avoid sending them over the edge by some totally innocuous and well-meant action.