R' Shalom Dov Ber of Lubavitch writes in the Kuntres HaAvoda:
He who worries about his soul, not to bring it to impurity, will fence himself in what respects to the eyes, and if this is difficult for him, he should strengthen himself with all his power and might. And he should know that his soul depends on this. And if we will not be careful about this, all his service is as nothing and nothing will he get out of all his service, and on the contrary, G-d forbid he will descend low very low…And he should not worry about what others say…and this is very hard in the beginning…but with time he will get used to guarding his eyes. And through this he will find rest for his soul from many evils and confusion and then he will be able to serve Hashem and he will work salvation for his soul with the Help of the Blessed One.