“ But the sturdiest offender of your peace and of the neighbourhood, if you rip up his claims, is as thin and timid as any; and the peace of society is often kept, because, as children say, one is afraid, and the other dares not. ”
 Ralph Waldo Emerson, Prudence (1841). copy citation

add

“Every man is actually weak, and apparently strong. To himself, he seems weak; to others, formidable. You are afraid of Grim; but Grim also is afraid of you. You are solicitous of the good-will of the meanest person, uneasy at his ill-will. But the sturdiest offender of your peace and of the neighbourhood, if you rip up his claims, is as thin and timid as any; and the peace of society is often kept, because, as children say, one is afraid, and the other dares not. Far off, men swell, bully, and threaten; bring them hand to hand, and they are a feeble folk.
It is a proverb, that 'courtesy costs nothing'; but calculation might come to value love for its profit. Love is fabled to be blind; but kindness is necessary to perception; love is not a hood, but an eye-water.” source