Using the hanging-punctuation CSS property it’s possible to control whether a punctuation mark should hang at the start or end of a line of text, so that they do not disrupt the ‘flow’ of a body of text or ‘break’ the margin of alignment.

Here’s a pen:
See the Pen mWJJNx by Joel Drapper (@joeldrapper) on CodePen.
☝️ If that’s not working for you, here’s why: at the time of writing hanging-punctuation is only supported on Safari.
To cater for browser that don’t support hanging-punctuation you can (somewhat) fake hanging-punctuation: first; using a negative text-indent, along with an override for browser that do (using @supports):
See the Pen Hanging Punctuation in CSS with @supports and Custom Properties by Dudley Storey (@dudleystorey) on CodePen.
“Hangable” characters targeted by hanging-punctuation are ', ", ‘, ’, “, ”, ,, ., ،, ۔, 、, 。, ,, ., ﹐, ﹑, ﹒, 。, 、, «, and »