Line beside text in H1 and H2 tags

if you open the heading in your layout – where you enter the content ( heading text ) of the heading one option under it you can choose the heading tag.
on the bottom of that tab there is the heading style. These littel borders belong to the classic style. – if you go and choose modern style these borders are gone.

But if you have already set many of these headings, changing them manually might be too time consuming.
Put this to your quick css:

.special-heading-border {
    display: none !important;
}