To answer this question, I'll have to delve into SPOILER territory -- be warned !
First off, just so you know, the Kotodama Devil's true name is Zagred. Zagred was immensely powerful -- and his atypical, magical biology tremendously helped him to power through enemy attacks. Every character who has been able to hurt him or impede him in some way have been connected to his native underworld.
Asta used anti-magic to negate his attacks and slash through his magic-enhanced defenses. His anti-magic powers stem directly from an unnamed demon, so it stands to reason that Asta can hurt Zagred.
Yuno exploited Zagred's magic emissions to enhance his own wind spells and attune them to the Kotodama Magic's "wavelength". He basically used Zagred's own power against himself.
Lemiel, Patolli and Licht were able to hurt Zagred because they were all tainted in some way by underworld magic. Lemiel had been sealed by a demonic spell from Secre, and both Patolli and Licht had been resurrected by the demon's spell. Their corruption allowed them to strike at Zagred.
Yami's Dark Magic is known to be linked to the Underworld. In Chapter 246, Dante tells us that Dark Magic is necessary to create the Tree of Qliphoth along with William Vengeance's World Tree Magic. The hows and whys haven't been explained yet, however.
In theory, any other character who has been tainted by demonic magic or who is able to use demonic magic in some way (by reflecting it like Yuno, for example) can hurt the Kotodama Devil. Since other devils will intervene in the story, we will no doubt discover new ways to hurt them as Black Clover progresses.