Why is it that the characters' faces do this at various points in the dialog and what is it meant to convey? I wonder if perhaps this is a Japanese culture or anime culture expression that I have not seen before?
Based on the actions of the characters and Yuma's unique personality, I believe it's more of a "I get the reason, but not the logic" like understanding that the kettle is hot without understanding why, or rather not caring why.
The little kid in Tamakoma does the same face, as Yuma when it come's to our world may as well be a child. he also frequently say's "naru hodo" which roughly translate to "now I get it" but in reality he doesn't.
This is a stylized squint. It is not specific to World Trigger or Yuma's personality.
This stylized squint is a variation on using horizontal lines to indicate eyelashes. This was popularized in shoujo manga, where the eyelashes were originally very thick.
Over time, eyelashes have generally become thinner.
And over time, increasingly abstract versions, such as a sketch of horizontal lines, have emerged.
Yuma's 3 horizontal lines is an abstraction of the standard horizontal-line eyelashes, which in this case conveys acting like one is trying to understand or agree: squinting indicates thinking so hard that you end up squinting from the strain on brain power and/or on account of focusing by shutting out your surroundings. Sometimes it is used when a character is thinking 「なるほど」 (naruhodo = "Oh, I see..." or "Oh, so that's what that means") whether or not he or she actually understands what happened/what was said. Here you can see Teruo from Ore Monogatari!! doing the same thing with an abstraction of 1 horizontal line.
Sometimes it is used for squinting for vision purposes, such as when a character takes his/her glasses off and cannot see clearly. This is based on real Japanese people imagining that squinting improves long-distance vision when they do not feel like wearing their glasses or if their glasses prescription is too low (a very frequent occurrence in the Japanese school classroom). The 3-shaped abstraction is a development in which the horizontal lines become a very stylized scribble (the artist does not pick up the pen between each horizontal line, so the squint is connected into a single line).
Sometimes it is used to indicate sleeping with closed eyes.