( Sorry about my bad grammar and vocabulary, feel free to edit my answer )
##Spoiler Warning
The content from this point on contains spoilers. Read it at your own risk.
From what i could figure, the most accepted answer for this kind of question is you can find it in the manga version. The anime version seems lack of explanation. And of course i have finished the manga.
I haven't watched the anime, well i do watch that but just some scenes, skipped some part.
( i watched the first minute part where shouya lost his bicycle and met nagatsuka. I watched it on youtube)
1. What is the significance of the X on people's faces from Ishida's perspective
It was started in Volume 1, Chapter 5, page 2-6 Here are some of the pages (read from right to left)
Page 2-3
Page 4-5
Page 6
He have an
anxiety disorder with other people. He feel pissed off because his own negative thought to the other. Look at the yellow marker, he hate everything, even the friend of friends that he hated. The "X" mark can be symbolized as "Not My Friend" or "Not interest to interact with them". So that's the reason why the entire class have "x" mark on their faces. For Nagatsuka why he got "X" mark from the first, there is something that Ishida hate from him. For The other people like Sahara that doesn't have "X" Mark when they met on the train, because Ishida have no reason to hate Sahara for some reason ( exclude his class mate nor neighborhood that he hate ). He will put the "X" mark down when he someone want to acknowledge him more and he trusted it.
2. Why does Ueno get the X on and off ?
In Volume 3, CHapter 18, Page 12-17 , Its explained (read from right to left)
page 12-13
page 14-15
page 16-17
Ishida said that he doesnt care about his ex-friend ( aka all his elementary school friends ) in page 12-13, but when he met ueno for the first time since in junior high school, he did care. At first, Uone's face didn't have the "X" Mark but in in chapter 21, page 9-10, Ueno did something terrible to Shouko, made Ishida really pissed of with Ueno attitude. So That's why Ueno got The 'X' mark on page 15-16.
Page 9-10
Page 15-16
TL;DR
So the summary of this Answer is:
The "X" mark can be symbolized as "Not My Friend" or "Not interest to interact with them". When He feel pissed off because ,act, attitude of someone/something, the "X" mark would be arise. And when would the "X" mark taken down ? When Ishida feel happy towards someone or something
Because Ueno is so hypocrite, that make Ishida so pissed and happy and the same time. The X mark on Uone's face was first marked in volume 3, And it was put off in Volume 7 ( The last volume ). When Ishida tried to change himself. In volume 7, all the x mark that everyone had on ishida perspective already have put off.