This is most certainly likely to be Ladies x Butlers!
From the character section of the wiki page:
Akiharu Hino (日野 秋晴 Hino Akiharu)
Voiced by: Kazuyuki Okitsu
The main protagonist of the story, he is often seen as a delinquent because of his red eyes and rough appearance (accented by a scar bisecting his left eyebrow). In contrast to his appearance, he has a caring personality, but is dense in regards to girls and matters of love. Because of everyone's less than friendly perception of him, he is often accused of performing perverted acts on the girls of the academy. He was tormented by his childhood friend Tomomi when they were younger and now that they have become reacquainted, Tomomi is often behind the schemes leading to these misunderstandings.
Here is the plot of it.
Akiharu Hino lost his parents when he was young and was adopted into his uncle's family. He decides to enter a boarding school, Hakureiryō Academy, because he no longer wants to burden his relatives. He decides to take the exam for that school and succeeds in entering the House Management Department, where the school trains servants for high society. He wants to become a butler, but his delinquent appearance frightens the girls, who make up the majority of the students. Being unable to get along with his classmates, Akiharu meets his childhood girlfriend Tomomi Saikyō, a girl with a two-faced character who traumatized him many times when they were kids. Another girl named Selnia Iori Flameheart chases after him because of his looks and calls him suspicious. Still, Akiharu starts finding his feet at the school and his relationships with the girls also get better.