The manga has 9 volumes and one extra volume (titled 'Bonus track').
The "I'm happy Sen is gone" scene with Kaoru failing to convey his feelings to Ritsuko and afterwards telling her that he'll be going to Tokyo is the end of the manga's 8th volume. This scene is at 6 minutes into the anime's last episode.
In short, the following content of manga volumes 9 and Bonus is:
Kaoru leaves to study medicine at a Tokyo university where he rekindles his love with jazz. He kind of makes up with Ritsuko, then finds out she seems to be with another man and cuts all ties. He becomes a doctor at a Kyushu hospital and through a hint by Yurika (who lives with Junichi and later has twins) finds out that Sen became a priest on a nearby island. Sen arrived there after initially having decided to leave everything behind after his sister's accident. Only after realizing how much he loves to work with children does he change his mind. Kaoru meets Ritsuko again many years later, learning she never was in a serious relationship. The manga ends with everyone meeting for a jam session on the island, listened to by Ritsuko who is pregnant with Kaoru's child.
Here's a more detailed breakdown by chapters. For the sake of brevity Ritsuko is R. and Kaoru is K.
- ch41: On graduation day R. ignores K. He keeps thinking about Sen, but tells people that all this will soon be behind him and he will focus on his studies in Tokyo. K. shouts his apology at R.'s window, same as in the anime. The following train scene is slightly less dramatic - they have time to exchange a few words. R.'s dad thinks that Sen might have gone to Tokyo, too.
- ch42: K. is at university in Tokyo. With the student strikes still underway life seems easy going. He meets a girl at a party who invites herself over to his place for the night, but thinking of R. he sends her away again without anything happening. While visiting his mother, she sings Lullaby of Birdland waking old memories in him.
- ch43: The strikes end and instead of partying, K. starts playing piano again after joining the jazz club at university and taking on a part-time job in a Shinjuku bar. The stress of both the job and upcoming university exams has K. at a low when he finds a photo of him and Sen in the bag R. gave him on his departure. The note on the back reads To the two idiots. Friendship is for life. Since 1966, and forever. Inspired by this K. changes his mind and begins asking around at work for Sen. He also finally answers a letter he received from R. in ch42.
- ch44: Junichi shows up at K.'s work and promises to help looking out for Sen as well. K. quits his job in favour of his studies but keeps exchanging letters with R. Over time, her's become less frequent and shorter. One day a telephone number is on a postcard from R. Upon calling the number K. is greeted by a man telling him to leave R. alone. K. assumes the obvious. At his grandfather's funeral K. is told the he is expected to inherit the formers hospital and become 'the pillar of the Nishimi family' (implying upcoming marriage interviews). K. sees Seji, who is now a celebrity, in the TV and realizes that unlike Seiji, he hasn't been following his dreams. Shortly after K. looses the photo of him and Sen and fails to find it again. Yurika (Junichi's girlfriend) visits K. in the hospital and gives him the photo of Sen as a priest together with the address on the island.
- ch45: We see Sen as a priest in training on the aforementioned island. He seems popular with the kids and islanders. The sound of Moanin' being played on the organ draws him back to the church where he meets K. and they promptly have to run away from the head priest angry at the unauthorized organ usage. After a jump in time we learn that K. turned down the inheritance of the hospital and instead joined a hospital in Kyushu. At Sachiko's wedding (in Sen's church), K. meets R. again and learns that she never really was and certainly isn't in a relationship with the guy on the phone anymore. R. learns that K. isn't going back to Tokyo.
Bonus Track (Volume 10):
- Track 1: Yurika is living with Junichi and is having trouble finding a job because she didn't graduate high-school. By chance, she finally finds one in painting sign-boards. While she is increasingly occupied with work, Junichi seems to act cold and shows that he does not seem to think of a future for him and Yurika. It takes another train station scene with Yurika on the verge of moving out and out of the city for Junichi to admit his feelings.
- Track 2: Kouta (one of Sen's brothers), now 14, is shown picking up jazz drumming in the record store's basement. He spends some time with R. who is over around the time of New Year's and they meet the guy who answered the phone when K. called R. back then. He doesn't take No for an answer which leads to an intervention of Kouta, and R. openly admitting to both of them that she still loves K.
- Track 3: Backstory of Tsutomu (R.'s dad). He was introduced to Jazz by a guy called Kenji. Kenji often played Jazz with his brother and Tsutomu used to stop to listen to the sound he heard from the street, until Kenji invites him in one day and starts teaching him bass cello. There he also meets Fumi who has an unnamed illness and sneaks away from the hospital to spend time at Kenji's. With the second world war intensifying, Kenji goes off to join the naval band and Tsutomu is drafted for factory work. Kenji dies in the war and Tsutomu spends some hours with Fumi (who is still in the hospital) after not having seen her for a long time. During the following air raid Tsutomu saves Fumi's life but afterwards finds himself devastated by the destruction. To cheer him up she drags the cello to him and he plays solemn notes to gathering survivors amidst the rubble. Opening the record shop is fulfilling the dream Kenji had ('to spread the music of the world to this town and gather musicians to play together every day').
- Track 4: Explains what happened to Sen after his sister's accident. Sen is found, almost drowned, by a fisherman. While staying at the latter's place to earn money for traveling on, Sen becomes friends with a younger reclusive orphan. Playing on improvised drums with her and the other kids in the village Sen remembers how much fun he has a around children and decides on his future career. it is initially suggested Sen might have tried to commit suicide by throwing himself into the sea. However, when he leaves the town we learn that he jumped into the water while trying to retrieve the orphan's necklace and passed out in the process.
- Track 5: Kaoru, Sen and Junichi all meet up for a Jazz session on the island and learn that Yurika had twins. Sen sees Junichi for the first time since their argument years ago, but seemingly overcame all bad feelings and congratulates the two. Shortly after, Tsutomu and the pregnant Ritsuko arrive as well. The guys start arguing about the instrument the child will pick up, everyone of course vouching for their own, until Ritsuko reconciles the discussions with her suggestions of the saxophone.
Concerning chirale's comment, I did not find anything specifically mentioning whether the series was originally planned to be longer. However, be aware that the Bonus Track Manga was announced only in January 2012, while the anime started airing already in April 2012. It is very possible of course, that the people involved were talking to eachother, but depending on how fleshed out the story for the additional Manga was (not) at this time, there probably wasn't enough time to fit it into the Anime as well.
In addition, as can be seen above, there isn't really enough story missing to warrant a second cour and some of the stories from the Bonus Track are rather episodic in nature, and therefore hard to adapt into a coherent episode.