According to Japanese Wikipedia,
Using volume 1 and volume 7 (the episode where Yuuta's doing daily life with the three sisters at an apartment in Hachioji) from the original work (light novel) as the base for the anime's original story, original characters were also added. Also, there are some different settings between the anime and the light novel, like
the Takanashi couple died instead of missing.
(Emphasis mine)
Since volume 7 is a collection of short stories and the anime's story is considered original, I can't really suggest continuing from volume 2, but instead just start from volume 1.
Regarding the 2 OVAs, I think both of them are original stories. Both OVAs were included as a bonus for the limited edition of volume 13 and 18 (the last volume), but Wikipedia doesn't mention if they adapted from them instead.