According to Japanese WikipediaJapanese Wikipedia:
- Volume 1 covers Season 1, Episode 1 (S1E1) - S1E6
- Volume 2 covers S1E7 - S1E10 and S2E3
- Volume 3 covers S2E1 - S2E5 except S2E3
- Volume 4 covers S2E6 - S2E10
So, you might want to start from Volume 5.
However, take a moment to read Is the Konosuba anime a close adaptation of the light novel? to decide whether you want to read from the beginning instead.