I've heard this term used in a few places, notably on one of the Flavor Day nominations. A quick Google search shows me that it started in 1985 but there doesn't seem to be anything extensively documented (mostly because there's a lot of noise in the search results). What is it?
The OVA boom describes the exploding popularity of direct to video anime during the 1980s and 1990s that started when some of the very first OVAs became commercially successful in the 1980s. Specifically the OVA "Megazone 23" became one of the first popular OVAs of the 1980s and sparked a chain reaction of studios making OVAs.
Back in the day the only serious way studios thought to make money off their anime was to either broadcast on television or make it into a movie. (Dragon Ball, Grave of Fireflies) Well, the introduction of the first successful "direct to video" animes showed studios that it was possible to make a lot of money off of sending anime straight to VHS or DVD for people to buy. This served as a milestone in the anime industry as the first time OVAs became a credible marketing model for anime.
This OVA boom in the 1980s did a number of things. First, it gave the anime studios a lot of wiggle room as to what they could turn into anime. When making anime for TV or movies the only choices are to make a long running plot or an incredibly short plot. With the OVAs an anime series could be of varying lengths and run times as long as they told the story. This gave anime writers more freedom as to what the plots of their anime could be since they didn't just have to write for TV or movies. Writers were no longer bound by the format of anime, they could write what they wanted and turn it into a 13 episode series for OVA release.
OVA popularity also meant that anime studios could spend more money making better quality anime since they could put more money into making 13 episodes as opposed to having to save up money to animate a long running TV series.
The OVA boom basically is the rise of the OVA in the 1980s as a result of studios seeing OVA as a credible way to make money off of their anime. If I got a couple of details wrong, I'm sorry about that but I hope this answered your question.