TL;DR: the anime studio doesn't own the complete rights to the anime, so in most cases there will be no problem in the case the studio goes under.
The owner of the rights to an anime in modern times is almost always a 制作委員会, which I usually translate as "production partnership".
This is a partnership that is set up in order to fund the anime, while splitting the risk with other sponsors. This includes companies like the publisher of the original work, the music company, figure companies, advertising companies, TV stations, etc. This may or may not include the anime production studio itself. Either way, in the case the studio ceases to exist, this would not affect any agreement with foreign publishers, as the rights are still being managed by the production partnership.
It should be noted that just because a company goes bankrupt doesn't necessarily mean it loses all capability to create anime (depending on the type of bankruptcy). There have been several cases like this, for example Tatsunoko Production. Mangrove, whose anime you have mentioned, has ceased to exist as of 2016, but there has been no removal of the anime (and Mangrove has never opted into the partnership, so they never had any rights to the anime itself anyway). It does seem that Mangrove's bankruptcy affected Gangsta's BD release, but I would assume that had more to do with the common practice for anime studios to brush up and fix TV anime for the disk releases.
There have been cases where studios go bankrupt and production partnerships enlisted a different studio to produce the sequel. Strike Witches and Saki are examples of this (in the case of Saki, Studio Gokumi is a studio founded by former Gonzo employees).
Sources (all in Japanese):