Was Deep Space Nine competing with Babylon Five?

Sure. Babylon 5 was one show that launched in syndication and later on cable. Babylon 5 tried to develop its own, different vision of the universe, whereas Deep Space Nine still lived within the Roddenberry vision of a more perfect, utopian universe

Babylon 5 was able to be a little freer and more adventuresome. And that appealed to some fans who had grown tired of the same old ethos of Star Trek.

For the most part, though, sci-fi had not taken off in network prime time. There have been very few successful sci-fi shows in prime time in the '90s and today. It just isn't a genre that generally does well. Star Trek is in the exception.