I have been reading Edward J. Watts' The Final Pagan Generation. In it tells of the lives of the last Roman pagans born before Emperor Constantine converted to Christianity (312 A.D.). He examines why this last generation, born to the traditional ways, and dominant in most positions of power, proved both unable to anticipate the imperially sponsored changes and unwilling to resist them.
At the point we are discussing here, the book has arrived at the time of Constantine's older son, Constantius's, reign (337 - 361 A.D). He has cracked down harder on the temples, at least officially closing the urban temples so that they may not be used for 'superstitious' sacrificial acts. So why don't the upper elite, still a majority of whom would be pagan, not object?
There was little direct criticism and no violent protests [led by pagan elites]. They had too much to lose and little sense that resistance was necessary. Constantius's political retribution touched only a few unlucky or stupid members of the elite, and his religious policies were largely ineffectual. Sacrifice continued despite Constantius's ban, temples remained open despite his injunction to the contrary, and the emperor himself even toured the (still open) temples in the city of Rome when he visited [from the new Capitol of Constantinople] in 357. The gods remained present everywhere in forms that could be seen, heard, smelled , and touched in every city across the empire. Constantius's policies may have been disagreeable, but they hardly seemed to be a pressing or universal threat (p.89).
Note, that there is the other side of the story, why it is that Christian's actively overcame the opposition, but the question is why the Pagan's didn't do anything when they still had a chance. Note, that it wasn't required that they wipe out the Christians. That would have been difficult. Simply that enough be done to keep the Emperors from feeling safe about pushing religious restrictions. Or better yet, simply deposing Christian Emperors.
You can say it is because the Emperor had the army behind him, but that just begs the question of why an almost certainly pagan army would go along with anti-pagan policies. The author doesn't go into the attitudes of the folks serving in the legions, but likely it is very similar to why the elites didn't do anything.
The problem is that the effective elites, and the military saw themselves as being on the winning side. Through luck or cunning they had chosen the winning Emperor (there were usually more than one contender during any reign) and they were compensated appropriately. Look how quickly the elected Egyptian President lasted when the military felt it's position threatened. The Christian Emperors would not have stayed in power without support from many pagans.
So in effect, the early Christian Emperor's policies accelerated the growth of Christianity, but the Pagan majority (likely 80% of the population at the start of the period) where not sufficiently alarmed enough to take action when they could. Constantine converted in 312, by 392 a mob of Christians are able to storm the great pagan temple, The Serapeum in Alexandria, and destroy it.