Observations of narrow radio-emitting filaments near the Galactic center have been interpreted in previous studies as evidence of a pervasive vertical (i.e. \perp to the Galactic plane) milliGauss magnetic field in the central 150 pc of the Galaxy. Such a magnetic field would have important implications for the central environments of spiral galaxies and would, for somewhat complex reasons, be a key piece of evidence in support of a pre-Galactic origin of the Galactic magnetic field. This paper addresses the question of whether such an intense vertical magnetic field, whose pressure significantly exceeds the thermal pressure of the surrounding medium, could be confined. A simple cylindrically symmetric model for the equilibrium in this central region is proposed in which horizontal (i.e. \parallel to the Galactic plane) magnetic fields embedded in an annular band of molecular material of radius 150 pc are wrapped around vertical magnetic fields threading hot plasma. Since orthogonal magnetic fields can not easily interpenetrate, the horizontal magnetic field in this equilibrium can transfer the weight of the molecular material to the hot plasma and vertical magnetic field, potentially providing confinement. The stability of this equilibrium is studied indirectly by treating the equilibrium as an infinite isothermal slab, with high-density plasma representing the partially ionized molecular clouds suspended above low-density plasma, and with the orthogonal magnetic fields in the high- and low-density plasmas perpendicular to the effective gravitational acceleration g (true gravitational acceleration minus the centripetal acceleration of Galactic rotation). Perturbations of the interface are found to be unstable on wavelengths shorter than a critical wavelength that depends upon g and the sound and Alfvén speeds in the high-density plasma. The results of the slab-equilibrium calculation are extrapolated on a qualitative basis to argue that the proposed cylindrically symmetric equilibrium is unstable if the magnetic pressure of the vertical magnetic field B_vert greatly exceeds the thermal pressure in the molecular material, or if B_vert \gg 50 mu G for Galactic-center parameters. If this qualitative argument is correct, then milliGauss vertical fields can not be stably confined using the weight of the molecular material in the kind of equilibrium that has been proposed. If an equilibrium can not be found that can stably confine pervasive vertical milliGauss magnetic fields at the Galactic center, then a powerful argument will be added in favor of other interpretations of the Galactic-center radio filaments.