We have observed a vast amount of high temperature (T 250 K) and low density (n 100 cm-3) gas with a large velocity dispersion in the central 200 pc of the Galactic center (the Central Molecular Zone; CMZ) that has not previously been reported. We used the infrared spectrum of H3^+ which is a sensitive probe of low density molecular gas. The observed large column density of H3^+ in the (J, K) = (3, 3) metastable rotational level (361 K above the lowest (1, 1) level) gives evidence for high temperature, and the observed small column density in the (2, 2) level (210 K below (3, 3)), gives evidence for low density. This remarkable non-thermal rotational distribution is caused for a low density gas by the fact that the spontaneous emission from the (3, 3) level is rigorously forbidden by the selection rules while that for the (2, 2) -> (1, 1) transition has a short lifetime of 27 days, corresponding to a low critical density of 200 cm-3. Observed H3^+ spectrum toward the brightest infrared source GCS 3-2, one of the Quintuplet Stars, has been analyzed in detail. Of the observed total H3^+ column density of 4.3 * 1015 cm-2, approximately 3.1 * 1015 cm-2 with high velocity dispersion is inferred to be in the CMZ while 1.2 * 1015 cm-2 is in the intervening spiral arms. Almost all of H3^+ in the CMZ are in diffuse clouds with high temperature. About a half of the gas has a velocity of - 100 km s-1 indicating that it is associated with the 180 pc Expanding Molecular Ring which approximately forms the boundary of the CMZ. The other half with lower velocities of - 50 km s-1 and 0 km s-1 is thought to be closer to the Galactic center. CO do not exist much in those clouds. The non-thermal rotational distribution of H3^+ has also been observed toward 7 other infrared sources within 40 pc of the Galactic center indicating that the hot and diffuse gas is ubiquitous in the CMZ. The spectrum toward GC IRS 3 near Sgr A* shows presence of the hot and diffuse gas in the ``50 km s-1 cloud", the complex of giant molecular clouds which plays a central role in the discussion of Sgr A* and its environment. The hot and diffuse gas has not been observed toward any of the dense and diffuse clouds in the Galactic disk where we have observed large column densities of H3^+ had been observed. The large observed H3^+ column density suggests an ionization rate in the CMZ which is an on the order of magnitude higher than in the diffuse interstellar medium in the Galactic disk if the C/H ratio is indeed as high as reported. Also the fact that the observed H3^+ in the CMZ are almost all in diffuse clouds, along with the reported relatively low visual extinctions (AV 25 - 40) and mass estimated from radio observations of molecules, indicate that dense clouds are more sparse than previously thought and the reported volume filling factor (f >= 0.1) is an overestimate by at least an order of magnitude.