We searched for infrared counterparts to the cluster of X-ray point sources discovered by Chandra in the Galactic Center Region (GCR). While the sources could be white dwarfs, neutron stars, or black holes accreting from stellar companions, their X-ray properties are consistent with magnetic Cataclysmic Variables, or High Mass X-ray Binaries (HMXB) at low accretion-rates. A direct way to decide between these possibilities and hence between alternative formation scenarios is to measure or constrain the luminosity distribution of the companions. Using infrared (J, H, K, Br gamma ) imaging, we searched for counterparts corresponding to typical HMXB secondaries: spectral type B0V with K<15 at the GCR. We found no significant excess of bright stars in Chandra error circles, indicating that HMXBs are not the dominant X-ray source population, and account for fewer than 10% of the hardest X-ray sources.