Diffuse 511 keV line emission, from the annihilation of cold positrons, has been observed in the direction of the Galactic Centre for more than 30 years. The latest high-resolution maps of this emission produced by the SPI instrument on INTEGRAL suggest at least one component of the emission is spatially coincident with the distribution of 70 luminous, low-mass X-ray binaries detected in the soft gamma-ray band. The X-ray band, however, is generally a more sensitive probe of X-ray binary populations. Recent X-ray surveys of the Galactic Centre have discovered a much larger population (>4000) of faint, hard X-ray point sources. We investigate the possibility that the positrons observed in the direction of the Galactic Centre originate in pair-dominated jets generated by this population of fainter accretion-powered X-ray binaries. We also consider briefly whether such sources could account for unexplained diffuse emission associated with the Galactic Centre in the microwave (the WMAP `haze') and at other wavelengths. Finally, we point out several unresolved problems in associating Galactic Centre 511 keV emission with the brightest X-ray binaries.