Recent analysis of data sets from two extensive air shower cosmic ray detectors shows tantalizing evidence of an anisotropic overabundance of cosmic rays towards the Galactic Center (GC) that ``turns on'' around 1018 eV. We demonstrate that the anisotropy could be due to neutrons created at the Galactic Center through charge-exchange in proton-proton collisions, where the incident, high energy protons obey an E-2 power law associated with acceleration at a strong shock. We show that the normalization supplied by the gamma-ray signal from EGRET GC source 3EG J1746-2851---ascribed to p-p--induced neutral pion decay at GeV energies---together with a very reasonable spectral index of 2.2, predicts a neutron flux at 1018 eV fully consistent with the extremely high energy cosmic ray data. Likewise, the normalization supplied by the very recent GC data from the HESS air-Cerenkov telescope at TeV energies is almost equally-well compatible with the 1018 eV cosmic ray data. Interestingly, however, the EGRET and HESS data appear to be themselves incompatible. We consider the implications of this discrepancy. We discuss why the Galactic Center environment can allow diffusive shock acceleration at strong shocks up to energies approaching the ankle in the cosmic ray spectrum. Finally, we argue that the shock acceleration may be occuring in the shell of Sagittarius A East, an unusual supernova remnant located very close to the Galactic Center. If this connection between the anisotropy and Sagittarius A East could be firmly established it would be the first direct evidence for a particular Galactic source of cosmic rays up to energies near the ankle.