We discuss the composition of dust and ice along the line of sight to the Galactic Center (GC) based on analysis of mid-infrared spectra (2.4-13 micron ) from the Short Wavelength Spectrometer on the Infrared Space Observatory (ISO). We have analyzed dust absorption features arising in the molecular cloud material and the diffuse interstellar medium along the lines of sight toward Sagittarius A* and the Quintuplet sources, GCS3 and GCS4. It is evident from the depth of the 3.0 micron H2O and the 4.27 micron CO2 ice features that there is more molecular cloud material along the line of sight toward Sgr A* than GCS3 and 4. In fact, Sgr A* has a rich infrared ice spectrum with evidence for the presence of solid Methane, NH3, and possibly HCOOH. Hydrocarbon dust in the diffuse interstellar medium along the line of sight to the GC is characterized by absorption features centered at 3.4 micron , 6.85 micron , and 7.25 micron . Ground-based studies have identified the 3.4 micron feature with aliphatic hydrocarbons, and ISO has given us the first meaningful observations of the corresponding modes at longer wavelengths. The integrated strengths of these three features suggest that hydrogenated amorphous carbon is their carrier. We attribute an absorption feature centered at 3.28 micron in the GCS3 spectrum to the C-H stretch in aromatic hydrocarbons. This feature is not detected, and its C-C stretch counterpart appears to be weaker, in the Sgr A* spectrum. One of the key questions which now arises is whether aromatics are a widespread component of the diffuse interstellar medium, analogous to aliphatic hydrocarbons.