Numerical simulations of the dynamical friction suffered by a Galactic center star cluster harboring an intermediate-mass black hole (IMBH) have been performed. Gerhard has suggested that dynamical friction, which causes a cluster to lose orbital energy and spiral in toward the Galactic center, may explain the presence of a cluster of very young stars in the central parsec, where star formation might be prohibitively difficult because of strong tidal forces. However, numerical simulations by Kim & Morris showed that this is only possible if the cluster initially has an extremely dense core. Hansen & Milosavljevi\'c recently suggested that the presence of an IMBH in the cluster core might stabilize the core against tidal disruption during the inspiral through dynamical friction, and thus might easily deliver young stars down to the central parsec. We find that the presence of an IMBH does lower the minimum initial core density required to transport young stars down to the central parsec, but this is possible only when the mass of the IMBH is at least 10 % of the total cluster mass. This fraction is significantly higher than that estimated by Portegies Zwart & McMillan with numerical simulations of IMBH formation by successive merging of stars in the cluster core, so it does not appear that a realistic IMBH can help transport young stars into the central parsec.