Information about the X-ray luminosity of the supermassive black hole located at the Galactic center (GC), Sgr A*, and its temporal variations in the past is imprinted in the scattered emission observed today in the direction towards giant molecular clouds (GMCs) located in our Galaxy. Due to light travel time effects these clouds probe the activity of Sgr A* at different times in the past depending on their position relative to the GC and the observer. In this paper we combine results of recent ASCA observations along the Galactic plane, providing upper limits for the scattered flux in the 4-10 keV range produced in a given direction, with data from CO surveys of the same regions. These CO surveys map the position and mass of the molecular gas which the GMCs are made up of. Demanding the scattered flux to be not larger than the observed one, this data enables us to derive upper limits for the 4-10 keV luminosity of Sgr A* at certain times during the last 40 000 years down to about 8*1040 erg s-1. At other times the limits are less tight, of the order of 1041-1042 erg s-1. For two periods of time of about 2000 and 4000 years duration 8000 and 14 000 years ago the currently available CO data is insensitive to any enhanced activity of the GC. Flares lasting longer than 3000 years fill these time gaps and therefore can be excluded to have occured during the last 40 000 years with a luminosity larger than a few 1042 erg s-1. The more extended and continuous HI distribution in the Galactic disk, which also scatters the radiation emitted by Sgr A*, allows us to extend the time coverage further into the past, back to about 110 000 years, albeit the limits are becoming less tight. We thereby can rule out a long term X-ray activity phase of Sgr A* at one per cent of its Eddington level ending less than about 80 000 years ago. The limits presented in this paper can be improved by observations of emission in the fluorescent iron K_ alpha -line. We study the feasibility of these methods to investigate past nuclear activity in other spiral galaxies observed with the angular resolution of X-ray telescopes like Chandra and XMM-Newton.