The near-infrared emission from the black hole at the Galactic center (Sgr A*) has unique properties. The most striking feature is a suggestive periodic sub-structure that has been observed in a couple of flares so far. Using near-infrared polarimetric observations and modelling the quasi-periodicity in terms of an orbiting blob, we try to constrain the three dimensional orientation of the Sgr A* system. We report on so far unpublished polarimetric data from 2003. They support the observations of a roughly constant mean polarization angle of 60 DEGr +/- 20 DEGr from 2004 - 2006. Prior investigations of the 2006 data are deepened. In particular, the blob model fits are evaluated such that constraints on the position angle of Sgr A* can be derived. Confidence contours in the position - inclination angle plane are derived. On a 3 sigma level the position angle of the equatorial plane normal is in the range 60 DEGr - 108 DEGr (east of north) in combination with a large inclination angle. This agrees well with recent independent work in which radio spectral/morphological properties of Sgr A* and X-ray observations, respectively, have been used. However, the quality of the presently available data and the uncertainties in our model bring some ambiguity to our conclusions.