We investigate in detail 13 early-type field galaxies with 0.2<z<0.7 drawn from the FORS Deep Field. Since the majority (9 galaxies) is at z ≈ 0.4, we compare the field galaxies to 22 members of three rich clusters with z=0.37 to explore possible variations caused by environmental effects. We exploit VLT/FORS spectra ( R≈1200) and HST/ACS imaging to determine internal kinematics, structures and stellar population parameters. From the Faber-Jackson and Fundamental Plane scaling relations we deduce a modest luminosity evolution in the B-band of 0.3-0.5 mag for both samples. We compare measured Lick absorption line strengths (H$\delta$, H$\gamma$, H$\beta$, Mgb, and Fe 5335) with evolutionary stellar population models to derive light-averaged ages, metallicities and the element abundance ratios Mg/Fe. We find that these three stellar parameters of the distant galaxies obey a scaling with velocity dispersion (mass) which is consistent with that of local nearby galaxies. In particular, the distribution of Mg/Fe ratios of local galaxies is matched by the distant ones, and their derived mean offset in age corresponds to the average lookback time. This indicates that there was little chemical enrichment and no significant star formation within the last ~5 Gyr. The calculated luminosity evolution of a simple stellar population model for the derived galaxy ages and lookback times is in most cases consistent with the mild brightening measured by the scaling relations.