Redshift space distortions (RSD) in the void-galaxy correlation ξs provide information on the linear growth rate of structure in low density environments. Accurate modelling of these RSD effects can also allow the use of voids in competitive Alcock-Paczynski measurements. Linear theory models of ξs are able to provide extremely good descriptions of simulation data on all scales provided the real space void positions are known. However, by reference to simulation data we demonstrate the failure of the assumptions implicit in current models of ξs for voids identified directly in redshift space, as would be simplest using real observational data. To overcome this problem we instead propose using a density-field reconstruction method based on the Zeldovich approximation to recover the real space void positions from redshift space data. We show that this recovers the excellent agreement between theory and data for ξs. Performing the reconstruction requires an input cosmological model so, to be self-consistent, we have to perform reconstruction for every model to be tested. We apply this method to mock galaxy and void catalogues in the Big MultiDark N-body simulation and consistently recover the fiducial growth rate to a precision of 3.4% using the simulation volume of (2.5 h−1Gpc)3.