We measure the projected number density profiles of galaxies and the splashback feature in clusters selected by the Sunyaev-Zel'dovich effect from the Advanced Atacama Cosmology Telescope (AdvACT) survey using galaxies observed by the Dark Energy Survey (DES). The splashback radius is consistent with CDM-only simulations and is located at 2.4-0.4+0.3 Mpc h-1. We split the galaxies on color and find significant differences in their profile shapes. Red and green-valley galaxies show a splashback-like minimum in their slope profile consistent with theory, while the bluest galaxies show a weak feature at a smaller radius. We develop a mapping of galaxies to subhalos in simulations and assign colors based on infall time onto their hosts. We find that the shift in location of the steepest slope and different profile shapes can be mapped to the average time of infall of galaxies of different colors. The steepest slope traces a discontinuity in the phase space of dark matter halos. By relating spatial profiles to infall time, we can use splashback as a clock to understand galaxy quenching. We find that red galaxies have on average been in clusters over 3.2 Gyr, green galaxies about 2.2 Gyr, while blue galaxies have been accreted most recently and have not reached apocenter. Using the full radial profiles, we fit a simple quenching model and find that the onset of galaxy quenching occurs after a delay of about a gigayear and that galaxies quench rapidly thereafter with an exponential timescale of 0.6 Gyr.