We use data from the Sloan Digital Sky Survey and visual classifications of morphology from the Galaxy Zoo project to study black hole growth in the nearby universe (z < 0.05) and to break down the active galactic nucleus (AGN) host galaxy population by color, stellar mass, and morphology. We find that the black hole growth at luminosities L[Oiii] >1040 erg s–1 in early- and late-type galaxies is fundamentally different. AGN host galaxies as a population have a broad range of stellar masses (1010-1011 M ☉), reside in the green valley of the color-mass diagram and their central black holes have median masses around 106.5 M ☉. However, by comparing early- and late-type AGN host galaxies to their non-active counterparts, we find several key differences: in early-type galaxies, it is preferentially the galaxies with the least massive black holes that are growing, while in late-type galaxies, it is preferentially the most massive black holes that are growing. The duty cycle of AGNs in early-type galaxies is strongly peaked in the green valley below the low-mass end (1010 M ☉) of the red sequence at stellar masses where there is a steady supply of blue cloud progenitors. The duty cycle of AGNs in late-type galaxies on the other hand peaks in massive (1011 M ☉) green and red late-types which generally do not have a corresponding blue cloud population of similar mass. At high-Eddington ratios (L/L Edd>0.1), the only population with a substantial fraction of AGNs are the low-mass green valley early-type galaxies. Finally, the Milky Way likely resides in the "sweet spot" on the color-mass diagram where the AGN duty cycle of late-type galaxies is highest. We discuss the implications of these results for our understanding of the role of AGNs in the evolution of galaxies.