Speakers reshape listeners' memories through at least two discrete means: (1) social contagion and (2) socially shared retrieval-induced forgetting (SS-RIF). Three experiments explored how social relationships between speaker and listener moderate these conversational effects, focusing specifically on two speaker characteristics, expertise and trustworthiness. We examined their effect on SS-RIF and contrasted, within-subjects, their effects on both SS-RIF and the previously studied social contagion. Experiments 1 and 2 explored the effects of perceived expertise; Experiment 3 explored trust. We found (1) that speakers who were perceived as experts induced greater levels of social contagion and lower levels of SS-RIF than non-expert speakers, and (2) that, likewise, trust in the speaker had similar mnemonic consequences, in that neutral speakers induced more social contagion and less SS-RIF than untrustworthy speakers. These findings suggest that how speakers shape listeners' memories depends on the social dynamic that exists between speaker and listener.