Specialization in pollination systems played a central role in angiosperm diversification, yet the evolution of specialization remains poorly understood. Competition through interspecific pollen transfer may select for specialization through costs to male fitness (pollen lost to heterospecific flowers) or female fitness (heterospecific pollen deposited on stigmas). Previous theoretical treatments of pollination focused solely on seed set, thus overlooking male fitness. Here we use individual-based models that explicitly track pollen fates to explore how competition affects the evolution of specialization. Results show that plants specialize on different pollinators when visit rates are high enough to remove most pollen from anthers; this increases male fitness by minimizing pollen loss to foreign flowers. At low visitation, plants generalize, which minimizes pollen left undispersed in anthers. A model variant in which plants can also evolve differences in sex allocation (pollen/ovule production) produces similar patterns of specialization. At low visitation, plants generalize and allocate more to female function. At high visitation, plants specialize and allocate equally to both sexes (in line with sex-allocation theory). This study demonstrates that floral specialization can be driven by selection through male function alone and more generally highlights the importance of community context in the ecology and evolution of pollination systems.