We study a set of universe models where the dark sector is described by a perfect fluid with an affine equation of state $P=P_0+\alpha \rho$, focusing specifically on cosmological perturbations in a flat universe. We perform a Monte Carlo Markov Chain analysis spanning the full parameter space of the model using the WMAP 5 years data and the SDSS LRG4 survey. The affine fluid can either play the role of a unified dark matter (UDM), accounting for both dark matter and a cosmological constant, or work alongside cold dark matter (CDM), as a form of dark energy. A key ingredient is the sound speed, that depends on the nature of the fluid and that, for any given background model, adds a degree of freedom to the perturbations: in the barotropic case the square of the sound speed is simply equal to the affine parameter $\alpha$; if entropic perturbations are present the effective sound speed has to be specified as an additional parameter. In addition to the barotropic case, we consider the two limiting cases of effective sound speed equal to 0 or 1. For $\alpha=c_s^2=0$ our UDM model is equivalent to the standard $\Lambda$CDM with adiabatic perturbations. Apart of a trivial subcase, all models considered satisfy the data constraints, with quite standard values for the usual cosmological parameters. In general our analysis confirms that cosmological datasets require both a collisionless massive and cold component to form the potential wells that lead to structure formation, and an effective cosmological constant that drives the late accelerated expansion.