In this paper, in order to improve both the performance and the efficiency of the conventional Gaussian Mixture Models (GMMs), generalized GMMs are firstly introduced by integrating the conventional GMMs and the active curve axis GMMs for fitting non-linear datasets, and then two types of Fuzzy Gaussian Mixture Models (FGMMs) with a faster convergence process are proposed based on the generalized GMMs, inspired from the mechanism of Fuzzy C-means (FCMs) which introduces the degree of fuzziness on the dissimilarity function based on distances. One is named as probability based FGMMs defining the dissimilarity as the multiplicative inverse of probability density function, and the other is distance based FGMMs which define the dissimilarity function focusing the degree of fuzziness only on the distances between points and component centres. Different from FCMs, both of the proposed dissimilarity functions are based on the exponential function of the distance. The FGMMs are compared with the conventional GMMs and the generalized GMMs in terms of the fitting degree and convergence speed. The experimental results show that the proposed FGMMs not only possess the non- linearity to fit datasets with curve manifolds but also have a much faster convergence process saving more than half computational cost than GMMs’.