A generic property of non-equilibrium systems maintained in a steady state by contact with several reservoirs of particles at unequal chemical potentials is the presence of long range correlations. In this talk, we will first discuss the equivalence of ensembles in a non-equilibrium framework and the impact of the long range correlations on this issue. In a second part of the talk, we will show for some specific models how the long range correlations become singular when approaching a phase transition.