Many inverse problems in science and engineering involve multi-experiment data and thus require a large number of forward simulations. Dimensionality reduction techniques aim at reducing the number of forward solves by (randomly) subsampling the data. In the special case of (non-linear) least-squares estimation, we can interpret this compression of the data as a (low-rank) approximation of the noise covariance matrix. We show that this leads to different design criteria for the subsampling process. Furthermore, the resulting low-rank structure can be exploited when designing matrix-free methods for estimating (properties of) the posterior covariance matrix. Finally, we discuss the possibility of estimating the noise covariance matrix itself.