In Wu et. al, the MLE objective is computationally infeasible due to the high number of images in the dataset. Non-parametric Softmax Negative Log-Likelihood With large n, the denominator in (2) is hard to compute. Therefore, they use NCE (Noise-Contrastive Estimation). The NCE Objective Essentially, they approximate the difficult loss in (3) with the easier to compute loss in (7). However, we end up estimating the denominator anyways in (8). Why not just approximate the denominator in (2) with (8)? I asked Claude about this and it said something about it being a biased estimator, but