Rydberg exciton-polaritons in the direct-gap semiconductor Cu2O imprint Rydberg interactions onto photons, enabling nonlinear energy shifts that already surpass those of GaAs, the previously best-performing solid-state platform. Here we examine the physical basis and current status of this system, identify the specific problems hindering the attainment of the regime of nonlinear quantum optics, and outline a roadmap toward photon blockade and deterministic gate operation.