Nonlinear modes of a macroscopic quantum oscillator