We study a large-$N$ bosonic quantum mechanical sigma-model with a spherical target space subject to disordered interactions, more colloquially known as the $p$-spin spherical model. Replica symmetry is broken at low temperatures and for sufficiently weak quantum fluctuations, which drives the system into a spin glass phase. The first half of this paper is dedicated to a discussion of this model’s thermodynamics, with particular emphasis on the marginally stable spin glass. This phase exhibits an emergent conformal symmetry in the strong coupling regime, which dictates its thermodynamic properties. It is associated with an extensive number of nearby states in the free energy landscape. We discuss in detail an elegant approximate solution to the spin glass equations, which interpolates between the conformal regime and an ultraviolet-complete short distance solution. In the second half of this paper we explore the real-time dynamics of the model and uncover quantum chaos as measured by out-of-time-order four-point functions, both numerically and analytically. We find exponential Lyapunov growth, which intricately depends on the model’s couplings and becomes strongest in the quantum critical regime. We emphasize that the spin glass phase also exhibits quantum chaos, albeit with parametrically smaller Lyapunov exponent than in the replica symmetric phase. An analytical calculation in the marginal spin glass phase suggests that this Lyapunov exponent vanishes in a particular infinite coupling limit. We comment on the potential meaning of these observations from the perspective of holography.