Essentially correct.

Any early "SAE XW-XX" started with lower HTHS viscosity than a similar SAE XX.

Additionally (as you suggest) the Viscosity Improvers (VI additives) were also pretty fragile and would break down over time and use. Login to read more.