Why did it take so long for Alia to succomb to the ego-memory of the Baron?

I would have expected that she would have succumbed at an early age, not lasting into adulthood before she lost control to the others who would control her.