Okay so I have An Idea! (spoiler break!)
Y’all know how the show has never shown more than, oh, a dozen actual Others/White Walkers? The TV show obviously has a ton of wights/ice zombies who fight the battles, but very few Others themselves. Likewise, I think the biggest congregation of Others in the book is in the opening scene beyond the Wall, and that’s still not that many. (I can’t quite call to mind the exact number, but I feel like it was around a dozen to around 20 individuals.) Probably the most we see in one place in the show is in season 5 when Craster’s child is turned into one by the Night’s King.
That’s weird, right? As in, there should be times that we would see more of them... or so I would think anyway. So two options there - there’s a whole population of them and we just never see them, or that’s really all Others/White Walkers that exist - let’s be generous and top out that population at about 30.
One more logical leap: in the show, we’ve seen the creation of a Baby Walker when Craster’s son is turned by the Night’s King. We’ve seen no other indications of Other reproduction.
Theory: abduction of infants and turning them into Others/White Walkers is the ONLY means of reproduction they have. Further theory: because of the low numbers they had after defeat at the end of the Long Night, their isolation north of the wall, and because they were held in check by north-of-the-wall Children of the Forest, they didn’t really recover a population to speak of. Let’s say there were single digit O/WW’s remaining around that period - enough to make occasional violent secretive mischief, but not enough to make themselves well known or tales of them believable south of the wall. However, the CoTF have been dwindling by their own admission. What if that enabled - bear with me here - increase in their harmful interactions with the northern wildlings? Which a wilding like Craster decided to avert by making his son-sacrifices - which then allow the population of O/WWs to grow to threat levels.
That means that there’s now a Craster-spawned young adult generation of O/WWs coming for everybody, and this whole fuckin’ thing is kind of Craster’s fault!
throw your tomatoes!