It might have escaped your notice, but I believe I might have heard here or there that when one takes an oath to enter the Kingsguard, vows of celibacy are usually spoken.

Hard to see where children would come into the mix.