Honestly, I'd agree, if William and Henry didn't seem so disinclined to associated with their father and his friends outside of culturally important events. William in particular seems to be really close with his wife's family, which likely speaks volumes. When Henry went on that Vegas bender I can't imagine exactly what was said, but I suspect they dropped the bomb on him that he wasn't Charles son in a less than loving or compassionate way.