You sound like my father. He was raised in the NC mountains, very poor with two younger brothers. They were both alcoholics and had no desire to move out of their town or further their careers. My father was always the care-giver for his younger brothers and fished them "out of the gutter".
He moved out of the his parents' boarding house and went to college at 15 y/o, as I mentioned in an earlier post. He was intelligent and determined to better himself. He became very successful. His younger brothers passed away very young due to complications related to alcoholism.
My father was an inspiration to me and I appreciate every day that I have a better, more comfortable life in adulthood due to his determination.
It is admirable that you worked so hard to build a better life for yourself and your family. Your children will certainly appreciate you for it.