Well, there is a reason many people can't digest milk. Humans are supposed to get one type of milk up to 1 year old. If we didn't get it by then, it's too late because we no longer produce the two enzymes needed to digest it.
The milk of other species (like cows) is designed for an animal that gains several hundred pounds in the first 6 months of life. And the composition is very different than human milk. That's why animals stop drinking mother's milk after they are weaned.
As for calcium, I know several elderly people who drink lots of milk on the advice of their doctor. Their osteoporosis keeps getting worse, so something's wrong with the advice they're getting.
Personally, I can have a spoon of yoghurt or cottage cheese now & then without a problem, but as for milk, it's only soy or almond milk.