Actual sugar although not great for you is not the problem per say. It's the artificial sugars and additives like corn syrup. And preservatives in a lot of pre made stuff.
Too many times I threw left over crumbs out on the lawn for the birds. Some of the premade stuff with preservatives last for weeks. Imagine how long it might preserve the contents of a fat cell in a human body.
Sugar is also very bad for you, but have you ever noticed how ants will not go near artificial sugars?