Many years ago I had to bail on someone at the last minute because I was suddenly stricken with the flu and was, among other things, VERY, very dizzy. There was no way it would have been safe for me to drive -- I could barely get to the bathroom. That flu flattened me for 10 days.
I figured, and still do, that it was kinder to leave him in the lurch than try to drive and get us both killed. He understood, of course, and actually helped me out while I was so sick.
Some things just can't be helped. BUT, having said that, if I had committed to drive the person, I would NEVER bail out just because I had decided I didn't want to do it or just because I was feeling a little icky.