When I start driving, my hands are sweating and I feel like I'm going to black out, as if I'm standing on the edge of a tall building. I think I'm sensible to avoid driving myself around the public roads at lethal speeds. I will always have to do without driving. This probably doesn't help you much.
We have autism in our family, and I've heard that a lack of 3-D coordination (and being good at maths) is an autistic trait.
I had ideas consistent with Erock. And maybe take time out just to drive on the freeway (back and forth, for no other reason) when the traffic is lightest.
My driving experience is hyper-vigilant, but not anxiety producing. Most of my younger years were on a motorcycle, where life and death were even more at risk. I just learned habits like checking the left and right streets of an intersection, even when the light's green, giving myself plenty of space in front of me (especially if there's a tailgater behind me), and just assume that all drivers are unpredictable. In some cases, you can just tell by the way someone's driving when they're on a cell phone, and not likely to be sufficiently aware or in control.
I had severe a fear of heights once, and ironically never got over it until I learned to fly! I gained an unexpected sense of control.
I like riding the bus and/or walking when possible. A few days ago my bus stopped at a pedestrian crossing, and after the pedestrian started walking, a car in the lane left of us zoomed by, barely missing the pedestrian. To top it off, the pedestrian shook his head and watched the car speeding away in disgust, without even looking to see if more idiot drivers might be targeting him accidentally. I'd say the pedestrian was a fricken idiot, too! I'm always aware of the hidden-pedestrian possibility, and if I were the bus driver, I would have been watching for traffic and hit my horn while the pedestrian was walking out.
My main point is, yeah, driving can be dangerous, and trying to be aware of whatever might possibly be going wrong around you can be stressful, but keep working at it. It's better to be stressed out than too relaxed.
OK, I know this is unlikely, but is there any way you can take some car racing lessons?
Afraid not. ;)
You have nothing to fear but fear itself.
Check out "the Brain That Changes Itself" by Norman Doidge http://www.amazon.com/The-Brain-That-Changes-Itself/dp/1491513357
Your brain makes connections that solidify (wrong word, but right concept) with repeated exposure. If you change the exposure, the connections rearrange to the new pattern.
You can work on brain plasticity to change behavior even in cases where there is limited physical damage to the brain, like from a stroke.