I would love to come to that party.
I know NIN has already been said but I think everyone needs to acknowledge this song as a boss for this discussion.
"Your god is dead and no one cares
If there is a hell I'll see you there"
..bow down before the one you serve...
I consider "head like a hole" to be neck and neck with "Mozart's requiem" to be my two most perfect sonic experiences ever.
I've always liked "The Devil wants to (****) me in the back of his car," though it has nothing to do with supporting anything I believe.
If you're into punk:
No such music exists..to my knowledge.
Don't know if anybody's said this yet, but I'll Follow You Into the Dark by Death Cab for Cutie is a great atheist song.