Richard Carrier seems to approach this free will business in a very down to earth, facts is facts manner, but it doesn't address this question I have - 

You could imagine a big bang maybe starting everything off, particles and energy zooming about and interacting with each other. Like a gigantic billiard table. But with billiards, I think it's generally recognised that if you hit a ball with a certain force and direction, it will go at a predictable speed in a predictable direction and if it hits another ball, the resulting speed and direction of each ball can be calculated. So, if we had a big enough computer to keep track of everything, we could predict everything.

Of course we don't have such a big computer - it would arguably have to be bigger than the universe it is keeping track of - so our predictions are limited. (There are some who say the universe IS a big computer, running a simulation...).

Anyway, my point is, never mind whether or how we could keep track of everything - if people could exert free will beyond the constraints of direct causation, do something their genetics and upbringing and environment didn't dictate, then where could that impulse come from? Aunt Bea referred to the homunculus, the little person inside of us sitting there deciding and controlling stuff.

I guess we'd have to accept a 'spirit' world, which interacts with and affects the 'real' world. But the only evidence for that would be miracles, events inexplicably disconnected from the normal causative forces.

I'm waiting.