I agree he didn't want to kill Hank, which is why he gave up (I mean, it's not like he would have been able to kill Jesse without trying to take down Hank, too) and why he panicked when the Nazi's showed up.
If Jesse had been face-to-face and said the things he said over the phone, Walt would have killed him, IMO. No way he contains that kind of rage, just like he didn't do it when Mike put him in his place.
Walt has put a lot of effort into making Jesse feel like he's a good guy, but I don't think that's because Walt truly cares about Jesse. It's because he's needed Jesse on his side.