Always wondered why he had such an intellectual crowd.History says that the networks will do what they choose to do. I believe that Letterman once had ties to Carson's company and was Carson's choice to succeed him. NBC went with Leno anyway.
NBC also did everything that they could to undermine Letterman when he went to CBS. They claimed intellectual property right to key segments of Letterman's show such as Viewer Mail and Larry "Bud" Melman. Letterman did an end run on many of these issues by simply changing the names of the segements.
For several years he killed Leno in the ratings. Leno and NBC then started to play hardball to turn the ratings around. They banned anyone associated with an NBC show from appearing on Letterman. They also cashed in all of their Hollywood chips and made it clear to the studious that the Leno show should be the 1st choice of promotional appearances. Many were told that if they appeared on Letterman, that they would never be booked on the Leno show. Leno ended up with the stronger guest list which had more to do with his climb in the ratings than anything that he did on the show.