No it doesn't. Academics is a factor just like football success, fan base, geography, etc. It is not irrelevant as you claim. It is also not the sole determinant.
I think it is silly to say otherwise considering that 10 university presidents are going to be the group that ends up making this decision. Each of them would love to add schools that, all else being equal, were Carnegie I, AAU members, highly ranked by USNWR, etc. The presidents believe that the conference's academic prestige reflects in some way on its own members, including their school.