Lots to unpack here, but I've played ODP and all that jazz and have a pretty strong background and knowledge on USMNT soccer and development. To answer some of your questions:
1) MLS has gotten better as a league, yes, but it has benefited other smaller, regional countries more than it has benefited the U.S. Thus Panama and Hondurous and Costa Rica etc are better because they have a real league to play in now.
2) MLS and USMNT are attached at the hip. US Soccer, literally because I deal with this for work, sell the rights to both when you acquire rights. They are pushing MLS guys onto USMNT to drive the MLS brand, thus more money, more TV sets etc. This is the sole reason we are not going to Russia. We didn't even go down playing with our best players, that is the problem.
3) Players are discouraged from playing overseas by the folks at US Soccer. They are encouraged to come back even though they have overseas offers (see: Bradley, Michael, Altidore, Jozy) Essentially saying, we will keep your spot on the NT if you help us grow MLS. **Michael Bradley tangent: He was a world class 6 when he played for Roma. His play has dropped so dramatically since returning to the US, its hard to remember how good he was. He also should never have been put in the position to be a single defensive midfielder. If only we had one playing in the Premier League (we have two)? Or perhaps another younger DM that could play beside him (no shortage here)? The best Bradley has looked all cycle was when he had Acosta playing next to him. 4-2-3-1 should be the formation, but it wasn't and hear we are. End tangent.**
4) College soccer is the reason US can't develop players domestically. The European and South American players have academies that identify at very early ages. Until the US is ready to do this, really ready to let kids give up on schooling and eat, breathe and play soccer all day, we will never get any better. IMF academy in Florida is something similar but every MLS team would have something like that. Pulisic went to Dortmound at 14 I think? Glad Carelton and Weah are going overseas. Glad Lyndon Gooch did it.
5) Look, Jurgen was a terrible tactician, and after the Costa Rica debacle I too was calling for his head, but in hindsight I was probably wrong. JK wasn't let go bc of that loss, he was let go because he was butting heads with US Soccer about prioritizing the MLS players over the European players. There is no reason that our back line last night should have been anything less than Yedlin, Cameron, Ream, Johnson. Period. Jurgen might not have qualified, but at least we would have gone down with our best players. Maybe out of position in a 2-6-1-1 formation, but still our best players.
6) Remember, we haven't played in the last two Olympics (which is a U-23 tournament) and those should be the guys playing during this WC cycle. That is an indictment on US Soccer and MLS inability to develop players.
Its a sad, sad day for US Soccer. Just keep this in mind, Germany got bounced out of Euro's in 2000 and had a 10 year plan and it worked. France missed two WC's in the 90s and they are the favorites this year. It takes some sole searching and changes, but good can come of this, but that good isn't going to be today, or next month, or sadly next Summer.
Go Switzerland! (My wife works for a Swiss based company)