Not just distance, but time.
How long has human civilization lasted? 10,000 years? How much longer until we go extinct? Another 10,000? If we are lucky?
20,000 years is an eyeblink to the billions of years old universe. Many intelligent civilizations could have popped up and petered out and never existed at the same time as another.
Right. I can buy the idea that other intelligent civilizations have existed before us, exist now, and will exist long after we're gone. And I also think that it's likely that even if more than one intelligent civilization ever existed at the same time they were never even close to being aware of each other. Space is that big and time is that long.
Of course, I got a C- in Astronomy so what the hell do I know?