To me Emperor Palpatine is more of a villain than Vader. Vader is a good guy twisted into evil by Palpatine. He even reconciled his evil ways in the end and brought the reign of terror to a halt.
Palpatine on the other hand took an entire galaxy that was thriving and drove it into regression, recession, oppression, whatever you want to call it. Twisted the "chosen one" into an evil pawn. And enslaved many races to achieve his own will and desires.
Vader may have had more screen time and been more menacing but Palpatine was the true villain behind the enforcer.
TLDR warning...
You could argue that Palpatine and Buffalo Bill are the pure villain in both movies.
The asylum warden in Silence of the Lambs is a pretty incredible nasty villain and Lecter actually takes him out while promising he'd never harm the hero of the movie. Not saying I like Lecter but he's clearly a little more complex than Buffalo Bill and wouldn't want to destroy an entire planet, he's also a naturalist, the opposite of blowing up Alderaan.
I think the most STUPID take that a lot of people have is that Return of the Jedi is the worst of the original trilogy because it's not dark and it's for kids with the Ewoks. That is 100% awful and nonsensical. Return of the Jedi is
by far the darkest movie in the original trilogy:
1. The Emperor is terrifying, pure evil, and it's the only movie he's in a lot in the original trilogy. Just the fact of how he controls Vader like a pet is scary, then his look and voice and the force lightning...
2. Luke handing himself over to these two villains willingly is a terrifying moment of risk.
3. Yoda dies.
4. Yeah the Ewoks are cute but a bunch of them get gunned down to death by laser blasters. It's like a movie that features killing 80 puppies. How is that cute and not dark?
5. Han is kept as a prisoner in a block of metal.
6. Leia is held as basically a sex slave until she kills her sex trafficker.
7. Tons of massive rebel ships in the armada space fight get destroyed.
8. Jabba executes another of his sex slaves to a gigantic Rancor monster and they all laugh while it eats her.
9. He tries to do the same thing to Luke unsuccessfully.
10. Vader's death isn't necessarily a hero dying but it's still very emotionally complex and not at all "cute kids movie".
I could really keep go on, it's the absolute most dumb movie take that for some reason a majority of people hold onto. The original script had Lando and the Falcon being blown up. For whatever reason people seem to want to hate the third movie in any trilogy. For years people thought the third Indy movie was the worst one or even hated it, then over time most realized it's either the best or tied with the original.