Alum does hurt pretty badly, but it works pretty well. It makes them look a little weird, though.
I've had pretty good luck with an OTC medicine called Glyoxide. It's more of an oral rinse than anything else, but if you apply it directly to the canker sore, it will foam up and clean it...