I usually don’t like to spread FUD by asking people to leave IE and migrate to this or that browser. However, I must admit that today it’s more secure to NOT use IE.
I think there’s a difference that comes from the market share and from the amount of “haters” that MS has. People with intention to do harm will focus on looking for vulnerabilities that can provide them a bigger return, and “MS haters” tend to not follow responsible disclosure guidelines when dealing with MS products. This, even if unrelated to the software quality, will make IE more insecure. I use Firefox and really like it. Making more people adopt it, if my line of thinking is correct, can even make IE more secure as they start to share the focus of attackers.
I believe that all of them have vulnerabilities, and with equal conditions (vuln research focus and responsible disclosure) a well oriented user will be able to use any with acceptable security.