While I'm sure it's not the case, the use of the term "-phobia" when describing prejudice against homosexuals does make a certain degree of sense when compared with racist language. When one looks at the main trends in racist rhetoric, there tends to be a mixture of fear and of a sense of biological superiority - Jews are weak and cowardly, but control all the money and are cunning; blacks are slow and stupid, but are strong and will commit crimes against you; hispanics are lazy and can't speak English, but are taking away jobs (note: I don't agree with these views, but I'm trying to present a simplified version of the rhetoric normally associated with these groups). While homophobic rhetoric tends to be more fear based than to argue any superiority - gays are destroying marriage or gays are corrupting/converting the youth - and when homophobic rhetoric does touch on the issue of superiority, it tends to be a religiously based superiority and since religiously based superiority from these groups are not only targeted at homosexuals but also at those who support civil liberties for homosexuals and against completely unrelated groups (i.e. people who are of other religions, atheists, people who don't care about church, people who decry censorship, etc.) it does not seem so special or problematic when targeted at homosexuals.
Of course, that fear implies a sense of superiority (what makes heterosexual life so special that it needs protecting?), but it does not directly state it. This may also be a result of the times - racism was largely a concern of politics 50-60 years ago and it was deemed more socially acceptable (in parts of the country) to say things like "Whites are superior to Blacks" whereas homosexual rights and homophobia have been more in issue starting only about 35 years ago, after the civil rights movement and after it stopped being socially acceptable to state, on the national stages, one group's inate superiority to another (just a theory). On the other hand, words and language evolves, so it's really not a big deal.