I'll admit I don't even really understand the OP's question, perhaps because I don't associate the same ideas to the terms used as he does.
To me being against homosexuality involves an opposition, a condemnation, or a rejection of either the sexual orientation, the sexual identity, or the sexual behavior of gay people.
If my understanding of the idea is correct, then I don't see how it cannot be homophobic.
If I have a negative opinion of a core component of someone's identity, then I have a negative opinion of that person, don't I? Unless the trait I dislike is a harmful one, how is my negative opinion not prejudiced? And if my prejudice is against homosexuality, how is not homophobia?