We're all a little confused - comes from those two fish trying to swim in opposite directions all the time.
(This is, in fact, a running joke between my husband and I, who share the 9th as a birthday. 'What do you want to do for dinner?' 'I don't know, what do you want to do?' 'FISH!' 'Takes one to know one!')