Unfortunately, you can't get away from political parties. He probable meant what we are facing now. Parties that look to their own future and ignore the country's.
Actually from what I have understood, and what scholars have written and debated, he believed that each man running for office should do so independently of any organizational structure or affiliation, running on their merits and abilities.
Shocking I know, but I believe Washington believed that a politician should only be held accountable to those that elected him, not to any groups, interests and political parties.
Washington himself never endorsed a party.
Here's the relevant text from his farewell address:
'All obstructions to the execution of the Laws, all combinations and associations, under whatever plausible character, with the real design to direct, control, counteract, or awe the regular deliberation and action of the constituted authorities, are destructive of this fundamental principle, and of fatal tendency. They [political parties] serve to organize faction, to give it an artificial and extraordinary force; to put, in the place of the delegated will of the nation, the will of a party, often a small but artful and enterprising minority of the community; and, according to the alternate triumphs of different parties, to make the public administration the mirror of the ill-concerted and incongruous projects of faction, rather than the organ of consistent and wholesome plans digested by common counsels, and modified by mutual interests.
"However combinations or associations of the above description may now and then answer popular ends, they are likely, in the course of time and things, to become potent engines, by which cunning, ambitious, and unprincipled men will be enabled to subvert the power of the people, and to usurp for themselves the reins of government; destroying afterwards the very engines, which have lifted them to unjust dominion."
The man had tremendous foresight, and has probably turned in his grave millions of times.