Pair-bonding is a fundamental trait of human sexuality actually, polygamy isn't natural and I'm not even sure if it's truly possible. Before you disagree with me, I implore you to actually read the rest of what I'm saying so you get a feel for what I mean. That's not to say there's anything wrong with polygamy or polyamorous people in general, but on a biological level "loving two people at once" in the same way isn't possible.
What you (the author) are experiencing with the woman you just met is a chemical response in your brain to "new love." When relationships and sexual opportunities blossom with new partners, incredible sensations occur in your brain. This is that burning, fiery, romantic love people always speak of, but it's fleeting, it only lasts about 9 months at the most. What you feel for your wife is based on emotional attachment and dutiful love, which is a very different mental and chemical process in the human brain.
Unless you've had problems with your wife before, stay with her. In time your fiery feelings for your new interest will wither away and you'll be back at square one, after an ugly divorce.