Honestly the safest way to slap someone into that category is based on their ethnic background. Skin color isn't all that useful for doing so, considering the wide variety of people that comes with being in the US. Take my sister and I for instance, for all intents and purposes, we are part of the Caucasian ethnicity, and yet when we were children we often were asked if my little 10 year old self was adopted from Mexico. This is due, apparently to the fact, that although both my sister and I have dark hair and eyes, she is far more fair skinned than I am. Though I don't get as much sun now as I did when I was a child, it could probably still happen. See, now I'm not sure which particular part of my background this comes from though I know I got it from my mother most likely, I tan, very very well. As a matter of fact, when I was a child, I used to get so damn tan that I really did have that dark, rich copper tone that people often times associate with those of Latin descent.
Also, Latino has a cultural connotation to it if I remember correctly, though don't hold me to that statement.