I think it's the inherent dignity of the Victorian era, as well as a fascination with tiny moving parts - a little romantic retro from this age of integrated circuits. There's also something appealing about an element of danger from experimental technology that doesn't quite work.
I definitely agree with the Victorian aspect. It's all so elegant, and the words/terminology flow so intricately. Those small Victorian touches (politeness, mannerisms, fashion, etc.) make everything graceful and lovely.
Combine this with clockwork contraptions, metal monstrosities, and romantic, crazy adventures, and we have a recipe for fun.
I had a chance to dabble in a systems game designed for the steampunk genre, called "Castle Faulkenstein." It's very well organized and puts heavy emphasis on keeping it Victorian in feel. For example, instead of dice, players draw cards from a regular card deck to determine moves, etc.