Most of those aren't 'games', though, but collaborative writing works/stories.
It really depends on how you define what a game is. It sounds to me like your referring to competition as a game. Having strict rules for competition makes sense but you don't have to have a competition for something to be a game. So rules, good or bad are not a necessity for a creative game to take place. IE: The act of participating in a game involving or where the focus is role playing.
The overlap with a freeform RPG is very close, but the mindset that drives each of them is a very different beast.
It really is more about the mindset of the players then it is the game itself. You are correct, which is why systems/rules are not really needed (they are optional.) You don't have to have that mindset to play an RPG (tabletop or otherwise.)
I'm not a competitive person and I value creativity over competition. As a result of that, rules have only as much meaning to me as I want them to.
I can Freeform then turn around and play with hard rules in the same game/same setting without it making a difference. My friends and I often used to do this when we'd take long road trips. We'd skip the dice, skip the rules and just freeform our role playing until we decided to pick the dice back up. If at all.
Now I'm not saying that you can completely do away with rules.. I'm just saying that the kinda rules that are involved in a "system" are not necessary.
The way for this to work, simply involves trusting your storyteller/GM's judgement and for the players to respect each other.
Plus, a good portion of the freeform stories here are 1v1 instead of group games, which also brings a different attitude.
I looked through the group games in several of the different categories here. Most of them look freeform to me. I see, magic, super heroes, fantasy settings, and lots of others that are all freeform/non-system.
So your wrong on that entirely.
Not to mention that fact that the system games themselves tend to not use the actual systems very much at all anyways. So it seems the ability of a system to function or not, if it's broken or not, doesn't matter much because people here at least don't really abide by them anyhow.
I think a better general statement would be 'prioritize good roleplaying over mechanics, but if you're going to use mechanics, make sure they're good mechanics. No mechanics is better than bad ones.'
I think good mechanics are nice to have but bad or good, a good GM/Storyteller should have no problem working with them.
I've been told by a number of people here on the forums how horrible the Palladium system is, yet I never had a problem with it in all my years of play. I never had a complaint from any of my players. No one ever complained about the system.. or any system that we ever played it. We laughed occasionally at some odd discrepancies in various systems but good or bad, the systems never made much of a difference to us as players or GM's(story tellers.)
So the only thing I really agree with is, "prioritize good roleplaying over mechanics."
Well, yes, but I think that's where there's a big difference. If it's just two people, it's easier to accomplish a lot of things that would otherwise ruin it. For instance, you can get what you want out of your individual character, but it's usually discussed first. Likewise, the overall telling of any story is discussed here-- while in a tabletop game, you don't know what's coming because it's always supposed to keep you on your feet and keep you from metagaming.
With group games, as system games often are, there's a competitive edge and no time to discuss what your character is going to do constantly, as well. That puts a small stunt on roleplaying-- you are always, in one way or another, going to step on another player's toes with some action. Mechanics are good because it keeps things from becoming painfully steamrolled by one character or battled for by more-- not that this doesn't happen, but man do players get neglected in non-system group games a lot.
I think this is the problem. There's some assumptions here.
1) Your assuming that people discuss stories in advance. That's not always the case. I know lots of people that respect each other and trust each other enough to enjoy the surprise of elements in a role play - on forum or in table top. I've done both for years. I've never had a problem.
2) Good role players don't metagame on purpose. Bad role players are usually just naive of how to role play and again, don't do it on purpose - they just don't know what they've done is wrong.
I rarely run into people that meta game on purpose in most role playing settings. If it happens, it's almost always by accident. Of course there is the occasional childish person that does it for various reasons but those people have been far and few in between in my experience. People just have a tendency to make a big dramatic deal out of it when it happens. It's better to stay calm, find out why it happened and help the person to not do it, or remove them from the game.
3) Group role play or One on One(duet) role play makes no difference. As I stated in my other responses above, there are lots of games taking place right here right now that are Freeform (more freeform then system games actually - just go look.)
4) Competition. I can see why people would like a system for competition. I never had that issue myself where a system was necessary but I did play with some really great groups of people throughout my many years of table top gaming. I've come to see that many people here have not shared that same good fortune of having good players to share games with.
However, don't make the mistake of assuming that a system is necessary for competition either. You said so yourself in an earlier post when talking about it being the Storyteller's(GM) responsibility to keep things in line and make it work fluidly. That's how I always approached my role playing and it's pretty much worked for me. Respect each other as players and go with the flow when things happen and most of the time it'll work out. A few good rules help but there's a difference between a few rules and an entire system of them.
As you also stated in an earlier post. Even the creators of those RPG systems usually state in the opening pages of their books that their system is a guideline only and encourage the players/GM's to do what they wish. So if that's the case then does it really matter if the system is flawed or broken in some way?
Now I admit, I agree with both of you about some issues that take place where having a good solid system can avoid those circumstances. Player conflicts do happen but we are all adults or in the case of younger table top gamers - they are all friends. If I have a conflict i a street game of basketball, I don't suddenly whip out a rule book and start dictating rules. We all know the basics.. a system isn't really necessary unless your just learning the game.
It also feels like we as players/GM's use the system as a crutch more often then use it for resolution of those issues. If the system doesn't work, it seems like many here, for some reason go into a panic and toss out the whole game instead of seeing past it to the good role playing possibilities.
It's only my opinion but there is far too much of a competition /battle simulation focus on system gaming. That or everyone wants to earn their XP points, get the best stats, roll the best dice, so on and so forth.
I'll keep my personal focus on role playing.