The idea would be that the prisoners basically have to work because there is next to no import of foodstuffs (not for them at any rate), and they don't have the kind of weapons that would allow a fair share of them to survive through hunting in the wild? So they are obliged to work in the fields if they want to survive, and as long as the system works they're not supposed to have any means to stop their overseers from confiscating most of the produce: that's the idea?
I guess even if there are no guards around most of the time, the prisoner colonists might be suspicious, or afraid, of informers, people who could be reporting to the guardsmen class when these come around, or who have some sort of intermediary function. Most slave societies, and many internment systems, have "favoured captives" who get some favours by cooperating with the guards, by being appointed as overseers or report men while still remaining prisoners or slaves. The presence of such a layer, more or less overt, would stimulate gang culture because those guys tend to be both envied and loathed by the other captives. It would also lead to opportunities for duplicity, and 'double agent' tactics, working both sides of the street as gangsters use to call it.
If you want it violent and igneous, you could suppose that the rulers of the planet, in the local governing city or higher up, have permitted some kind of gladiatorial games to instill fear in the captives and provide entertainment. It's easy to us to think of Roman gladiator games as just entertainment, like boxing, but it was likely also a way of showing the ultimate cost of opposition to the ruling social order and striking a blend of terror and fascination into the hearts of people; don't attack your betters, or Caesar, unless you want to die in the arena. So if you have a planet filled with a convict population whose rulers are confident that these guys (only males?) are scum and also sure that they will never be able to rise and threaten them, then gladiatorial games or something like it would be logical - and they would provide opportunities for a revolt, even if the gladiators were not supposed to have free access to really good weapons.