That's not accurate from what I know. There's a very high heritability factor for autism according to studies (90% from some sources in 2007, though more data is needed as the number is probably exaggerated), which establishes the disease as primarily genetic.
Only in rare cases has autism been genuinely linked to materials known to cause birth defects, that's more of the exception than the rule, and it could merely be that exposure to those substances during formative years results in the same kind of damage than the underlying genetic issue does -- that's certainly the most plausible explanation as far as I know.
One thing that isn't clear however is the exact mechanism for how it occurs. Even if it is seen as a primarily genetic disorder, how that disorder rising from genetic differences becomes expressed is what we understand very little about.
From my brief review of the literature, I get the impression that autism occurs from irregular brain development. That's where the phrase "wired differently" comes into account because it literally describes how autism seems to manifest: some in the autistic brain are connected more heavily than usual (and others are less connected), some sectors are given more emphasis for maturation, etc.