There is an excellent link to constituonal annotations on findlaw, http://caselaw.lp.findlaw.com/data/constitution/article06/02.html#1
it discusses the supremacy clause and the supreme court upholding that clause to strike down state laws that conflict with the supremacy of the constittion and federal laws.
As Chief Justice Marshall wrote "...The nullity of an act, inconsistent with the Constitution, is produced by the declaration, that the Constitution is the supreme law..."
You are correct in that ruling a law unconstituional doesn't really remove it from the books, it just makes it essentially unenforceable as the courts will simply dismiss cases made under those laws.
But... Lets get to the heart of how the court ended up with the power of Judicial Review...
The year is 1803. John Adams lost the 1800 election to Jefferson and decided to pack the lower courts with federalist appointees. Because of an administrative goof at the last minute, Marbury, one of the appointees didn't recieve his written commission before Jefferson took office. Even though Marbury had been confirmed by the senate, Jefferson refused to issue the commission to Marbury. So, in the manner of a true American, Marbury sued. Marbury didn't sue Jefferson though, he sued Madison, then Secretary of State who was the official who technically held the contested issue. Using the Judiciary Act of 1789, Marbury filed his case directly with the Supreme Court.
Now, Chief Justice Marshall was in a bind here... If he found in Jefferson's favor, it would appear he was bowing to the president; if he found for Marbury, Jefferson would just ignore him in all probability, and the courts had no way to enforce their ruling. Either way would relegate the court to a second tier status. Marshall, an excellent politician and legal mind, found a third way out...
First, he declared that Marbury SHOULD have recieved the commission and chastied the president, then went on. Summing up his feelings towards the Judiciary Act thusly: "The question, whether an act, repugnant to the consitutition, can become the law of the land, is a question deeply interesting to the united states." In his political masterstroke, Marshall then declared the Act of 1789 unconstitutional. This removed the ability of the Supreme Court to even hear the case in the first place, thus denying Marbury his commission and avoiding a confrontation with Jefferson, as well as establishing the right of the Court to void laws it found to be in violation of the constitution.
In his conclusion he wrote: "Certainly, all those who have framed written constitutions ontemplate them as forming the fundamental and paramount law of the nations, and, consequently, the theory of every such government must be, that an act of the legislature, repugnant to the constitution, is void."
This ruling established the Supreme Courts right via judicial precedent to void laws it found conflicted with the supreme law of the land, the constitution. Just think for a moment if we didn't have that last arbiter in the US... Any legislative body would only be constrained in the laws it passed by the fear of losing the next election (if it didn't simply legislate them away!). Lawmakers could eliminate free speech, gerrymander to their parties content, favor certain races or religions in law, etc. Because of this decision, ANY single individual can take a case to court, invoking the judiciary to measure laws and decisions against constitutional standards. In effect, any citizen can be a leading cause in having an unjust law overturned, giving citizens the power to directly control government, instead of vice versa.
And we all held hands and did the happy dance :)