All Paul ever meant by “justification by faith and not by works” (he never actually says “alone”) is that one is justified as a proper candidate for baptism by faith (or the profession of faith) alone.
That is, Paul never teaches that we don’t have to be baptized. Quite the contrary, Paul says “we are children of God by faith in Christ Jesus because as many of us as were baptized into Christ have put on Christ” (Galatians 3:26-27)
His thing with faith vs works is that no works, especially ceremonial works from the Law, can be placed in front of baptism as a test of your “justification” as a proper candidate for baptism.
Now, I posted a blogpost yesterday (called More on 9marks “church membership” nonsense) about http://9marks.org/interview/church-membership-in-theory-and-practice-with-ligon-duncan/ where two 9marks gurus (Ligon Duncan and Mark Dever) discuss their views on “church membership.” One thing their strange views are heavily based on is the idea that baptism must be preceded by a “long period of catechization.” Well, what is that? That’s a precise rejection of what Paul means by “justification by faith and not by works” because he is talking about what justifies us as a candidate for baptism, which is faith alone (or profession of faith alone), but these guys are putting some test, some works, something you have to do to prove yourself worthy of baptism, in front of baptism. This is precisely what Paul was against, and all he was against so far as the phrase “justification by faith and not by works” goes. (Otherwise, he would be contradicting himself. Of course if anyone wants to press the point, I’ll allow him to contradict himself, and summarily dismiss the epistles. But if you want to consider Paul a valid apostle, this has to be all he meant.)