Unlike, say, Soviet spies working under diplomatic cover, terrorists are hard to identify. Yet they are vastly more dangerous. Monitoring their likely communications channels is the best way to track and stop them. Building evidence to prove past crimes, as in the civilian criminal system, is entirely beside the point. The best way to find an al Qaeda operative is to look at all email, text and phone traffic between Afghanistan and Pakistan and the U.S.While Yoo doesn't come out and say it, the far more obvious difference between KGB spies and Al Qaeda operatives is that the Russians probably used strong encryption, and not, say, a shared Hotmail account.
The US government snooped on the communications of millions of Americans because Joe Terrorist still doesn't know how to use Pretty Good Privacy. If Al Qaeda's communications were all protected by strong encryption, it probably would have been much tougher to justify (even inside the permissive Yoo/Gonzales Department of Justice) the disgraceful warrantless interception and "other programs" which we still have yet to learn much about.