Former CIA analyst Maja Lehnus told Michael Morell in this podcast about lessons learned from the Iraq WMD debacle:

I’ll start with the Iraq WMD lesson of how important it is to understand and explain the quality of the intelligence sources on which judgments are based. In the broadest context, you need to understand and explain the quality of the information or perhaps the data sets that underpin your judgments. A key finding of the Iraq WMD review was that analysts failed to adequately represent the sourcing, on which important judgments were based. In fact, analysts were often unaware that there were issues or uncertainties regarding key sources and therefore their confidence in the information was much stronger than it otherwise would have been.

Another Iraq WMD lesson is the importance of examining alternative analytical possibilities or outcomes. No alternative analysis had been done on the Iraq WMD topic prior to the intelligence failure. No mechanism existed to force analysts to ask whether there might be another explanation for what they were observing or to ask themselves, ‘Could we be wrong? What might Alternative X look like? Hmm?’

A third lesson is the importance of clearly articulating key judgments and the levels of confidence for each judgment. This is a means of alerting customers to uncertainties and key intelligence gaps. In the Iraq WMD review, we found that key gaps and uncertainties were not highlighted and confidence levels were not clearly articulated. Moreover, when probabilistic language was used, such as possibly, probably and likely the review found that the analysts intended meaning and the readers’ interpretation were not always the same.

A final lesson I’ll mention that came out of the Iraq WMD review was the importance of conducting analytic line reviews, especially as accounts transitioned to other analysts or other offices right on related topics. History matters; you need to understand the historical context in which you’re working. It is fine to change an analytic assessment, but it’s important to highlight to your customers the fact that you’re doing so and explain the basis for the change. In the Iraq WMD review, we found unintentional shifts in the key analytic lines were occurring regularly. This was because the analyst lacked the historical context. The analyst writing a piece lacked awareness of what had been previously written by analysts from different part of the organization.