Special Master Analyzes Privilege Search Terms, Addresses Objection that they were Overbroad
Dornoch Holdings Int’l, LLC v. Conagra Foods Lamb Weston, Inc., No. 1:10-cv-00135 TJH, 2013 WL 2384235 (D. Idaho May 1, 2013)
In this case, a Special Master was directed to obtain ESI (more than one million documents) from a bankruptcy trustee, to review it for privilege, and to prepare a privilege log. The documents were screened utilizing keyword search terms. Upon production of the resulting privilege log, Defendants objected that the terms used were overly broad and that the log contained non-privileged documents. The Special Master therefore conducted an analysis of the terms used and made recommendations to address the objection.
The at-issue ESI was initially screened with the assistance of an outside vendor—FTI Consulting, Inc (“FTI”)—using keyword search terms that were proposed by Plaintiffs’ counsel and then refined by the Special Master with the help of FTI. “The total number of documents selected by the privilege screen was 28,894 (this rose to 43,264 when email attachments and family members were included), or approximately 3% of the population.” Following production of the resulting privilege log, Defendants objected that (among other things) the “privilege log was created using overly broad search terms and ha[d] not been substantively reviewed, thus, the log contain[ed] numerous non-privileged documents.” To reach a recommendation regarding that objection, the district court held that “the Special Master could review a statistically significant number of randomly selected documents from the privilege log to confirm the accuracy of the screening method employed to create it.”
Sparing some of the technical details (which are available in the Special Master’s recommendation, attached below), the Special Master and a small number of Spanish-speaking reviewers reviewed a statistically valid sample of documents chosen randomly by the vendor and determined that 68.9% were not privileged.
The review revealed that some search terms were more effective than others and it was “readily apparent that the results were less effective when more general search terms were used.” Analysis of the results showed that “73% of the search terms in the privilege filter proved to be well correlated with actual privilege, and many were highly correlated.” “The difference between the relatively high percentage of terms that were reasonably precise (73%) compared to the relatively low percentage of actual privilege found (31.1%) indicate[d] the imprecise, general terms provided the bulk of the false positive documents.” Thus, the Special Master concluded it was “unlikely” that the terms could be refined “to have higher precision. However to tighten the screen, the less precise terms could be removed.”
“After the Special Master’s analysis of the sampling results with FTI’s technical assistance, it was determined that those terms which identified a correlation with actual privilege of 59% or greater, showed a strong correlation with privilege.” (Once again, more detail is available in the attached recommendation.) When documents below that threshold were excluded, the effectiveness of the privilege screen increased to 87%. Thus, the Special Master recommended that documents meeting the 59% threshold continue to be withheld as privileged, and that “the documents associated with the less precise terms that fall beneath a 59% correlation with privilege be released.” The Special Master further recommended:
If Defendants wish to further challenge the assertion of privilege over documents that have demonstrated a 59% or greater correlation with privilege from the test sample. [sic] Defendants can request further human review of withheld documents above that threshold. The Special Master would review the associated documents across the whole population (less the documents already reviewed in the sample set). This could be targeted at particular search terms, to refine the review. The Defendants would be responsible for cost of this further analysis.
The Special Master further recommends that the documents associated with the less precise terms that fall beneath a 59% correlation with privilege be released. This will result in a known release of some privileged documents but requires no additional cost or review. Federal Rule of Evidence 502(b) provides protection for inadvertently disclosed documents. Federal Rule of Evidence 502(b) (3) specifically identifies compliance with Federal Rule of Civil Procedure 26(b)(5)(B) as an example of reasonable post-production remedial action. In addition, a “claw back” agreement between the Parties, entered in the early stages of discovery also requires the return of inadvertently disclosed privileged documents.
Plaintiffs also have the option of conducting a further privilege review of all or a portion of the population to be released. This would require that the Plaintiffs create a supplemental privilege log, for the additional documents selected as privileged below the 59% threshold. The Plaintiffs would be responsible for the cost of this further analysis, if Plaintiffs chose to conduct it.
(Footnote omitted.)
The recommendations of the Special Master were adopted in full by the district court. Notably, however, in that same opinion, Defendants’ Motion for Sanctions was granted, and Plaintiffs’ case was dismissed with prejudice, upon the court’s determination that “Plaintiffs willfully deceived the Court and engaged in conduct utterly inconsistent with the orderly administration of justice.” More details of Plaintiffs’ specific misbehavior are available in the court’s opinion, attached below.