Introduction
Technology Assisted Review or TAR is one of the few definitions used within the legal industry to describe the type of artificial intelligence, named ‘supervised machine learning’. In this blog, we are taking a closer look at how TAR works, the differences between TAR 1.0, TAR 2.0, and TAR 3.0, as well as their key benefits. Furthermore, we share some best practices on in-housing Technology Assisted Review and successfully applying it as part of legal teams’ document review process.
Contents
What is Technology Assisted Review (TAR)?
- How does Technology Assisted Review (TAR) work?
- What does the TAR project workflow entail?
- How does TAR 2.0 compare to TAR 1.0?
- What does the TAR 3.0 workflow look like?
What are the benefits of in-housing Technology Assisted Review?
What role does TAR play in streamlining the document review phase of eDiscovery?
Four tips for successful use of Technology Assisted Review (TAR)
In conclusion
What is Technology Assisted Review (TAR)?
Technology Assisted Review (TAR) is a way of handling the review phase of eDiscovery by deploying algorithms that can classify documents based on input from expert reviewers. It can provide statistics, categorization, and reporting data that is superior to human-only review.
In the early stages of a case, lawyers use TAR to collect and review documents that are likely to be produced by discovery. This information can be subsequently used to advise clients on the best strategy. Legal professionals can also use Technology Assisted Review to structure their cases, develop strategic defences, etc. It empowers legal teams to make decisions rapidly by prioritizing the most critical documents.
Former U.S. Magistrate Judge Peck, arguably the most notable proponent of analytics and assisted review, wrote in his landmark 2012 case, Monique Da Silva Moore, et al. v. Publicis Groupe & MSL Group, the first judicial decision approving the use of technology-assisted review, and stated, “…assisted review is an acceptable way to search for relevant ESI [Electronically Stored Information] in appropriate cases”. Judge Peck wrote again in his 2015 case, Rio Tinto PLC v. Valle S. A., the decision that it was black-letter law that if the responding party wished to use TAR, courts would allow it.
How does Technology Assisted Review (TAR) work?
The process of Technology Assisted Review relies on the following steps:
- A subject matter expert screening through tranches of documents;
- Once each document tranche is reviewed, coding is applied to similar documents across the dataset;
- After each review round, reporting is provided and any discrepancies are highlighted;
- The iterative coding process; review; and report continues until the reviewer accepts the technology-applied coding designations;
There are different variations of TAR: TAR 1.0 (known as ‘predictive coding’); TAR 2.0 (known as ‘continuous active learning’; with multiple reviewers in order to remove decision bias and add consistency); and TAR 3.0 (the newest adaptation of TAR, which is like Continuous Active Learning, but applied to cluster centers only).
What does the TAR project workflow entail?
The TAR project workflow includes three review “rounds”:
- Training: reviewers evaluate documents in a training set and tag them with TAR document tags (Responsive, Not Responsive, and optionally, Do Not Use as Example).
- Validation: a reviewer evaluates a sample of remaining documents to identify additional responsive/nonresponsive documents in the project. This round is repeated on different samples until the administrator is confident about the level of agreement between the human review and the TAR evaluation.
- Certification (optional): when the goal of TAR is to limit the number of documents that humans need to review (unreviewed documents are considered unresponsive), a reviewer evaluates a final set of documents.
How does TAR 2.0 compare to TAR 1.0?
TAR 1.0 or predictive coding is a method of document review in which a senior lawyer first analyzes a randomly selected “seed set” of documents.
The lawyer’s decisions are then fed into the predictive coding software to train the algorithm to identify similar responsive documents. The hallmark of TAR 1.0 is one-time training. Once the TAR algorithm reaches stable quality results, it stops learning and starts working, ranking the remaining document set.
This method then creates multiple validation rounds where reviewers can agree or disagree with the predictions. After a stable prediction or reasonable precision and recall, the review team can make the decision to continue or stop reviewing documents and accept the rest of the TAR predictions. If such a decision is made, there are certification or validation rounds that can be done by sampling documents to ensure completeness.
The more advanced approach – TAR 2.0 swaps simple learning with continuous active learning, meaning that the algorithms continuously learn as the document review is performed. This can begin with a seed set of documents and then expand to the main corpus. It does not focus on one subject matter expert, but rather allows for multiple reviewers in order to remove bias.
That said, it can also lead to many conflicts or overturn documents that must be resolved immediately to ensure the system continues to be trained properly.
While TAR 2.0 is the preferred option for large-scale eDiscovery productions, there are certain cases when TAR 1.0 better suits the review objective. For instance, in cases when you need only one expert or SME to conduct a fraud or security investigation, as that one person holds all the needed information, it might be reasonable to opt for TAR 1.0.
What does the TAR 3.0 workflow look like?
TAR 3.0 is an approach that trains the system and predicts much faster than TAR 2.0. It follows a formal pattern approach rather than a random selection of documents to train.
After grouping the documents in clusters, the training begins using only a few documents from each cluster in order to speed up the identity of where the most relevant documents are located. For example, one cluster will have all documents related to holidays, office parties, etc. – once two or more documents are marked as ‘not relevant’, all documents in this cluster will be omitted and predicted as not relevant.
This new approach is also likely to be viewed as a combination of TAR 2.0’s continuous active learning advantages with techniques that minimize the risk in content and in costs.
What are the benefits of Technology Assisted Review?
TAR delivers much-needed savings in eDiscovery costs and efficiency. TAR is quicker and less costly than traditional manual review, giving lawyers the opportunity to focus on more strategic tasks as the tedious document review tasks are completed using machine learning and automation. Some other key benefits of in-housing Technology Assisted Review include:
- You can prioritize your set of relevant documents and be better prepared for an interview, meet and confer, a deposition or settlement discussion.
- You can perform ECA on all predicted documents and begin to strategize on your case before the review of all documents is done;
- TAR increases the likelihood of early case resolutions: litigators can make informed decisions in relation to a case early on in the process. This way, a client’s risk, and potential exposure can be mitigated without any additional unnecessary litigation expenses.
- You can continue to use the traditional review approach by using search terms, but you can also set TAR in the background to validate results or identify documents that may conflict with human review and ensure your review quality is impeccable and no documents are missing.
- Legal teams are better protected from legal fines and sanctions. Many teams struggle to collect and produce electronic data timely, risking to get multi-million fines and sanctions as well as possible reputational damage.
- TAR makes courts more widely accessible. Since TAR reduces the time and costs associated with document review, civil litigations that would have been otherwise settled to avoid lengthy battles, are now being taken to trial.
What role does TAR play in streamlining the document review phase of eDiscovery?
The myth that exhaustive manual review is the most efficient, and therefore the most defensible approach to document review is fading away. Technology Assisted Review has proven itself as yielding more accurate results with much lower effort when it comes to document review.
Before we delve into how to successfully apply TAR to improve your eDiscovery review process, let’s first cover some basics.
What does Document Review look like?
There isn’t a set-in-stone document review process that all legal teams must follow. Depending on the size of the organization or legal team as well as available resources, the document review process will vary – from a detailed process that involves thought-out search terms and analytics to one that requires a large team of reviewers and multiple coding tools.
Before moving into the Document Review phase, legal teams need to first collect, process, cull, and produce their data. However, even when they identify all the relevant documents and reach the Document Review phase, much of the effort and time will still be spent on analyzing and organizing files. That said, the process of culling and producing continues as the document review process takes place.
In many cases, legal teams move into the Document Review phase blindly, without having a thought-out strategy defined. Oftentimes, they assume that relevant documents can be identified as they move through their datasets. Instead, first, it is essential for teams to assess the set of documents they collected and make the judgment of how likely that collection is to produce responsive documents. To be able to make that call, one needs to apply a strategic rationale and analytical tools.
With good TAR recall and precision, the document review process gets simplified to attorneys tagging what’s relevant and not. The review continues, with the computer learning from each human decision, until the system reaches a point of diminishing returns where the remaining documents are not relevant to the review.
Four tips for successful use of Technology Assisted Review (TAR)
To successfully apply Technology Assisted Review to your workflow, be sure to follow the following guidelines:
- Continuously monitor and re-train the system
TAR implies an iterative process, requiring legal teams to take their time to consistently tweak and tune the machine-reviewed documents to teach algorithms to identify relevant documents more efficiently.
- Involve senior lawyers in reviewing smaller training sets
To ensure the high quality of the reviewed documents, subject matter experts should be involved to ensure the consistency of the approach and “guide” the algorithm in providing quality results by reviewing and verifying a random set of documents coded for relevance by less experienced reviewers.
- Be ready for possible blind spots, as TAR is not perfect
While TAR provides great benefits, it clearly has its blind spots. Therefore, it is important to keep your eyes open on which documents to define relevance for, as many times relevance is driven by context, not only content. For instance, sometimes, documents might be relevant for a case because they don’t contain certain content. As TAR’s logic mostly establishes relevance based on content, manual review is a must to ensure nothing of potential relevance is left behind.
- Consider using ‘example documents’ to guide your algorithm
To ease and facilitate the training of your algorithm, consider identifying ‘example documents’ and allowing the algorithm to capture similar content in order to test the system. This way, such ‘dummy’ documents can be used as springboards for similarity, email threading, and clustering tools.
In conclusion
While TAR will never fully replace legal professionals, it surely opens new possibilities for faster, more efficient, and more informed document review workflows. Moreover, it helps organizations drive down their review-associated costs and risks, improving the overall eDiscovery process. If you would like to learn more about how your organization can leverage legal AI software to take your review to the next level, contact IPRO experts.
