By Joshua Croye, Product Manager Enterprise, IPRO Product Management
With the release of 2021.9.0. IPRO introduced Active Learning to its ever-growing toolkit in OPEN DISCOVERY. Active learning brings the power of Technology Assisted Review (TAR) without all the setup, and the best part is that it’s always on, whether or not your reviewers utilize it.
You can control how you use Active Learning, by creating an Active Learning Enabled or a Disabled Review Pass. With an Active Learning Enabled Review Pass, you may batch out the entire database. With an Active Learning Disabled Review Pass, you must first create a Saved Search. Even if you disable Active Learning upon creating a review pass, it’s running in the background and can be enabled at any time.
All you need to start taking advantage of Active Learning is to create a unique tag and a coding form to go along with a new review pass and then start tagging relevant documents with the tag you selected for the Primary Review Purpose. As soon as a document is tagged responsive to the primary review purpose and is checked in with a batch, the system starts learning.
Active Learning Metrics and Reporting
As soon as you start checking in documents, you’ll get metrics which will help you make the ever-important decision of when to produce.
Let’s take a look at some of the metrics and exactly what they mean.
Document Predictions: The predicted positive and negative documents are displayed in the overview tab. These numbers will fluctuate in the early stages of review but as the model settles in, they will begin to solidify.
Document On Hold: These are the documents placed on hold manually by your reviewers. Generally, these are documents that require an additional set of eyes and can be quickly navigated to by clicking on the search link provided on the overview tab.
Precision: This displays the percentage of predicted positive documents that were actually positive.
Recall: This displays the percentage of actually positive documents that were correctly predicted to be positive.
Active Learning Status: A status ranging from poor to excellent is used to indicate the status of the current active learning model. Many of the metrics are less reliable until this status reaches at least “good”.
Prevalence: This represents the estimated percentage of documents in the review pass that are positive.
Responsive Documents: These are the predicted range of positive documents in the review pass.
Review Progress Chart: This displays a breakdown of documents being reviewed by day, week, or month. Clicking into one of the bars on this chart will show a breakdown by user.
Relevancy Scores Chart: This displays all documents in the review pass based on their current relevancy score.
Desired Recall Chart: This displays the available options for setting a desired recall based on the review that has been done so far. Desired recall is a configurable setting when creating a review pass and sets the goal for percentage of positive documents to be identified. Setting a higher desired recall will introduce more false positives while setting it lower will lower the number of false positives.
Estimated Predicted Relevant Documents Chart: This displays the predicted relevant documents against the number of manually reviewed documents. It can be used to see the number of predicted documents leveling off as the model is refined.
Conclusion:
All these different metrics and the reports that coincide with them help reviewers make informed document review decisions.
The bottom line is that Active Learning allows you to find your relevant documents faster by serving up the documents that are most likely deemed to be relevant first. This saves time by ensuring that your reviewers are reviewing the right documents first. It also empowers you to produce when the model levels out,meaning you could produce without needing to review all of the documents in the review pass, saving time and money.
If you’d like to learn more about Active Learning, please visit IPRO’s Help Center, where you can find documentation on all IPRO products: https://my.staging.ipro.com/help
If you have topics you’d like for us to cover, please feel free to drop us a line and we will incorporate your ideas into future blog posts.
