Since the inception of ChatGPT in 2023, artificial intelligence has become more accessible and used within a legal context. Various important legal use cases exist, such as chatting with your documents, extracting meaningful information from documents, and managing the time-consuming and manual process of reviewing contracts. The capabilities shown by Large Langues Models (LLMs) have been impressive for legal use cases, where we have seen an exponential increase in reasoning capabilities for these models. See, for example, the recent announcement from OpenAI about their GPT-o1 / strawberry models.

In this post, we will discuss how AI can assist when conducting contract reviews, some critical considerations when using a contract review solution, some potential risks, and how Miramis (formerly Pocketlaw) approaches this problem using the latest and most excellent LLM / AI models and intuitive and holistic UX. 

How can AI assist with Contract Reviews

Before exploring AI's role in the Contract Review process, we need to understand what most companies mean by contract review and the role of a legal playbook.

Contract Review: is the process of carefully examining the terms and conditions of a contract to ensure that all parties’ rights, obligations, and risks are clearly defined and legally enforceable. The goal is to identify potential issues, ambiguities, or unfavorable clauses that could result in legal or financial consequences. Contract review often involves assessing payment terms, liability clauses, termination rights, confidentiality provisions, and dispute resolution mechanisms. Legal teams or contract managers conduct this review to confirm compliance with regulations and alignment with a company’s strategic objectives.

The role of a playbook in contract review refers to a predefined set of guidelines, best practices, and standardized clauses that legal teams use to streamline the review process. A contract playbook serves as a reference document outlining acceptable language, red flag terms, and negotiation strategies. It ensures consistency across contract reviews and empowers non-legal teams to handle basic contract edits within set boundaries. Playbooks are particularly useful for speeding up reviews, maintaining compliance, and consistently addressing critical risks.

TLDR: The playbook is coupled to the Contract Review process, where you might have various playbooks for contracts such as NDAs, Investment Agreements, Marketing agreements, etc. 

For an AI or AI agent (that we will get back to later in the post) to be able to solve the Contract Review problem, it requires various skills such as:

• Named Entity Recognition

• Classification

• Semantic Analysis and Understanding

• Risk Assessment

The contract review problem can also be seen as an entailment problem in AI parlance. For instance, when reviewing contracts, AI might need to:

• Entailment: Determine if a specific clause implies a condition or consequence (e.g., does a termination clause imply termination for convenience)

• Contraction: Identify if two clauses contradict each other (e.g., the governing law is Spain, but the contract states Portugal)

• Neutral: Determine whether a clause is unrelated or carries no bearing for a specific condition.

AI and LLM models can be excellent tools for finding clauses that either imply various conditions, including multiple contradictions, or find irrelevant provisions.

Important Considerations for a Contract Review Solution

When building or selecting an AI-powered Contract Review solution, it is essential to understand that the quality of the review depends not only on the performance of the underlying model but also on how it is implemented and customized for legal workflows. 

Below are some factors to consider:

1. The solution needs to be able to review the contract based on a Playbook 

2. The solution needs to be able to reason around the entailment of a contract

3. The solution needs to be able to verify Contract Adherence 

All these steps also need to be done promptly and accurately. To provide as much value as possible to you as a legal professional. 

1. Review the contract

The Contract Review Agent needs to review and understand the contract based on the playbook rules. Some critical tasks here are:

• Review/familiarize itself with the contract

• Compare the contract clause by clause and in its entirety against the playbook

• Analyze and deduce whether each contract clause aligns with the playbook rules.

2. Reasoning/entailment about a contract

The Contract Review Agent needs to be able to explain the contract’s entailment regarding specific playbook rules. Some critical tasks here are:

• Reasoning around entailment 

• Evaluate the level of alignment or deviation between a clause and the playbook rule.

• Reflection on understanding the contract and how it relates to the relevant playbook rule.

3. Verify Contract Adherence

Finally, the Contract Review Agent must analyze the contract's adherence to the playbook. Some critical tasks here are:

• Writing a contract review report detailing the contract adherence

• Generate potential issues with the contract by the playbook rules

• Offer clear and actionable recommendations for the problems identified.

These three components form the foundation for an AI-powered contract review solution that provides meaningful feedback and flags potential issues. Additionally, the solution should support playbook customization, ensure regulatory compliance, and integrate seamlessly with legal workflows, such as automatically generating redlines or enabling e-signatures.

At Miramis, we incorporate this thinking into the core of our solution. Blending AI’s power with legal expertise through human-in-the-loop oversight ensures every review is precise, relevant, and compliant with your specific playbooks and standards. This approach—combining automation with professional validation—is a critical differentiator that makes Miramis stand out as a trusted and comprehensive contract review platform.

Risk with using LLMs for contract review and why you need a Contract Review Solution 

While LLMs such as GPT-4o or Claude Sonnet 3.5 have demonstrated impressive performance in understanding and generating text, they have been lackluster regarding reasoning capabilities on a human level, especially in high-stakes fields such as legal, finance, and healthcare. 

Below are a few challenges to consider:

1.  Risk of Hallucination

Large Language Models (LLMs) sometimes generate factually incorrect or irrelevant content—called “hallucinations.” In the legal domain, this poses significant risks, as hallucinations can introduce errors into contracts, misinterpret clauses, or suggest legally unsound actions. At Miramis, we have mitigating actions in place, such as guardrails to limit the amount of hallucinations provided.

2. Risk of Non-deterministic behavior

Due to their probabilistic nature, LLMs can produce different results when asked the same question multiple times. This lack of consistency can be problematic in contract reviews, where precision and repeatability are crucial. Having a solution that limits variability by applying strict legal frameworks and rules is essential. At Miramis, we employ Data Science and ML techniques and best practices to minimize the non-deterministic behavior of LLMs.

3. Confidentially and Data Privacy

Contracts often contain sensitive information. While using LLMs for contract reviews, there’s a risk that data may be exposed to third parties, especially if processed on shared or unsecured infrastructure. A robust solution should ensure that all data is encrypted and processed in compliance with data protection regulations like GDPR.

4.  Legal domain: complexity and ambiguity

Legal language can be highly nuanced, with many contracts containing ambiguities or provisions that depend on context. LLMs often struggle with these complexities, leading to misinterpretations. A contract review solution should have domain-specific optimizations to handle legal intricacies effectively. At Miramis, we ground all our solutions in deep domain knowledge and complained with technical expertise.

5. No E2E automation of legal workflows

While LLMs can assist in parts of the contract review process, they often fail to provide complete automation of legal workflows—from document generation to e-signatures. A dedicated solution should handle the entire contract lifecycle, ensuring full automation and reducing manual work. We at Miramis have excelled at this since the early 2020s.

To help address the challenges discussed earlier, these are the reasons why you need a dedicated Contract Review Solution like Miramis:

1. Mitigate Legal Risks with Custom Playbooks:

Miramis is designed to mitigate risks by embedding your legal playbook into the AI’s reasoning. This allows our solution to reflect your organization’s legal guidelines, policies, and industry standards while flagging issues specific to your requirements.

2. Seamless, End-to-End Workflow Automation:

Our intuitive user experience (UX) is built around the entire contract lifecycle, from contract creation and automatic markups to redlining and e-signing. This ensures that legal teams and non-legal users alike can easily navigate the review process and act on insights.

3. Clear Explanations of Non-Compliance:

Miramis doesn’t just flag non-compliant clauses—it provides detailed reasoning behind why a clause is problematic, the associated risk level, and the potential consequences of not addressing it. This transparency empowers teams to make informed decisions with confidence.

4. Data Privacy and Security:

Your data is always your own. Miramis ensures that your sensitive contract data is stored securely in a dedicated, inaccessible environment to other clients. You can trust that your confidential information remains protected throughout the contract lifecycle.

5 . Industry-Specific Legal Expertise:

Miramis leverages internal legal experts to ensure the AI is optimized and fine-tuned for the legal industry. Our domain experts know the pain points and risks that matter most, ensuring our AI solution aligns with real-world legal challenges.

Why Prompt Engineering Alone is insufficient for Contract Review

While prompt engineering can be helpful, relying on it alone for contract review has several limitations:

1. Complex Reasoning: Contract reviews require deep legal reasoning, such as identifying contradictions or interpreting legal clauses in context. Prompt engineering cannot handle these complexities alone.

2. Customization: Every organization has unique legal playbooks, standards, and risk tolerances. Prompt engineering lacks the flexibility to adapt to these specific rules without deeper integration fully.

3. Contextual Understanding: Legal contracts often contain ambiguities or clauses that depend on nuanced interpretation. Prompt engineering does not provide the contextual understanding required for precise legal review.

4. Ensuring Accuracy: Without human validation, prompt-engineered responses may miss essential issues or generate hallucinations, where the AI produces incorrect information. This can be especially risky in legal contexts.

5. Regulatory Compliance: Contract reviews must ensure compliance with industry regulations, which vary across sectors. Prompt engineering alone does not address this need for highly specialized legal understanding.

Learn more about AI for contract reviewing

Are you interested in learning more about AI and how it can assist you in the contract review process? Book a meeting with us.