Formal Verification and Agentic Coding

  • 3 minute read
  • 5 code samples

Also available in Română

On this page

Introduction

Hello everyone! 👋

I learned about TLA+ and Lean from Boris Cherny when he tweeted about TLA+ and Lean used to verify the Claude Code SDK.

TLA was invented by Leslie Lamport in order to verify that distributed systems work the way they should. Until now, it was pretty hard to model a system with it, for me at least it was unheard of. You would have to be very skilled and spend a lot of time to create a formal verification model of the target system system.

Now, If you’re using a coding agent it’s straightforwad and easy to do.

Formal Verification

All you have to do is to prompt you agent to do it for you, and after the model is done, you can add it as an additional step in the CI system.

Note

Treat the generated specification as a draft: a checker verifies the behavior and assumptions you modeled. Compare it with the implementation before trusting a passing result.

The following prompts work:

Use TLA+ to model the transaction payment logic.
Use Lean to model the transaction payment logic.

This is just an example prompt, you could also ask the model for a prompt and it will give you one like this:

Analyze the existing payment transaction implementation and create a TLA+ specification.Model authorization, capture, settlement, refunds, retries, concurrent requests, timeouts, duplicate events, and service failures where applicable.Define and verify safety properties, including prevention of duplicate captures, preservation of transaction state consistency, and prohibition of invalid transitions. Include relevant liveness properties with explicit fairness assumptions.Run TLC or Apalache and report the checking scope, assumptions, explored bounds, counterexamples, and limitations.Do not silently change requirements to make verification pass. Identify discrepancies between the model and implementation.Add a reproducible CI verification step. Keep the specification version-controlled and require review of changes to assumptions, invariants, and behavior.
Tip

Start with one important property, such as preventing duplicate captures, and a small version of the workflow. Add retries and concurrent requests once the first model is understandable.

Experiment

Add a second capture request after a successful capture. Check whether the model checker finds a counterexample to the no-duplicate-capture property, then compare the trace with the implementation’s idempotency behavior.

You’d normally use TLA+ for distributed systems, state machines and concurrency protocols and Lean for mathematical proofs, algorithms and formally specified properties.

When I used these in a project I got pretty good results and some subtle bugs and race conditions surfaced, which were fixed by the model shortly after.

Keeping the formal verification model in sync

One challenge I see with this approach is that the formal verification model can get out of sync with the existing code base.

A strategy would be to add a step to AGENTS.md to tell the model than whenever a system with a formal verification model is changed, the formal verification model needs to be updated as well.

# Formal VerificationWhenever modifying code that is covered by a formal specification (e.g., TLA+ or Lean):- Update the formal specification to reflect any changes in system behavior.- Review existing invariants, assumptions, and properties for continued correctness.- Run the appropriate model checker or proof checker.- Do not weaken or remove verification properties simply to make checks pass.- Report any discrepancies between the implementation and its formal specification.

If you’re using Claude Code you can create a REVIEW.md file in which to specify that as well.

# ReviewWhen reviewing code changes, verify the following:Formal verification consistency: If a change affects a system, component, or workflow that has an existing formal specification (e.g., TLA+ or Lean), verify that the specification has been updated to reflect the new behavior. Ensure that relevant invariants, assumptions, and properties remain valid, and that the corresponding model checker or proof checker passes. Flag any behavioral changes that are not reflected in the formal model.

Once the feature is implemented by the agent you can always run /review or /verify to check that the change is quality.

Conclusion

Formal verification is an advanced techinque for verifying the corectness of a software application or system. It was mainly used in academia, research or mission critical systems. By having access to a state of the art frontier model everyone can verify it’s software for corectness using Lean and TLA+.

Thank you for reading! This hope this small tip will improve you agentic coding experience!