Improving the system to discover and verify more complex proofs would require a substantial increase in inference resources. Given this constraint, I plan to prioritize expanding its range of applications rather than pushing purely on raw proof complexity for now.

Parallelization

To improve throughput and avoid bottlenecks from slow sessions, I introduced a parallel execution scheme:

• Problems are taken from open_problems and processed through the pipeline (formalization of the statement → natural language proof → formal proof → expansion) with up to n problems running concurrently.
• The system does not wait for slower sessions; it proceeds with other available problems.
• The main_theorem_session runs in parallel as a dedicated single slot, separate from the n slots for open_problems.

This allows the system to maintain steady progress without being blocked by particularly difficult instances.

Expanding Applications

Originally, this repository aimed to construct theories in unexplored domains. However, this raises a natural question: what is the value of building theories in areas that humans themselves do not yet understand?

Rather than viewing this as a limitation, I now interpret it more positively. The system can be seen as a tool that expands a researcher’s imagination when they conceive a new, unexplored theme.

In this sense, the goal is not merely to target niche or underexplored fields, but to support the early-stage development of entirely new research directions.

As a concrete step in this direction, I am interested in collaborating with researchers working on:

• a comprehensive analysis of the expressive power of individual rules in combinatory categorial grammar, and
• a structural understanding of the landscape surrounding mildly context-sensitive grammars.

I believe this system has the potential to accelerate such investigations by systematically generating and organizing candidate statements and their formal properties.

Outlook

I will continue exploring new application domains where this framework can contribute meaningfully, while refining the system to better support theory construction workflows.

Overview

Note: This section documents newly introduced design considerations as part of ongoing progress in the system.

The search loop in automated theory construction tends to exhibit a strong bias toward generating transformed statements derived from existing theorems, including generalizations and technical lemmas. While such statements are useful for extracting local properties of a theory, they do not necessarily contribute to a unified or compressed global structure.

To address this limitation, we introduce a main-loop session centered around identifying and proving structurally significant theorems (“main theorems”). This mechanism is designed to periodically reorganize and compress the accumulated theory.

Main-Loop Session

The main-loop session operates as follows:

1. Trigger Condition Every time N new lemmas are added to Derived.lean, a main-theorem session is triggered.

2. Candidate Suggestion The system analyzes Derived.lean and uses main_theorem_suggester.md to propose at most one candidate for a main theorem. Strict filtering criteria are imposed, and proposing no candidate is explicitly allowed.

3. Proof Planning If a candidate is proposed, main_theorem_planner.md is used to construct a natural-language proof plan.

4. Formalization Loop Based on .codex/agents.md and SKILL.md, the system attempts to formalize the theorem in Lean. The loop continues until all sorry placeholders are eliminated.

5. Post-Success Expansion If formalization succeeds:

   • The theorem is appended to Derived.lean.
   • post_theorem_expander.md is invoked to generate five new open problems.

This process introduces periodic global restructuring pressure into the otherwise local search dynamics.

Pick-Up Policy (Open Problem Prioritization)

To improve search efficiency, we introduce a prioritization scheme for open problems.

Priority Levels

Each open problem is assigned one of three priority levels: high, medium, or low, determined by open_problem_prioritizer.md.

Core Rubric

• high

  • Connects existing theorem clusters.
  • Gives a strong equivalence, characterization, or existence statement.
  • Looks likely to unlock many future problems or reorganize the theory.

• medium

  • A natural local extension or useful nearby consequence.
  • Likely to help only one or two nearby problems.

• low

  • Cosmetic variant, shallow restatement, obvious weakening, or low-utility statement in the current Derived.lean context.
  • Already looks effectively covered by current verified theorems up to a shallow reformulation.

Additional Policies

• If an open problem fails twice, it is removed from the pool.

• Priorities are periodically refreshed:

  • After every M new additions.
  • After each successful main-loop formalization.

This prioritization ensures that computational resources are allocated toward structurally meaningful progress.

Scope and Generality

To improve general applicability, we relax the requirement that the system must always start strictly from axioms. However, to avoid drift during exploration, we impose the requirement that a core Lean theory file defining the domain must always be present.

This balance allows:

• Controlled exploration within a defined domain.
• Flexibility in incorporating partially structured or pre-existing theories.

Research Direction

The long-term objective of this framework is to:

• Enable systematic rediscovery and exploration of niche or underdeveloped domains.
• Revisit and restructure “mature” or seemingly exhausted areas of theory.
• Improve usability and accessibility of automated theory construction systems.

By combining local search with periodic global restructuring, the system aims to produce theories that are not only larger, but also more coherent and interpretable.

Summary

The introduction of the main-loop session and prioritization policies transforms the search process from purely accumulative to structurally aware. Instead of merely growing Derived.lean, the system actively attempts to compress, reorganize, and elevate the theory through strategically selected main theorems.

AI-generated questions can still improve, but even at this stage, it has become possible to automatically explore a surprisingly wide range of properties by manipulating expressions and checking what follows from them. That already changes the shape of the workflow.

There seems to be two different modes of doing mathematics: exploration mode and problem-solving mode.

Formalizing an important theorem is, of course, a valuable goal. But when working in an area that is still largely unexplored, it is just as important to build a broader understanding of the terrain around the result you care about. You need to know not only how to prove one theorem, but also what kinds of structures, patterns, and neighboring facts are present in the surrounding theory.

This repository is not mainly about formalizing one major theorem. Its more concrete aim is to give AI a kind of mathematical curiosity: to let it automatically discover, organize, and refine interesting facts within a domain. In the long run, I want this system to help build Basic.lean file for areas that have not yet been systematically developed.

That is the goal. At the moment, there are still some obvious challenges.

Interpretability

Because I have been pushing toward a fully automated workflow, even I sometimes lose track of what the generated output is actually doing. The system may produce results, but the overall picture can become difficult to interpret from the developer’s side.

Where To Start

I still do not have a good answer to a basic question: how can we make the system write clean, well-motivated definitions, as you find in Mathlib?

Since then, I have been interested in the idea of generating mathematical structure automatically from simple foundations. Traditional automated theorem proving has been powerful for proving individual statements, but it has been less effective as a tool for exploring and extending theories at a higher level.

Recently, the mathematical abilities of large language models have improved substantially. They can assist with proofs, suggest reformulations, and sometimes propose useful follow-up questions. Motivated by this, I started the automated-theory-construction-lean4 repository to explore how LLMs can complement formal verification and help with theory construction itself.

Overview

The basic idea is simple: start from a small axiom system, introduce a few seed propositions, and then repeatedly try to formalize, verify, and extend the resulting theory.

Each iteration looks roughly like this:

1. Put a base axiom system and a collection of elementary seed propositions into an open problem queue.
2. Retrieve one proposition from the queue and ask Codex CLI to formalize either a proof or a counterexample within a fixed time budget.
3. If the proposition is successfully formalized and verified, add it to Derived.lean as a theorem.
4. Use the logs produced during the attempt to decide what to do next: if the proposition was not formalized, put it back into the queue together with its subgoals; if it was formalized, generate more general candidate problems and add them to the queue.

In this way, the system alternates between proving statements and proposing new directions for exploration.

The most important part is the prompt used to generate follow-up problems after a successful result:

When the current problem is solved and verified (`verify_success = true` and `result = proof|counterexample`):
- Prefer outward-looking follow-up problems that extend the theory rather than merely staying near the last proof script.
- Favor, in roughly this order:
  1. natural generalizations or reusable abstractions
  2. converses, strict separations, or failure-of-converse statements
  3. existence, uniqueness, impossibility, or rigidity phenomena
  4. sharp boundary phenomena, minimal-hypothesis thresholds, or reusable structural dichotomies
  5. adjacent structural consequences that clarify the global shape of the theory
- It is good to return at least one candidate that meaningfully broadens, reinterprets, or reuses the verified result beyond the immediate local target.
- Prefer candidates whose resolution would teach something non-obvious about the theory or its models, rather than merely restating the solved fact in slightly altered form.
- If a more informative structural or threshold-style follow-up is available, prefer it over a nearby local rewrite.
- Also favor follow-up problems that vary the assumptions or structure of the theory to reveal robustness, thresholds, or failure modes.

The goal is not merely to stay close to the last proof, but to push outward toward statements that clarify the structure of the theory.

How It Works

The core loop lives in scripts/run_loop.py. At the moment, the implementation uses fixed paths:

• theory: AutomatedTheoryConstruction/Theory.lean
• accumulated theorems: AutomatedTheoryConstruction/Derived.lean
• temporary verification file: AutomatedTheoryConstruction/Scratch.lean
• initial seeds: AutomatedTheoryConstruction/seeds.jsonl
• runtime state: data/

So this is not yet a fully generic multi-theory runner. Switching to a different theory currently requires editing these files directly.

Each iteration proceeds as follows:

1. Select the next open problem deterministically.
2. If the problem is not already in Lean form, use prover_statement to translate it into a formal statement.
3. Use prover to attempt a proof, a counterexample, or determine that the problem is currently stuck.
4. Run formalize, then verify the result with:

lake env lean AutomatedTheoryConstruction/Scratch.lean

1. If verification fails, invoke repair repeatedly until the retry budget is exhausted.
2. If verification succeeds, append the resulting theorem to Derived.lean.
3. Run expand to generate additional candidate problems.
4. Update the system state deterministically (open, solved, counterexamples).

Open problems may be either Lean-formal statements or semi-formal natural language prompts. Problems that cannot yet be formalized remain in the queue.

Three-Stage Formalization

Proof formalization is split into three stages:

1. formalization of the statement
2. natural language proof generation
3. formalization of the proof in Lean

This decomposition helps separate semantic understanding from syntactic verification.

Current Experiment

At the moment, I am experimenting with code adapted from SnO2WMaN/provability-toy, which I incorporated into Theory.lean. I also added the following four propositions to seeds.jsonl:

∀ {α : Type u} [ACR α] [ACR.Prov α] [ACR.Reft α] [ACR.APS α], ∀ g : ACR.GödelFixpoint α, g.1 ≤ □g.1
∀ {α : Type u} [ACR α] [ACR.Prov α] [ACR.Reft α] [ACR.APS α] [ACR.C5 α], ∀ g h : ACR.GödelFixpoint α, g.1 ≡ h.1
∀ {α : Type u} [ACR α] [ACR.Prov α] [ACR.Reft α] [ACR.APS α] [ACR.C5 α] [Nonempty (ACR.GödelFixpoint α)], ⊠(⊤ : α) ≡ ⊠⊠(⊤ : α)
∀ {α : Type u} [ACR α] [ACR.Prov α] [ACR.Reft α] [ACR.APS α] [ACR.C5 α] [Nonempty (ACR.GödelFixpoint α)], ∃ g : ACR.GödelFixpoint α, g.1 ≡ ⊠(⊤ : α)

These are formalized versions of statements from Sections 2 and 3 of arXiv:1602.05728v1.

The results obtained so far are available in this gist. I am currently receiving feedback on the generated code.

I attach the ChatGPT response for reference.

What I Still Do Not Fully Understand

I am not a specialist in provability logic, so there are still parts of the underlying mathematics that I do not fully understand. For now, I see this project primarily as an experiment in how LLMs and Lean can be combined to support theory exploration.

I plan to study this area more carefully and write a more detailed explanation later. If you work in provability logic or related areas, I would be very happy to hear your thoughts.

Goals

One major goal is to enable the system to incorporate more general propositions into the theory as theorems, especially statements that are less tightly tied to a specific internal language.

Ideally, I would also like the resulting theories to acquire a consistent style and structure, closer to files such as Basic.lean in mathlib.

In logic, language theory, and type theory, it is common to come up with small axiom systems whose importance is not immediately clear. As a result, they tend to be deprioritized. I hope this project can serve as a tool for exploring what kinds of theories emerge from such systems and for deciding which ones are worth developing further.

I plan to keep improving the system as AI tools continue to advance. If you have small axiom systems you would like to experiment with, feel free to let me know.

Acknowledgments

I am grateful to SnO2WMaN for publishing provability-toy.