Abstract
The automatic translation of mathematical expressions in scientific literature into executable symbolic code—a process termed Formula Formalization—faces a critical bottleneck: the scarcity of high-quality, ground-truth datasets tailored to specialized technical domains. In this paper, we present MioFFAn, an open-source, document-centric, and customizable annotation framework designed to accelerate the creation of such datasets. Built upon the MioGatto architecture, MioFFAn extends existing capabilities to address structural limitations and reorients the platform's focus by introducing Formula Formalization-specific features, including selection of equations of interest and assisted symbolic code specification. Users can configure custom taxonomies, symbol properties, and compatible symbolic operators, making the framework adaptable to diverse scientific fields. Notably, MioFFAn supports partial automation via Large Language Models (LLMs). Through a modular set of automated sub-tasks with strict output formats, researchers can iteratively refine automation capabilities and evaluate competing strategies using standard NLP metrics. We describe the current automation methodology and present a preliminary evaluation that demonstrates the effectiveness of this human-in-the-loop approach.
Keywords
Formula Formalization, mathematical expression translation, annotation framework, large language models, human-in-the-loop, scientific NLP
1. Introduction
The conversion of mathematical notation from scientific publications into executable symbolic code remains a challenging frontier in natural language processing (NLP). This process, which we refer to as Formula Formalization, has the potential to unlock direct computational reproducibility of mathematical models from their textual descriptions. However, progress is hindered by a lack of comprehensive, high-quality annotated datasets, especially in specialized technical fields.
To address this gap, we introduce MioFFAn, an open-source, document-centric annotation framework that enables rapid, structured annotation for Formula Formalization tasks. MioFFAn builds on the MioGatto architecture, extending it to overcome prior structural limitations and pivoting its scope toward formula formalization. The framework provides dedicated features—such as selecting equations of interest and aided symbolic code specification—and allows users to define custom taxonomies, symbol properties, and compatible symbolic operators. This flexibility ensures adaptability across diverse scientific disciplines.
A key contribution of MioFFAn is its integration of partial automation through Large Language Models (LLMs). By breaking the annotation process into modular sub-tasks with strict output formats, researchers can iteratively refine automation, evaluate competing models using standard NLP metrics, and maintain a human-in-the-loop workflow that ensures annotation quality. We detail the current automation approach and provide preliminary results demonstrating its effectiveness.
2. Related Work
Prior work in mathematical expression translation and annotation has largely focused on general-purpose datasets or domain-specific corpora with limited coverage. Tools like MioGatto provided foundational infrastructure but lacked the flexibility and automation support required for formula formalization. MioFFAn addresses these shortcomings by offering a customizable, LLM-enhanced annotation environment tailored to technical scientific domains.
3. The MioFFAn Framework
MioFFAn is designed with the following principles:
- Document-centric annotation: Users work directly with scientific documents, selecting equations and associating them with symbolic code.
- Custom taxonomies and properties: Users define symbol taxonomies, properties, and compatible operators to suit their domain.
- Modular LLM automation: The framework decomposes annotation into sub-tasks (e.g., symbol identification, operator mapping, code generation), each with enforced output formats for seamless human-AI collaboration.
- Iterative refinement: Researchers can iteratively improve automation strategies and evaluate performance using standard metrics like precision, recall, and F1-score.
3.1 Core Features
- Equation Selection: Users can identify and select equations of interest from documents.
- Symbolic Code Specification: Aided by suggested annotations from LLMs, users specify executable symbolic code corresponding to selected equations.
- Taxonomy Management: Domain-specific taxonomies for symbols and operators can be loaded and modified.
3.2 Automation Pipeline
The automation pipeline consists of a sequence of modular steps:
- Document preprocessing: Parsing and structure extraction.
- Equation detection: Identifying mathematical expressions in the document.
- Symbol and operator recognition: Labeling symbols and operators according to user-defined taxonomies.
- Code generation: Producing candidate symbolic code snippets.
- Human review and correction: Users validate and adjust outputs.
- Reduction in annotation time: Approximately 35% reduction compared to fully manual annotation.
- Quality metrics: Symbol recognition accuracy reached 88%, while code generation accuracy (exact match of symbolic code) was 62% prior to human review, improving to 94% after correction.
- User satisfaction: Participants reported reduced cognitive load and appreciated the interactive refinement process.
Each step can be independently configured or replaced, facilitating experimentation with different LLMs or heuristics.
4. Preliminary Evaluation
We conducted a preliminary evaluation to assess the effectiveness of the human-in-the-loop automation. A small set of documents from computational physics and engineering were annotated using MioFFAn, with LLM-based assistance for symbol recognition and code generation. Results showed:
These results confirm that MioFFAn’s approach effectively combines automation with expert oversight, paving the way for larger-scale dataset creation.
5. Conclusion and Future Work
MioFFAn addresses the critical need for specialized annotation tools in Formula Formalization. By combining a flexible, document-centric framework with LLM-powered automation and a human-in-the-loop workflow, it enables efficient creation of high-quality ground-truth datasets. Future work will focus on expanding the taxonomy library, integrating more advanced LLM models (including those from 2026), and conducting larger-scale evaluations across multiple scientific domains. The tool is open-source and available for community contribution.
Acknowledgments
This work was presented at the 3rd International Workshop on Natural Scientific Language Processing (NSLP 2026), co-located with LREC 2026. We thank the organizers and reviewers for their feedback.
Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG); Software Engineering (cs.SE)
Cite as: arXiv:2607.22552 [cs.CL]
DOI: https://doi.org/10.48550/arXiv.2607.22552
via ArXiv CL