GROBID

GROBID is an open-source machine-learning system that converts scholarly PDFs into structured, machine-readable documents. AI enters the research workflow at the corpus-preparation stage, after documents are collected and before text mining, bibliometrics, information extraction or other computational analysis. A cascade of sequence-labelling models works on text combined with layout information to identify document components: header metadata, authors and affiliations, section structure, paragraphs, figures, tables, bibliographic references, citation contexts, and funding and licence statements. The default models are Conditional Random Fields, and deep-learning architectures (RNNs and transformers) can be selected for tasks where they measurably improve accuracy. The output is TEI XML, which can be linked back to coordinates in the original PDF. The practice is the long-term development and open provision of this parser as shared infrastructure, which lets researchers in any discipline build structured corpora from the scientific literature.

While the original document supported are scholarly publications and patents, Grobid offers also specialised processes that would allow processing different types of documents. For example IETF or 3GPP standards documents, as well as a simplified structure (title, authors, and body text) allow the coverage over more generic data extraction.

GROBID began in 2008 as a personal ‘hobby’ project by Patrice Lopez. It was released as open source in 2011 and has since been developed as a sustained side project with ongoing support from Inria. The initial idea arose from a practical bottleneck. While scholarly PDFs preserve page layout, they do not preserve machine-readable structure. This meant that large-scale computational work on the literature required a reliable, reusable parser rather than bespoke extraction scripts.

Several design choices have made broad adoption possible. Firstly, the system is modular, with separate models handling segmentation, header metadata, references, names, affiliations, and full text. This means that users can apply only the components they need. Secondly, the training data comprises small, high-quality sets that are annotated by hand rather than being derived from publisher XML. This keeps the models general across layouts and avoids dependence on proprietary sources. Thirdly, engineering priorities favour speed and scalability on standard hardware. CRF models are the default option, and deep-learning alternatives can be enabled where GPUs are available.

Standardisation and evaluation promote trust and interoperability. Output follows the TEI XML standard. The project provides a training and evaluation framework featuring n-fold cross-validation and continuously updated, end-to-end benchmarks on thousands of documents. It also provides tools for semi-automated generation of training data, enabling users to correct outputs and retrain models for their own document collections. Access is provided via a web service API, Docker images, batch processing, and client libraries in Python, Java, and Node.js, thereby lowering the entry barrier for domain researchers without AI engineering expertise. Support for users is provided through a community mailing list and Discord channel.

GROBID has become the shared infrastructure for converting scientific literature into research data. It is currently in use in several research institutes, including ResearchGate, Semantic Scholar, the HAL open archive, Internet Archive Scholar, INIST-CNRS, CERN (Invenio). Companies of different sizes also use Grobid in their infrastructure, like the STM publishers, Scite.ai, and many others. The variety of domains it covers is demonstrated by the fact that modules built on the same parsing pipeline can recognise astronomical entities, superconductor materials, properties, physical quantities, mentions of datasets and software, in scientific articles.

This approach enables processing on a scale that manual or bespoke extraction could not achieve, while using modest resources. For example, a single 16-CPU machine without GPUs processed around 915,000 PDFs per day, while 11.3 million PDFs were processed in six days using two servers. This frugality is the result of deliberate design choices. The ML models run by default on standard hardware and the use of GPUs is optional.

Openness and transparency are built into the process. The code, documentation, and training data are openly licensed, and the software is archived with a Software Heritage persistent identifier. Performance is reported through continuously updated, end-to-end benchmarks. For example, reference extraction achieves an F1 score of approximately 0.87 on PubMed Central documents and around 0.90 on bioRxiv documents. Users can therefore select models based on documented evidence and link extracted structures back to their position in the original PDF. GROBID also identifies the copyright holder and licence of each document, supporting researchers in assessing the conditions under which they can reuse the documents they collect. While newer document-parsing systems increasingly rely on large vision–language models, GROBID shows that a mature, continuously maintained pipeline of lightweight models, now extended with transformer options, can still deliver traceable structured data at scale, including as input for current LLM-based research workflows.

Different from LLM-based PDF processing, Grobid analyse the front-stream of a PDF document, allowing modules such as scientific RAG systems, to visualise the retrieved passages used to ground the LLM response back to the original PDF document via bounding boxes.

GROBID repository: https://github.com/grobidOrg/grobid 
Licence: Apache Licence 2.0
∙ Documentation: https://grobid.readthedocs.io/ 

∙ Benchmarking results: https://grobid.readthedocs.io/en/latest/benchmarks/Benchmarking/ 

∙ Public demo: https://grobidOrg-grobid.hf.space/ 

∙ Software Heritage archive: https://archive.softwareheritage.org/browse/origin/?origin_url=https://github.com/grobidOrg/grobid 

∙ Example of domain reuse: Luhmann & Burghardt (2022), Journal of the Association for Information Science and Technology, https://doi.org/10.1002/asi.24533  

Examples of domain module: grobid-superconductors, https://github.com/lfoppiano/grobid-superconductors