Skip to content

Get started with plugins

The main way to customize a NOMAD installation is through the use of plugins. A NOMAD plugin is a Python package that an administrator can install into a NOMAD distribution to add custom features. This page contains shows you how to create, develop and publish a NOMAD plugin. For a high-level overview of the plugin mechanism, see the NOMAD plugin system -page.

Plugin anatomy

Tip

We provide a template repository which you can use to create the initial plugin layout for you.

Plugin Git repositories should roughly follow this layout:

├── nomad-example
│   ├── src
|   │   ├── nomad_example
|   |   │   ├── apps
|   |   │   │   ├── __init__.py
|   |   │   ├── normalizers
|   |   │   │   ├── mynormalizer.py
|   |   │   │   ├── __init__.py
|   |   │   ├── schema_packages
|   |   │   │   ├── mypackage.py
|   |   │   │   ├── __init__.py
|   |   │   ├── parsers
|   |   │   │   ├── myparser.py
|   |   │   │   ├── __init__.py
│   ├── docs
│   ├── tests
│   ├── pyproject.toml
│   ├── LICENSE.txt
│   ├── README.md

We suggest using the following convention for naming the repository name and the plugin package:

  • repository name: nomad-<plugin name>
  • package name: nomad_<plugin name>

In the folder structure you can see that a single plugin can contain multiple types of customizations: apps, parsers, schema packages and normalizers. These are called a plugin entry points and you will learn more about them next.

Plugin entry points

Plugin entry points represent different types of customizations that can be added to a NOMAD installation. Entry points contain configuration, but also a resource, which lives in a separate Python module. This split enables lazy-loading: the configuration can be loaded immediately, while the resource is loaded later when/if it is required. This can significantly improve startup times, as long as all time-consuming initializations are performed only when loading the resource. This split also helps to avoid cyclical imports between the plugin code and the nomad-lab package.

For example the entry point configuration for a parser is contained in .../parsers/__init__.py and it contains e.g. the name, version and any additional entry point-specific parameters that control its behaviour. The entry point has a load method than can be called lazily to return the resource, which is a Parser instance defined in .../parsers/myparser.py.

In pyproject.toml you can expose plugin entry points for automatic discovery. E.g. to expose an app and a package, you would add the following to pyproject.toml:

[project.entry-points.'nomad.plugin']
myapp = "nomad_example.parsers:myapp"
mypackage = "nomad_example.schema_packages:mypackage"

Here it is important to use the nomad.plugin group name in the project.entry-points header. The value on the right side ("nomad_example.schema_packages:mypackage") must be a path pointing to a plugin entry point instance inside the python code. This unique key will be used to identify the plugin entry point when e.g. accessing it to read some of it's configuration values. The name on the left side (mypackage) can be set freely.

You can read more about how to write different types of entry points in their dedicated documentation pages or learn more about the Python entry point mechanism.

Plugin configuration

The plugin entry point configuration is an instance of a pydantic model. This base model may already contain entry point-specific fields (such as the file extensions that a parser plugin will match) but it is also possible to extend this model to define additional fields that control your plugin behaviour.

Here is an example of a new plugin entry point configuration class and instance for a parser, that has a new custom parameter configuration added as a pydantic Field:

from pydantic import Field
from nomad.config.models.plugins import ParserEntryPoint


class MyParserEntryPoint(ParserEntryPoint):
    parameter: int = Field(0, description='Config parameter for this parser.')

myparser = MyParserEntryPoint(
    name = 'MyParser',
    description = 'My custom parser.',
    mainfile_name_re = '.*\.myparser',
)

The plugin entry point behaviour can be controlled in nomad.yaml using plugins.entry_points.options:

plugins:
  entry_points:
    options:
      "nomad_example.parsers:myparser":
        parameter: 47

Note that the model will also validate the values coming from nomad.yaml, and you should utilize the validation mechanisms of pydantic to provide users with helpful messages about invalid configuration.

Plugin resource

The configuration class has a load method that returns the entry point resource. This is typically an instance of a class, e.g. Parser instance in the case of a parser entry point. Here is an example of a load method for a parser:

class MyParserEntryPoint(ParserEntryPoint):

    def load(self):
        from nomad_example.parsers.myparser import MyParser

        return MyParser(**self.dict())

Often when loading the resource, you will need access to the final entry point configuration defined in nomad.yaml. This way also any overrides to the plugin configuration are correctly taken into account. You can get the final configuration using the get_plugin_entry_point function and the plugin name as defined in pyproject.toml as an argument:

from nomad.config import config

configuration = config.get_plugin_entry_point('nomad_example.parsers:myparser')
print(f'The parser parameter is: {configuration.parameter}')

Controlling loading of plugin entry points

By default, plugin entry points are automatically loaded, and as an administrator you only need to install the Python package. You can, however, control which entry points to load by explicitly including/excluding them in your nomad.yaml. For example, if a plugin has the following pyproject.toml:

[project.entry-points.'nomad.plugin']
myparser = "nomad_example.parsers:myparser"

You could disable the parser entry point in your nomad.yaml with:

plugins:
  entry_points:
    exclude: ["nomad_plugin.parsers:myparser"]

Plugin development guidelines

Linting and formatting

While developing NOMAD plugins, we highly recommend using a Python linter, such as Ruff, to analyze and enforce coding standards in your plugin projects. This also ensures smoother integration and collaboration. If you have used our template repository, you will automatically have ruff defined as a development dependency with suitable defaults set in pyproject.toml together with a GitHub actions that runs the linting and formatting checks on each push to the Git repository.

Testing

For testing, you should use pytest, and a folder structure that mimics the package layout with test modules named after the tested module. For example, if you are developing a parser in myparser.py, the test folder structure should look like this:

├── nomad-example-plugin
│   ├── src
|   │   ├── nomad_example
|   |   │   ├── parsers
|   |   │   │   ├── myparser.py
|   |   │   │   ├── __init__.py
│   ├── tests
|   │   ├── parsers
|   |   │   ├── test_myparser.py
|   |   │   ├── conftest.py
|   │   ├── conftest.py

Any shared test utilities (such as pytest fixtures) should live in conftest.py modules placed at the appropriate level in the folder hierarchy, i.e. utilities dealing with parsers would live in tests/parsers/conftest.py, while root level utilities would live in tests/conftest.py. If you have used our template repository, you will automatically have an initial test folder structure, pytest defined as a development dependency in pyproject.toml and a GitHub action that runs the test suite on each push to the Git repository.

In the pytest framework, test cases are created by defining functions with the test_ prefix, which perform assertions. A typical test case could look like this:

def test_parse_file():
    parser = MyParser()
    archive = EntryArchive()
    parser.parse('tests/data/example.out', archive, logging)

    sim = archive.data
    assert len(sim.model) == 2
    assert len(sim.output) == 2
    assert archive.workflow2.x_example_magic_value == 42

You can run all the tests in the tests/ directory with:

python -m pytest -svx tests

Documentation

As your plugin matures, you should also think about documenting its usage. We recommend using mkdocs to create your documentation as a set of markdown files. If you have used our template repository, you will automatically have an initial documentation folder structure, mkdocs defined as a development dependency in pyproject.toml and a GitHub action that builds the docs to a separate gh-pages branch each push to the Git repository. Note that if you wish to host the documentation using GitHub pages, you need to enable this in the repository settings.

Publishing a plugin

Attention

The standard processes for publishing plugins and using plugins from other developers are still being worked out. The "best" practices mentioned in the following are preliminary. We aim to set up a dedicated plugin registry that allows you to publish your plugin and find plugins from others.

GitHub repository

The simplest way to publish a plugin is to have it live in a publicly shared Git repository. The package can then be installed with:

pip install git+https://<repository_url>

Note

If you develop a plugin in the context of FAIRmat or the NOMAD CoE, put your plugin repositories in the corresponding GitHub organization.

PyPI/pip package

You may additionally publish the plugin package in PyPI. Learn from the PyPI documentation how to create a package for PyPI. We recommend to use the pyproject.toml-based approach.

The PyPI documentation provides further information about how to publish a package to PyPI. If you have access to the MPCDF GitLab and NOMAD's presence there, you can also use the nomad-FAIR package registry:

pip install twine
twine upload \
    -u <username> -p <password> \
    --repository-url https://gitlab.mpcdf.mpg.de/api/v4/projects/2187/packages/pypi \
    dist/nomad-example-plugin-*.tar.gz

Installing a plugin

See our documentation on How to install plugins into a NOMAD Oasis.