Configuration¶

Introduction¶

Many aspects of NOMAD and its operation can be modified through configuration. Most configuration items have reasonable defaults and typically only a small subset has to be overwritten.

Configuration items get their value in the following order:

The item is read from the environment. This has the highest priority and will overwrite values in a nomad.yaml file or the NOMAD source-code.
The value is given in a nomad.yaml configuration file.
There is no custom value, and the value hard-coded in the NOMAD sources will be used.

Configuration items are structured. The configuration is hierarchical and items are aggregated in potentially nested section. For example the configuration item services.api_host denotes the attribute api_host in the configuration section services.

Setting values from the environment¶

NOMAD services will look at the environment. All environment variables starting with NOMAD_ are considered. The rest of the name is interpreted as a configuration item. Sections and attributes are concatenated with a _. For example, the environment variable NOMAD_SERVICES_API_HOST will set the value for the api_host attribute in the services section.

Setting values from a `nomad.yaml`¶

NOMAD services will look for a nomad.yaml file. By default, they will look in the current working directory. This location can be overwritten with the NOMAD_CONFIG environment variable.

The configuration sections and attributes can be denoted with YAML objects and attributes. Here is an example nomad.yaml file:

services:
  api_host: 'localhost'
  api_base_path: '/nomad-oasis'

oasis:
  is_oasis: true
  uses_central_user_management: true

north:
  jupyterhub_crypt_key: '978bfb2e13a8448a253c629d8dd84ff89587f30e635b753153960930cad9d36d'

meta:
  deployment: 'oasis'
  deployment_url: 'https://my-oasis.org/api'
  maintainer_email: 'me@my-oasis.org'

logtransfer:
  enabled: false

mongo:
    db_name: nomad_oasis_v1

elastic:
    entries_index: nomad_oasis_entries_v1
    materials_index: nomad_oasis_materials_v1

When overwriting an object in the configuration, the new value will be merged with the default value. The new merged object will have all of the attributes of the new object in addition to any old attributes that were not overwritten. This allows you to simply change an individual setting without having to provide the entire structure again, which simplifies customization that happens deep in the configuration hierarchy. When overwriting anything else (numbers, strings, lists etc.) the new value completely replaces the old one.

User interface customization¶

Many of the UI options use a data model that contains the following three fields: include, exclude and options. This structure allows you to easily disable, enable, reorder and modify the UI layout with minimal config rewrite. Here are examples of common customization tasks using the search columns as an example:

Disable item:

ui:
  apps:
    options:
      entries:
        columns:
          exclude: ['upload_create_time']

Explicitly select the shown items and their order

ui:
  apps:
    options:
      entries:
        columns:
          include: ['entry_id', 'upload_create_time']

Modify existing option

ui:
  apps:
    options:
      entries:
        columns:
          options:
            upload_create_time:
              label: "Uploaded"

Add a new item that does not yet exist in options. Note that by default all options are shown in the order they have been declared unless the order is explicitly given in include.

ui:
  apps:
    options:
      entries:
        columns:
          options:
            upload_id:
              label: "Upload ID"

The following is a reference of all configuration sections and attributes.

Services¶

services¶

Contains basic configuration of the NOMAD services (app, worker, north).

name	type
console_log_level	`Union[int, str]`	The log level that controls console logging for all NOMAD services (app, worker, north). The level is given in Python `logging` log level numbers. default: `30`
api_host	`str`	The external hostname that clients can use to reach this NOMAD installation. default: `localhost`
api_port	`int`	The port used to expose the NOMAD app and api to clients. default: `8000`
api_base_path	`str`	The base path prefix for the NOMAD app and api. default: `/fairdi/nomad/latest`
api_secret	`str`	A secret that is used to issue download and other tokens. default: `defaultApiSecret`
api_timeout	`int`	If the NOMAD app is run with gunicorn as process manager, this timeout (in s) is passed and worker processes will be restarted, if they do not respond in time. default: `600`
https	`int`	Set to `True`, if external clients are using SSL to connect to this installation. Requires to setup a reverse-proxy (e.g. the one used in the docker-compose based installation) that handles the SSL encryption. default: `False`
https_upload	`int`	Set to `True`, if upload curl commands should suggest the use of SSL for file uploads. This can be configured independently of `https` to suggest large file via regular HTTP. default: `False`
admin_user_id	`str`	The admin user `user_id`. All users are treated the same; there are no particular authorization information attached to user accounts. However, the API will grant the user with the given `user_id` more rights, e.g. using the `admin` owner setting in accessing data. default: `00000000-0000-0000-0000-000000000000`
encyclopedia_base	`str`	This enables links to the given encyclopedia installation in the UI. default: `https://nomad-lab.eu/prod/rae/encyclopedia/#`
optimade_enabled	`int`	If true, the app will serve the optimade API. default: `True`
dcat_enabled	`int`	If true the app will serve the DCAT API. default: `True`
h5grove_enabled	`int`	If true the app will serve the h5grove API. default: `True`
upload_limit	`int`	The maximum allowed unpublished uploads per user. If a user exceeds this amount, the user cannot add more uploads. default: `10`
force_raw_file_decoding	`int`	By default, text raw-files are interpreted with utf-8 encoding. If this fails, the actual encoding is guessed. With this setting, we force to assume iso-8859-1 encoding, if a file is not decodable with utf-8. default: `False`
max_entry_download	`int`	There is an inherent limit in page-based pagination with Elasticsearch. If you increased this limit with your Elasticsearch, you can also adopt this setting accordingly, changing the maximum amount of entries that can be paginated with page-base pagination. Page-after-value-based pagination is independent and can be used without limitations. default: `50000`
unavailable_value	`str`	Value that is used in `results` section Enum fields (e.g. system type, spacegroup, etc.) to indicate that the value could not be determined. default: `unavailable`
app_token_max_expires_in	`int`	Maximum expiration time for an app token in seconds. Requests with a higher value will be declined. default: `2592000`
html_resource_http_max_age	`int`	Used for the max_age cache-control directive on statically served html, js, css resources. default: `60`
image_resource_http_max_age	`int`	Used for the max_age cache-control directive on statically served image resources. default: `2592000`
upload_members_group_search_enabled	`int`	If true, the GUI will show a search for groups as upload members. default: `False`

meta¶

Metadata about the deployment and how it is presented to clients.

name	type
deployment_url	`str`	The NOMAD deployment's url. If not explicitly set, will default to the (api url) read from the configuration.
label	`str`	An additional log-stash data key-value pair added to all logs. Can be used to differentiate deployments when analyzing logs.
beta	`dict`	Additional data that describes how the deployment is labeled as a beta-version in the UI. default: Complex object, default value not displayed.
version	`str`	The NOMAD version string. default: `0.0`
commit	`str`	The source-code commit that this installation's NOMAD version is build from. default: ""
deployment	`str`	Human-friendly name of this nomad deployment. default: `devel`
service	`str`	Name for the service that is added to all logs. Depending on how NOMAD is installed, services get a name (app, worker, north) automatically. default: `unknown nomad service`
name	`str`	Web-site title for the NOMAD UI. default: `NOMAD` deprecated
homepage	`str`	Provider homepage. default: `https://nomad-lab.eu` deprecated
source_url	`str`	URL of the NOMAD source-code repository. default: `https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR` deprecated
maintainer_email	`str`	Email of the NOMAD deployment maintainer. default: `markus.scheidgen@physik.hu-berlin.de`

oasis¶

Settings related to the configuration of a NOMAD Oasis deployment.

name	type
allowed_users	`List[str]`	A list of usernames or user account emails. These represent a white-list of allowed users. With this, users will need to login right-away and only the listed users might use this deployment. All API requests must have authentication information as well.
is_oasis	`int`	Set to `True` to indicate that this deployment is a NOMAD Oasis. default: `False`
uses_central_user_management	`int`	Set to True to use the central user-management. Typically the NOMAD backend is using the configured `keycloak` to access user data. With this, the backend will use the API of the central NOMAD (`central_nomad_deployment_url`) instead. default: `False`
central_nomad_deployment_url	`str`	The URL of the API of the NOMAD deployment that is considered the central NOMAD. default: `https://nomad-lab.eu/prod/v1/api`

north¶

Settings related to the operation of the NOMAD remote tools hub service north.

name	type
enabled	`int`	Enables or disables the NORTH API and UI views. This is independent of whether you run a jupyter hub or not. default: `True`
hub_connect_ip	`str`	Overwrites the default hostname that can be used from within a north container to reach the host system. Typically has to be set for non Linux hosts. Set this to `host.docker.internal` on windows/macos.
hub_connect_url	`str`	This setting is forwarded to jupyterhub; refer to the jupyterhub documentation.
docker_network	`str`	This setting is forwarded to jupyterhub; refer to the jupyterhub documentation.
jupyterhub_crypt_key	`str`	This setting is forwarded to jupyterhub; refer to the jupyterhub documentation.
nomad_host	`str`	The NOMAD app host name that spawned containers use.
nomad_access_token_expiry_time	`int`	All tools are run with an access token for the NOMAD api in the NOMAD_CLIENT_ACCESS_TOKEN environment variable. This token will be automatically used by the nomad-lab Python package, e.g. if you use the ArchiveQuery to access data. This option sets the amount of seconds that this token is valid for. default: `86400`
tools	`NORTHTools`	The available north tools. Either the tools definitions as dict or a path to a .json file. default: Complex object, default value not displayed.
hub_service_api_token	`str`	A secret token shared between NOMAD and the NORTH jupyterhub. This needs to be the token of an admin service. default: `secret-token`
hub_ip	`str`	This setting is forwarded to jupyterhub; refer to the jupyterhub documentation. default: `0.0.0.0`
hub_host	`str`	The internal host name that NOMAD services use to connect to the jupyterhub API. default: `localhost`
hub_port	`int`	The internal port that NOMAD services use to connect to the jupyterhub API. default: `9000`
windows	`int`	Enable windows OS hacks. default: `True`

NORTHTools¶

name	type
include	`List[str]`	List of included options. If not explicitly defined, all of the options will be included by default.
exclude	`List[str]`	List of excluded options. Has higher precedence than include.
options	`Dict[str, NORTHTool]`	The available plugin. default: Complex object, default value not displayed.

NORTHTool¶

name	type
short_description	`str`	A short description of the tool, e.g. shown in the NOMAD GUI.
description	`str`	A description of the tool, e.g. shown in the NOMAD GUI.
image	`str`	The docker image (incl. tags) to use for the tool.
cmd	`str`	The container cmd that is passed to the spawner.
image_pull_policy	`str`	The image pull policy used in k8s deployments. default: `Always`
privileged	`int`	Whether the tool needs to run in privileged mode. default: `False`
default_url	`str`	An optional path prefix that is added to the container URL to reach the tool, e.g. "/lab" for jupyterlab.
path_prefix	`str`	An optional path prefix that is added to the container URL to reach the files, e.g. "lab/tree" for jupyterlab.
with_path	`int`	Whether the tool supports a path to a file or directory. This also enables tools to be launched from files in the NOMAD UI. default: `False`
file_extensions	`List[str]`	The file extensions of files that this tool should be launchable for. default: `[]`
mount_path	`str`	The path in the container where uploads and work directories will be mounted, e.g. /home/jovyan for Jupyter containers.
icon	`str`	A URL to an icon that is used to represent the tool in the NOMAD UI.
maintainer	`List[NORTHToolMaintainer]`	The maintainers of the tool. default: `[]`
external_mounts	`List[NORTHExternalMount]`	Additional mounts to be added to tool containers. default: `[]`

NORTHExternalMount¶

name	type
host_path	`str`
bind	`str`
mode	`str`	default: `ReadMode.ro` options: - `ro` - `rw`

NORTHToolMaintainer¶

name	type
name	`str`
email	`str`

Files, databases, external services¶

rabbitmq¶

Configures how NOMAD is connecting to RabbitMQ.

name	type
host	`str`	The name of the host that runs RabbitMQ. default: `localhost`
user	`str`	The RabbitMQ user that is used to connect. default: `rabbitmq`
password	`str`	The password that is used to connect. default: `rabbitmq`

fs¶

name	type
staging_external	`str`
public_external	`str`
north_home_external	`str`
archive_version_suffix	`Union[str, List[str]]`	This allows to add an additional segment to the names of archive files and thereby allows different NOMAD installations to work with the same storage directories and raw files, but with separate archives. If this is a list, the first string is used. If the file with the first string does not exist on read, the system will look for the file with the next string, etc. default: `['v1.2', 'v1']`
external_working_directory	`str`
tmp	`str`	default: `.volumes/fs/tmp`
staging	`str`	default: `.volumes/fs/staging`
public	`str`	default: `.volumes/fs/public`
north_home	`str`	default: `.volumes/fs/north/users`
local_tmp	`str`	default: `/tmp`
prefix_size	`int`	default: `2`
working_directory	`str`	default: `/app`

elastic¶

name	type
username	`str`
password	`str`
host	`str`	default: `localhost`
port	`int`	default: `9200`
timeout	`int`	default: `60`
bulk_timeout	`int`	default: `600`
bulk_size	`int`	default: `1000`
entries_per_material_cap	`int`	default: `1000`
entries_index	`str`	default: `nomad_entries_v1`
materials_index	`str`	default: `nomad_materials_v1`

keycloak¶

name	type
public_server_url	`str`
client_secret	`str`
server_url	`str`	default: `https://nomad-lab.eu/fairdi/keycloak/auth/`
realm_name	`str`	default: `fairdi_nomad_prod`
username	`str`	default: `admin`
password	`str`	default: `password`
client_id	`str`	default: `nomad_public`

mongo¶

Connection and usage settings for MongoDB.

name	type
host	`str`	The name of the host that runs mongodb. default: `localhost`
port	`int`	The port to connect with mongodb. default: `27017`
db_name	`str`	The used mongodb database name. default: `nomad_v1`
username	`str`
password	`str`

logstash¶

name	type
level	`Union[int, str]`	default: `10`
enabled	`int`	default: `False`
host	`str`	default: `localhost`
tcp_port	`str`	default: `5000`

mail¶

name	type
cc_address	`str`
enabled	`int`	default: `False`
with_login	`int`	default: `False`
host	`str`	default: ""
port	`int`	default: `8995`
user	`str`	default: ""
password	`str`	default: ""
from_address	`str`	default: `support@nomad-lab.eu`

datacite¶

name	type
mds_host	`str`	default: `https://mds.datacite.org`
enabled	`int`	default: `False`
prefix	`str`	default: `10.17172`
user	`str`	default: `*`
password	`str`	default: `*`

rfc3161_timestamp¶

name	type
cert	`str`	Path to the optional rfc3161ng timestamping server certificate.
username	`str`
password	`str`
server	`str`	The rfc3161ng timestamping host. default: `http://zeitstempel.dfn.de`
hash_algorithm	`str`	Hash algorithm used by the rfc3161ng timestamping server. default: `sha256`

Processing¶

celery¶

name	type
max_memory	`float`	default: `64000000.0`
timeout	`int`	default: `1800`
acks_late	`int`	default: `False`
routing	`str`	default: `queue`
priorities	`dict`	default: Complex object, default value not displayed.

normalize¶

name	type
normalizers	`Options`	default: Complex object, default value not displayed.
springer_db_path	`str`	default: `/app/nomad/config/models/normalizing/data/springer.msg`
system_classification_with_clusters_threshold	`int`	The system size limit for running the dimensionality analysis. For very large systems the dimensionality analysis will get too expensive. default: `64`
clustering_size_limit	`int`	The system size limit for running the system clustering. For very large systems the clustering will get too expensive. default: `600`
symmetry_tolerance	`float`	Symmetry tolerance controls the precision used by spglib in order to find symmetries. The atoms are allowed to move this much from their symmetry positions in order for spglib to still detect symmetries. The unit is angstroms. The value of 0.1 is used e.g. by Materials Project according to https://pymatgen.org/pymatgen.symmetry.html#pymatgen.symmetry.analyzer.SpacegroupAnalyzer default: `0.1`
prototype_symmetry_tolerance	`float`	The symmetry tolerance used in aflow prototype matching. Should only be changed before re-running the prototype detection. default: `0.1`
max_2d_single_cell_size	`int`	Maximum number of atoms in the single cell of a 2D material for it to be considered valid. default: `7`
cluster_threshold	`float`	The distance tolerance between atoms for grouping them into the same cluster. Used in detecting system type. default: `2.5`
angle_rounding	`float`	Defines the "bin size" for rounding cell angles for the material hash in degree. default: `10.0`
flat_dim_threshold	`float`	The threshold for a system to be considered "flat". Used e.g. when determining if a 2D structure is purely 2-dimensional to allow extra rigid transformations that are improper in 3D but proper in 2D. default: `0.1`
k_space_precision	`float`	The threshold for point equality in k-space. Unit: 1/m. default: `150000000.0`
band_structure_energy_tolerance	`float`	The energy threshold for how much a band can be on top or below the fermi level in order to still detect a gap. Unit: Joule. default: `8.01088e-21`

Options¶

Common configuration class used for enabling/disabling certain elements and defining the configuration of each element.

name	type
include	`List[str]`	List of included options. If not explicitly defined, all of the options will be included by default.
exclude	`List[str]`	List of excluded options. Has higher precedence than include.
options	`Dict[str, Any]`	Contains the available options. default: Complex object, default value not displayed.

process¶

name	type
redirect_stdouts	`int`	True will redirect lines to stdout (e.g. print output) that occur during processing (e.g. created by parsers or normalizers) as log entries. default: `False`
store_package_definition_in_mongo	`int`	Configures whether to store the corresponding package definition in mongodb. default: `False`
add_definition_id_to_reference	`int`	Configures whether to attach definition id to `m_def`, note it is different from `m_def_id`. The `m_def_id` will be exported with the `with_def_id=True` via `m_to_dict`. default: `False`
write_definition_id_to_archive	`int`	Write `m_def_id` to the archive. default: `False`
index_materials	`int`	default: `True`
reuse_parser	`int`	default: `True`
metadata_file_name	`str`	default: `nomad`
metadata_file_extensions	`tuple`	default: `('json', 'yaml', 'yml')`
auxfile_cutoff	`int`	default: `100`
parser_matching_size	`int`	default: `12000`
max_upload_size	`int`	default: `34359738368`
use_empty_parsers	`int`	default: `False`
rfc3161_skip_published	`int`	default: `False`

reprocess¶

Configures standard behaviour when reprocessing. Note, the settings only matter for published uploads and entries. For uploads in staging, we always reparse, add newfound entries, and delete unmatched entries.

name	type
rematch_published	`int`	default: `True`
reprocess_existing_entries	`int`	default: `True`
use_original_parser	`int`	default: `False`
add_matched_entries_to_published	`int`	default: `True`
delete_unmatched_published_entries	`int`	default: `False`
index_individual_entries	`int`	default: `False`

bundle_export¶

Controls behaviour related to exporting bundles.

name	type
default_cli_bundle_export_path	`str`	Default path used when exporting bundles using the CLI command. default: `./bundles`
default_settings	`BundleExportSettings`	General default settings. default: Complex object, default value not displayed.
default_settings_cli	`BundleExportSettings`	Additional default settings, applied when exporting using the CLI. This allows to override some of the settings specified in the general default settings above.

BundleExportSettings¶

name	type
include_raw_files	`int`	If the raw files should be included in the export default: `True`
include_archive_files	`int`	If the parsed archive files should be included in the export default: `True`
include_datasets	`int`	If the datasets should be included in the export default: `True`

bundle_import¶

Controls behaviour related to importing bundles.

name	type
required_nomad_version	`str`	Minimum NOMAD version of bundles required for import. default: `1.1.2`
default_cli_bundle_import_path	`str`	Default path used when importing bundles using the CLI command. default: `./bundles`
allow_bundles_from_oasis	`int`	If oasis admins can "push" bundles to this NOMAD deployment. default: `False`
allow_unpublished_bundles_from_oasis	`int`	If oasis admins can "push" bundles of unpublished uploads. default: `False`
default_settings	`BundleImportSettings`	General default settings. default: Complex object, default value not displayed.
default_settings_cli	`BundleImportSettings`	Additional default settings, applied when importing using the CLI. This allows to override some of the settings specified in the general default settings above. default: Complex object, default value not displayed.

BundleImportSettings¶

name	type
include_raw_files	`int`	If the raw files should be included in the import default: `True`
include_archive_files	`int`	If the parsed archive files should be included in the import default: `True`
include_datasets	`int`	If the datasets should be included in the import default: `True`
include_bundle_info	`int`	If the bundle_info.json file should be kept (not necessary but may be nice to have. default: `True`
keep_original_timestamps	`int`	If all timestamps (create time, publish time etc) should be imported from the bundle. default: `False`
set_from_oasis	`int`	If the from_oasis flag and oasis_deployment_url should be set. default: `True`
delete_upload_on_fail	`int`	If False, it is just removed from the ES index on failure. default: `False`
delete_bundle_on_fail	`int`	Deletes the source bundle if the import fails. default: `True`
delete_bundle_on_success	`int`	Deletes the source bundle if the import succeeds. default: `True`
delete_bundle_include_parent_folder	`int`	When deleting the bundle, also include parent folder, if empty. default: `True`
trigger_processing	`int`	If the upload should be processed when the import is done (not recommended). default: `False`
process_settings	`Reprocess`	When trigger_processing is set to True, these settings control the reprocessing behaviour (see the config for `reprocess` for more info). NOTE: reprocessing is no longer the recommended method to import bundles. default: Complex object, default value not displayed.

Reprocess¶

Configures standard behaviour when reprocessing. Note, the settings only matter for published uploads and entries. For uploads in staging, we always reparse, add newfound entries, and delete unmatched entries.

name	type
rematch_published	`int`	default: `True`
reprocess_existing_entries	`int`	default: `True`
use_original_parser	`int`	default: `False`
add_matched_entries_to_published	`int`	default: `True`
delete_unmatched_published_entries	`int`	default: `False`
index_individual_entries	`int`	default: `False`

archive¶

name	type
block_size	`int`	In case of using blocked TOC, this is the size of each block. default: `1048576`
read_buffer_size	`int`	GPFS needs at least 256K to achieve decent performance. default: `1048576`
copy_chunk_size	`int`	The chunk size of every read of binary data. It is used to copy data from one file to another. A small value will result in more syscalls, a large value will result in higher peak memory usage. default: `16777216`
toc_depth	`int`	Depths of table of contents in the archive. default: `10`
small_obj_optimization_threshold	`int`	For any child of lists/dicts whose encoded size is smaller than this value, no TOC will be generated. default: `1048576`
fast_loading	`int`	When enabled, this flag determines whether to read the whole dict/list at once when a certain mount of children has been visited. This reduces the number of syscalls although data may be repeatedly read. Otherwise, always read children one by one. This may slow down the loading as more syscalls are needed. default: `True`
fast_loading_threshold	`float`	If the fraction of children that have been visited is less than this threshold, fast loading will be used. default: `0.6`
trivial_size	`int`	To identify numerical lists. default: `20`

User Interface¶

These settings affect the behaviour of the user interface. Note that the preferred way for creating custom apps is by using app plugin entry points.

ui¶

Used to customize the user interface.

name	type
app_base	`str`	This is automatically set.
north_base	`str`	This is automatically set.
theme	`Theme`	Controls the site theme and identity.
unit_systems	`UnitSystems`	Controls the available unit systems.
entry	`Entry`	Controls the entry visualization.
apps	`Apps`	deprecated
north	`NORTHUI`	NORTH (NOMAD Remote Tools Hub) UI configuration. default: Complex object, default value not displayed.
example_uploads	`ExampleUploads`	Controls the available example uploads. default: Complex object, default value not displayed.

NORTHUI¶

NORTH (NOMAD Remote Tools Hub) UI configuration.

name	type
enabled	`int`	Whether the NORTH tools are available in the UI. The default value is read from the root-level NORTH configuration. default: `True`

Entry¶

Controls the entry visualization.

name	type
cards	`Cards`	Controls the cards that are displayed on the entry overview page.

Cards¶

Contains the overview page card definitions and controls their visibility.

name	type
include	`List[str]`	List of included options. If not explicitly defined, all of the options will be included by default.
exclude	`List[str]`	List of excluded options. Has higher precedence than include.
options	`Dict[str, Card]`	Contains the available card options.

Card¶

Definition for a card shown in the entry overview page.

name	type
error	`str`	The error message to show if an error is encountered within the card.

ExampleUploads¶

Controls the availability of example uploads.

name	type
include	`List[str]`	List of included options. If not explicitly defined, all of the options will be included by default.
exclude	`List[str]`	List of excluded options. Has higher precedence than include.

Theme¶

Theme and identity settings.

name	type
title	`str`	Site name in the browser tab.

Apps¶

Contains App definitions and controls their availability.

name	type
include	`List[str]`	List of included options. If not explicitly defined, all of the options will be included by default.
exclude	`List[str]`	List of excluded options. Has higher precedence than include.
options	`Dict[str, App]`	Contains the available app options.

UnitSystems¶

Controls the available unit systems.

name	type
include	`List[str]`	List of included options. If not explicitly defined, all of the options will be included by default.
exclude	`List[str]`	List of excluded options. Has higher precedence than include.
options	`Dict[str, UnitSystem]`	Contains the available unit systems.
selected	`str`	Selected option.

UnitSystem¶

name	type
label	`str`	Short, descriptive label used for this unit system.
units	`Dict[str, UnitSystemUnit]`	Contains a mapping from each dimension to a unit. If a unit is not specified for a dimension, the SI equivalent will be used by default. The following dimensions are available: - dimensionless - length - mass - time - current - temperature - luminosity - luminous_flux - substance - angle - information - force - energy - power - pressure - charge - resistance - conductance - inductance - magnetic_flux - magnetic_field - frequency - luminance - illuminance - electric_potential - capacitance - activity

UnitSystemUnit¶

name	type
definition	`str`	The unit definition. Can be a mathematical expression that combines several units, e.g. `(kg * m) / s^2`. You should only use units that are registered in the NOMAD unit registry (`nomad.units.ureg`).
locked	`int`	Whether the unit is locked in the unit system it is defined in. default: `False`

Others¶

logtransfer¶

Configuration of logtransfer and statistics service.

When enabled (enabled) an additional logger will write logs to a log file (log_file). At regular intervals (transfer_interval) a celery task is scheduled. It will log a set of statistics. It will copy the log file (transfer_log_files). Transfer the contents of the copy to the central NOMAD (oasis.central_nomad_deployment_url) and delete the copy. The transfer is only done if the the log file has a certain size (transfer_threshold). Only a maximum amount of logs are transferred (transfer_capacity). Only logs with a certain level (level) are considered. The files will be stored in fs.tmp.

name	type
enabled	`int`	If enabled this starts process that frequently generates logs with statistics. default: `False`
transfer_threshold	`int`	The minimum size in bytes of stored logs before logs are transferred. 0 means transfer at every transfer interval. default: `0`
transfer_capacity	`int`	The maximum number of bytes of stored logs that are transferred. Excess is dropped. default: `1000000`
transfer_interval	`int`	Time interval in seconds after which stored logs are potentially transferred. default: `600`
level	`Union[int, str]`	The min log level for logs to be transferred. default: `20`
log_file	`str`	The log file that is used to store logs for transfer. default: `nomad.log`
transfer_log_file	`str`	The log file that is used to copy logs for transfer. default: `.transfer.log`
file_rollover_wait_time	`float`	Time in seconds to wait after log file was "rolled over" for transfer. default: `1`

tests¶

name	type
assume_auth_for_username	`str`	Will assume that all API calls with no authentication have authentication for the user with the given username.
default_timeout	`int`	default: `60`

resources¶

name	type
enabled	`int`	default: `False`
db_name	`str`	default: `nomad_v1_resources`
max_time_in_mongo	`float`	Maxmimum time a resource is stored in mongodb before being updated. default: `31536000.0`
download_retries	`int`	Number of retries when downloading resources. default: `2`
download_retry_delay	`int`	Delay between retries in seconds default: `10`
max_connections	`int`	Maximum simultaneous connections used to download resources. default: `10`

client¶

name	type
user	`str`
password	`str`
access_token	`str`
url	`str`	default: `http://nomad-lab.eu/prod/v1/api`

gitlab¶

name	type
private_token	`str`	default: `not set`

plugins¶

name	type
entry_points	`EntryPoints`	Used to control plugin entry points.
plugin_packages	`Dict[str, PluginPackage]`	Contains the installed installed plugin packages with the package name used as a key. This is autogenerated and should not be modified.

EntryPoints¶

name	type
include	`List[str]`	List of included options. If not explicitly defined, all of the options will be included by default.
exclude	`List[str]`	List of excluded options. Has higher precedence than include.
options	`Dict[str, Union[Schema, Normalizer, Parser, SchemaPackageEntryPoint, ParserEntryPoint, NormalizerEntryPoint, AppEntryPoint, ExampleUploadEntryPoint, APIEntryPoint]]`	The available plugin entry points. default: Complex object, default value not displayed.

Normalizer¶

A Normalizer describes a NOMAD normalizer that can be loaded as a plugin.

name	type
plugin_type	`str`	The type of the plugin. This has to be the string `normalizer` for normalizer plugins. default: `normalizer`
id	`str`	The unique identifier for this plugin.
name	`str`	A short descriptive human readable name for the plugin.
description	`str`	A human readable description of the plugin.
plugin_documentation_url	`str`	The URL to the plugins main documentation page.
plugin_source_code_url	`str`	The URL of the plugins main source code repository.
python_package	`str`	Name of the python package that contains the plugin code and a plugin metadata file called `nomad_plugin.yaml`.
normalizer_class_name	`str`	The fully qualified name of the Python class that implements the normalizer. This class must have a function `def normalize(self, logger)`.

Parser¶

A Parser describes a NOMAD parser that can be loaded as a plugin.

The parser itself is referenced via python_name. For Parser instances python_name must refer to a Python class that has a parse function. The other properties are used to create a MatchingParserInterface. This comprises general metadata that allows users to understand what the parser is, and metadata used to decide if a given file "matches" the parser.

name	type
plugin_type	`str`	The type of the plugin. This has to be the string `parser` for parser plugins. default: `parser`
id	`str`	The unique identifier for this plugin.
name	`str`	A short descriptive human readable name for the plugin.
description	`str`	A human readable description of the plugin.
plugin_documentation_url	`str`	The URL to the plugins main documentation page.
plugin_source_code_url	`str`	The URL of the plugins main source code repository.
python_package	`str`	Name of the python package that contains the plugin code and a plugin metadata file called `nomad_plugin.yaml`.
parser_class_name	`str`	The fully qualified name of the Python class that implements the parser. This class must have a function `def parse(self, mainfile, archive, logger)`.
parser_as_interface	`int`	By default the parser metadata from this config (and the loaded nomad_plugin.yaml) is used to instantiate a parser interface that is lazy loading the actual parser and performs the mainfile matching. If the parser interface matching based on parser metadata is not sufficient and you implemented your own is_mainfile parser method, this setting can be used to use the given parser class directly for parsing and matching. default: `False`
mainfile_contents_re	`str`	A regular expression that is applied the content of a potential mainfile. If this expression is given, the parser is only considered for a file, if the expression matches.
mainfile_name_re	`str`	A regular expression that is applied the name of a potential mainfile. If this expression is given, the parser is only considered for a file, if the expression matches. default: `.*`
mainfile_mime_re	`str`	A regular expression that is applied the mime type of a potential mainfile. If this expression is given, the parser is only considered for a file, if the expression matches. default: `text/.*`
mainfile_binary_header	`bytes`	Matches a binary file if the given bytes are included in the file.
mainfile_binary_header_re	`bytes`	Matches a binary file if the given binary regular expression bytes matches the file contents.
mainfile_alternative	`int`	If True, the parser only matches a file, if no other file in the same directory matches a parser. default: `False`
mainfile_contents_dict	`dict`	Is used to match structured data files like JSON, HDF5 or csv/excel files. In case of a csv/excel file for example, in order to check if certain columns exist in a given sheet, one can set this attribute to `'__has_all_keys': [<column names>]`. In case the csv/excel file contains comments that are supposed to be ignored, use this reserved key-value pair `'__has_comment': '<symbol>'` at the top level of the dictionary. Also in order to check if a certain sheet name with specific column names exist, one may set this attribute to: {'': {'__has_all_keys': []}}. Available options are: __has_key: str __has_all_keys: List[str] __has_only_keys: List[str] __has_comment: str (only for csv/xlsx files)
supported_compressions	`List[str]`	Files compressed with the given formats (e.g. xz, gz) are uncompressed and matched like normal files. default: `[]`
domain	`str`	The domain value `dft` will apply all normalizers for atomistic codes. Deprecated. default: `dft`
level	`int`	The order by which the parser is executed with respect to other parsers. default: `0`
code_name	`str`
code_homepage	`str`
code_category	`str`
metadata	`dict`	Metadata passed to the UI. Deprecated.

ExampleUploadEntryPoint¶

Base model for example upload plugin entry points.

name	type
id	`str`	Unique identifier corresponding to the entry point name. Automatically set to the plugin entry point name in pyproject.toml.
entry_point_type	`str`	Determines the entry point type. default: `example_upload`
name	`str`	Name of the plugin entry point.
description	`str`	Longer description of the example upload.
plugin_package	`str`	The plugin package from which this entry points comes from.
category	`str`	Category for the example upload.
title	`str`	Title of the example upload.
resources	`Union[List[Union[UploadResource, str]], UploadResource, str, NoneType]`	List of data resources for this example upload.
path	`str`	deprecated
url	`str`	deprecated
from_examples_directory	`int`	Whether this example upload should be read from the "examples" directory. default: `False`

UploadResource¶

Represents a request to include a certain resource into an example upload. Can point to a local folder/file, or alternatively to an online resource that can be downloaded.

name	type
path	`str`	Path to a file/folder within the python package (filepaths should start from the package root directory) or to an online URL.
target	`str`	File path within the upload where the file should be stored. default: ""

Schema¶

A Schema describes a NOMAD Python schema that can be loaded as a plugin.

name	type
plugin_type	`str`	The type of the plugin. This has to be the string `schema` for schema plugins. default: `schema`
id	`str`	The unique identifier for this plugin.
name	`str`	A short descriptive human readable name for the plugin.
description	`str`	A human readable description of the plugin.
plugin_documentation_url	`str`	The URL to the plugins main documentation page.
plugin_source_code_url	`str`	The URL of the plugins main source code repository.
python_package	`str`	Name of the python package that contains the plugin code and a plugin metadata file called `nomad_plugin.yaml`.
package_path	`str`	Path of the plugin package. Will be determined using python_package if not explicitly defined.
key	`str`	Key used to identify this plugin.

ParserEntryPoint¶

Base model for parser plugin entry points.

name	type
id	`str`	Unique identifier corresponding to the entry point name. Automatically set to the plugin entry point name in pyproject.toml.
entry_point_type	`str`	Determines the entry point type. default: `parser`
name	`str`	Name of the plugin entry point.
description	`str`	A human readable description of the plugin entry point.
plugin_package	`str`	The plugin package from which this entry points comes from.
level	`int`	Integer that determines the execution order of this parser. Parser with lowest level will attempt to match raw files first. default: `0`
aliases	`List[str]`	List of alternative parser names. default: `[]`
mainfile_contents_re	`str`	A regular expression that is applied the content of a potential mainfile. If this expression is given, the parser is only considered for a file, if the expression matches.
mainfile_name_re	`str`	A regular expression that is applied the name of a potential mainfile. If this expression is given, the parser is only considered for a file, if the expression matches. default: `.*`
mainfile_mime_re	`str`	A regular expression that is applied the mime type of a potential mainfile. If this expression is given, the parser is only considered for a file, if the expression matches. default: `.*`
mainfile_binary_header	`bytes`	Matches a binary file if the given bytes are included in the file.
mainfile_binary_header_re	`bytes`	Matches a binary file if the given binary regular expression bytes matches the file contents.
mainfile_alternative	`int`	If True, the parser only matches a file, if no other file in the same directory matches a parser. default: `False`
mainfile_contents_dict	`dict`	Is used to match structured data files like JSON or HDF5.
supported_compressions	`List[str]`	Files compressed with the given formats (e.g. xz, gz) are uncompressed and matched like normal files. default: `[]`

APIEntryPoint¶

Base model for API plugin entry points.

name	type
id	`str`	Unique identifier corresponding to the entry point name. Automatically set to the plugin entry point name in pyproject.toml.
entry_point_type	`str`	Specifies the entry point type. default: `api`
name	`str`	Name of the plugin entry point.
description	`str`	A human readable description of the plugin entry point.
plugin_package	`str`	The plugin package from which this entry points comes from.
prefix	`str`	The prefix for the API. The URL for the API will be the base URL of the NOMAD installation followed by this prefix. The prefix must not collide with any other API prefixes. There is no default, this field must be set.

AppEntryPoint¶

Base model for app plugin entry points.

name	type
id	`str`	Unique identifier corresponding to the entry point name. Automatically set to the plugin entry point name in pyproject.toml.
entry_point_type	`str`	Determines the entry point type. default: `app`
name	`str`	Name of the plugin entry point.
description	`str`	A human readable description of the plugin entry point.
plugin_package	`str`	The plugin package from which this entry points comes from.
app	`App`	The app configuration.

NormalizerEntryPoint¶

Base model for normalizer plugin entry points.

name	type
id	`str`	Unique identifier corresponding to the entry point name. Automatically set to the plugin entry point name in pyproject.toml.
entry_point_type	`str`	Determines the entry point type. default: `normalizer`
name	`str`	Name of the plugin entry point.
description	`str`	A human readable description of the plugin entry point.
plugin_package	`str`	The plugin package from which this entry points comes from.
level	`int`	Integer that determines the execution order of this normalizer. Normalizers are run in order from lowest level to highest level. default: `0`

SchemaPackageEntryPoint¶

Base model for schema package plugin entry points.

name	type
id	`str`	Unique identifier corresponding to the entry point name. Automatically set to the plugin entry point name in pyproject.toml.
entry_point_type	`str`	Specifies the entry point type. default: `schema_package`
name	`str`	Name of the plugin entry point.
description	`str`	A human readable description of the plugin entry point.
plugin_package	`str`	The plugin package from which this entry points comes from.

PluginPackage¶

name	type
name	`str`	Name of the plugin Python package, read from pyproject.toml.
description	`str`	Package description, read from pyproject.toml.
version	`str`	Plugin package version, read from pyproject.toml.
homepage	`str`	Link to the plugin package homepage, read from pyproject.toml.
documentation	`str`	Link to the plugin package documentation page, read from pyproject.toml.
repository	`str`	Link to the plugin package source code repository, read from pyproject.toml.
entry_points	`List[str]`	List of entry point ids contained in this package, read form pyproject.toml