Skip to content

Schema annotations

Definitions in a schema can have annotations. These annotations provide additional information that NOMAD can use to alter its behavior around these definitions. Annotations are named blocks of key-value pairs:

definitions:
  sections:
    MyAnnotatedSection:
      m_annotations:
        annotation_name:
          key1: value
          key2: value

Many annotations control the representation of data in the GUI. This can be for plots or data entry/editing capabilities.

ELN annotations

These annotations control how data can be entered and edited. Use the key eln to add this annotations. For example:

class Sample(EntryData):
    sample_id = Quantity(type=str, a_eln=dict(component='StringEditQuantity'))`)

or in YAML schemas:

Sample:
  quantities:
    sample_id:
      type: str
      m_annotations:
        eln:
          component: StringEditQuantity

An eln annotation can be added to section and quantity definitions to different effects. In both cases, it controls how sections and quantities are represented in the GUI with different parameters; see below.

The UI gives an overview about all ELN edit annotations and components here.

name type
component str The form field component that is used to make the annotated quantity editable. If no component is given, the quantity won't be editable. This can be used on quantities only.
The supported values are:
StringEditQuantity: For editing simple short string values.
URLEditQuantity: For editing strings that are validated to be URLs.
EnumEditQuantity: For Editing enum values. Uses a dropdown list with enum values. This component may be used for short enumerates.
RadioEnumEditQuantity: For Editing enum values. Uses radio buttons.
AutocompleteEditQuantity: For editing enum values. Uses an autocomplete form with dropdown list. This component may be used for longer enumerates.
FileEditQuantity: For editing a reference to a file. Will allow to choose a file or upload a file.
BoolEditQuantity: For editing boolean choices.
NumberEditQuantity: For editing numbers with our without unit.
SliderEditQuantity: For editing numbers with a horizontal slider widget.
DateTimeEditQuantity: For editing datetimes.
RichTextEditQuantity: For editing long styled text with a rich text editor.
ReferenceEditQuantity: For editing references to other sections.
UserEditQuantity: For entering user information. Lets you choose a nomad user or enter information manually.
AuthorEditQuantity: For entering author information manually.
options:
- StringEditQuantity
- URLEditQuantity
- EnumEditQuantity
- RadioEnumEditQuantity
- AutocompleteEditQuantity
- FileEditQuantity
- BoolEditQuantity
- NumberEditQuantity
- SliderEditQuantity
- DateTimeEditQuantity
- RichTextEditQuantity
- ReferenceEditQuantity
- UserEditQuantity
- AuthorEditQuantity
label str [Deprecated] ELN label annotation has been deprecated and it is advised to utilize display annotation instead. Custom label for the quantity shown on the form field. It is recommended to adhere to the convention of using lowercase letters for the label, except for abbreviations which could be capitalized.
props Dict[str, typing.Any] A dictionary with additional props that are passed to the edit component.
default typing.Any Prefills any set form field component with the given value. This is different from the quantities default property. The quantities default is not stored in the data; the default value is assumed if no other value is given. The ELN form field default value will be stored, even if not changed.
defaultDisplayUnit str This attribute is deprecated, use the unit attribute of display annotation instead. Allows to define a default unit to initialize a NumberEditQuantity with. The unit has to be compatible with the unit of the annotation quantity and the annotated quantity must have a unit. Only applies to quantities and with component=NumberEditQuantity.
deprecated
minValue Union[int, float] Allows to specify a minimum value for quantity annotations with number type. Will show an error, if outside numbers are entered. Only works on quantities and in conjunction with component=NumberEditQuantity.
maxValue Union[int, float] Allows to specify a maximum value for quantity annotations with number type. Will show an error, if outside numbers are entered. Only works on quantities and in conjunction with component=NumberEditQuantity.
showSectionLabel int To customize the ReferenceEditQuantity behaviour. If true the section label will be shown instead of referenced file name and the path to the section.
hide List[str] This attribute is deprecated. Use visible attribute of display annotation instead. Allows you to hide certain quantities from a section editor. Give a list of quantity names. Quantities must exist in the section that this annotation is added to. Can only be used in section annotations.
deprecated
overview int Shows the annotation section on the entry's overview page. Can only be used on section annotations.
lane_width Union[str, int] Value to overwrite the css width of the lane used to render the annotation section and its editor.
properties SectionProperties The value to customize the quantities and sub sections of the annotation section. The supported keys: visible: To determine the visible quantities and sub sections by their names
editable: To render things visible but not editable, e.g. in inheritance situations
order: # To order things, properties listed in that order first, then the rest

SectionProperties

The display settings for quantities and subsections. (Deprecated)

name type
visible Filter Defines the visible quantities and subsections. (Deprecated)
default: 1
editable Filter Defines the editable quantities and subsections. (Deprecated)
order List[str] To customize the order of the quantities and subsections. (Deprecated)

Filter

A filter defined by an include list or and exclude list of the quantities or subsections.

name type
include List[str] The list of quantity or subsection names to be included.
exclude List[str] The list of quantity or subsection names to be excluded.

Browser

The browser annotation allows to specify if the processed data browser needs to display a quantity differently. It can be applied to quantities. For example

    class Experiment(EntryData):
        description = Quantity(type=str, a_browser=dict(render_value='HtmlValue'))

or in yaml

Experiment:
  quantities:
    description:
      type: str
      m_annotations:
        browser:
          render_value: HtmlValue
name type
adaptor str Allows to change the Adaptor implementation that is used to render the lane for this quantity. Possible values are:
RawFileAdaptor: An adopter that is used to show files, including all file actions, like file preview.
options:
- RawFileAdaptor
render_value str Allows to change the Component used to render the value of the quantity. Possible values are:
HtmlValue: Renders a string as HTML.
JsonValue: Renders a dict or list in a collapsable tree.
options:
- JsonValue
- HtmlValue

Display annotations

Display annotation for quantities

This annotations control how quantities are displayed in the GUI. Use the key display to add this annotation. For example in Python:

class Example(EntryData):
    sample_id = Quantity(type=str, a_display={'visible': False})

or in YAML:

definitions:
  Example:
    quantities:
      sample_id:
        type: str
        m_annotations:
          display:
            visible: false

name type
visible Filter Defines the visible quantities and subsections.
default: 1
editable Filter Defines the editable quantities and subsections.
display_unit str To determine the default display unit for quantity.

Display annotation for sections

This annotations control how sections are displayed in the GUI. Use the key display to add this annotation. For example in Python:

class Example(MSection):
    m_def = Section(a_display={
        'visible': False
    })

or in YAML:

definitions:
  sections:
    Example:
      m_annotations:
        display:
          visible: false

name type
visible Filter Defines the visible quantities and subsections.
default: 1
editable Filter Defines the editable quantities and subsections.
order List[str] To customize the order of the quantities and subsections.

label_quantity

This annotation goes in the section that we want to be filled with tabular data, not in the single quantities. It is used to give a name to the instances that might be created by the parser. If it is not provided, the name of the section itself will be used as name. Many times it is useful because, i. e., one might want to create a bundle of instances of, say, a "Substrate" class, each instance filename not being "Substrate_1", "Substrate_2", etc., but being named after a quantity contained in the class that is, for example, the specific ID of that sample.

MySection:
  more:
    label_quantity: my_quantity
  quantities:
    my_quantity:
      type: np.float64
      shape: ['*']
      description: "my quantity to be filled from the tabular data file"
      unit: K
      m_annotations:
        tabular:
          name: "Sheet1/my header"
        plot:
          x: timestamp
          y: ./my_quantity

Important

The quantity designated as label_quantity should not be an array but a integer, float or string, to be set as the name of a file. If an array quantity is chosen, the parser would fall back to the use of the section as name.

Tabular data

tabular

Allows to map a quantity to a row or a column of a spreadsheet data-file. Should only be used in conjunction with tabular_parser.

name type
name str The column name that should be mapped to the annotation quantity. Has to be the same string that is used in the header, i.e. first .csv line or first excel file row. For excel files with multiple sheets, the name can have the form <sheet name>/<column name>. Otherwise, only the first sheets is used. Has to be applied to the quantity that a column should be mapped to.
unit str The unit of the value in the file. Has to be compatible with the annotated quantity's unit. Will be used to automatically convert the value. If this is not defined, the values will not be converted. Has to be applied to the quantity that a column should be mapped to.

Each and every quantity to be filled with data from tabular data files should be annotated as the following example. A practical example is provided in How To section.

my_quantity:
  type: np.float64
  shape: ['*']
  description: "my quantity to be filled from the tabular data file"
  unit: K
  m_annotations:
    tabular:
      name: "Sheet1/my header"
    plot:
      x: timestamp
      y: ./my_quantity

tabular_parser

One special quantity will be dedicated to host the tabular data file. In the following examples it is called data_file, it contains the tabular_parser annotation, as shown below.

Instructs NOMAD to treat a string valued scalar quantity as a file path and interprets the contents of this file as tabular data. Supports both .csv and Excel files.

name type
parsing_options TabularParsingOptions Options on how to extract the data from csv/xlsx file. Under the hood, NOMAD uses pandas Dataframe to parse the data from tabular files. These are the available options that can be passed down to the parser.
The supported values are:
skiprows: Number of rows to be skipped while reading the file.
sep: The character used to separate cells (specific to csv files).
comment: The character denoting the commented lines.
separator: An alias for sep.

default: Complex object, default value not displayed.
mapping_options List[TabularMappingOptions] A list of directives on how to map the extracted data from the csv/xlsx file to NOMAD. Each directive is a distinct directive, which allows for more modular definition of your tabular parser schema. If no item is provided, the entire schema is treated to be parsed under column mode.
The supported values in each item of this list are:
mapping_mode: A list of paths to the repeating sub-sections where the tabular quantities are to be filled from individual rows of the excel/csv file (i.e. in the row mode). Each path is a / separated list of nested sub-sections. The targeted sub-sections, will be considered when mapping table rows to quantities. Has to be used to annotate the quantity that holds the path to the .csv or excel file.
file_mode: The character used to separate cells (specific to csv files).
sections: The character denoting the commented lines.

default: []

TabularParsingOptions

name type
skiprows Union[List[int], int] Number of rows to skip
sep str Character identifier of a separator
comment str Character identifier of a commented line
separator str Alias for sep

TabularMappingOptions

name type
mapping_mode str This controls the behaviour of mapping of the extracted data onto NOMAD schema.
The supported values are:
row: A list of paths to the repeating sub-sections where the tabular quantities are to be filled from individual rows of the excel/csv file (i.e. in the row mode). Each path is a / separated list of nested sub-sections. The targeted sub-sections, will be considered when mapping table rows to quantities. Has to be used to annotate the quantity that holds the path to the .csv or excel file.
column: A list of paths to the sub-sections where the tabular quantities are to be filled from the entire column of the excel/csv file (i.e. in the column mode). Each path is a / separated list of nested sub-sections. The targeted sub-sections, will be considered when mapping table columns to quantities. Has to be used to annotate the quantity that holds the path to the .csv or excel file.
enrty: A list of paths to the (sub)sections where the tabular quantities are to be filled from individual rows of the excel/csv file, to create distinct entries. Each path is a / separated list of nested sub-sections. The targeted (sub)sections, will be considered when mapping table rows to quantities. The schema of the resultant entry follows the (sub)section's schema. In order to parse the entire schema using entry mode, then set the first item in this list to root. Has to be used to annotate the quantity that holds the path to the .csv or excel file.

default: column
options:
- row
- column
file_mode str This controls the behaviour of the parser towards working physical files in file system.
The supported values are:
current_entry: Processing the data into the same NOMAD entry.
single_new_entry: Creating a new entry and processing the data into this new NOMAD entry.
multiple_new_entries: Creating many new entries and processing the data into these new NOMAD entries.

options:
- current_entry
- single_new_entry
- multiple_new_entries
sections List[str] A list of paths to the (sub)sections where the tabular quantities are to be filled from the data extracted from the tabular file.

Available Combinations

Tutorial ref. file_mode mapping_mode sections How to ref.
1 current_entry column root HowTo
2 current_entry column my path HowTo
np1 current_entry row root Not possible
3 current_entry row my path HowTo
np2 single_new_entry column root Not possible
4 single_new_entry column my path HowTo
np3 single_new_entry row root Not possible
5 single_new_entry row my path HowTo
np4 multiple_new_entries column root Not possible
np5 multiple_new_entries column my path Not possible
6 multiple_new_entries row root HowTo
7 multiple_new_entries row my path HowTo
data_file:
  type: str
  description: "the tabular data file containing data"
  m_annotations:
    tabular_parser:
      parsing_options:
        comment: '#'
      mapping_options:
      - mapping_mode: column
        file_mode: single_new_entry
        sections:
        - my_section/my_quantity

Plot

The PlotSection base section serves as an additional functionality to your sections. This base section is designed to simplify the process of creating various types of plots, making it easy to use Plotly Express, Plotly Subplot, and the general Plotly graph objects.

Features:

  • Plotly Express: Create simple and quick plots with a high-level, expressive API.
  • Plotly Subplot: Organize multiple plots into subplots for more complex visualizations.
  • General Plotly Graph Objects: Fine-tune your plots by working directly with Plotly's graph objects.

Usage:

  • Inherit from this base section to leverage its plot functionality.
  • Customize your plots using the annotations plotly-express, plotly-subplots, or/and plotly-graph-object.

The PlotSection class makes it possible to define plots that are shown alongside your data. Underneath, we use the Plotly Open Source Graphing Libraries to control the creation of the plots, and you can find many useful examples in their documentation.

In Python schemas, the PlotSection class gives you full freedom to define plots programmatically. For example, you could use plotly.express and plotly.graph_objs to define plots like this:

from nomad.datamodel.metainfo.plot import PlotSection, PlotlyFigure
from nomad.datamodel.data import EntryData
import plotly.express as px
import plotly.graph_objs as go
from plotly.subplots import make_subplots

class CustomSection(PlotSection, EntryData):
    m_def = Section()
    time = Quantity(type=float, shape=['*'], unit='s', a_eln=dict(component='NumberEditQuantity'))
    substrate_temperature = Quantity(type=float, shape=['*'], unit='K', a_eln=dict(component='NumberEditQuantity'))
    chamber_pressure = Quantity(type=float, shape=['*'], unit='Pa', a_eln=dict(component='NumberEditQuantity'))

    def normalize(self, archive, logger):
        super(CustomSection, self).normalize(archive, logger)

        first_line = px.scatter(x=self.time, y=self.substrate_temperature)
        second_line = px.scatter(x=self.time, y=self.chamber_pressure)
        figure1 = make_subplots(rows=1, cols=2, shared_yaxes=True)
        figure1.add_trace(first_line.data[0], row=1, col=1)
        figure1.add_trace(second_line.data[0], row=1, col=2)
        figure1.update_layout(height=400, width=716, title_text="Creating Subplots in Plotly")
        self.figures.append(PlotlyFigure(label='figure 1', figure=figure1.to_plotly_json()))

        figure2 = px.scatter(x=self.substrate_temperature, y=self.chamber_pressure, color=self.chamber_pressure, title="Chamber as a function of Temperature")
        self.figures.append(PlotlyFigure(label='figure 2', index=1, figure=figure2.to_plotly_json()))

        heatmap_data = [[None, None, None, 12, 13, 14, 15, 16],
             [None, 1, None, 11, None, None, None, 17],
             [None, 2, 6, 7, None, None, None, 18],
             [None, 3, None, 8, None, None, None, 19],
             [5, 4, 10, 9, None, None, None, 20],
             [None, None, None, 27, None, None, None, 21],
             [None, None, None, 26, 25, 24, 23, 22]]

        heatmap = go.Heatmap(z=heatmap_data, showscale=False, connectgaps=True, zsmooth='best')
        figure3 = go.Figure(data=heatmap)
        figure_json = figure3.to_plotly_json()
        figure_json['config'] = {'staticPlot': True}
        self.figures.append(PlotlyFigure(label='figure 3', index=0, figure=figure_json)

To customize the plot configuration in python one can add the config to the generated json by to_plotly_json().

figure_json['config'] = {'staticPlot': True}

In YAML schemas, plots can be defined by using the PlotSection as a base class, and additionally utilizing different flavours of plot annotations. The different annotation options are described below.

PlotlyGraphObjectAnnotation

Allows to plot figures using plotly graph object.

    Example:
      base_sections:
        - 'nomad.datamodel.metainfo.plot.PlotSection'
      m_annotations:
        plotly_graph_object:
        - data:
            x: '#xArr'
            y: '#xArr'
          layout:
            title:
              text: 'Plotly Graph Object'
          label: 'Plotly Graph Object'
          index: 1
name type
label str Figure label
data Dict Plotly data
layout Dict Plotly layout
config Dict Plotly config

PlotlyExpressAnnotation

Allows to plot multi trace figures using plotly Express.

  sections:
    Example:
      base_sections:
        - 'nomad.datamodel.metainfo.plot.PlotSection'
      m_annotations:
        plotly_express:
          method: scatter
          x: '#xArr'
          y: '#yArr'
          label: 'Example Express Plot'
          index: 0
          layout:
            title:
              text: 'Example Express Plot'
            xaxis:
              title:
                text: 'x axis'
            yaxis:
              title:
                text: 'y axis'
          traces:
            - method: scatter
              x: '#xArr'
              y: '#zArr'
name type
method str Plotly express plot method
layout Dict Plotly layout
x Union[List[float], List[str], str] Plotly express x
y Union[List[float], List[str], str] Plotly express y
z Union[List[float], List[str], str] Plotly express z
color Union[List[float], List[str], str] Plotly express color
symbol str Plotly express symbol
title str Plotly express title
label str Figure label
traces List[PlotlyExpressTraceAnnotation] List of traces added to the main trace defined by plotly_express method
default: []

PlotlyExpressTraceAnnotation

Allows to plot figures using plotly Express.

name type
method str Plotly express plot method
layout Dict Plotly layout
x Union[List[float], List[str], str] Plotly express x
y Union[List[float], List[str], str] Plotly express y
z Union[List[float], List[str], str] Plotly express z
color Union[List[float], List[str], str] Plotly express color
symbol str Plotly express symbol
title str Plotly express title

PlotlySubplotsAnnotation

Allows to plot figures in subplots.

    Example:
      base_sections:
        - 'nomad.datamodel.metainfo.plot.PlotSection'
      m_annotations:
        plotly_subplots:
          parameters:
            rows: 2
            cols: 2
          layout:
            title:
              text: 'All plots'
          plotly_express:
            - method: scatter
              x: '#xArr'
              y: '#yArr'
              title: 'subplot 1'
            - method: scatter
              x: '#xArr'
              y: '#zArr'
              title: 'subplot 2'
            - method: scatter
              x: '#zArr'
              y: '#xArr'
              title: 'subplot 3'
            - method: scatter
              x: '#zArr'
              y: '#yArr'
              title: 'subplot 4'
name type
label str Figure label
layout Dict Plotly layout
parameters Dict plotly.subplots.make_subplots parameters i.e. rows, cols, shared_xaxes, shared_xaxes, horizontal_spacing , ... See plotly make_subplots documentation for more information.
plotly_express List[PlotlyExpressAnnotation] List of subplots defined by plotly_express method
default: []

plot annotations in python

For simple plots in Python schema one could use the annotations without normalizer:

from nomad.datamodel.metainfo.plot import PlotSection
from nomad.metainfo import Quantity, Section
from nomad.datamodel.data import EntryData

class CustomSection(PlotSection, EntryData):
    m_def = Section(
        a_plotly_graph_object=[
            {
                'label': 'graph object 1',
                'data': {'x': '#time', 'y': '#chamber_pressure'},
                'layout': {
                    'title': {
                        'text': 'Plot in section level'
                    },
                    'xaxis': {
                        'title': {
                            'text': 'x data'
                        }
                    },
                    'yaxis': {
                        'title': {
                            'text': 'y data'
                        }
                    }
                }
            }, {
                'label': 'graph object 2',
                'data': {'x': '#time', 'y': '#substrate_temperature'}
            }
        ],
        a_plotly_express={
            'label': 'fig 2',
            'index': 2,
            'method': 'scatter',
            'x': '#substrate_temperature',
            'y': '#chamber_pressure',
            'color': '#chamber_pressure'
        },
        a_plotly_subplots={
            'label': 'fig 1',
            'index': 1,
            'parameters': {'rows': 2, 'cols': 2},
            'layout': {
                'title': {
                    'text': 'All plots'
                }
            },
            'plotly_express': [
                {
                    'method': 'scatter',
                    'x': '#time',
                    'y': '#chamber_pressure',
                    'color': '#chamber_pressure'
                },
                {
                    'method': 'scatter',
                    'x': '#time',
                    'y': '#substrate_temperature',
                    'color': '#substrate_temperature'
                },
                {
                    'method': 'scatter',
                    'x': '#substrate_temperature',
                    'y': '#chamber_pressure',
                    'color': '#chamber_pressure'
                },
                {
                    'method': 'scatter',
                    'x': '#substrate_temperature',
                    'y': '#chamber_pressure',
                    'color': '#substrate_temperature'
                }
            ]
        }
    )
    time = Quantity(type=float, shape=['*'], unit='s', a_eln=dict(component='NumberEditQuantity'))
    substrate_temperature = Quantity(type=float, shape=['*'], unit='K', a_eln=dict(component='NumberEditQuantity'))
    chamber_pressure = Quantity(type=float, shape=['*'], unit='Pa', a_eln=dict(component='NumberEditQuantity'))

PlotAnnotation (Deprecated)

The PlotAnnotation is now deprecated and will be removed in future releases. We recommend transitioning to the use of PlotSection and PlotlyGraphObjectAnnotation for your plotting needs.

This annotation can be used to add a plot to a section or quantity. Example:

class Evaporation(MSection):
    m_def = Section(a_plot={
        'label': 'Temperature and Pressure',
        'x': 'process_time',
        'y': ['./substrate_temperature', './chamber_pressure'],
        'config': {
            'editable': True,
            'scrollZoom': False
        }
    })
    time = Quantity(type=float, shape=['*'], unit='s')
    substrate_temperature = Quantity(type=float, shape=['*'], unit='K')
    chamber_pressure = Quantity(type=float, shape=['*'], unit='Pa')

You can create multi-line plots by using lists of the properties y (and x). You either have multiple sets of y-values over a single set of x-values. Or you have pairs of x and y values. For this purpose the annotation properties x and y can reference a single quantity or a list of quantities. For repeating sub sections, the section instance can be selected with an index, e.g. "sub_section_name/2/parameter_name" or with a slice notation start:stop where negative values index from the end of the array, e.g. "sub_section_name/1:-5/parameter_name".

The interactive examples of the plot annotations can be found here.

name type
label str Is passed to plotly to define the label of the plot.
x Union[List[str], str] A path or list of paths to the x-axes values. Each path is a / separated list of sub-section and quantity names that leads from the annotation section to the quantity. Repeating sub sections are indexed between two /s with an integer or a slice start:stop.
y Union[List[str], str] A path or list of paths to the y-axes values. list of sub-section and quantity names that leads from the annotation section to the quantity. Repeating sub sections are indexed between two /s with an integer or a slice start:stop.
lines List[dict] A list of dicts passed as traces to plotly to configure the lines of the plot. See https://plotly.com/javascript/reference/scatter/ for details.
layout dict A dict passed as layout to plotly to configure the plot layout. See https://plotly.com/javascript/reference/layout/ for details.
config dict A dict passed as config to plotly to configure the plot functionality. See https://plotly.com/javascript/configuration-options/ for details.