Metric – defining what to optimize from the output datafiles

A nextnano simulation produces physical data – band profiles, potentials, energy spectra, alloy profiles, etc. An optimizer, on the other hand, works with plain numbers: it needs one objective value per design, or a few of them for a multi-objective search. Metric is the mediator between the simulation tools and the optimizer. It contains the definition of the optimization objective: one or more numerical values that are extracted from the simulation results.

Connection to IO

IO runs the simulation and loads the files listed in target_output_paths as nextnanopy.DataFile objects. Metric receives exactly those datafiles and turns them into objective values:

DataFiles from IO
      |
extraction function
      |
objective value(s)
      |
   optimizer

The datafiles arrive in the same order as the target_output_paths given to IO. That order is the only link between the two objects. Therefore, the extraction function relies on precisely that order.

Extraction function

For each optimization problem, you need to define how the simulation results are converted into an objective value. You define this conversion in the extraction_function. It is a normal Python function with a simple contract:

input  -> nextnanopy.DataFile objects
output -> numpy.ndarray of objective values

A simple example of the extraction function reads as follows:

import numpy as np

def extract_target(datafiles):
    datafile = datafiles[0]
    potential = datafile.variables["potential"].value

    return np.array([potential.max() - potential.min()])

In the example above the objective is the potential drop across the device. datafiles[0], datafiles[1], … are the output files defined in target_output_paths, in the order they were listed there. For example, with

target_output_paths = [
    ("bias_00000", "potential.dat"),
    ("Structure", "AlloyConcentration.dat"),
]

the extraction function receives two datafiles: datafiles[0] is potential.dat and datafiles[1] is AlloyConcentration.dat.

Reading a datafile

A nextnanopy.DataFile contains data read from nextnano simulation output files, such as .dat files for 1D simulation output, .fld or .vtr files for 2D and 3D simulation output.

Variables and coordinates

The physical quantities are stored in variables, while the grid on which they are defined is stored in coords.

For example, for a 1D potential profile the stored variable is potential, and the coordinate is x:

potential = datafile.variables["potential"]
x = datafile.coords["x"]

For a 2D potential profile, the stored variable is potential, and the coordinates are x and y:

potential = datafile.variables["potential"]
x = datafile.coords["x"]
y = datafile.coords["y"]

Each coordinate and variable has a name, unit and value:

potential.name # string, e.g. "potential"
potential.unit # string, e.g. "Volt"
potential.value # numpy.ndarray, e.g. array([-0.1, -0.2, -0.3])

Processing the data

For an extraction function, value is usually the important part. It is a regular numpy.ndarray containing the numerical data, so it can be processed directly with NumPy:

objective = np.array([potential.value.max() - potential.value.min()])

Coordinates can be used together with variable values when the computation of the objective requires the numerical grid. For example, a one-dimensional integral can be computed from the variable values and the corresponding grid values:

x = datafile.coords["x"].value
density = datafile.variables["density"].value

integral = np.trapezoid(density, x)

More details can be found on the page Reading simulation output.

Input and output shape

Besides the extraction function, Metric is given two additional parameters: input_shape, how many datafiles the extraction function receives, and output_shape, how many objective values it returns. Both are checked on every evaluation:

from nextnano_optimizers.metric import Metric

metric = Metric(
    input_shape=(2,), # two datafiles per evaluation
    output_shape=(1,), # one objective value per evaluation
    extraction_function=extract_target,
)

Note

input_length and output_length are accepted as shorthand for the one-dimensional case: Metric(input_length=1, output_length=1, ...) is the same as Metric(input_shape=(1,), output_shape=(1,), ...).

Input shape

input_shape describes how many DataFile objects are passed to the extraction_function and how they are arranged.

In the usual case input_shape=(k,), where k is the number of output files configured in target_output_paths of IO – one datafile per target.

io = IO(
    input_file_path="device.nnp",
    variable_names=["well_width", "barrier_width"],
    target_output_paths=[
        ("bias_00000", "potential.dat"),
        ("bias_00000", "electric_field.dat"),
        ("Structure", "AlloyConcentration.dat"),
    ],
)

def extract_target(datafiles):
    potential = datafiles[0]       # bias_00000/potential.dat
    electric_field = datafiles[1]  # bias_00000/electric_field.dat
    concentration = datafiles[2]   # Structure/AlloyConcentration.dat
    ...

metric = Metric(
    input_shape=(3,),
    output_shape=(1,),
    extraction_function=extract_target,
)

In the example above, 3 entries in target_output_paths mean that the extraction function receives 3 datafiles, so input_shape=(3,). input_shape has to match what IO actually hands over.

Input shape with sweeps

If the IO uses sweep to vary additional input variables for each objective evaluation, the input_shape will have one axis per sweep key, in the order the keys were given: (k, n1, n2, ...), where n1, n2, ... are the numbers of swept values.

For example, with sweep={"temperature": [250, 300, 350], "voltage": [1, 2]}, the extraction function receives k datafiles for every combination of sweep values. The resulting array has a shape input_shape=(k, 3, 2).

io = IO(
    input_file_path="device.nnp",
    variable_names=["well_width", "barrier_width"],
    target_output_paths=[
        ("bias_00000", "potential.dat"),
        ("bias_00000", "electric_field.dat"),
        ("Structure", "AlloyConcentration.dat"),
    ],
    sweep={"temperature": [250, 300, 350], "voltage": [1, 2]},
)

def extract_target(datafiles):
    datafiles.shape     # (3, 3, 2)

    datafiles[0, 0, 0]  # potential.dat           at 250 K, 1 V
    datafiles[0, 2, 1]  # potential.dat           at 350 K, 2 V
    datafiles[2, 1, 0]  # AlloyConcentration.dat  at 300 K, 1 V
    ...

metric = Metric(
    input_shape=(3, 3, 2),
    output_shape=(1,),
    extraction_function=extract_target,
)

Output shape

output_shape is the shape of the objective array the extraction function returns. (1,) for a single-objective optimization; (p,) for a multi-objective optimization with p objectives. The Metric expects the extraction_function to return an array of that shape, and will raise an error if it does not.

def extract_target(datafiles):
    potential = datafiles[0].variables["potential"].value
    density = datafiles[1].variables["density"].value

    potential_drop = potential.max() - potential.min()
    max_density = density.max()

    return np.array([potential_drop, max_density])  # shape (2,)

metric = Metric(
    input_shape=(2,),
    output_shape=(2,),
    extraction_function=extract_target,
)

In the example above, the extraction_function returns two objective values, so output_shape=(2,).

Testing the Metric setup

During an optimization, the optimizer automatically runs the nextnano simulations through IO and evaluates the resulting objective values through Metric. You can perform these steps manually before starting a full optimization run to make sure that the extraction function works correctly with real simulation output:

io.run_simulation([7.0, 3.0])
datafiles = io.load_output_datafiles()

objective = metric.extract(datafiles)
print(objective)

Use metric.extract(datafiles) rather than calling the extraction function directly: it is the same entry point the optimizer uses, and it checks that the datafiles match input_shape and that what comes back matches output_shape. Any mismatch can be detected here instead of the first evaluation in the optimization workflow.

Using input variables in the objective

Sometimes the objective depends not only on the simulation output but also on the input values that produced it. For example, the objective may combine a simulated quantity with the layer width used in the simulation. To achieve this, the Metric needs access to the input values. The nextnano solvers output provides a way to retrieve them.

The simulators write the input variables and their values to an additional output file. This file can be added to target_output_paths and read by the extraction function like any other datafile. For nextnano++, the file is variables_input.txt:

io = IO(
    input_file_path="device.nnp",
    variable_names=["well_width"],
    target_output_paths=[
        ("bias_00000", "density.dat"),
        ("variables_input.txt",),
    ],
)

def extract_target(datafiles):
    density = datafiles[0].variables["density"].value
    x = datafiles[0].coords["x"].value
    well_width = datafiles[1].variables["well_width"].value

    integrated_density = np.trapezoid(density, x)
    return np.array([integrated_density / well_width])

metric = Metric(
    input_shape=(2,),
    output_shape=(1,),
    extraction_function=extract_target,
)

The example above first computes the integrated density from the simulation output, then divides it by the well width read from the input file to compute average density.

Additional information

See Metric for the exact argument types, the validation rules and the attributes of Metric.