Advanced Features

Note

PINNICLE is under active development, and several advanced/experimental features are listed here:

Mini-batch

PINNICLE supports mini-batch training when using the pytorch backend. In the current implementation, mini-batches are resampled at every epoch to enhance convergence stability. To activate mini-batch training, simply add the following setting to the hyperparameter dictionary. PINNICLE will automatically generate mini-batches from the data specified in hp["data"]. And in the data section, you can set the "data_size" to a large number or simply "MAX", to cover a large amount of your data.

hp["data"] = {"ISSM": {"data_size": {"u":100, "v":"MAX", "s":100, "H":10, "C":None}}}
hp["mini_batch"] = mini_batch

Avoid setting mini_batch too large, as this can lead to inefficiencies and excessive memory usage.
Avoid setting mini_batch too small, as this can result in wasted computational resources due to the increased overhead of frequent resampling.

Based on our numerical experiments, we recommend choosing a batch size in the range of 500 to 10,000.

Resample collocation points

PINNICLE supports the resampling of collocation points during training. To activate this feature, set the following in your hyperparameter dictionary:

hp["period"] = 100

This will instruct PINNICLE to resample the collocation points every 100 epochs.

Training with a list of optimizers

PINNICLE now supports training with a list of optimizers, allowing users to combine the strengths of different optimization strategies within a single training workflow. Users can define an ordered sequence of optimizers, each with its own training configuration, such number of epochs.

hp["optimizers"] = ["adam", "L-BFGS"]
hp["epochs"] = [100000, 20000]

Note

This feature is not supported by JAX due to the backend restriciton.

Load Settings from Previous Experiment

Each PINNICLE experiment will save the entire settings to the folder defined in hp["save_path"] if hp["is_save"]=True. The settings are saved into a params.json file containing all the non-default settings from hp.

To repeat a new experiment with the same settings as in some folder_path, you can easily load the settings by:

experiment = pinnicle.PINN(loadFrom=folder_path)

However, if you want to run the experiment and save the results in a new folder, you will need to set the "save_path" and update the experiment:

experiment.update_parameters({"save_path": new_folder})

Then, compile and train the model.

Load Weights from Previously Trained Neural Network

PINNICLE can also load the weights from a previous experiment. You can either load the settings from a saved experiment or set up a completely new one. After that, you can load the weights of the neural network from the folder_path which contains the entire experiment (including params.json, a folder called pinn, and some figures).

experiment.load_model(path=folder_path)

After loading the weights, compiling and training are the same as for other experiments.

Dummy Equations

PINNICLE provides pinnicle.physics.dummy physics, which allows you to train the neural network using data only. Here is an example:

dummy = {}
dummy["output"] = ['u', 's', 'C']
hp["equations"] = {"DUMMY": dummy}

In this example, we define a dict with a key output, where the value is a list of three output variables. Then, we add this dict to hp['equations'] with the key DUMMY (all uppercase). Additionally, you need to provide the data for u, s, and C in the data section, similar to other examples. The neural network will then be trained solely with the provided data.

By default, the Dummy physics already has x and y as input. If there is no need to change this, only the output needs to be defined.