reservoirpy.nodes.LocalPlasticityReservoir#

class reservoirpy.nodes.LocalPlasticityReservoir( units: int | None = None, local_rule: ~typing.Literal['oja', 'anti-oja', 'hebbian', 'anti-hebbian', 'bcm'] = 'oja', eta: float = 0.001, bcm_theta: float = 0.0, synapse_normalization: bool = False, epochs: int = 1, sr: float = 1.0, lr: float = 1.0, input_scaling: float | ~typing.Sequence = 1.0, input_connectivity: float = 0.1, rc_connectivity: float = 0.1, Win: ~numpy.ndarray | ~scipy.sparse._base.sparray | ~typing.Callable = _bernoulli(), W: ~numpy.ndarray | ~scipy.sparse._base.sparray | ~typing.Callable = _uniform(), bias: ~numpy.ndarray | ~scipy.sparse._base.sparray | ~typing.Callable = _bernoulli(), activation: str | ~typing.Callable = <function tanh>, input_dim: int | None = None, seed: int | ~numpy.random._generator.Generator | None = None, dtype: type = <class 'numpy.float64'>, name: str | None = None, )[source]#

A reservoir that learns its recurrent weights W through a local learning rule selected by the learning_rule hyperparameter.

Reservoir states are updated with the external equation:

\[\begin{split}& r[t+1] = (1 - lr)*r[t] + lr*(W r[t] + W_{in} u[t+1] + bias) \\ & x[t+1] = f(r[t+1])\end{split}\]

Where \(f\) is the activation function.

Then the local rule is applied each timestep to update W.

\[W_{ij} \leftarrow W_{ij} + \Delta W_{ij}\]

Supported rules:

oja:: \(\Delta W_{ij} = \eta y (x - y W_{ij})\)
anti-oja [1] [2] [3] :: \(\Delta W_{ij} = - \eta y (x - y W_{ij})\)
hebbian [4] :: \(\Delta W_{ij} = \eta x y\)
anti-hebbian:: \(\Delta W_{ij} = - \eta x y\)
bcm [2] :: \(\Delta W_{ij} = \eta x y (y - \theta_{BCM})\)

Where \(x\) represents the pre-synaptic state and \(y\) represents the post-synaptic state of the neuron.

For “bcm”, you can set a threshold bcm_theta (default 0.0).

If synapse_normalization=True, then after each local-rule update on a row i of W, the row is rescaled to unit L2 norm. [4]

Parameters:

units (int, optional) – Number of reservoir units. If None, the number of units will be inferred from the W matrix shape.
local_rule (str, default to oja) – One of “oja”, “anti-oja”, “hebbian”, “anti-hebbian”, “bcm”.
eta (float, default to 1e-3.) – Local learning rate for the weight update.
bcm_theta (float, default to 0.0) – The threshold used in the “bcm” rule.
synapse_normalization (bool, default to True) – If True, L2-normalize each row of W after its update.
epochs (int, default to 1) – Number of training iterations.
sr (float, default to 1.0) – Spectral radius of recurrent weight matrix.
lr (float or array-like of shape (units,), default to 1.0) – Neurons leak rate. Must be in \([0, 1]\).
input_scaling (float or array-like of shape (features,), default to 1.0.) – Input gain. An array of the same dimension as the inputs can be used to set up different input scaling for each feature.
input_connectivity (float, default to 0.1) – Connectivity of input neurons, i.e. ratio of input neurons connected to reservoir neurons. Must be in \(]0, 1]\).
rc_connectivity (float, default to 0.1) – Connectivity of recurrent weight matrix, i.e. ratio of reservoir neurons connected to other reservoir neurons, including themselves. Must be in \(]0, 1]\).
Win (callable or array-like of shape (units, features), default to bernoulli()) – Input weights matrix or initializer. If a callable (like a function) is used, then this function should accept any keywords parameters and at least two parameters that will be used to define the shape of the returned weight matrix.
W (callable or array-like of shape (units, units), default to uniform()) – Recurrent weights matrix or initializer. If a callable (like a function) is used, then this function should accept any keywords parameters and at least two parameters that will be used to define the shape of the returned weight matrix.
bias (callable or array-like of shape (units, 1), default to bernoulli()) – Bias weights vector or initializer. If a callable (like a function) is used, then this function should accept any keywords parameters and at least two parameters that will be used to define the shape of the returned weight matrix.
activation ({"tanh", "sigmoid"}, default to "tanh") – Reservoir units activation function.
input_dim (int, optional) – Input dimension. Can be inferred at first call.
seed (int or numpy.random.Generator, optional) – A random state seed, for noise generation.
dtype (Numpy dtype, default to np.float64) – Numerical type for node parameters.
name (str, optional) – Node name.

References

Example

>>> reservoir = LocalPlasticityReservoir(
...     units=100, sr=0.9, local_rule="hebbian",
...     eta=1e-3, epochs=5, synapse_normalization=True
... )
>>> # Fit on data timeseries
>>> reservoir.fit(X_data, warmup=10)
>>> # Then run
>>> states = reservoir.run(X_data)

Methods

`__init__`([units, local_rule, eta, ...])
`fit`(x[, y, warmup])	Offline fitting method of a Node.
`initialize`(x)	Define input and output dimensions, and instantiate variables.
`predict`([x, iters, workers])	Alias for `run()`
`reset`()	Reset all Node state
`run`([x, iters, workers])	Run the Node on a sequence of data.
`step`([x])	Call the Node function on a single step of data and update the state of the Node.

Attributes

`input_dim`	Expected dimension of the Node input.
`name`	Optional name of the Node.
`output_dim`	Expected dimension of the Node input.
`units`	Number of neuronal units in the reservoir.
`eta`	Local learning rate for the weight update.
`bcm_theta`	The threshold used in the "bcm" rule.
`synapse_normalization`	If True, L2-normalize each row of W after its update.
`epochs`	Number of training iterations.
`lr`	Leaking rate (1.0 by default) (\(\mathrm{lr}\)).
`sr`	Spectral radius of `W` (optional).
`input_scaling`	Input scaling (float or array) (1.0 by default).
`input_connectivity`	Connectivity (or density) of `Win` (0.1 by default).
`rc_connectivity`	Connectivity (or density) of `Wfb` (0.1 by default).
`Win`	Input weights matrix (\(\mathbf{W}_{in}\)).
`W`	Recurrent weights matrix (\(\mathbf{W}\)).
`bias`	Bias vector (\(\mathbf{b}\)).
`activation`	Activation of the reservoir units (tanh by default) (\(f\)).
`dtype`	Type of matrices elements.
`rng`	A random state generator.
`initialized`	True if the Node has been initialized
`state`	Current state of the Node.

W: ndarray | sparray#: Recurrent weights matrix (\(\mathbf{W}\)).

Win: ndarray | sparray#: Input weights matrix (\(\mathbf{W}_{in}\)).

activation: Callable#: Activation of the reservoir units (tanh by default) (\(f\)).

bcm_theta: float#: The threshold used in the “bcm” rule.

bias: ndarray | sparray#: Bias vector (\(\mathbf{b}\)).

dtype: type#: Type of matrices elements. By default, np.float64.

epochs: int#: Number of training iterations.

eta: float#: Local learning rate for the weight update.

fit( x: array(t, d) | array(s, t, d) | ~typing.Sequence[array(t, d)], y: None = None, warmup: int = 0, ) → LocalPlasticityReservoir[source]#

Offline fitting method of a Node.

Parameters:

x (list or array-like of shape ([series, ] timesteps, input_dim), optional) – Input sequences dataset.
y (list or array-like of shape ([series], timesteps, output_dim), optional) – Teacher signals dataset. If None, the method will try to fit the Node in an unsupervised way, if possible.
warmup (int, default to 0) – Number of timesteps to consider as warmup and discard at the beginning of each timeseries before training.

Returns:

Node trained offline.

Return type:

Node

initialize( x: array(t, d) | array(s, t, d) | ~typing.Sequence[array(t, d)] | array(d) | None, )[source]#

Define input and output dimensions, and instantiate variables.

Only called once, before fitting or running the node.

Parameters:

x (array of shape (input_dim,) or (timestep, input_dim)) – Input data to the node.
y (None) – Training data to the node. As it is not a trainable node, y is expected to be None.

initialized: bool#: True if the Node has been initialized

input_connectivity: float#: Connectivity (or density) of Win (0.1 by default).

input_dim: int = None#: Expected dimension of the Node input. Can be None before initialization

input_scaling: float | Sequence#: Input scaling (float or array) (1.0 by default).

lr: float#: Leaking rate (1.0 by default) (\(\mathrm{lr}\)).

name: str | None = None#: Optional name of the Node.

output_dim: int = None#: Expected dimension of the Node input. Can be None before initialization

Alias for run()

Run the Node on a sequence of data. Can update the state of the Node several times.

Parameters:

x (array-like of shape ([n_inputs,] timesteps, input_dim) or list of) – arrays of shape (timesteps, input_dim), optional A sequence of data of shape (timesteps, features).
iters (int, optional) – If x is None, a dimensionless timeseries of length iters is used instead.

Returns:

A sequence of output vectors.

Return type:

array of shape ([n_inputs,] timesteps, output_dim) or list of arrays

rc_connectivity: float#: Connectivity (or density) of Wfb (0.1 by default).

reset() → dict[str, ndarray][source]#

Reset all Node state

Returns:: dict[str, np.array]
Return type:: previous state of the Node.

rng: Generator#: A random state generator. Used for generating Win and W.

Run the Node on a sequence of data. Can update the state of the Node several times.

Parameters:

x (array-like of shape ([n_inputs,] timesteps, input_dim) or list of arrays of shape (timesteps, input_dim), optional) – A timeseries, array of shape (timesteps, features), or a sequence of timeseries. Input of the Node.
iters (int, optional) – If x is None, a dimensionless timeseries of length iters is used instead.
workers (int, default to 1) – Number of workers used for parallelization. If set to -1, all available workers (threads or processes) are used.

Returns:

A sequence of output vectors.

Return type:

array of shape ([n_inputs,] timesteps, output_dim) or list of arrays

sr: float#: Spectral radius of W (optional).

state: dict[str, ndarray]#: Current state of the Node. Must have “out” as one of the keys.

step(x: array(d) | None = None)[source]#

Call the Node function on a single step of data and update the state of the Node.

Parameters:: x (array of shape (input_dim,), optional) – One single step of input data. If None, an empty array is used instead and the Node is assumed to have an input_dim of 0
Returns:: An output vector.
Return type:: array of shape (output_dim,)

synapse_normalization: bool#: If True, L2-normalize each row of W after its update.

units: int#: Number of neuronal units in the reservoir.