reservoirpy.nodes.LMS#

Single layer of neurons learning connections using Least Mean Squares algorithm.

The learning rules is well described in [1].

Parameters:

learning_rate (float or Python generator or iterable, default to 1e-6) – Learning rate. If an iterable or a generator is provided, the learning rate can be changed at each timestep of training. A new learning rate will be drawn from the iterable or generator at each timestep.
Wout (callable or array-like of shape (units, targets), default to zeros()) – Output weights matrix or initializer. If a callable (like a function) is used, then this function should accept any keywords parameters and at least two parameters that will be used to define the shape of the returned weight matrix.
bias (callable or array-like of shape (units, 1), default to zeros()) – Bias weights vector or initializer. If a callable (like a function) is used, then this function should accept any keywords parameters and at least two parameters that will be used to define the shape of the returned weight matrix.
fit_bias (bool, default to True) – If True, then a bias parameter will be learned along with output weights.
input_dim (int, optional) – Number of input dimensions in the readout, can be inferred at first call.
output_dim (int, optional) – Number of units in the readout, can be inferred at first call.
name (str, optional) – Node name.

Examples

>>> x = np.random.normal(size=(100, 3))
>>> noise = np.random.normal(scale=0.01, size=(100, 1))
>>> y = x @ np.array([[10], [-0.2], [7.]]) + noise + 12.

>>> from reservoirpy.nodes import LMS
>>> lms_node = LMS(alpha=1e-1)

>>> lms_node.train(x[:50], y[:50])
>>> print(lms_node.Wout.T, lms_node.bias)
[[ 9.156 -0.967   6.411]] [[11.564]]
>>> lms_node.train(x[50:], y[50:])
>>> print(lms_node.Wout.T, lms_node.bias)
[[ 9.998 -0.202  7.001]] [[12.005]]

References

Methods

`__init__`([learning_rate, Wout, bias, ...])
`fit`(x[, y, warmup])	Offline fitting method of a Node.
`initialize`(x[, y])	Define input and output dimensions, and instantiate variables.
`partial_fit`(x, y)	Fit the Node in an online fashion.
`predict`([x, iters, workers])	Alias for `run()`
`reset`()	Reset all Node state
`run`([x, iters, workers])	Run the Node on a sequence of data.
`step`([x])	Call the Node function on a single step of data and update the state of the Node.

Attributes

`input_dim`	Expected dimension of the Node input.
`name`	Optional name of the Node.
`output_dim`	Expected dimension of the Node input.
`Wout`	Learned output weights (\(\mathbf{W}_{out}\)).
`bias`	Learned bias (\(\mathbf{b}\)).
`learning_rate`	Learning rate (\(\alpha\)) (\(1\cdot 10^{-6}\) by default).
`fit_bias`	If True, learn a bias term (True by default).
`initialized`	True if the Node has been initialized
`state`	Current state of the Node.

Wout: ndarray | sparray#: Learned output weights (\(\mathbf{W}_{out}\)).

bias: ndarray | sparray#: Learned bias (\(\mathbf{b}\)).

Offline fitting method of a Node.

Parameters:

x (list or array-like of shape ([series, ] timesteps, input_dim), optional) – Input sequences dataset.
y (list or array-like of shape ([series], timesteps, output_dim), optional) – Teacher signals dataset. If None, the method will try to fit the Node in an unsupervised way, if possible.
warmup (int, default to 0) – Number of timesteps to consider as warmup and discard at the beginning of each timeseries before training.

Returns:

Node trained offline.

Return type:

Node

fit_bias: bool#: If True, learn a bias term (True by default).

Define input and output dimensions, and instantiate variables.

Only called once, before fitting or running the node.

Parameters:

x (array of shape (input_dim,) or (timestep, input_dim)) – Input data to the node.
y (None) – Training data to the node. As it is not a trainable node, y is expected to be None.

initialized: bool#: True if the Node has been initialized

input_dim: int = None#: Expected dimension of the Node input. Can be None before initialization

learning_rate: float#: Learning rate (\(\alpha\)) (\(1\cdot 10^{-6}\) by default).

name: str | None = None#: Optional name of the Node.

output_dim: int = None#: Expected dimension of the Node input. Can be None before initialization

partial_fit(x: array(t, d), y: array(t, d))[source]#

Fit the Node in an online fashion.

This method both trains the Node parameters and produce predictions on the run. Calling partial_fit() updates the Node without resetting the parameters, unlike fit().

Parameters:

x (array-like of shape (timesteps, input_dim)) – Input sequence of data.
y (array-like of shape (timesteps, output_dim), optional.) – Target sequence of data. If None, the Node will train in an unsupervised way, if possible.

Returns:

All outputs computed during the training.

Return type:

array of shape (timesteps, output_dim)

Alias for run()

Run the Node on a sequence of data. Can update the state of the Node several times.

Parameters:

x (array-like of shape ([n_inputs,] timesteps, input_dim) or list of) – arrays of shape (timesteps, input_dim), optional A sequence of data of shape (timesteps, features).
iters (int, optional) – If x is None, a dimensionless timeseries of length iters is used instead.

Returns:

A sequence of output vectors.

Return type:

array of shape ([n_inputs,] timesteps, output_dim) or list of arrays

reset() → dict[str, ndarray][source]#

Reset all Node state

Returns:: dict[str, np.array]
Return type:: previous state of the Node.

Run the Node on a sequence of data. Can update the state of the Node several times.

Parameters:

x (array-like of shape ([n_inputs,] timesteps, input_dim) or list of arrays of shape (timesteps, input_dim), optional) – A timeseries, array of shape (timesteps, features), or a sequence of timeseries. Input of the Node.
iters (int, optional) – If x is None, a dimensionless timeseries of length iters is used instead.
workers (int, default to 1) – Number of workers used for parallelization. If set to -1, all available workers (threads or processes) are used.

Returns:

A sequence of output vectors.

Return type:

array of shape ([n_inputs,] timesteps, output_dim) or list of arrays

state: dict[str, ndarray]#: Current state of the Node. Must have “out” as one of the keys.

step(x: array(d) | None = None)[source]#

Call the Node function on a single step of data and update the state of the Node.

Parameters:: x (array of shape (input_dim,), optional) – One single step of input data. If None, an empty array is used instead and the Node is assumed to have an input_dim of 0
Returns:: An output vector.
Return type:: array of shape (output_dim,)