reservoirpy.nodes.IPReservoir#

class reservoirpy.nodes.IPReservoir( units: int | None = None, sr: float | None = None, lr: float | ~numpy.ndarray = 1.0, mu: float = 0.0, sigma: float = 1.0, learning_rate: float = 0.0005, epochs: int = 1, input_scaling: float | ~typing.Sequence = 1.0, input_connectivity: float = 0.1, rc_connectivity: float = 0.1, Win: ~numpy.ndarray | ~scipy.sparse._base.sparray | ~typing.Callable = _bernoulli(), W: ~numpy.ndarray | ~scipy.sparse._base.sparray | ~typing.Callable = _uniform(), bias: ~numpy.ndarray | ~scipy.sparse._base.sparray | ~typing.Callable = _bernoulli(), activation: ~typing.Literal['tanh', 'sigmoid'] = 'tanh', input_dim: int | None = None, dtype: type = <class 'numpy.float64'>, seed: int | ~numpy.random._generator.Generator | None = None, name=None, )[source]#

Pool of neurons with random recurrent connexions, tuned using Intrinsic Plasticity.

Intrinsic Plasticity is applied as described in [1] and [2].

Reservoir neurons states, gathered in a vector \(\mathbf{x}\), follow the update rule below:

\[\mathbf{r}[t+1] = (1 - \mathrm{lr}) * \mathbf{r}[t] + \mathrm{lr} * (\mathbf{W}_{in} \cdot \mathbf{u}[t+1] + \mathbf{W} \cdot \mathbf{x}[t] + \mathbf{b}_{in})\]

\[\mathbf{x}[t+1] = f(\mathbf{a}*\mathbf{r}[t+1]+\mathbf{b})\]

Parameters \(\mathbf{a}\) and \(\mathbf{b}\) are updated following two different rules:

1. Neuron activation is tanh:

In that case, output distribution should be a Gaussian distribution of parameters (\(\mu\), \(\sigma\)). The learning rule to obtain this output distribution is described in [2].

2. Neuron activation is sigmoid:

In that case, output distribution should be an exponential distribution of parameter \(\mu = \frac{1}{\lambda}\). The learning rule to obtain this output distribution is described in [1] and [2].

where:

\(\mathbf{x}\) is the output activation vector of the reservoir;
\(\mathbf{r}\) is the internal activation vector of the reservoir;
\(\mathbf{u}\) is the input timeseries;
\(f\) and \(g\) are activation functions.

Parameters:

units (int, optional) – Number of reservoir units. If None, the number of units will be inferred from the W matrix shape.
sr (float, optional) – Spectral radius of recurrent weight matrix.
lr (float or array-like of shape (units,), default to 1.0) – Neurons leak rate. Must be in \([0, 1]\).
mu (float, default to 0.0) – Mean of the target distribution.
sigma (float, default to 1.0) – Variance of the target distribution.
learning_rate (float, default to 5e-4) – Learning rate.
epochs (int, default to 1) – Number of training iterations.
input_scaling (float or array-like of shape (features,), default to 1.0.) – Input gain. An array of the same dimension as the inputs can be used to set up different input scaling for each feature.
input_connectivity (float, default to 0.1) – Connectivity of input neurons, i.e. ratio of input neurons connected to reservoir neurons. Must be in \(]0, 1]\).
rc_connectivity (float, default to 0.1) – Connectivity of recurrent weight matrix, i.e. ratio of reservoir neurons connected to other reservoir neurons, including themselves. Must be in \(]0, 1]\).
Win (callable or array-like of shape (units, features), default to bernoulli()) – Input weights matrix or initializer. If a callable (like a function) is used, then this function should accept any keywords parameters and at least two parameters that will be used to define the shape of the returned weight matrix.
W (callable or array-like of shape (units, units), default to uniform()) – Recurrent weights matrix or initializer. If a callable (like a function) is used, then this function should accept any keywords parameters and at least two parameters that will be used to define the shape of the returned weight matrix.
bias (callable or array-like of shape (units, 1), default to bernoulli()) – Bias weights vector or initializer. If a callable (like a function) is used, then this function should accept any keywords parameters and at least two parameters that will be used to define the shape of the returned weight matrix.
activation ({"tanh", "sigmoid"}, default to "tanh") – Reservoir units activation function.
input_dim (int, optional) – Input dimension. Can be inferred at first call.
dtype (Numpy dtype, default to np.float64) – Numerical type for node parameters.
seed (int or numpy.random.Generator, optional) – A random state seed, for noise generation.
name (str, optional) – Node name.

References

Example

>>> from reservoirpy.nodes import IPReservoir
>>> reservoir = IPReservoir(
...                 100, mu=0.0, sigma=0.1, sr=0.95, activation="tanh", epochs=10)

We can fit the intrinsic plasticity parameters to reach a normal distribution of the reservoir activations. Using the narma() timeseries:

>>> from reservoirpy.datasets import narma
>>> x = narma(1000)
>>> _ = reservoir.fit(x, warmup=100)
>>> states = reservoir.run(x)

../../_images/intrinsic_plasticity_example.png

Methods

`__init__`([units, sr, lr, mu, sigma, ...])
`exp_gradients`(y, a, mu)	KL loss gradients of neurons with sigmoid activation (~ Exponential(lambda=1/mu)).
`fit`(x[, y, warmup])	Offline fitting method of a Node.
`gaussian_gradients`(y, a, mu, sigma)	KL loss gradients of neurons with tanh activation (~ Normal(mu, sigma)).
`initialize`(x[, y])	Define input and output dimensions, and instantiate variables.
`partial_fit`(x)
`predict`([x, iters, workers])	Alias for `run()`
`reset`()	Reset all Node state
`run`([x, iters, workers])	Run the Node on a sequence of data.
`step`([x])	Call the Node function on a single step of data and update the state of the Node.

Attributes

`initialized`	True if the Node has been initialized
`input_dim`	Expected dimension of the Node input.
`name`	Optional name of the Node.
`output_dim`	Expected dimension of the Node input.
`W`	Recurrent weights matrix (\(\mathbf{W}\)).
`Win`	Input weights matrix (\(\mathbf{W}_{in}\)).
`bias`	Bias vector (\(\mathbf{bias}\)).
`a`	Gain of reservoir activation (\(\mathbf{a}\)).
`b`	Bias of reservoir activation (\(\mathbf{b}\)).
`lr`	Leaking rate (1.0 by default) (\(\mathrm{lr}\)).
`sr`	Spectral radius of W.
`mu`	Mean of the target distribution (0.0 by default) (\(\mu\)).
`sigma`	Variance of the target distribution (1.0 by default) (\(\sigma\)).
`learning_rate`	Learning rate (5e-4 by default).
`epochs`	Number of epochs for training (1 by default).
`input_scaling`	Input scaling (float or array) (1.0 by default).
`rc_connectivity`	Connectivity (or density) of W (0.1 by default).
`input_connectivity`	Connectivity (or density) of Win (0.1 by default).
`activation`	Activation of the reservoir units (tanh by default) (\(f\)).
`units`	Number of neuronal units in the reservoir.
`rng`	A random state generator.
`state`	Current state of the Node.

W: ndarray | sparray#: Recurrent weights matrix (\(\mathbf{W}\)).

Win: ndarray | sparray#: Input weights matrix (\(\mathbf{W}_{in}\)).

a: float#: Gain of reservoir activation (\(\mathbf{a}\)).

activation: Literal['tanh', 'sigmoid']#: Activation of the reservoir units (tanh by default) (\(f\)).

b: float#: Bias of reservoir activation (\(\mathbf{b}\)).

bias: ndarray | sparray#: Bias vector (\(\mathbf{bias}\)).

epochs: int#: Number of epochs for training (1 by default).

exp_gradients(y, a, mu)[source]#: KL loss gradients of neurons with sigmoid activation (~ Exponential(lambda=1/mu)).

fit( x: array(t, d) | array(s, t, d) | ~typing.Sequence[array(t, d)], y: None = None, warmup: int = 0, ) → IPReservoir[source]#

Offline fitting method of a Node.

Parameters:

x (list or array-like of shape ([series, ] timesteps, input_dim), optional) – Input sequences dataset.
y (list or array-like of shape ([series], timesteps, output_dim), optional) – Teacher signals dataset. If None, the method will try to fit the Node in an unsupervised way, if possible.
warmup (int, default to 0) – Number of timesteps to consider as warmup and discard at the beginning of each timeseries before training.

Returns:

Node trained offline.

Return type:

Node

gaussian_gradients(y, a, mu, sigma)[source]#: KL loss gradients of neurons with tanh activation (~ Normal(mu, sigma)).

initialize( x: array(t, d) | array(s, t, d) | ~typing.Sequence[array(t, d)] | array(d), y: None = None, )[source]#

Define input and output dimensions, and instantiate variables.

Only called once, before fitting or running the node.

Parameters:

x (array of shape (input_dim,) or (timestep, input_dim)) – Input data to the node.
y (None) – Training data to the node. As it is not a trainable node, y is expected to be None.

initialized: bool = False#: True if the Node has been initialized

input_connectivity: float#: Connectivity (or density) of Win (0.1 by default).

input_dim: int = None#: Expected dimension of the Node input. Can be None before initialization

input_scaling: float | Sequence#: Input scaling (float or array) (1.0 by default).

learning_rate: float#: Learning rate (5e-4 by default).

lr: float | ndarray#: Leaking rate (1.0 by default) (\(\mathrm{lr}\)).

mu: float#: Mean of the target distribution (0.0 by default) (\(\mu\)).

name: str | None = None#: Optional name of the Node.

output_dim: int = None#: Expected dimension of the Node input. Can be None before initialization

Alias for run()

Run the Node on a sequence of data. Can update the state of the Node several times.

Parameters:

x (array-like of shape ([n_inputs,] timesteps, input_dim) or list of) – arrays of shape (timesteps, input_dim), optional A sequence of data of shape (timesteps, features).
iters (int, optional) – If x is None, a dimensionless timeseries of length iters is used instead.
workers (int, default to 1) – Number of workers used for parallelization. If set to -1, all available workers (threads or processes) are used.

Returns:

A sequence of output vectors.

Return type:

array of shape ([n_inputs,] timesteps, output_dim) or list of arrays

rc_connectivity: float#: Connectivity (or density) of W (0.1 by default).

reset() → dict[str, ndarray][source]#

Reset all Node state

Returns:: dict[str, np.array]
Return type:: previous state of the Node.

rng: Generator#: A random state generator.

Run the Node on a sequence of data. Can update the state of the Node several times.

Parameters:

x (array-like of shape ([n_inputs,] timesteps, input_dim) or list of arrays of shape (timesteps, input_dim), optional) – A timeseries, array of shape (timesteps, features), or a sequence of timeseries. Input of the Node.
iters (int, optional) – If x is None, a dimensionless timeseries of length iters is used instead.
workers (int, default to 1) – Number of workers used for parallelization. If set to -1, all available workers (threads or processes) are used.

Returns:

A sequence of output vectors.

Return type:

array of shape ([n_inputs,] timesteps, output_dim) or list of arrays

sigma: float#: Variance of the target distribution (1.0 by default) (\(\sigma\)).

sr: float#: Spectral radius of W.

state: dict[str, ndarray]#: Current state of the Node. Must have “out” as one of the keys.

step(x: array(d) | None = None)[source]#

Call the Node function on a single step of data and update the state of the Node.

Parameters:: x (array of shape (input_dim,), optional) – One single step of input data. If None, an empty array is used instead and the Node is assumed to have an input_dim of 0
Returns:: An output vector.
Return type:: array of shape (output_dim,)

units: int#: Number of neuronal units in the reservoir.