Weights initialization (reservoirpy.mat_gen)#

Quick tools for weight matrices initialization.

This module provides simple tools for reservoir internal weights and input/feedback weights initialization. Spectral radius of the internal weights, input scaling and sparsity are fully parametrizable.

Because most of the architectures developed in Reservoir Computing involve sparsely-connected neuronal units, the preferred format for all generated matrices is a scipy.sparse format (in most cases csr). Sparse arrays allow fast computations and compact representations of weights matrices, and remains easily readable. They can be parsed back to simple Numpy arrays just by calling their toarray() method.

All functions can take as parameter a numpy.random.Generator instance, or a seed number, to ensure reproducibility. Both distribution of weights and distribution of non-zero connections are controlled with the seed.

random_sparse

Create a random matrix.

uniform

Create an array with uniformly distributed values.

normal

Create an array with values distributed following a Gaussian distribution.

bernoulli

Create an array with values equal to either 1 or -1.

zeros

Create an array filled with 0.

ones

Create an array filled with 1.

generate_internal_weights

Generate the weight matrix that will be used for the internal connections of a

generate_input_weights

Generate input or feedback weights for a reservoir.

fast_spectral_initialization

Fast spectral radius (FSI) approach for weights initialization [1] of square matrices.

Initializer(func[, autorize_sr, ...])

Base class for initializer functions.

Example

Random sparse matrix initializer from uniform distribution, with spectral radius to 0.9 and connectivity of 0.1.

Matrix creation can be delayed…

In [1]: from reservoirpy.mat_gen import random_sparse

In [2]: initializer = random_sparse(dist="uniform", sr=0.9, connectivity=0.1)

In [3]: matrix = initializer(100, 100)

In [4]: print(type(matrix), "\n", matrix[:5, :5])
<class 'scipy.sparse._csr.csr_matrix'> 
   (0, 2)	0.08447685374716976
  (1, 2)	0.13192027394906106

…or can be performed right away.

In [5]: matrix = random_sparse(100, 100, dist="uniform", sr=0.9, connectivity=0.1)

In [6]: print(type(matrix), "\n", matrix[:5, :5])
<class 'scipy.sparse._csr.csr_matrix'> 
   (2, 2)	0.17201487357329942
  (3, 3)	0.06307703659462487

Random sparse matrix from Gaussian distribution, with mean of 0 and variance of 0.5 and an out-degree of 2:

In [7]: from reservoirpy.mat_gen import normal

In [8]: matrix = normal(7, 10, degree=2, direction="out", loc=0, scale=0.5)

In [9]: print(type(matrix), "\n", matrix)
<class 'scipy.sparse._csr.csr_matrix'> 
   (0, 0)	-0.550810914787131
  (1, 0)	0.3329165489033579
  (1, 1)	0.006256879710765227
  (1, 3)	-0.7171111037288016
  (1, 4)	-0.528132704145334
  (1, 6)	0.19824340870545915
  (1, 8)	0.0340504817188594
  (2, 3)	-0.3777703593702642
  (2, 5)	0.5495919968926554
  (2, 6)	0.10432479093668451
  (2, 7)	0.5998648901720852
  (3, 4)	0.37762588798805036
  (3, 7)	0.23486366986752424
  (3, 9)	0.05759044157639503
  (4, 1)	0.4441787624173251
  (5, 2)	-0.7108810345235578
  (5, 5)	0.17565483799395887
  (5, 8)	0.7402265169653427
  (5, 9)	-0.15217030245896082
  (6, 2)	0.3950377747361157

Dense matrix from Gaussian distribution, with mean of 0 and variance of 0.5:

In [10]: from reservoirpy.mat_gen import normal

In [11]: matrix = normal(50, 100, loc=0, scale=0.5)

In [12]: print(type(matrix), "\n", matrix[:5, :5])
<class 'numpy.ndarray'> 
 [[-0.4189  0.3515  0.6185 -0.0253  0.654 ]
 [-0.0157 -0.7217 -0.4813 -0.0747 -0.0826]
 [ 0.1426 -0.1903  0.71   -0.9205  0.0472]
 [-0.5243 -0.376   0.1111 -0.0135  0.1077]
 [-0.0406 -0.2424 -0.2802  0.1792 -0.4469]]

Sparse matrix from uniform distribution in [-0.5, 0.5], with connectivity of 0.9 and input_scaling of 0.3:

In [13]: from reservoirpy.mat_gen import uniform

In [14]: matrix = uniform(200, 60, low=0.5, high=0.5, connectivity=0.9, input_scaling=0.3)

In [15]: print(type(matrix), "\n", matrix[:5, :5])
<class 'scipy.sparse._csr.csr_matrix'> 
   (0, 0)	0.15
  (0, 1)	0.15
  (0, 2)	0.15
  (0, 3)	0.15
  (0, 4)	0.15
  (1, 0)	0.15
  (1, 1)	0.15
  (1, 2)	0.15
  (1, 3)	0.15
  (1, 4)	0.15
  (2, 0)	0.15
  (2, 2)	0.15
  (2, 3)	0.15
  (2, 4)	0.15
  (3, 0)	0.15
  (3, 1)	0.15
  (3, 3)	0.15
  (3, 4)	0.15
  (4, 0)	0.15
  (4, 1)	0.15
  (4, 2)	0.15
  (4, 3)	0.15
  (4, 4)	0.15

Sparse matrix from a Bernoulli random variable giving 1 with probability p and -1 with probability 1-p, with p=0.5 (by default) with connectivity of 0.2 and fixed seed, in Numpy format:

In [16]: from reservoirpy.mat_gen import bernoulli

In [17]: matrix = bernoulli(10, 60, connectivity=0.2, sparsity_type="dense")

In [18]: print(type(matrix), "\n", matrix[:5, :5])
<class 'numpy.ndarray'> 
 [[ 0.  0.  0.  0.  1.]
 [ 0.  0.  0.  0. -1.]
 [ 0.  0.  0.  0.  0.]
 [ 0.  0.  0.  0.  1.]
 [ 0.  0.  0.  0. -1.]]

References