Weights initialization (reservoirpy.mat_gen)#

Quick tools for weight matrices initialization.

This module provides simple tools for reservoir internal weights and input/feedback weights initialization. Spectral radius of the internal weights, input scaling and sparsity are fully parametrizable.

Because most of the architectures developed in Reservoir Computing involve sparsely-connected neuronal units, the preferred format for all generated matrices is a scipy.sparse format (in most cases csr). Sparse arrays allow fast computations and compact representations of weights matrices, and remains easily readable. They can be parsed back to simple Numpy arrays just by calling their toarray() method.

All functions can take as parameter a numpy.random.Generator instance, or a seed number, to ensure reproducibility. Both distribution of weights and distribution of non-zero connections are controlled with the seed.

random_sparse

Create a random array.

uniform

Create an array with uniformly distributed values.

normal

Create an array with values distributed following a Gaussian distribution.

bernoulli

Create an array with values equal to either 1 or -1.

ring

Create a lower cyclic shift matrix.

line

Create a lower shift matrix.

orthogonal

Create a random orthogonal matrix, drawn from the O(N) Haar distribution (the only uniform distribution on O(N)).

cluster

Create a cluster matrix with given distribution and p_in/p_out parameters.

small_world

Create a small-world network using the Watts-Strogatz model.

zeros

Create an array filled with 0.

ones

Create an array filled with 1.

fast_spectral_initialization

Fast spectral radius (FSI) approach for weights initialization [1] of square matrices.

Initializer(func[, allow_sr, ...])

Base class for initializer functions.

Example

Random sparse matrix initializer from uniform distribution, with spectral radius to 0.9 and connectivity of 0.1.

Matrix creation can be delayed…

In [1]: from reservoirpy.mat_gen import random_sparse

In [2]: initializer = random_sparse(dist="uniform", sr=0.9, connectivity=0.1)

In [3]: matrix = initializer(100, 100)

In [4]: print(type(matrix), "\n", matrix[:5, :5])
<class 'scipy.sparse._csr.csr_array'> 
 <Compressed Sparse Row sparse array of dtype 'float64'
	with 2 stored elements and shape (5, 5)>
  Coords	Values
  (3, 4)	0.05692392581980639
  (4, 2)	0.009979950498368863

…or can be performed right away.

In [5]: matrix = random_sparse(100, 100, dist="uniform", sr=0.9, connectivity=0.1)

In [6]: print(type(matrix), "\n", matrix[:5, :5])
<class 'scipy.sparse._csr.csr_array'> 
 <Compressed Sparse Row sparse array of dtype 'float64'
	with 4 stored elements and shape (5, 5)>
  Coords	Values
  (1, 2)	0.02912189457227911
  (2, 4)	0.009020101608383362
  (3, 4)	0.018572765904679975
  (4, 1)	0.12880485797736843

Random sparse matrix from Gaussian distribution, with mean of 0 and variance of 0.5 and an out-degree of 2:

In [7]: from reservoirpy.mat_gen import normal

In [8]: matrix = normal(7, 10, degree=2, direction="out", loc=0, scale=0.5)

In [9]: print(type(matrix), "\n", matrix)
<class 'scipy.sparse._csr.csr_array'> 
 <Compressed Sparse Row sparse array of dtype 'float64'
	with 20 stored elements and shape (7, 10)>
  Coords	Values
  (0, 0)	-0.13961303685290227
  (0, 1)	-0.12191574059017937
  (0, 6)	-0.12354300773705544
  (0, 9)	-1.0682239420435844
  (1, 2)	0.620937093224141
  (2, 2)	0.29644813229256173
  (2, 5)	-0.026359681842368487
  (3, 6)	0.3600055849105935
  (3, 8)	0.6161927417055968
  (4, 0)	-0.30533281937169987
  (4, 1)	0.2158133997344339
  (4, 3)	-0.6937652947071925
  (4, 5)	0.5792934032927515
  (4, 8)	1.2135744004086464
  (5, 4)	0.35864553118031767
  (5, 7)	-0.2853212714086524
  (6, 3)	0.7123710727795848
  (6, 4)	-0.8484586232360364
  (6, 7)	0.9911992576191182
  (6, 9)	0.6038804698423382

Dense matrix from Gaussian distribution, with mean of 0 and variance of 0.5:

In [10]: from reservoirpy.mat_gen import normal

In [11]: matrix = normal(50, 100, loc=0, scale=0.5)

In [12]: print(type(matrix), "\n", matrix[:5, :5])
<class 'numpy.ndarray'> 
 [[-0.595   0.3154 -0.5559 -0.2361 -0.0191]
 [-0.1662  0.4188  0.0361  0.5791 -1.1602]
 [-0.2324 -0.4988 -0.1529  0.0109  0.0518]
 [-0.0955 -0.5784  0.5121 -0.6383  0.8014]
 [-1.1495 -0.3511 -0.3394 -0.1327  0.3214]]

Sparse matrix from uniform distribution in [-0.5, 0.5], with connectivity of 0.9 and input_scaling of 0.3:

In [13]: from reservoirpy.mat_gen import uniform

In [14]: matrix = uniform(200, 60, low=0.5, high=0.5, connectivity=0.9, input_scaling=0.3)

In [15]: print(type(matrix), "\n", matrix[:5, :5])
<class 'scipy.sparse._csr.csr_array'> 
 <Compressed Sparse Row sparse array of dtype 'float64'
	with 23 stored elements and shape (5, 5)>
  Coords	Values
  (0, 0)	0.15
  (0, 1)	0.15
  (0, 2)	0.15
  (0, 4)	0.15
  (1, 0)	0.15
  (1, 1)	0.15
  (1, 2)	0.15
  (1, 3)	0.15
  (1, 4)	0.15
  (2, 0)	0.15
  (2, 1)	0.15
  (2, 2)	0.15
  (2, 3)	0.15
  (2, 4)	0.15
  (3, 0)	0.15
  (3, 1)	0.15
  (3, 2)	0.15
  (3, 4)	0.15
  (4, 0)	0.15
  (4, 1)	0.15
  (4, 2)	0.15
  (4, 3)	0.15
  (4, 4)	0.15

Sparse matrix from a Bernoulli random variable giving 1 with probability p and -1 with probability 1-p, with p=0.5 (by default) with connectivity of 0.2 and fixed seed, in Numpy format:

In [16]: from reservoirpy.mat_gen import bernoulli

In [17]: matrix = bernoulli(10, 60, connectivity=0.2, sparsity_type="dense")

In [18]: print(type(matrix), "\n", matrix[:5, :5])
<class 'numpy.ndarray'> 
 [[ 0.  0.  0.  0.  0.]
 [ 1.  0.  0.  0.  0.]
 [ 0. -1.  0. -1.  0.]
 [ 1.  0.  0.  0.  0.]
 [ 0.  0.  0.  0.  0.]]

References