{ "cells": [ { "cell_type": "markdown", "id": "749a2d02", "metadata": {}, "source": [ "# Introduction to the Jax backend\n", "\n", "ReservoirPy v0.4.1 introduces the Jax backend, an implementation of the classical ReservoirPy nodes using the Jax library instead of NumPy. Jax offers several machine learning features, such as differentiation, and allows for Python programs to be executed either on the CPU or on the GPU through function computation, which allows for a significant speed improvement on complex tasks.\n", "\n", "An overview of the jax library can be found at https://docs.jax.dev/en/latest/." ] }, { "cell_type": "code", "execution_count": null, "id": "fa86f058", "metadata": {}, "outputs": [], "source": [ "%pip install \"reservoirpy[jax]\"" ] }, { "cell_type": "code", "execution_count": 2, "id": "dfdc743c", "metadata": {}, "outputs": [], "source": [ "# Imports\n", "import time\n", "import tqdm\n", "\n", "import numpy as np\n", "import jax\n", "import jax.numpy as jnp\n", "import matplotlib.pyplot as plt\n", "\n", "rng = np.random.default_rng(seed=260_418)" ] }, { "cell_type": "code", "execution_count": 3, "id": "bce0d129", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "TFRT_CPU_0\n" ] } ], "source": [ "# Depending on your machine, it is sometimes better to only use the CPU\n", "jax.config.update('jax_platform_name', 'cpu')\n", "print(jax.numpy.ones(3).device)" ] }, { "cell_type": "markdown", "id": "d8f55f46", "metadata": {}, "source": [ "### Importing Jax nodes\n", "\n", "Jax implementations of classical ReservoirPy classes and methods are mirrored from `reservoirpy` to `reservoirpy.jax`:" ] }, { "cell_type": "code", "execution_count": 4, "id": "3f7a4d3e", "metadata": {}, "outputs": [], "source": [ "import reservoirpy as rpy\n", "import reservoirpy.jax as rjx\n", "\n", "# Data definition\n", "X = rng.uniform(size=(1200, 2))\n", "X_train, X_test = X[:1000], X[1000:]\n", "Y = X @ rng.uniform(size=(2, 1)) + rng.normal(scale=0.1, size=(1200, 1)) # noisy linear combination of X\n", "Y_train, Y_test = Y[:1000], Y[1000:]" ] }, { "cell_type": "code", "execution_count": 5, "id": "6c06ba99", "metadata": {}, "outputs": [], "source": [ "numpy_model = rpy.nodes.Reservoir(100, lr=0.7) >> rjx.nodes.Ridge(1e-2)\n", "y_pred_np = numpy_model.fit(X_train, Y_train, warmup=10).run(X_test)\n", "\n", "jax_model = rjx.nodes.Reservoir(100, lr=0.7) >> rjx.nodes.Ridge(1e-2)\n", "y_pred_jax = jax_model.fit(X_train, Y_train, warmup=10).run(X_test)" ] }, { "cell_type": "markdown", "id": "a4fc8260", "metadata": {}, "source": [ "Or, with the `ESN` model:" ] }, { "cell_type": "code", "execution_count": 6, "id": "3651041b", "metadata": {}, "outputs": [], "source": [ "numpy_model = rpy.ESN(units=100, lr=0.7, ridge=1e-2)\n", "jax_model = rjx.ESN(units=100, lr=0.7, ridge=1e-2)" ] }, { "cell_type": "markdown", "id": "bc3c29b9", "metadata": {}, "source": [ "Models with feedback work the same:" ] }, { "cell_type": "code", "execution_count": 7, "id": "38a2e53c", "metadata": {}, "outputs": [], "source": [ "reservoir, readout = rjx.nodes.Reservoir(100), rjx.nodes.Ridge()\n", "fb_model = (reservoir >> readout) & (reservoir << readout)\n", "_ = fb_model.fit(X_train, Y_train, warmup=10).run(X_test)" ] }, { "cell_type": "markdown", "id": "1d80275d", "metadata": {}, "source": [ "As well as big models with complex interactions (as shown in the Advanced Features guide):" ] }, { "cell_type": "code", "execution_count": 8, "id": "c3039387", "metadata": {}, "outputs": [], "source": [ "reservoir1 = rjx.nodes.Reservoir(100, name=\"res1-3\")\n", "reservoir2 = rjx.nodes.Reservoir(100, name=\"res2-3\")\n", "reservoir3 = rjx.nodes.Reservoir(100, name=\"res3-3\")\n", "\n", "readout1 = rjx.nodes.Ridge(name=\"readout2\")\n", "readout2 = rjx.nodes.Ridge(name=\"readout1\")\n", "\n", "model = [reservoir1, reservoir2] >> readout1 & \\\n", " [reservoir2, reservoir3] >> readout2" ] }, { "cell_type": "markdown", "id": "e7229fb8", "metadata": {}, "source": [ "Note that the `ScikitLearnNode` interface does not have a Jax implementation, since `scikit-learn` only has a NumPy\n", "backend. But you can seamlessly compose Jax nodes with NumPy nodes:" ] }, { "cell_type": "code", "execution_count": null, "id": "0249e526", "metadata": {}, "outputs": [], "source": [ "from sklearn.linear_model import RidgeCV\n", "\n", "reservoir = rjx.nodes.Reservoir(100)\n", "readout = rpy.nodes.ScikitLearnNode(RidgeCV)\n", "\n", "model = reservoir >> readout # Reservoir in jax, RidgeCV in numpy + scikit-learn" ] }, { "cell_type": "markdown", "id": "d35e67b2", "metadata": {}, "source": [ "Moreover, nodes and models from the ReservoirPy-Jax backend interoperate nicely with all ReservoirPy modules\n", "(`hyper`, `observables`, ...)" ] }, { "cell_type": "markdown", "id": "762469b1", "metadata": {}, "source": [ "### Performances\n", "\n", "Thanks to Jax code compilation and optimization, and its ability to run on GPUs, you can expect a significant performance boost on large models and with large datasets, making it ideal of resource intensive tasks.\n", "\n", "Here, we are illustrating this on ESN with reservoirs of various size, processing a timeseries of 100.000 timesteps, both using numpy and jax." ] }, { "cell_type": "code", "execution_count": 9, "id": "f9e47820", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Running on device: TFRT_CPU_0\n", "10 neurons:\n", "\tNumPy time: 1.06s\tJax time: 0.953s\n", "31 neurons:\n", "\tNumPy time: 1.1s\tJax time: 0.892s\n", "100 neurons:\n", "\tNumPy time: 1.24s\tJax time: 0.538s\n", "316 neurons:\n", "\tNumPy time: 1.83s\tJax time: 1.07s\n", "1000 neurons:\n", "\tNumPy time: 3.57s\tJax time: 1.72s\n", "3162 neurons:\n", "\tNumPy time: 10.6s\tJax time: 4.21s\n", "10000 neurons:\n", "\tNumPy time: 1.21e+02s\tJax time: 20.0s\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjsAAAHHCAYAAABZbpmkAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy80BEi2AAAACXBIWXMAAA9hAAAPYQGoP6dpAAB9k0lEQVR4nO3dd3hT5dsH8G9Wk+5S6ITSlrJL2bIFZBVQhiIIVGUpDoZYF7gYKhQExYHwExVw+yqCg1mRIVP2phYom7ZA98543j/ShKaLJE2bNv1+rqtXmues+zw5kLvn3Oc5EiGEABEREZGDkto7ACIiIqLKxGSHiIiIHBqTHSIiInJoTHaIiIjIoTHZISIiIofGZIeIiIgcGpMdIiIicmhMdoiIiMihMdkhIiIih8Zkh6iakEgkmDNnjr3DMDF+/HiEhITYOwy6h969e6N37972DoOo2mKyQw5t9erVkEgkxh+5XI769etj/PjxuH79ur3DqxZu3LiBOXPm4NixY/YOpdrYsWOHyXFT9Gf//v0l5t+7dy969OgBFxcX+Pv7Y/r06cjKyjKZx3AsHjp0yKQ9PT0dnTp1gkqlwubNm8uM6cyZM5gzZw4uXbpkk32sSvPnz8f69evtHQbVYnJ7B0BUFebNm4fQ0FDk5eVh//79WL16NXbv3o1Tp05BpVLZOzwAQG5uLuTyqv8neePGDcydOxchISFo27atybSVK1dCp9NVeUzVxfTp03HfffeZtDVu3Njk/bFjx9C3b1+0aNECH3zwAa5du4bFixcjPj4emzZtKnf9GRkZGDBgAE6cOIF169Zh4MCBZc575swZzJ07F7179y5xtm3r1q2W7VgVmz9/Ph599FEMHz7c3qFQLcVkh2qFQYMGoWPHjgCAp556CvXq1cPChQvx+++/Y9SoUXaOTq+6JF1FKRQKe4dgV/fffz8effTRcud5/fXXUadOHezYsQMeHh4AgJCQEDz99NPYunUrBgwYUOpymZmZiIyMxLFjx/Drr79i0KBBVsfp5ORk9bJEtQEvY1GtdP/99wMALly4YGwrq+6heN3KpUuXIJFIsHjxYnz++ecICwuDUqnEfffdh4MHD5ZY1s3NDdevX8fw4cPh5uYGHx8fvPzyy9BqtSbzFq/ZmTNnDiQSCc6fP4/x48fDy8sLnp6emDBhAnJyckyWzc3NxfTp01GvXj24u7tj6NChuH79+j3rgHbs2GE8czFhwgTjpZrVq1ffc9+XLVuGRo0awcXFBQMGDMDVq1chhMA777yDBg0awNnZGcOGDUNKSkqJ7W7atAn3338/XF1d4e7ujgcffBCnT58uM86iLl68iJEjR8Lb2xsuLi7o0qULNmzYUGK/JBIJ/u///g/vvfceGjRoAJVKhb59++L8+fNmbccgMzMTGo2m1GkZGRmIjY3F448/bkx0AODJJ5+Em5sb/u///q/U5bKysjBw4EAcOXIEa9euxYMPPlhuDKtXr8bIkSMBAA888IDxc9qxYweAksdu0f2fO3cu6tevD3d3dzz66KNIT09Hfn4+ZsyYAV9fX7i5uWHChAnIz88vsd1vv/0WHTp0gLOzM7y9vTF69GhcvXrVZJ74+HiMGDEC/v7+UKlUaNCgAUaPHo309HQA+uM6Ozsba9asMcY9fvx44/LXr1/HxIkT4efnB6VSifDwcHz11Vcm2zDsz08//YTXX38d/v7+cHV1xdChQy2Oh2onntmhWslQ91CnTh2r1/H9998jMzMTzzzzDCQSCRYtWoRHHnkEFy9eNDkjotVqERkZic6dO2Px4sX466+/sGTJEoSFheG5556753ZGjRqF0NBQLFiwAEeOHMEXX3wBX19fLFy40DjP+PHj8X//93944okn0KVLF+zcufOeX6AA0KJFC8ybNw9vv/02Jk+ebEwCu3XrVu5y3333HQoKCjBt2jSkpKRg0aJFGDVqFPr06YMdO3bgtddew/nz5/HJJ5/g5ZdfNvny+uabbzBu3DhERkZi4cKFyMnJwfLly9GjRw8cPXq03ILopKQkdOvWDTk5OZg+fTrq1q2LNWvWYOjQofjll1/w8MMPm8wfExMDqVSKl19+Genp6Vi0aBGioqJw4MCBe/YNoE8As7KyIJPJcP/99+P99983niEEgJMnT0Kj0Zi0AfozLW3btsXRo0dLrDM7OxuDBg3CwYMH8csvv+Chhx66Zxw9e/bE9OnT8fHHH+P1119HixYtAMD4WpYFCxbA2dkZM2fONH4eCoUCUqkUqampmDNnjvGybmhoKN5++23jsu+99x7eeustjBo1Ck899RRu3bqFTz75BD179sTRo0fh5eWFgoICREZGIj8/H9OmTYO/vz+uX7+OP//8E2lpafD09MQ333yDp556Cp06dcLkyZMBAGFhYQD0n2eXLl0gkUgwdepU+Pj4YNOmTZg0aRIyMjIwY8YMk/157733IJFI8NprryE5ORlLly5Fv379cOzYMTg7O5sVD9VSgsiBrVq1SgAQf/31l7h165a4evWq+OWXX4SPj49QKpXi6tWrxnl79eolevXqVWId48aNE8HBwcb3CQkJAoCoW7euSElJMbb/9ttvAoD4448/TJYFIObNm2eyznbt2okOHTqYtAEQs2fPNr6fPXu2ACAmTpxoMt/DDz8s6tata3x/+PBhAUDMmDHDZL7x48eXWGdpDh48KACIVatWmb3vPj4+Ii0tzdg+a9YsAUC0adNGqNVqY/uYMWOEk5OTyMvLE0IIkZmZKby8vMTTTz9tsp3ExETh6elZor24GTNmCADin3/+MbZlZmaK0NBQERISIrRarRBCiO3btwsAokWLFiI/P98470cffSQAiJMnT5a7nT179ogRI0aIL7/8Uvz2229iwYIFom7dukKlUokjR44Y5/v5558FALFr164S6xg5cqTw9/c3vjcci8HBwUKhUIj169eXG0Nxhm1t3769xLTix65h/1u1aiUKCgqM7WPGjBESiUQMGjTIZPmuXbuafM6XLl0SMplMvPfeeybznTx5UsjlcmP70aNHBQDx888/lxu7q6urGDduXIn2SZMmiYCAAHH79m2T9tGjRwtPT0+Rk5Njsj/169cXGRkZxvn+7//+TwAQH330kUXxUO3Dy1hUK/Tr1w8+Pj4ICgrCo48+CldXV/z+++9o0KCB1et87LHHTM4MGc6KXLx4scS8zz77rMn7+++/v9T5SlPasnfu3EFGRgYAGO/gef75503mmzZtmlnrt8bIkSNN/kru3LkzAODxxx83KbLu3LkzCgoKjHe+xcbGIi0tDWPGjMHt27eNPzKZDJ07d8b27dvL3e7GjRvRqVMn9OjRw9jm5uaGyZMn49KlSzhz5ozJ/BMmTDCpZynvMyqqW7du+OWXXzBx4kQMHToUM2fOxP79+yGRSDBr1izjfLm5uQAApVJZYh0qlco4vaikpCSoVCoEBQWVG4MtPPnkkyZnGTt37gwhBCZOnGgyX+fOnXH16lXj5bpff/0VOp0Oo0aNMvmc/P390aRJE+PnZDgGtmzZUuLS6r0IIbB27VoMGTIEQgiT7URGRiI9PR1HjhwpsT/u7u7G948++igCAgKwcePGCsdDjo3JDtUKy5YtQ2xsLH755RcMHjwYt2/fLvULyhINGzY0eW9IfFJTU03aVSoVfHx8SsxbfD5rt3P58mVIpVKEhoaazFf8riFbKh6T4Uum+Be4od0Qa3x8PACgT58+8PHxMfnZunUrkpOTy93u5cuX0axZsxLthss5ly9fLjfOsj4jczRu3BjDhg3D9u3bjfVWzs7OAFBqvUteXp5xelH/+9//4OTkhIEDByIuLs7iOCxhyeek0+mMdS3x8fEQQqBJkyYlPqezZ88aP6fQ0FBER0fjiy++QL169RAZGYlly5aZVR9z69YtpKWl4fPPPy+xjQkTJgBAieOhSZMmJu8lEgkaN25svCxdkXjIsbFmh2qFTp06Gesqhg8fjh49emDs2LGIi4uDm5sbAP1/nEKIEssWLyQ2kMlkpbYXX0dZ85nL3O1UpbJiuleshtvYv/nmG/j7+5eYz9a33tu674KCglBQUIDs7Gx4eHggICAAAHDz5s0S8968eROBgYEl2lu2bImNGzeib9++6N+/P/bs2VNpZ3kq8jlJJBJs2rSp1HkN/2YAYMmSJRg/fjx+++03bN26FdOnT8eCBQuwf//+cs+cGo6Fxx9/HOPGjSt1ntatW5e5fFmsjYccG5MdqnVkMhkWLFiABx54AJ9++ilmzpwJQP9Xf2mXN4qfLahugoODodPpkJCQYPKXr7l3HUkkksoKrQRDYaqvry/69etn8fLBwcGlng05d+6ccXplunjxIlQqlfHLvlWrVpDL5Th06JDJEAYFBQU4duxYmcMadOrUCevXr8eDDz6I/v37459//ilx9q+4qv6chBAIDQ1F06ZN7zl/REQEIiIi8Oabb2Lv3r3o3r07VqxYgXfffRdA6bH7+PjA3d0dWq3W7GPBcGbQQAiB8+fPl0iK7hUP1T68jEW1Uu/evdGpUycsXboUeXl5APT/wZ87dw63bt0yznf8+HHs2bPHXmGaJTIyEgDw2WefmbR/8sknZi3v6uoKAEhLS7NpXKWJjIyEh4cH5s+fD7VaXWJ60b4vzeDBg/Hvv/9i3759xrbs7Gx8/vnnCAkJQcuWLW0SZ2lxHD9+HL///jsGDBgAqVT/X6enpyf69euHb7/9FpmZmcZ5v/nmG2RlZRlvFy9N37598cMPP+D8+fMYOHCgsQarLFX5OT3yyCOQyWSYO3duibNgQgjcuXMHgP7W++K35UdEREAqlZpc2nN1dS0Rt0wmw4gRI7B27VqcOnWqRAylfQZff/21ST//8ssvuHnzpnGMInPjodqHZ3ao1nrllVcwcuRIrF69Gs8++ywmTpyIDz74AJGRkZg0aRKSk5OxYsUKhIeH3/OLyJ46dOiAESNGYOnSpbhz547x1vP//vsPwL3PCISFhcHLywsrVqyAu7s7XF1d0blz5xI1QLbg4eGB5cuX44knnkD79u0xevRo+Pj44MqVK9iwYQO6d++OTz/9tMzlZ86ciR9++AGDBg3C9OnT4e3tjTVr1iAhIQFr1641JiEV9dhjj8HZ2RndunWDr68vzpw5g88//xwuLi6IiYkxmfe9995Dt27d0KtXL0yePBnXrl3DkiVLMGDAgHJHRAaAhx9+GCtXrjQWQm/evLnMwSXbtm0LmUyGhQsXIj09HUqlEn369IGvr69N9rmosLAwvPvuu5g1axYuXbqE4cOHw93dHQkJCVi3bh0mT56Ml19+GX///TemTp2KkSNHomnTptBoNPjmm2+MiYxBhw4d8Ndff+GDDz5AYGAgQkND0blzZ8TExGD79u3o3Lkznn76abRs2RIpKSk4cuQI/vrrrxJjNHl7e6NHjx6YMGECkpKSsHTpUjRu3BhPP/00AJgdD9VC9rgFjKiqGG73PXjwYIlpWq1WhIWFibCwMKHRaIQQQnz77beiUaNGwsnJSbRt21Zs2bKlzNuv33///RLrRLFbvceNGydcXV1LzGe4rby8ZQ3z3Lp1q9R9SkhIMLZlZ2eLKVOmCG9vb+Hm5iaGDx8u4uLiBAARExNTXhcJIfS3zbds2VLI5XKT29DN3XfDrcHFb/ktq/+3b98uIiMjhaenp1CpVCIsLEyMHz9eHDp06J6xXrhwQTz66KPCy8tLqFQq0alTJ/Hnn3+aFY8h/tJusy/qo48+Ep06dRLe3t5CLpeLgIAA8fjjj4v4+PhS5//nn39Et27dhEqlEj4+PmLKlCkmt0iX1xdCCLF48WIBQDz00EMmt+4Xt3LlStGoUSMhk8lMbkMv69Zzcz+Pso61tWvXih49eghXV1fh6uoqmjdvLqZMmSLi4uKEEEJcvHhRTJw4UYSFhQmVSiW8vb3FAw88IP766y+T9Zw7d0707NlTODs7CwAmt6EnJSWJKVOmiKCgIKFQKIS/v7/o27ev+Pzzz0vszw8//CBmzZolfH19hbOzs3jwwQfF5cuXjfOZGw/VPhIh7FjlSESV5tixY2jXrh2+/fZbREVF2TscIqvt2LEDDzzwAH7++ed7Pr6DqDSs2SFyAKWN57J06VJIpVL07NnTDhEREVUfrNkhcgCLFi3C4cOH8cADD0Aul2PTpk3YtGkTJk+eXCWD1xERVWdMdogcQLdu3RAbG4t33nkHWVlZaNiwIebMmYM33njD3qEREdkda3aIiIjIobFmh4iIiBwakx0iIiJyaKzZgf4ZLTdu3IC7u3uVDslORERE1hNCIDMzE4GBgeUOKspkB8CNGzd4xwoREVENdfXq1XIf9MpkB4C7uzsAfWd5eHjYbL1qtRpbt27FgAEDoFAobLZeR8N+shz7zHzsK/Oxr8zHvjJfZfZVRkYGgoKCjN/jZWGyg7vPDvLw8LB5suPi4gIPDw/+YygH+8ly7DPzsa/Mx74yH/vKfFXRV/cqQWGBMhERETk0JjtERETk0JjsEBERkUNjzY6ZdDodCgoKLFpGrVZDLpcjLy8PWq22kiKr+dhP5VMoFJDJZPYOg4ioxmKyY4aCggIkJCRAp9NZtJwQAv7+/rh69SrH7ykH++nevLy84O/vz/4hIrICk517EELg5s2bkMlkCAoKKnfQouJ0Oh2ysrLg5uZm0XK1DfupbEII5OTkIDk5GQAQEBBg54iIiGoeJjv3oNFokJOTg8DAQLi4uFi0rOHSl0ql4pd4OdhP5XN2dgYAJCcnw9fXl5e0iIgsxG+WezDUkDg5Odk5EqrNDIm2Wq22cyRERDUPkx0zsVaC7InHHxGR9ZjsEBERkUNjskM1xqVLlyCRSHDs2LFK3Y5EIsH69esrdRvFVdW+ERFVtdRMNY5c9URqpv0uwzPZcVDjx4+HRCJBTEyMSfv69eur7JKIRCIx/nh6eqJ79+74+++/q2TbRERUPaRmanD0mhdSMzV2i8Guyc6uXbswZMgQBAYGlvhrWq1W47XXXkNERARcXV0RGBiIJ598Ejdu3DBZR0pKCqKiouDh4QEvLy9MmjQJWVlZVbwn5knJUOP72JtIyaia7FalUmHhwoVITU2tku2VZtWqVbh58yb27NmDevXq4aGHHsLFixftFg8REVWt67fz7R2CfZOd7OxstGnTBsuWLSsxLScnB0eOHMFbb72FI0eO4Ndff0VcXByGDh1qMl9UVBROnz6N2NhY/Pnnn9i1axcmT55cVbtgkdRMNX7cllRlp/L69esHf39/LFiwoMx55syZg7Zt25q0LV26FCEhIcb348ePx/DhwzF//nz4+fnBy8sL8+bNg0ajwSuvvAJvb280aNAAq1atKrF+w2B4rVq1wvLly5Gbm4vY2Fh8/fXXqFu3LvLzTf8RDB8+HE888US5+3Xu3Dl069YNKpUKrVq1ws6dO43TtFotJk2ahNDQUDg7O6NZs2b46KOPSqzjq6++Qnh4OJRKJQICAjB16tQytzd79mwEBATgxIkTAIDdu3fj/vvvh7OzM4KCgjB9+nRkZ2cb5w8JCcH8+fMxceJEuLu7o2HDhvj8889N1vnvv/+iXbt2UKlU6NixI44ePVruPhMR1SQpGWpcuJ6DC9dz8OWGRADA7pPpxraq+qPfwK7j7AwaNAiDBg0qdZqnpydiY2NN2j799FN06tQJV65cQcOGDXH27Fls3rwZBw8eRMeOHQEAn3zyCQYPHozFixcjMDDQ5jELIZCvNm8kZZ1OIL9Ah7wCHaTSu8vlq3XIK7D8sQhKhdSiS1AymQzz58/H2LFjMX36dDRo0MDibRr8/fffaNCgAXbt2oU9e/Zg0qRJ2Lt3L3r27IkDBw7gp59+wjPPPIP+/fuXuR3DeDEFBQV48sknMX36dPz+++8YMWIEAP04Mhs2bMDWrVvLjeWVV17B0qVL0bJlS3zwwQcYMmQIEhISULduXeh0OjRo0AA///wz6tati71792Ly5MkICAjAqFGjAADLly9HdHQ0YmJiMGjQIKSnp2PPnj0ltiOEwPTp0/Hnn3/in3/+QePGjXHhwgUMHDgQ7777Lr766ivcunULU6dOxdSpU02SvSVLluCdd97B66+/jl9++QXPPfccevXqhWbNmiErKwsPPfQQ+vfvj2+//RYJCQl44YUXrPpciIiqo80HbuPHbUkmbb/tvoPfdt8BAIzu64ex/atukNQaNahgeno6JBIJvLy8AAD79u2Dl5eXMdEB9GczpFIpDhw4gIcffrjU9eTn55ucUcjIyACgv3RWfBwTtVoNIQR0Oh10On3iMnrOqQrtx8wV561a7sc5raByMu9knBACQggMGzYMbdu2xdtvv40vvvjC+MgLw6sQwuR9aW1CCHh7e2Pp0qWQSqVo0qQJFi1ahJycHMycORMA8NprryEmJga7du3C6NGjjesy9FtOTg7eeOMNyGQy3H///VAqlRgzZgy++uorPPLIIwCAb7/9Fg0bNkTPnj1LfTSHoW3KlCnGz3bZsmXYvHkzvvjiC7zyyiuQyWSYPXu2cZng4GDs3bsXP/30Ex599FEAwLvvvovo6GhMmzbNOF+HDh1MtllQUICoqCgcO3YMu3btQv369aHT6UySRwAICwvD0qVL8cADD2DZsmVQqVQA9In8s88+C0CfnH344YfYtm0bmjRpgm+//RY6nQ4rV66ESqVCixYtcOXKFUyZMsXYX6XtuxACarUaMpnMeJxy3J17Y1+Zj31lPvZV+fp18ESHpq44+l8mvt92CwAw+SE/NAlyBQDUcZfbpO/MXUeNSXby8vLw2muvYcyYMfDw8AAAJCYmwtfX12Q+uVwOb29vJCYmlrmuBQsWYO7cuSXat27dWmKUZLlcDn9/f2RlZaGgoAD5BZY9H8uWMjMyUGBmsqNWq6HRaJCRkYE333wTw4YNwzPPPIPc3FwAdxO8/Px8aLVa43tA39c6nc4kCWzatKlJLVTdunXRtGlTk+Xq1KmDq1evmrRFRUVBJpMhNzcX9erVwyeffIKQkBBkZGRgzJgx6NOnD+Li4hAYGIhVq1Zh9OjRyMzMLHWfDNuPiIgw2UabNm1w4sQJY9vKlSvx3Xff4dq1a8jLy0NBQYFxmVu3buHGjRvo0qWLyTqKe/HFF6FUKhEbGwt3d3fjvEePHsXp06fx/fffG+c1JMMnT55Es2bNoNPpSvSNj48Prl27hoyMDJw4cQItW7ZEQUGB8eGyERERAPSXdkuLq6CgALm5udi1axc0mrtFfsXPflLZ2FfmY1+Zj31VvsMXvAG4AwCSLh+D9o5lD9S+l5ycHLPmqxHJjlqtxqhRoyCEwPLlyyu8vlmzZiE6Otr4PiMjA0FBQRgwYIAxkTLIy8vD1atX4ebmBpVKBSEEfpzTyqztCCFw7WYa1FBCAgkSbubi8z9uYPKQQIQG6C/peLnLUcddYdb6lAqJ2ZexFAoF5HI5PDw8MGjQIAwYMADz58/HuHHjAMC4n87OzpBKpSb7LZPJTNoUCgWcnZ1N5lEoFHB1dS2xnJOTk0nbkiVL0K9fP3h6esLHx8ckxh49eqBNmzZYt24dunfvjnPnzmHy5MklPgMDNzc3ACixXblcDoVCAQ8PD/z44494++23sXjxYnTp0gXu7u5YvHgx/v33X3h4eBj7z8XFpcztAMCAAQPw448/Yu/evYiKijK25+bmYvLkySZnhQwaNmwIJycnSKVSuLu7lxmjk5OT8bO5174Z5OXlwdnZGT179oRKpYJarUZsbCz69+8PhcK846e2Yl+Zj31lPvaVeTZ/FA9An+B06dIZTRu623T95f3RWlS1T3YMic7ly5fx999/m3wR+Pv7Gx+QaKDRaJCSkgJ/f/8y16lUKqFUKku0KxSKEgetVquFRCKBVCo1PrfJxcxHE+l0Ovh5K+DhoX/ApUqpX7BFiBvC6lv2nC1LGW75NsS8cOFCtG3bFs2bNwcAY7uvry8SExON8wPA8ePHTeYpvq6i27hXW2BgIJo2bVpmnE899RSWLl2KS5cuoW/fvggODi5zXsN6//33X/Tu3RuA/vM+cuQIpk6dCqlUin379qFbt26YMmWKcTnD3V9SqRSenp4ICQnB9u3b0bdv3zK3NWzYMAwdOhRjx46FQqEwXppr3749zp49W+4+ldYPRdtatmyJb7/91vg8MMM+GWIs7flgUqm+Xqv4MVraMUulY1+Zj31lPvZV2VIz1bh+u/DsdUA6fOqobN5X5q6vWo+zY0h04uPj8ddff6Fu3bom07t27Yq0tDQcPnzY2Pb3339Dp9Ohc+fOVR1utRYREYGoqCh8/PHHJu29e/fGrVu3sGjRIly4cAHLli3Dpk2bqiyusWPH4tq1a/j6668xYcIEs5ZZtmwZ1q1bh3PnzmHKlClITU3FxIkTAQBNmjTBoUOHsGXLFvz333946623cPDgQZPl58yZgyVLluDjjz9GfHw8jhw5gk8++aTEdh5++GF88803mDBhAn755RcA+tqkvXv3YurUqTh27Bji4+Px22+/lXs3V2n7LJFI8PTTT+PMmTPYuHEjFi9ebPbyREQ1wZlL+rtUg/2U6BSSZvZVjMpg12QnKysLx44dM44am5CQgGPHjuHKlStQq9V49NFHcejQIXz33XfQarVITExEYmKisc6hRYsWGDhwIJ5++mn8+++/2LNnD6ZOnYrRo0dXyp1YFVXHXYHRff3s9oHPmzevRPFrixYt8Nlnn2HZsmVo06YN/v33X7z88stVFpOnpyceeeQRuLq6Yvjw4WYtExMTg5iYGLRp0wa7d+/G77//jnr16gEAnnnmGTzyyCN47LHH0LlzZ9y5cwfPP/+8yfLjxo3D0qVL8dlnnyE8PBwPPfQQ4uPjS93Wo48+ijVr1uCJJ57Ar7/+itatW2Pnzp3477//cP/996Ndu3Z4++23LTre3Nzc8Mcff+DkyZNo164d3njjDSxcuNDs5YmIaoJTF/V1li1DXO0cCQBhR9u3bxcASvyMGzdOJCQklDoNgNi+fbtxHXfu3BFjxowRbm5uwsPDQ0yYMEFkZmZaFEd6eroAINLT00tMy83NFWfOnBG5ubkW759WqxWpqalCq9VavGxt0qdPHzF58mT2UzmKH4cFBQVi/fr1oqCgwM6RVX/sK/Oxr8zHvrq3aR+eFUNeOyp2HrlVaX1V3vd3UXat2endu7fxNufSlDfNwNvb2+TOGKo5UlNTsWPHDuzYsaPEYy2IiKjmyszR4HJSHgCgRYgr9l6xbzzVvkCZHFe7du2QmpqKmJgYNGnSxN7hEBGRjZy5lA0hgAY+Sni52T/VsH8EVGtdunQJAEzG9CEioprPUK8THupm50j0qvXdWERERFTznL6kT3ZaNWKyQ0RERA4mJ0+Li9f1o/WHh1aDO7HAZIeIiIhs6OzlbOgE4O/thHqeTvYOBwCTHSIiIrKhUwnVq14HYLJDRERENnQ6QT9ycqtqcgkLYLJDRERENpJfoMP5a/onkYdXk+JkgMkOVdD48ePNfsyDtebMmYO2bdtW6jZKUxX7RkTkSM5dyYZGK1DPUwG/OtWjXgdgsuOwSvuiXrBgAWQyGd5///0S87/22msICQlBZmamSfuQIUPQs2fPEs/UIiIiKu50kXodiURi52juYrJThXKS83Dkk3PISc6zy/a/+uorvPrqq/jqq69KTJs3bx7c3NwQHR1tMv/27duxatUqSKU8VIiIqHyGep3qcsu5Ab/BqlDOrTwcXRaHnFtVn+zs3LkTubm5mDdvHjIyMrB3716T6UqlEmvWrMGaNWuwefNmXLlyBS+++CIWLVqEsLCwe65/7ty58PHxgYeHB5599lnjk+kBYPPmzejRowe8vLxQt25dPPTQQ7hw4YLJ8tevX8fYsWPh7e0NV1dXdOzYEQcOHCh1WxcuXECjRo0wdepUCCGQn5+Pl19+GfXr14erqys6d+6MHTt2GOdfvXo1vLy8sGXLFrRo0QJubm4YOHAgbt68aZxHq9UiOjraGOOrr75q1rPZiIhIT63RIe5KYXFyNarXAZjsWEwIAXWOxuwfTa727u95WgCAJk9r0ToMPxX58v3yyy8xZswYKBQKjBkzBl9++WWJeTp06IBZs2bhqaeewhNPPIFOnTrhueeeu+e6t23bhrNnz2LHjh344Ycf8Ouvv2Lu3LnG6dnZ2YiOjsahQ4ewbds2SKVSPPzww8ZLY1lZWXjooYdw48YN/P777zh+/DheffXVUi+dnThxAj169MDYsWPx6aefQiKRYOrUqdi3bx9+/PFHnDhxAiNHjsTAgQMRHx9vXC4nJweLFy/GN998g127duHKlSt4+eWXjdOXLFmC1atX46uvvsLu3buRkpKCdevWWdTHRES1Wfy1HBRoBLzc5KhfT2nvcEzw2VgW0uRq8XX7DRVax4ao3VYt9+SRB6Fwsfwjy8jIwC+//IJ9+/YBAB5//HHcf//9+Oijj+DmZpp9v/nmm1i1ahUOHDiA//77z6xrrk5OTvjqq6/g4uKC8PBwzJs3D6+88greeecdSKVSjBgxwmT+r776Cj4+Pjhz5gxatWqF77//Hnfu3MHBgwdRr149AEDjxo1LbGfv3r146KGH8MYbb+Cll14CAFy5cgWrVq3ClStXEBgYCAB4+eWXsXnzZqxatQrz588HAKjVaqxYscJ4lmrq1KmYN2+ecd1Lly7FrFmz8MgjjwAAVqxYgS1btty7c4mICIDp87CqU70OwDM7tcIPP/yAsLAwtGnTBgDQtm1bBAcH46effioxb2xsLBITE6HT6XDw4EGz1t+mTRu4uLgY33ft2hVZWVm4evUqACA+Ph5jxoxBo0aN4OHhgZCQEAD6RAUAjh8/joiICHh7e5e5jStXrqB///54++23jYkOAJw8eRJarRZNmzaFm5ub8Wfnzp0ml8pcXFxMLscFBAQgOTkZAJCeno6bN2+ic+fOxulyuRwdO3Y0a/+JiAg4VU3rdQCe2bGY3FmGJ488aNa8Op0Oty7dgTRPDqlEipRz6dj3zkl0fSsC3s09AQDO9ZRw8VGZvW1rfPnllzh9+jTk8rsft06nw1dffYVJkyYZ21JTU/H000/jzTffhBACzz//PHr16mU822KtIUOGIDg4GCtXrkRgYCB0Oh1atWplrOtxdna+5zp8fHwQGBiIH374ARMnToSHhwcA/SUwmUyGw4cPQyYz7Z+iZ60UCoXJNIlEwpocIiIb0WgFzl2unvU6AJMdi0kkErMvJel0Org1cIaHhwekUinkKv2XsW9bb9QL96rEKO86efIkDh06hB07dpicOUlJSUHv3r1x7tw5NG/eHAAwbdo0+Pv74/XXXwcA/Pbbb5gyZUqpZ4CKOn78OHJzc41Jy/79++Hm5oagoCDcuXMHcXFxWLlyJe6//34AwO7dppfxIiIi8MUXXyAlJaXMxMrZ2Rl//vknBg8ejMjISGzduhXu7u5o164dtFotkpOTjeu3lKenJwICAnDgwAH07NkTAKDRaHD48GG0b9/eqnUSEdUmF67nIK9AB3cXGRr6mvcHfFXiZSwH9+WXX6JTp07o2bMnWrVqZfzp2bMn7rvvPmOh8rp16/Dzzz9jzZo1kMvlkMvlWLNmDdavX4+1a9eWu42CggJMmjQJZ86cwcaNGzF79mxMnToVUqkUderUQd26dfH555/j/Pnz+Pvvv01ubweAMWPGwM/PD4888gj27NmDixcvYu3atcYaIwNXV1ds2LABcrkcgwYNQlZWFpo2bYqoqCg8+eST+PXXX5GQkIB///0XCxYswIYN5tdWvfDCC4iJicH69etx7tw5PP/880hLSzN7eSKi2sxwy3nLEFdIpdWrXgdgslOlXHxUaDelmdmXrSpCp9NBKpXi22+/LVEgbDBixAh8/fXXuHXrFp599lnMnj0brVq1Mk6PiIjA7Nmz8fzzz+P27dtlbqtv375o0qQJevbsicceewxDhw7FnDlzAABSqRQ//vgjDh8+jFatWuHFF18sMaihk5MT1q5dCx8fHwwePBgRERGIiYkpcVkK0F+a2rRpE4QQePDBB5GdnY1Vq1bhySefxEsvvYRmzZph+PDhOHjwIBo2bGh2f7300kt44oknMG7cOHTt2hXu7u54+OGHzV6eiKg2q44P/yxKIli4gIyMDHh6eiI9Pd1YC2KQl5eHhIQEhIaGQqWyLEnR6XTIyMgwXsaqSgMHDkTjxo3x6aefVul2rWHPfqopih+HarUaGzduxODBg0vUI5Ep9pX52FfmY1/dpdUJPD7vJLLzdPhgalM0buBiMr0y+6q87++i+M3iYFJTU/Hnn39ix44d6Nevn73DISIiB3fpZi6y83RwVkoRGnDvG07sgQXKDmbixIk4ePAgXnrpJQwbNsze4RARkYMrWq8jk1W/eh2AyY7D4ai/RERUlap7vQ7Ay1hERERkJSGE8UnnrZjs1Hys4yZ74vFHRNXR1eQ8ZOZo4aSQIKx+9azXAZjs3JPh9ueiT/Emqmo5OTkASo4ETURkT4ZHRLQIdoVCXn1TCtbs3INcLoeLiwtu3boFhUJh0a3ROp0OBQUFyMvL4y3V5WA/lU0IgZycHCQnJ8PLy6vUsYeIiOzl9MXqX68DMNm5J4lEgoCAACQkJODy5csWLSuEMD5Gobo9AbY6YT/dm5eXF/z9/e0dBhGRUU2p1wGY7JjFyckJTZo0sfhSllqtxq5du9CzZ09efigH+6l8CoWCZ3SIqNq5eacAKZkayGUSNA1yufcCdsRkx0xSqdTiEZRlMhk0Gg1UKhW/xMvBfiIiqnkMt5w3DXKBk6J6lyBU7+iIiIioWjLU67RqVL0vYQFMdoiIiMgKp2vAYIIGTHaIiIjIIsmpBUhOU0MmBVoEV+96HYDJDhEREVnIcFYnrL4LVE7V/wYKJjtERERkkVM1qF4HYLJDREREFjp9ST9ycnioq50jMQ+THSIiIjJbSoYaN27nQyIBWobwzA4RERE5GMP4OqEBznBVVf96HYDJDhEREVmgpjwioigmO0RERGS2UxdrVr0OwGSHiIiIzJSepcHV5DwANWMwQQMmO0RERGSW05f0l7Aa+qng4VpzHq/JZIeIiIjMUhPrdQAmO0RERGSmmlivAzDZISIiIjNk5WpwKTEXQM2q1wHsnOzs2rULQ4YMQWBgICQSCdavX28yXQiBt99+GwEBAXB2dka/fv0QHx9vMk9KSgqioqLg4eEBLy8vTJo0CVlZWVW4F0RERI7v7KVsCAEE1lPC20Nh73AsYtdkJzs7G23atMGyZctKnb5o0SJ8/PHHWLFiBQ4cOABXV1dERkYiLy/POE9UVBROnz6N2NhY/Pnnn9i1axcmT55cVbtARERUK5xK0F/CalXDLmEBgF1LqQcNGoRBgwaVOk0IgaVLl+LNN9/EsGHDAABff/01/Pz8sH79eowePRpnz57F5s2bcfDgQXTs2BEA8Mknn2Dw4MFYvHgxAgMDq2xfiIiIHJmhOLmmXcIC7JzslCchIQGJiYno16+fsc3T0xOdO3fGvn37MHr0aOzbtw9eXl7GRAcA+vXrB6lUigMHDuDhhx8udd35+fnIz883vs/IyAAAqNVqqNVqm+2DYV22XKcjYj9Zjn1mPvaV+dhX5qttfZWbr8X56zkAgGZBSov2uzL7ytx1VttkJzExEQDg5+dn0u7n52eclpiYCF9fX5Ppcrkc3t7exnlKs2DBAsydO7dE+9atW+Hi4lLR0EuIjY21+TodEfvJcuwz87GvzMe+Ml9t6atraSrodH5wU2pwcO9fVq2jMvoqJyfHrPmqbbJTmWbNmoXo6Gjj+4yMDAQFBWHAgAHw8PCw2XbUajViY2PRv39/KBQ1q5irKrGfLMc+Mx/7ynzsK/PVtr76/q8k4OxtdGxRF4MHt7Fo2crsK8OVmXuptsmOv78/ACApKQkBAQHG9qSkJLRt29Y4T3JysslyGo0GKSkpxuVLo1QqoVQqS7QrFIpKOWgra72Ohv1kOfaZ+dhX5mNfma+29NXZy/pbziPCPKze38roK3PXV23H2QkNDYW/vz+2bdtmbMvIyMCBAwfQtWtXAEDXrl2RlpaGw4cPG+f5+++/odPp0Llz5yqPmYiIyNHkq3X476r+clFNLE4G7HxmJysrC+fPnze+T0hIwLFjx+Dt7Y2GDRtixowZePfdd9GkSROEhobirbfeQmBgIIYPHw4AaNGiBQYOHIinn34aK1asgFqtxtSpUzF69GjeiUVERGQD/13JhkYr4O2hQEBdJ3uHYxW7JjuHDh3CAw88YHxvqKMZN24cVq9ejVdffRXZ2dmYPHky0tLS0KNHD2zevBkqlcq4zHfffYepU6eib9++kEqlGDFiBD7++OMq3xciIiJHdDrh7iMiJBKJnaOxjl2Tnd69e0MIUeZ0iUSCefPmYd68eWXO4+3tje+//74ywiMiIqr1TtXQh38WVW1rdoiIiMi+1Bodzl0xnNlhskNEREQO5vz1XBSoBTxcZQjyLXkXc03BZIeIiIhKZXxERIhbja3XAZjsEBERURlOXSys12lUcy9hAUx2iIiIqBRarcDZy3fvxKrJmOwQERFRCRdv5iI3XwdXlQzB/s72DqdCmOwQERFRCYZLWC1DXCGT1tx6HYDJDhEREZXCUJxc0+t1ACY7REREVIxOJ3DmkmPU6wBMdoiIiKiYy0l5yMrVQuUkRVigi73DqTAmO0RERGTidGG9TotgV8hkNbteB2CyQ0RERMWculQ4mKAD1OsATHaIiIioCCGE8UnnrRygXgdgskNERERFXL+Vj/QsDZzkEjRpUPPrdQAmO0RERFTEqcJbzps1dIVC7hhpgmPsBREREdmE4RKWI9xybsBkh4iIiADo63UMIyeHhzpGcTLAZIeIiIgKJaUU4E6GGnKZBM0b8swOERERORhDvU6TBi5QOjlOiuA4e0JEREQV4oj1OgCTHSIiIip0yoEe/lkUkx0iIiLCrbQCJKUUQCoFmgfzzA4RERE5mNOFZ3XCAl3gopTZORrbYrJDREREDluvAzDZISIiIhSp13Gg8XUMmOwQERHVcqmZaly/lQ+JBGgZwjM7RERE5GAMl7BC/FVwc5HbORrbY7JDRERUyxmKkx3pERFFMdkhIiKq5Ry5XgdgskNERFSrZWRrcDkxDwDQ0gHvxAKY7BAREdVqZy7p63WCfJXwclPYOZrKwWSHiIioFnP0eh2AyQ4REVGtxmSHiIiIHFZ2nhYXb+QCAFo5aL0OwGSHiIio1jp3ORs6AfjXdUJdTyd7h1NpmOwQERHVUqcuFl7CCnHcS1gAkx0iIqJay1Cv06oRkx0iIiJyMHkFWsRfywHgmE86L4rJDhERUS0UdyUHWh1Qz1MBvzqOW68DMNkhIiKqlYz1OqFukEgkdo6mcjHZISIiqoVqS70OYINkJz8/3xZxEBERURUpUOsQd1Vfr+PI4+sYWJzsbNq0CePGjUOjRo2gUCjg4uICDw8P9OrVC++99x5u3LhRGXESERGRjcRfy4FaI+DlJkdgPaW9w6l0Zic769atQ9OmTTFx4kTI5XK89tpr+PXXX7FlyxZ88cUX6NWrF/766y80atQIzz77LG7dulWZcRMREZGVDPU6rRo5fr0OYEGys2jRInz44Ye4fv06vvzySzzzzDMYMmQI+vXrh1GjRmHevHnYvn07Lly4AC8vL3z77bcVDk6r1eKtt95CaGgonJ2dERYWhnfeeQdCCOM8Qgi8/fbbCAgIgLOzM/r164f4+PgKb5uIiMhRnS580rmj33JuIDd3xn379pk1X/369RETE2N1QEUtXLgQy5cvx5o1axAeHo5Dhw5hwoQJ8PT0xPTp0wHok7CPP/4Ya9asQWhoKN566y1ERkbizJkzUKlUNomDiIjIUWi0Aucu65OdVg788M+ibHI3llarxbFjx5CammqL1Rnt3bsXw4YNw4MPPoiQkBA8+uijGDBgAP79918A+rM6S5cuxZtvvolhw4ahdevW+Prrr3Hjxg2sX7/eprEQERE5ggvXc5BXoIO7iwxBvrXjpIBVyc6MGTPw5ZdfAtAnOr169UL79u0RFBSEHTt22Cy4bt26Ydu2bfjvv/8AAMePH8fu3bsxaNAgAEBCQgISExPRr18/4zKenp7o3Lmz2WeiiIiIapNTCXefhyWVOn69DmDBZayifvnlFzz++OMAgD/++AMJCQk4d+4cvvnmG7zxxhvYs2ePTYKbOXMmMjIy0Lx5c8hkMmi1Wrz33nuIiooCACQmJgIA/Pz8TJbz8/MzTitNfn6+yS3zGRkZAAC1Wg21Wm2T2A3rK/pKpWM/WY59Zj72lfnYV+aryX116kImAKB5sKpK4q/MvjJ3nVYlO7dv34a/vz8AYOPGjRg5cqTxTq2PPvrImlWW6v/+7//w3Xff4fvvv0d4eDiOHTuGGTNmIDAwEOPGjbN6vQsWLMDcuXNLtG/duhUuLi4VCblUsbGxNl+nI2I/WY59Zj72lfnYV+araX2lE8CJC0EApEi9dgwbNxZU2bYro69ycnLMms+qZMfPzw9nzpxBQEAANm/ejOXLlxs3KpPJrFllqV555RXMnDkTo0ePBgBERETg8uXLWLBgAcaNG2dMuJKSkhAQEGBcLikpCW3bti1zvbNmzUJ0dLTxfUZGBoKCgjBgwAB4eHjYLH61Wo3Y2Fj0798fCoXCZut1NOwny7HPzMe+Mh/7ynw1ta8u3siFev9FuCiliHq0L2RVcBmrMvvKcGXmXqxKdiZMmIBRo0YhICAAEonEWDNz4MABNG/e3JpVlionJwdSqWlZkUwmg06nAwCEhobC398f27ZtMyY3GRkZOHDgAJ577rky16tUKqFUlhxESaFQVMpBW1nrdTTsJ8uxz8zHvjIf+8p8Na2vzl3V30jUMsQNKmXVPvyzMvrK3PVZlezMmTMHrVq1wtWrVzFy5Ehj4iCTyTBz5kxrVlmqIUOG4L333kPDhg0RHh6Oo0eP4oMPPsDEiRMBABKJBDNmzMC7776LJk2aGG89DwwMxPDhw20WBxERkSM4fbF2ja9jYFWyAwCPPvpoibaK1NGU5pNPPsFbb72F559/HsnJyQgMDMQzzzyDt99+2zjPq6++iuzsbEyePBlpaWno0aMHNm/ezDF2iIiIitDpBE5fKrwTqxY8/LMos289//HHH81e6dWrV21yR5a7uzuWLl2Ky5cvIzc3FxcuXMC7774LJ6e7p94kEgnmzZuHxMRE5OXl4a+//kLTpk0rvG0iIiJHcjU5D5k5WigVUjSub/ubcaozs5Od5cuXo0WLFli0aBHOnj1bYnp6ejo2btyIsWPHon379rhz545NAyUiIiLrnU7QX8JqHuwCuax2jK9jYPZlrJ07d+L333/HJ598glmzZsHV1RV+fn5QqVRITU1FYmIi6tWrh/Hjx+PUqVMlxr4hIiIi+zEOJlhLHhFRlEU1O0OHDsXQoUNx+/Zt7N6923h5qV69emjXrh3atWtX4u4pIiIisi8hBE4XJju15XlYRVlVoFyvXj3e7URERFRD3Lidj9RMDRRyCZoG1a56HcBGDwIlIiKi6stQr9M0yAVOitr31V/79piIiKiWqc31OgCTHSIiIodXm+t1ACY7REREDi0pJR+30tSQSfW3nddGFUp2CgoKEBcXB41GY6t4iIiIyIYM9TqNG7hA5WS7h3XXJFYlOzk5OZg0aRJcXFwQHh6OK1euAACmTZuGmJgYmwZIRERE1jtVyy9hAVYmO7NmzcLx48exY8cOk2dQ9evXDz/99JPNgiMiIqKKOV3Li5MBK8fZWb9+PX766Sd06dIFEsndIafDw8Nx4cIFmwVHRERE1ruTocbNOwWQSoAWIbXrSedFWXVm59atW/D19S3Rnp2dbZL8EBERkf0YzuqEBjrDVVU763UAK5Odjh07YsOGDcb3hgTniy++QNeuXW0TGREREVXIqYus1wGsvIw1f/58DBo0CGfOnIFGo8FHH32EM2fOYO/evdi5c6etYyQiIiIrGO7ECg+tvZewACvP7PTo0QPHjh2DRqNBREQEtm7dCl9fX+zbtw8dOnSwdYxERERkofQsDa4m5wEAWobwzI5VwsLCsHLlSlvGQkRERDZy+pL+ElawnwoerlZ/3TuECu19cnIykpOTodPpTNpbt25doaCIiIioYk4X1uuEN6rdZ3UAK5Odw4cPY9y4cTh79iyEECbTJBIJtFqtTYIjIiIi65wqrNdpVcvrdQArk52JEyeiadOm+PLLL+Hn58fbzYmIiKqRrBwNLiXmAqjdgwkaWJXsXLx4EWvXrkXjxo1tHQ8RERFV0NnL2RACqF9PiTruCnuHY3dW3Y3Vt29fHD9+3NaxEBERkQ0YnofFeh09q87sfPHFFxg3bhxOnTqFVq1aQaEwzRqHDh1qk+CIiIjIcqcucnydoqxKdvbt24c9e/Zg06ZNJaaxQJmIiMh+cvK1uHAjBwBHTjaw6jLWtGnT8Pjjj+PmzZvQ6XQmP0x0iIiI7Ofc5WzodIBvHSf4eDnZO5xqwapk586dO3jxxRfh5+dn63iIiIioAk7zlvMSrEp2HnnkEWzfvt3WsRAREVEFGR7+yVvO77KqZqdp06aYNWsWdu/ejYiIiBIFytOnT7dJcERERGS+fLUO8dcK63V4J5aR1Xdjubm5YefOnSWeci6RSJjsEBER2UHclWxotALeHgr4e7Nex8CqZCchIcHWcRAREVEFFa3X4dMN7rKqZoeIiIiqH9brlM7sMzvR0dF455134Orqiujo6HLn/eCDDyocGBEREZlPrdEh7krhmR3W65gwO9k5evQo1Gq18XciIiKqPuKv5aBAI+DpKkcDH6W9w6lWzE52it5qztvOiYiIqhdDvU4463VKsKpmZ+LEicjMzCzRnp2djYkTJ1Y4KCIiIrKM4eGffERESVYlO2vWrEFubm6J9tzcXHz99dcVDoqIiIjMp9UKnLtkOLPDZKc4i249z8jIgBACQghkZmZCpVIZp2m1WmzcuBG+vr42D5KIiIjKdvFGLnILdHBVyRDsr7r3ArWMRcmOl5cXJBIJJBIJmjZtWmK6RCLB3LlzbRYcERER3ZvhElZ4qCukUtbrFGdRsrN9+3YIIdCnTx+sXbsW3t7exmlOTk4IDg5GYGCgzYMkIiKisp1O4Pg65bEo2enVqxcA/QjKDRs2ZLU3ERGRnel0osjIyUx2SmPV4yKCg4NtHQcRERFZ4XJiHrLztHB2kqJRoLO9w6mW+LgIIiKiGsxwCat5iCtkMl5xKQ2THSIiohqM4+vcG5MdIiKiGkoIYTJyMpWOyQ4REVENde1WPtKzNXCSS9CkgYu9w6m2rEp2kpKS8MQTTyAwMBByuRwymczkx5auX7+Oxx9/HHXr1oWzszMiIiJw6NAh43QhBN5++20EBATA2dkZ/fr1Q3x8vE1jICIiqo5OXdRfwmrW0BUKOc9flMWqu7HGjx+PK1eu4K233kJAQECl3YKempqK7t2744EHHsCmTZvg4+OD+Ph41KlTxzjPokWL8PHHH2PNmjUIDQ3FW2+9hcjISJw5c8ZkhGciIiJHYyhObtWI9TrlsSrZ2b17N/755x+0bdvWxuGYWrhwIYKCgrBq1SpjW2hoqPF3IQSWLl2KN998E8OGDQMAfP311/Dz88P69esxevToSo2PiIjIXoQQOMV6HbNYdc4rKCgIQghbx1LC77//jo4dO2LkyJHw9fVFu3btsHLlSuP0hIQEJCYmol+/fsY2T09PdO7cGfv27av0+IiIiOwlMaUAKRlqyGUSNGvIZKc8Vp3ZWbp0KWbOnIn//e9/CAkJsXFId128eBHLly9HdHQ0Xn/9dRw8eBDTp0+Hk5MTxo0bh8TERACAn5+fyXJ+fn7GaaXJz89Hfn6+8X1GRgYAQK1WQ61W2yx+w7psuU5HxH6yHPvMfOwr87GvzFcd+up4fDoAoHF9Z0ihhVqttVss5anMvjJ3nRJhxSmaOnXqICcnBxqNBi4uLlAoFCbTU1JSLF1lqZycnNCxY0fs3bvX2DZ9+nQcPHgQ+/btw969e9G9e3fcuHEDAQEBxnlGjRoFiUSCn376qdT1zpkzp9QHln7//fdwcWE1OxERVX+7ztdF/C03tKmfjo4N0+wdjl3k5ORg7NixSE9Ph4eHR5nzWX1mpyoEBASgZcuWJm0tWrTA2rVrAQD+/v4A9HeHFU12kpKSyq0nmjVrFqKjo43vMzIyEBQUhAEDBpTbWZZSq9WIjY1F//79SySEdBf7yXLsM/Oxr8zHvjJfdeirP5b8B0CNIX1bo12T6lugXJl9Zbgycy9WJTvjxo2zZjGLde/eHXFxcSZt//33n/HZXKGhofD398e2bduMyU1GRgYOHDiA5557rsz1KpVKKJXKEu0KhaJSDtrKWq+jYT9Zjn1mPvaV+dhX5rNXX91KK0BymhpSKdAqzAMKhW2HfakMldFX5q7PqmQHALRaLdavX4+zZ88CAMLDwzF06FCbjrPz4osvolu3bpg/fz5GjRqFf//9F59//jk+//xzAIBEIsGMGTPw7rvvokmTJsZbzwMDAzF8+HCbxUFERFSdGB4RERboAhdl9U907M2qZOf8+fMYPHgwrl+/jmbNmgEAFixYgKCgIGzYsAFhYWE2Ce6+++7DunXrMGvWLMybNw+hoaFYunQpoqKijPO8+uqryM7OxuTJk5GWloYePXpg8+bNHGOHiIgcluEREa0a8S4sc1iV7EyfPh1hYWHYv38/vL29AQB37tzB448/junTp2PDhg02C/Chhx7CQw89VOZ0iUSCefPmYd68eTbbJhERUXV2+iIf/mkJq5KdnTt3miQ6AFC3bl3ExMSge/fuNguOiIiITKVmqnH9dj4kEqBFCM/smMOqQQWVSiUyMzNLtGdlZcHJyanCQREREVHpDI+ICPF3hpuz1aW3tYpVyc5DDz2EyZMn48CBAxBCQAiB/fv349lnn8XQoUNtHSMREREVMjwiohUfEWE2q5Kdjz/+GGFhYejatStUKhVUKhW6d++Oxo0b46OPPrJ1jERERFTIUK8Tzod/ms2q819eXl747bffEB8fj3PnzgHQD/bXuHFjmwZHREREd2Vka3A5KQ8AEB7CZMdcFbrY16RJEzRp0sRWsRAREVE5zlzSn9UJ8lXB0431OuYyu6eio6PxzjvvwNXV1eRRC6X54IMPKhwYERERmTKMrxPOeh2LmJ3sHD161Ph00aNHj1ZaQERERFQ6w8jJrVivYxGzk53t27eX+jsRERFVvuw8LRJu5AIAwjmYoEWsuhtr4sSJpY6zk52djYkTJ1Y4KCIiIjJ19lI2dAIIqOuEuh58UKslrEp21qxZg9zc3BLtubm5+PrrryscFBEREZkyDCbIszqWs6iUOyMjwziIYGZmpsnDNrVaLTZu3AhfX1+bB0lERFTbGet1mOxYzKJkx8vLCxKJBBKJBE2bNi0xXSKRYO7cuTYLjoiIiIC8Ai3OX8sBwDuxrGFRsrN9+3YIIdCnTx+sXbvW5EGgTk5OCA4ORmBgoM2DJCIiqs3OXc6BVgf4eCng5620dzg1jkXJTq9evQAACQkJaNiwISQSSaUERURERHedYr1OhVg1/OLly5dx+fLlMqf37NnT6oCIiIjI1GnW61SIVclO7969S7QVPcuj1WqtDoiIiIjuKlDr8N9V1utUhFW3nqemppr8JCcnY/PmzbjvvvuwdetWW8dIRERUa/13NQdqjUAddzkC67FexxpWndnx9PQs0da/f384OTkhOjoahw8frnBgREREZFqvw1pZ61h1Zqcsfn5+iIuLs+UqiYiIajXW61ScVWd2Tpw4YfJeCIGbN28iJiYGbdu2tUVcREREtZ5GK3DuMut1KsqqZKdt27aQSCQQQpi0d+nSBV999ZVNAiMiIqrtzl/PQb5aB3cXGYJ8VfdegEplVbKTkJBg8l4qlcLHx8fk8RFERERUMacv3q3XkUpZr2Mtq5Kd4OBgW8dBRERExZxOyAbAS1gVZVWB8vTp0/Hxxx+XaP/0008xY8aMisZERERU62l1AmcusTjZFqxKdtauXYvu3buXaO/WrRt++eWXCgdFRERU2126mYucfB1cVVKEBDjbO5wazapk586dO6WOtePh4YHbt29XOCgiIqLa7lRhvU6LEDfIWK9TIVYlO40bN8bmzZtLtG/atAmNGjWqcFBERES1Het1bMeqAuXo6GhMnToVt27dQp8+fQAA27Ztw5IlS7B06VJbxkdERFTr6HQCp1mvYzNWJTsTJ05Efn4+3nvvPbzzzjsAgJCQECxfvhxPPvmkTQMkIiKqba4m5yEzRwuVkxRh9V3sHU6NZ1WyAwDPPfccnnvuOdy6dQvOzs5wc2PmSUREZAuG52E1D3aFXMZ6nYqy+tlYGo0Gf/31F3799VfjSMo3btxAVlaWzYIjIiKqjYz1OiGs17EFq87sXL58GQMHDsSVK1eQn5+P/v37w93dHQsXLkR+fj5WrFhh6ziJiIhqBSGE8U6sVo141cQWrDqz88ILL6Bjx45ITU2Fs/Pde/8ffvhhbNu2zWbBERER1TY3bucjLUsDhVyCJg1Yr2MLVp3Z+eeff7B37144OTmZtIeEhOD69es2CYyIiKg2OlV4CatZkAucFFZXm1ARVvWiTqeDVqst0X7t2jW4u7tXOCgiIqLa6nTC3Yd/km1YlewMGDDAZDwdiUSCrKwszJ49G4MHD7ZVbERERLUK63Uqh1WXsZYsWYLIyEi0bNkSeXl5GDt2LOLj41GvXj388MMPto6RiIioVkhKLcDtdDVkUqBZQ9br2IpVyU6DBg1w/Phx/PTTTzh+/DiysrIwadIkREVFmRQsExERkfkMt5w3aeAClZPMztE4DquSnVu3bsHHxwdRUVGIiooymXby5ElERETYJDgiIqLaxHAJi/U6tmVVzU5ERAQ2bNhQon3x4sXo1KlThYMiIiKqjYzPw2K9jk1ZlexER0djxIgReO6555Cbm4vr16+jb9++WLRoEb7//ntbx0hEROTw7qQXIPFOAaQSoEUwR062JauSnVdffRX79u3DP//8g9atW6N169ZQKpU4ceIEHn74YVvHSERE5PAM4+s0CnSGi4r1OrZk9WhFjRs3RqtWrXDp0iVkZGTgscceg7+/vy1jIyIiqjU4vk7lsSrZ2bNnD1q3bo34+HicOHECy5cvx7Rp0/DYY48hNTXV1jESERE5PCY7lceqZKdPnz547LHHsH//frRo0QJPPfUUjh49iitXrlTqnVgxMTGQSCSYMWOGsS0vLw9TpkxB3bp14ebmhhEjRiApKanSYiAiIrK1tCw1ribnAwDCQ1mvY2tWJTtbt25FTEwMFAqFsS0sLAx79uzBM888Y7Pgijp48CD+97//oXXr1ibtL774Iv744w/8/PPP2LlzJ27cuIFHHnmkUmIgIiKqDGcK63WC/VVwd7FqVBgqh1XJTq9evUpfmVSKt956q0IBlSYrKwtRUVFYuXIl6tSpY2xPT0/Hl19+iQ8++AB9+vRBhw4dsGrVKuzduxf79++3eRxERESV4VThJaxWvIRVKSxKdgYPHoz09HTj+5iYGKSlpRnf37lzBy1btrRZcAZTpkzBgw8+iH79+pm0Hz58GGq12qS9efPmaNiwIfbt22fzOIiIiCoD63Uql0XnyrZs2YL8/Hzj+/nz52PUqFHw8vICAGg0GsTFxdk0wB9//BFHjhzBwYMHS0xLTEyEk5OTcfsGfn5+SExMLHOd+fn5JvuRkZEBAFCr1VCr1bYJvHB9RV+pdOwny7HPzMe+Mh/7yny27KusXC0uJeYBAJoFKR2u/yvzuDJ3nRYlO0KIct/b2tWrV/HCCy8gNjYWKpXKZutdsGAB5s6dW6J969atcHGx/YPXYmNjbb5OR8R+shz7zHzsK/Oxr8xni766kuIMIXzhqVJj766tNoiqeqqM4yonJ8es+ap1FdThw4eRnJyM9u3bG9u0Wi127dqFTz/9FFu2bEFBQQHS0tJMzu4kJSWVO+bPrFmzEB0dbXyfkZGBoKAgDBgwAB4eHjaLX61WIzY2Fv379zcp5iZT7CfLsc/Mx74yH/vKfLbsqzWbE4G4O+jUyheDB7e1TYDVSGUeV4YrM/diUbIjkUggkUhKtFWWvn374uTJkyZtEyZMQPPmzfHaa68hKCgICoUC27Ztw4gRIwAAcXFxuHLlCrp27VrmepVKJZRKZYl2hUJRKf/AK2u9job9ZDn2mfnYV+ZjX5nPFn119nIuACAizN2h+70yjitz12fxZazx48cbE4W8vDw8++yzcHXVjwlQtA7GFtzd3dGqVSuTNldXV9StW9fYPmnSJERHR8Pb2xseHh6YNm0aunbtii5dutg0FiIiIlvLydfiwg39pRgWJ1cei5KdcePGmbx//PHHS8zz5JNPViwiC3344YeQSqUYMWIE8vPzERkZic8++6xKYyAiIrLGucvZ0OkAP28n+Hg52Tsch2VRsrNq1arKisNsO3bsMHmvUqmwbNkyLFu2zD4BERERWenURY6vUxWsfhAoERERVczpwpGT+YiIysVkh4iIyA7yC3SIv6av1+GZncrFZIeIiMgO4q5mQ6MVqOuhgJ8363UqE5MdIiIiOzDW6zRyq9RhXIjJDhERkV2wXqfqMNkhIiKqYmqNDnFX9MkO63UqH5MdIiKiKhZ/LQcFGgFPNznq+5Qc0Z9si8kOERFRFTtV5BIW63UqH5MdIiKiKnbaUJwcwktYVYHJDhERURXSagXOXi48s9OIyU5VYLJDRERUhS7cyEFegQ5uzjIE+6nsHU6twGSHiIioChluOW8Z4gqplPU6VYHJDhERURUyDCYYzlvOqwyTHSIioiqi1QmcuXR35GSqGkx2iIiIqsjlxFxk5+ngrJSiUYCzvcOpNZjsEBERVRFDvU6LYFfIZKzXqSpMdoiIiKpI0Yd/UtVhskNERFQFhBA4fYnFyfbAZIeIiKgKXE3OR0a2Fk4KCRrXZ71OVWKyQ0REVAVOJ+jP6jRv6AqFnF+/VYm9TUREVAVOFSY7rXgJq8ox2SEiIqpkQgjjnVjhoa52jqb2YbJDRERUyW7eKUBKhhpymQRNGzLZqWpMdoiIiCqZoV6naZALlAp+9VY19jgREVElY72OfTHZISIiqmSs17EvJjtERESV6FZaAZJTCyCVAs2DmezYA5MdIiKiSmS4hNW4vguclTI7R1M7MdkhIiKqRKcv8hER9sZkh4iIqBKdKqzXacV6HbthskNERFRJUjLUuHE7HxIJ0CKEyY69MNkhIiKqJIbxdUL8neHmLLdzNLUXkx0iIqJKYrjlvFUjntWxJyY7RERElcRwJxaLk+2LyQ4REVElyMjW4EpSHgAgPITJjj0x2SEiIqoEhnqdIF8VPN1Yr2NPTHaIiIgqAet1qg8mO0RERJWAD/+sPpjsEBER2Vh2nhYJN3MBsDi5OmCyQ0REZGNnLmVBCCCwnhLeHgp7h1PrMdkhIiKyMUO9TjhHTa4WmOwQERHZ2KnCh3+2asRLWNUBkx0iIiIbys3X4sL1HACs16kumOwQERHZ0LkrOdDqAF8vBXzrONk7HAKTHSIiIps6zUdEVDvVOtlZsGAB7rvvPri7u8PX1xfDhw9HXFycyTx5eXmYMmUK6tatCzc3N4wYMQJJSUl2ipiIiGo7Q71OOOt1AAA5t/KQHat/tZdqnezs3LkTU6ZMwf79+xEbGwu1Wo0BAwYgOzvbOM+LL76IP/74Az///DN27tyJGzdu4JFHHrFj1EREVFsVqHX476q+XoeDCerl3spH7l/6V3up1g/r2Lx5s8n71atXw9fXF4cPH0bPnj2Rnp6OL7/8Et9//z369OkDAFi1ahVatGiB/fv3o0uXLvYIm4iIaqn/ruZAoxXwdpcjoC7rdaqLap3sFJeeng4A8Pb2BgAcPnwYarUa/fr1M87TvHlzNGzYEPv27WOyQ0REVepUkXodiURi52jsJyc5Dzm38iCEwLnvLwEA7pxJh1yuTztcfFRw8VVVWTw1JtnR6XSYMWMGunfvjlatWgEAEhMT4eTkBC8vL5N5/fz8kJiYWOa68vPzkZ9/93RaRkYGAECtVkOtVtssZsO6bLlOR8R+shz7zHzsK/Oxr8xXVl+dvJAJAGge7Fyr+/H0DxdwYvl5k7b9c08Zf2/9XGO0fb5phbdjbh/XmGRnypQpOHXqFHbv3l3hdS1YsABz584t0b5161a4uLhUeP3FxcbG2nydjoj9ZDn2mfnYV+ZjX5mvaF9pdcCZS0EApLhz5TA23q6dyY4uC8jaU6RBBkALuA4HFA31Tdfcz+PGxvOlLG2ZnJwcs+arEcnO1KlT8eeff2LXrl1o0KCBsd3f3x8FBQVIS0szObuTlJQEf3//Mtc3a9YsREdHG99nZGQgKCgIAwYMgIeHh83iVqvViI2NRf/+/aFQ8NkoZWE/WY59Zj72lfnYV+Yrra/iruRAeyABHi4yjB3Rr9ZdxtKpdTj342UcXx4PdaYGANBoaH00HOCLHVOPosfIzvBrXdem2zRcmbmXap3sCCEwbdo0rFu3Djt27EBoaKjJ9A4dOkChUGDbtm0YMWIEACAuLg5XrlxB165dy1yvUqmEUqks0a5QKCrlH3hlrdfRsJ8sxz4zH/vKfOwr8xXtq3NX9bdWtwx1g5NT7SpOvrHvFva9dxJp5/WX8eq29ETXN1vDr703Eo/fBgDI5XKbH1fmrq9aJztTpkzB999/j99++w3u7u7GOhxPT084OzvD09MTkyZNQnR0NLy9veHh4YFp06aha9euLE4mIqIqZRhMsDbdcp55LQf/LjqFS1tvAgBUdZzQ4cUWaDoiGFKZ/syWs48Szv30r/ZSrZOd5cuXAwB69+5t0r5q1SqMHz8eAPDhhx9CKpVixIgRyM/PR2RkJD777LMqjpSIiGozrVbgzKXCJ52HOv6TzjV5Wpz4Ih4nVsZDm6+DRCZBizEhaD+tOZSepme1XHxUcO2vf7WXap3sCCHuOY9KpcKyZcuwbNmyKoiIiIiopISbucjN18FVJUVIgLO9w6k0Qghcjr2JAzGnkHUjFwAQ0KkeurwRAe9mtqt5tbVqnewQERHVBIbxdVqEuEEmdczC5NTzGdj/3knc2KevwXENcEbn18IREhlY7YuxmewQERFVkCPX6xRkqnHk03M4820ChFZA5iRFxFON0fqpJlC41Iw0omZESUREVE3pdI5ZryN0AvHrruDgB2eRd0c/EG9wP390eq0VPIJq1n4y2SEiIqqAK8l5yMzRQuUkRVh92w9Maw/Jx1Ow792TuH0yDQDgGeqGLm9EoEEPX/sGZiUmO0RERBVw6mJhvU6wK+Sy6l27ci+5t/NwcMkZxK+7CgBQuMrRbkoztHy8EWROUjtHZz0mO0RERBVwOqHmX8LSqXU4810Cjnx6Duos/ejHTR4OQsfolna9ZdxWmOwQERFZSQhhvBOrVaOaWZx8fU8y9s8/ibQL+v2o18oLXd+MgG9bbztHZjtMdoiIiKx043YB0rM0cJJL0KRBzarXybyWjQMLT+NybOHox95O6BjdEk0faQiJg90+z2SHiIjISqcL78Jq1tAVCnnNqGnR5GpwYuV5nPjy7ujHLaNC0W5qcyg9HPOZaEx2iIiIrHT2Ug6AmlGvI4TApS03cWDRKWQbRj/uUg9d34hAnSbVd/RjW2CyQ0REZAUh7p7ZCa/mgwmmxmdg33sncXN/4ejHgc7o/ForhAwIqPajH9sCkx0iIiIrZOXLcSdDA7lMguYNq+eZnfwMNY58cg5nvy8c/VgpReunmqD1U40hd649KUDt2VMiIiIbupmhBAA0buACZTUbg0boBP5bewWHPjiDvNQCAEBI/wB0eq0V3GtYIbUtMNkhIiKyQmKGfvyZ6lavk3yscPTjU2kAAK8wN3R5PQL1u9fM0Y9tgckOERGRFRILz+xUl4d/5tzKw6ElZxC/vnD0Yzc52k9tjpZRoZAqqteZp6rGZIeIiMhCd9LVyMxXQCrRPybCnrQFOpz59iKOLouDOrtw9ONHGuK+6BZwrlfzRz+2BSY7REREFjp4NgMAEOSnhItKZrc4ru1Oxv73TiK9cBTnehFe6PZWa/i0rmO3mKojJjtEREQWOln4PKyGvvY5c5JxNRsHYk7hyrZEAICqrhL3RbdAk4cdb/RjW2CyQ0REZIGcfC3OX9cPyhcWWLXJjiZXg+Ofx+Pkl+ehLSgc/fjxRmg/tRmc3B1z9GNbYLJDRERUDrVGh7grOdh/Og0nLmThSlIedEI/TSIBLlzXj6Jcx10B70p63IIQAgmbb+DfRaeRfVOfaAV2rYcub0SgTmPHHv3YFpjsEBERFaHTCVxKzMXx81k4fj4TpxOyka/WlTrvqk1JAJIAAKP7+mFs/wCbx5MSl4H9753EzX/1ox+7BTqj88xWCO5fO0Y/tgUmO0REVKsJIZCYUoDj5zNx/HwWTl7MREa21mQeTzc5mjd0QbC/M5o2cMad9AIs/+0GnhsWiKYN9bee17HxZaT89AL96Mc/XLo7+vHTTdD6qSaQ27EouiZiskNERLVOWpYaJy5k4Xh8Jo5fyEJy4SjDBs5OUoQ3ckObxm5o09gdwX4qk7MocZf1d2M1ClQhrL5tRyTWaQX+W3sZhz88e3f04wGFox/beFu1BZMdIiJyeDn5WpxOyMKJ81k4dj4TlxPzTKbLZRI0a+iCNmHuaN3YDU2DXCGXVf0loqQjKdj/3gncPp0OAPBq7I6ub0QgsKtPlcfiSJjsEBGRw1FrdPjvao7x0tR/V7OhLVZ2ExrgbDxzEx7qCpWT+ZeG6rjL0a5BGuq42+ZrNCc5DweXnMb5364BAJzc5Wg3rTlajuHox7bAZIeIiGq84kXFZy5lI6/ANLvx93ZCm8buaNPYDRGN3OHpZv1XYB13BdoHpVe4TkdboMPpby7g2LI4qHO0gARoOqIhOr7YEs51lRVaN93FZIeIiGqkxDv5OH5Bn9ycuFBKUbGrHK0buxkvTfl7V6/k4do/SfrRjy/pByj0aVMHXd+MgE8ERz+2NSY7RERUIxiKik+cz8Sx8yWLilVOUrQK1RcVty4sKpZWw9GEM64Ujn78d5HRj19qiSbDgzj6cSVhskNERNVSbr4WpxOyC+tuMnGpWFGxTAo0a+hqvDTVpIELFPLqW9+iztHg+P/+w8mvLkCn1kEilyD88UZoN4WjH1c2JjtERFQtaLQC/13NxrH4TJy4kIW4K6UVFavQOswdbZu4o2WIK5yV1X+8GSEELm68joPvn0Z2YcIW2M0HXd+IgFeYu52jqx2Y7BARkV3odAKXE/Nw/EImTpzPwqmErBJFxX6GouIwN0SEucHLrWadAUmJS8e+d04i8dAdAIBbfRd0ntUKwX39OfpxFWKyQ0REVSYxJR/Hz+vrbk5cyEJ6tsZkuqerHK3D9DU3bcLc4F9D70jKTyvA4Y/P4dyPCRA6QKaSoc3kJoiY2JijH9sBkx0iIqo06VkanLiQabxrKimlZFFxeGhh3U2YO4L9q2dRsbl0WoH/fr6MQ0vPIj9Nv6+hAwNx3yvhHP3YjpjsEBGRzeTma3HmkqHuJhMJN0svKm4d5oa2TdyrfVGxJZKO3MG+d0/izpnC0Y+bFI5+3IWjH9sbkx0iIrKaoajYMJhfaUXFIf6qwjum3NEy1BUuNaCo+F5ybuUhOxbIuS8PcpkGBxefwYU/Ckc/9lCg/bTmaDEmBFIHSeRqOiY7REQEAEjNVOPIVU90zVTD17v0QmCdTuByUh5OFD6G4XRCFnKLFRX71nFC28KxblrXwKJic+TeykfuX8Ap3ws4v/4aNIbRjx8NRscXW8C5mg1gWNsx2SEiIgBAaqYGR695ITVTA1/vu+1JhUXFxy8UFhVnmRYVe7jK0DrM3XjXVE0tKi6PEAIF6Wrk3M5D7u18XNufDAA49/1lABz9uLpjskNERCaycrXYfSLVeGkqsVhRsVJRpKi4sRtC/J1rZFGxEALqbA1yb+cX/ugTmZwivxvb7+RDpxYl1iF3lSFiYmME9fKDq6+zHfaCzMFkh4ioFhBCIDdfh6xcLbJztcjK1SA7T4uklALcTi9ATr4ON27pi4nnrr5ssqxUCjQLcjU+IbxpUPUuKtbkapB7Jx85t/KRe6cwWSnye86tfOTd0Sc12jztvVdY3raytTj6SRyOfhKHdlOaof205jbaC7IlJjtERDWEWqNDdp62SMKiLeO9BtnF2rJztdCVPDFxTz3beOH5R4LsXlSsLdDdTVwMZ1zu6JOYnNuFicwt/TR1sbF77kXhIoOqngou9ZRwNv6ojL+7FP6u0+kvZQFA8skU7JtzEl3nRMA3Qn/Nz8VHZfP9JttgskNEDs2cotuqIoRAboHubhJifNWUk7jcfZ+v1t17I/cgl0ng5iyDq7MMbs4yOMklkMskcFHJkZOnwdH4LIwb6Ie2TTwBAHXcFZWW6Oi0AnkpZlxCup2P/MIkw1wyJ6k+WfEpTFrqFiYtPvrfjclMXSUUrhZ8FTbQv2g0+oTKu4UH6oV7WRQbVT0mO0Tk0MoqurWWRitMLgOVlpwUv1RkeJ+dp4Wu4vkKXJTSIgmLHK6qu8lLiVdV4TxFkpuyHlNw8sgt5Px+DU1H1UeYlQPgCZ1AfnpBkUtFZV9CykvJByw42ySRSwqTFqXpmZi6xZIaHxUUbnI+joGMmOxUour0F2V1xn6yHPvMfAV38tHsbBoK7uQDwfqzK3kFumJJSSmXf/K0Jc/A5GlLPLvJGnKZpFgyUjI5Kdmmf++ikkFWScXABSkFaB6XjoJiBclCCBRkako941L8ElJuSj6ExpIMBvpkpW7Zl48M7UpPBSTVpBDa2UcJ5376V6r+mOxUoqRL2ciJBZK6ZcPXm8OEl4X9ZDlbn62oLFqdgFYroCnyo9UJqDWFvxvadQIajQ4aHaDR6qDRFM6nLbm8ybrKaM/J0yIvX6df75VMtIxLx7KvLyP995vIzddZVbtSnItSWubZkxIJjPFVDjeVDE6Kss+uWEMIAaEV0Kl10Bbo9K9q/XvTNtP3xrYC/WtGnH7k38SfLyP212smyY3WwiRP6akwOdviXE8Fl6KXkAp/V9VxqpED77n4qODan3U6NQWTnUpU1l9JZKo29ZMQAkIUnrkXgE7ov3V1Qv9eQBT5XX9JQAD6ZQqnCwHcuZqDZmfTkHgpCy7OTtBoSk8aDMlAaUmDtrC9vKTB2oTE8N4WSUVFeaap0RLQF+nm3f3CNp5dKbwEVHqiItdfMlJK4SyXwkUmgUomgUoqAXQC2gItdIVJRdGkQafWQZdZAO2duwlFZoEOaYZpBeKeiYe+zbCNu8lLWfNbcjnoXhJ33iq1XeEmL3kJqZ5p/YuLjwoqbyVkTjUvgSHH5TDJzrJly/D+++8jMTERbdq0wSeffIJOnTpVeRwpGWqkZuoL6bZvuI46AGJ/uYqde+8AkEAuB5wU0pL/Lxn+yhMAJKX8vyXRf9kZCEnRNolxHuP0wjWIUv56NK5GcveN6XyFS0uKzCgKt1l0+8XjKb7+Ir+LYvunVuug1uqnam9koxGAXzbexNoTGRASQCaXQK6QQSe5u6+6wo0IcfdLVJ8ElJ4Q3P0RRZIL/S9FEwpdYfZRenKhf6MzTC9MVgAU1l4Ik3lLxmO6PlvxTMtH77h0fPHDVaR7JdtuxZVJCMgFoJAAcon+VSGR3H0P/X9IcgkghwQyISCXFL4CkAGQCkAmhMnvUgFIdPpXqRCQ6gB1lhrabA0kOoGC1HwAQO8LGXD2cIJUCCidpHBSSIskDQK6Aq3J2ZDswgTF1olEVZEqpJApJJA6yfSvCqn+x0kKWeHvMoUUWYm5yLqWU+Z6mo5oiLbPNYNzPSWf1k01lkMkOz/99BOio6OxYsUKdO7cGUuXLkVkZCTi4uLg6+tbpbFs2noDm/9KBAB02af/EvLZkVSlMdRUjTdeL3e6gD7xERJJ4WuR36WFrwB0Uok+PyucppMAMP5eyrLWtkkBgSJtxhgKp1VgG6XFDIl+34QEUOXq7wRxydZASCSQ6gSc5RK4K2WQS/SJgQwCckggL0wOZKLwB4VJQWFyICt8leiKv+p/178KQHv31fAjiv0utDoITeHvGh10GgGhEdBpKi9hMJyrKW20FIEi/8ldzUEu9F/q2RXcplQhhcxJejehKPLe8HN3ummCof+95DIl5iu+DoUEMicZpIZlS51H/15SThFycTnJecgpHF+nrNupXXx5qYZqNodIdj744AM8/fTTmDBhAgBgxYoV2LBhA7766ivMnDmzSmMJuZSF3jsSy57BTQ6JR7GC0tJOhRhPudw9wVL0zIDEOFnop5eyDv00/ekfIUxO/BRZlzBuw2TTpcRRaryGlZVoK2XZomeCNLq731JmkqBwv0V5wdQunQ7ettm6BEpPGCqTVCGBVC6FVH73VaKQlmgzJhHF2+TFli9cn7ZAB22BFlKZFLeuZSNl323U6+WDwGaekMmlUHrrL7kYkoS7ScfdJKSspMOSRKImcPG9m8zwdmpyVDU+2SkoKMDhw4cxa9YsY5tUKkW/fv2wb9++UpfJz89Hfn6+8X1GRgYAQK1WQ622bCyH4lqNbYiw/n4AgLN7knHx43g0mt4ELbrrzzA5+yhZ0Ab9E4Nzb+k/g6L91LybD4ROwNlbCWUdJ/1lJcNZAx2g0+ggdDCeRdAZfzfMU1ioWfz3Iu+FVkCn059xKG3+u/NAf6ZCq7+8pdMUm0dXZLly1lVi3UX3Q1d4BqTofpQRd0G2Brr8sjNEhbtc/wUul0Ai1589kBRPGgwJRRnJgsQkmTDMU8qypSQcJu1Fk4/CdUiKtsslkMiqJmmI35+Mfftuo8njwWjSpSJnegW00AKWjVdXoxiSHY1GU+H/Cx2doX/YT/dWmX1l7jprfLJz+/ZtaLVa+Pn5mbT7+fnh3LlzpS6zYMECzJ07t0T71q1b4eJiu7uBbufou/diTgIyLsXrGy/ZbPUOw6SfLhf209Uq2LC08MfO/wok0F9auhddBqDL1P+edlEG/KkFHpLBq5H+fIzUXQOph+k3seFsjc3P2BhWmn+vGe1PU3h19OzJQ4hPsW8s1Z0uA3DuBxw8cwDSa/aOpmaIjY21dwg1RmX0VU5O2fVmRdX4ZMcas2bNQnR0tPF9RkYGgoKCMGDAAHh4eNhsO/H7k7Hvi0Po2rltBf+idGzsJ8vF70/Gvj8Poeuwduyze8i4kYUtZ3ah94M94RHoZu9wqjW1Wo1Yj1j0798fCgXHbyqPWq1GbCz7yhyV2VeGKzP3UuOTnXr16kEmkyEpybQIOCkpCf7+/qUuo1QqoVSWHAhKoVDY9IOoH+YB5376V/5jKBv7yXLeHkrjK/usfB6BbnDtr39lX5nH1v8XOjL2lfkqo6/MXV+NHwjByckJHTp0wLZt24xtOp0O27ZtQ9euXe0YGQedMhf7yXIcvZWIyHw1/swOAERHR2PcuHHo2LEjOnXqhKVLlyI7O9t4dxaRo2GCSERkPodIdh577DHcunULb7/9NhITE9G2bVts3ry5RNEyERER1T4OkewAwNSpUzF16lR7h0FERETVTI2v2SEiIiIqD5MdIiIicmhMdoiIiMihMdkhIiIih8Zkh4iIiBwakx0iIiJyaEx2iIiIyKEx2SEiIiKH5jCDClaEEAKA+U9PNZdarUZOTg4yMjL4oLhysJ8sxz4zH/vKfOwr87GvzFeZfWX43jZ8j5eFyQ6AzMxMAEBQUJCdIyEiIiJLZWZmwtPTs8zpEnGvdKgW0Ol0uHHjBtzd3SGRSEqd57777sPBgwctmpaRkYGgoCBcvXoVHh4eNo25MpW3r5WxrW3bttmkn6yN29LlzJ3fnPnuNQ+PrYpvp6J9VZGYLVnWlvPyuKr8bdWU48qS+Sv6f5Y9jishBDIzMxEYGAiptOzKHJ7ZASCVStGgQYNy55HJZGV+SOVNAwAPD48a9R/HvfansrZV0X6yNm5LlzN3fnPmu9c8PLZstx1r+6oiMVuyrC3n5XFVdduq7seVJfNX9P8sex1X5Z3RMWCBspmmTJli1bSaqCr3x5bbsnZdli5n7vzmzHeveXhs2X87FVmXJcvacl4eV9V/W1V1XFkyf0X/z6rOxxUvY1WijIwMeHp6Ij09vUb9lVTV2E+WY5+Zj31lPvaV+dhX5qsOfcUzO5VIqVRi9uzZUCqV9g6lWmM/WY59Zj72lfnYV+ZjX5mvOvQVz+wQERGRQ+OZHSIiInJoTHaIiIjIoTHZISIiIofGZIeIiIgcGpMdG9i1axeGDBmCwMBASCQSrF+/3mS6EAJvv/02AgIC4OzsjH79+iE+Pt4+wdrJ8uXL0bp1a+OgUl27dsWmTZuM0z///HP07t0bHh4ekEgkSEtLs1+wdmKL4yglJQVRUVHw8PCAl5cXJk2ahKysrCrci8pnq2Npw4YN6Ny5M5ydnVGnTh0MHz68anagklXVcXTixAncf//9UKlUCAoKwqJFiyp712yqKo6j48ePY8yYMQgKCoKzszNatGiBjz76qJL3zDaq03H0888/o3nz5lCpVIiIiMDGjRst3h8mOzaQnZ2NNm3aYNmyZaVOX7RoET7++GOsWLECBw4cgKurKyIjI5GXl1fFkdpPgwYNEBMTg8OHD+PQoUPo06cPhg0bhtOnTwMAcnJyMHDgQLz++ut2jtR+bHEcRUVF4fTp04iNjcWff/6JXbt2YfLkyVW1C1XCFsfS2rVr8cQTT2DChAk4fvw49uzZg7Fjx1bVLlSqqjiOMjIyMGDAAAQHB+Pw4cN4//33MWfOHHz++eeVvn+2UhXH0eHDh+Hr64tvv/0Wp0+fxhtvvIFZs2bh008/rfT9q6jqchzt3bsXY8aMwaRJk3D06FEMHz4cw4cPx6lTpyzbIUE2BUCsW7fO+F6n0wl/f3/x/vvvG9vS0tKEUqkUP/zwgx0irD7q1KkjvvjiC5O27du3CwAiNTXVPkFVE9YcR2fOnBEAxMGDB43zbNq0SUgkEnH9+vUqi90eLDmW1Gq1qF+/fon5HVFlHUefffaZqFOnjsjPzzfO89prr4lmzZpV8h5Vrqo4jp5//nnxwAMPVDTUKmXP42jUqFHiwQcfNImnc+fO4plnnrFoH3hmp5IlJCQgMTER/fr1M7Z5enqic+fO2Ldvnx0jsx+tVosff/wR2dnZ6Nq1q73DqRHMOY727dsHLy8vdOzY0ThPv379IJVKceDAgSqPuSpYcywdOXIE169fh1QqRbt27RAQEIBBgwZZ/pdiDWSr42jfvn3o2bMnnJycjPNERkYiLi4OqampVbQ3tlOVx1F6ejq8vb1tEbbdVOVxtG/fPpPtGOax9PuTyU4lS0xMBAD4+fmZtPv5+Rmn1RYnT56Em5sblEolnn32Waxbtw4tW7a0d1g1gjnHUWJiInx9fU2my+VyeHt7O9yxVpFj6eLFiwCAOXPm4M0338Sff/6JOnXqoHfv3khJSanMsO3OVsdRYmJiqesouo2aoKqPo7179+Knn36q8ZeWq/I4KmseS48zJjtUZZo1a4Zjx47hwIEDeO655zBu3DicOXPG3mFRDVSRY0mn0wEA3njjDYwYMQIdOnTAqlWrIJFI8PPPP1dm2FTNVOVxdOrUKQwbNgyzZ8/GgAEDbLofdG9MdiqZv78/ACApKcmkPSkpyTittnByckLjxo3RoUMHLFiwAG3atKkxdybYmznHkb+/P5KTk02mazQapKSkONyxVpFjKSAgAABM/oJXKpVo1KgRrly5UinxVhe2Oo78/f1LXUfRbdQEVXUcnTlzBn379sXkyZPx5ptv2m4H7KQqj6Oy5rH0OGOyU8lCQ0Ph7++Pbdu2GdsyMjJw4MCBWl+votPpkJ+fb+8wagRzjqOuXbsiLS0Nhw8fNs7z999/Q6fToXPnzlUec1Wy5Fjq0KEDlEol4uLijG1qtRqXLl1CcHBwZYVYLdjqOOratSt27doFtVptnCc2NhbNmjVDnTp1qmhvbK8yjqPTp0/jgQcewLhx4/Dee+/ZPGZ7qMrjqGvXribbMcxj8fenReXMVKrMzExx9OhRcfToUQFAfPDBB+Lo0aPi8uXLQgghYmJihJeXl/jtt9/EiRMnxLBhw0RoaKjIzc21c+RVZ+bMmWLnzp0iISFBnDhxQsycOVNIJBKxdetWIYQQN2/eFEePHhUrV64UAMSuXbvE0aNHxZ07d+wcedWxxXE0cOBA0a5dO3HgwAGxe/du0aRJEzFmzBh77VKlsMWx9MILL4j69euLLVu2iHPnzolJkyYJX19fkZKSYq/dspmqOI7S0tKEn5+feOKJJ8SpU6fEjz/+KFxcXMT//ve/Kt9fa1XFcXTy5Enh4+MjHn/8cXHz5k3jT3Jysl322RLV5Tjas2ePkMvlYvHixeLs2bNi9uzZQqFQiJMnT1q0P0x2bMBwa2Lxn3Hjxgkh9LfpvfXWW8LPz08olUrRt29fERcXZ9+gq9jEiRNFcHCwcHJyEj4+PqJv377G/1SEEGL27Nml9uGqVavsF3QVs8VxdOfOHTFmzBjh5uYmPDw8xIQJE0RmZqYd9qby2OJYKigoEC+99JLw9fUV7u7uol+/fuLUqVN22Bvbq6rj6Pjx46JHjx5CqVSK+vXri5iYmKraRZuoiuOorHUEBwdX4Z5apzodR//3f/8nmjZtKpycnER4eLjYsGGDxfsjEUIIy84FEREREdUcrNkhIiIih8Zkh4iIiBwakx0iIiJyaEx2iIiIyKEx2SEiIiKHxmSHiIiIHBqTHSIiInJoTHaI7OzSpUuQSCQ4duyYvUMxOnfuHLp06QKVSoW2bdvaO5xqYc6cOZXWFxKJBOvXr6+UdTuKHTt2QCKRIC0tzd6hUA3EZIdqvfHjx0MikSAmJsakff369ZBIJHaKyr5mz54NV1dXxMXFlXguTW318ssvV7gvKjNhcnTdunXDzZs34enpae9QqAZiskMEQKVSYeHChUhNTbV3KDZTUFBg9bIXLlxAjx49EBwcjLp169owqoqryH5VhJubW7XrC0totVrodLoq366tPi8nJyf4+/uX+QeIvfaPagYmO0QA+vXrB39/fyxYsKDMeUr7q3zp0qUICQkxvh8/fjyGDx+O+fPnw8/PD15eXpg3bx40Gg1eeeUVeHt7o0GDBli1alWJ9Z87dw7dunWDSqVCq1atsHPnTpPpp06dwqBBg+Dm5gY/Pz888cQTuH37tnF67969MXXqVMyYMQP16tVDZGRkqfuh0+kwb948NGjQAEqlEm3btsXmzZuN0yUSCQ4fPox58+ZBIpFgzpw5pa6nd+/emD59Ol599VV4e3vD39+/xLxpaWl46qmn4OPjAw8PD/Tp0wfHjx8v0V9FzZgxA717977nfu3cuROdOnWCUqlEQEAAZs6cCY1GY3Z8QgjMmTMHDRs2hFKpRGBgIKZPn17qvgIlP39D7IsXL0ZAQADq1q2LKVOmmDzBuajVq1dj7ty5OH78OCQSCSQSCVavXm2cfvv2bTz88MNwcXFBkyZN8Pvvv5ssf6/Pv7TteXl54ffff0fLli2hVCpx5coV5Ofn4+WXX0b9+vXh6uqKzp07Y8eOHcblLl++jCFDhqBOnTpwdXVFeHg4Nm7caHYcpX1eY8eOxWOPPWYSn1qtRr169fD1118DAPLz8zF9+nT4+vpCpVKhR48eOHjwoHH+4pexyto/otIw2SECIJPJMH/+fHzyySe4du1ahdb1999/48aNG9i1axc++OADzJ49Gw899BDq1KmDAwcO4Nlnn8UzzzxTYjuvvPIKXnrpJRw9ehRdu3bFkCFDcOfOHQD6pKFPnz5o164dDh06hM2bNyMpKQmjRo0yWceaNWvg5OSEPXv2YMWKFaXG99FHH2HJkiVYvHgxTpw4gcjISAwdOhTx8fEAgJs3byI8PBwvvfQSbt68iZdffrnMfV2zZg1cXV1x4MABLFq0CPPmzUNsbKxx+siRI5GcnIxNmzbh8OHDaN++Pfr27YuUlBSL+rT4fl2/fh2DBw/Gfffdh+PHj2P58uX48ssv8e6775od39q1a/Hhhx/if//7H+Lj47F+/XpERERYFNf27dtx4cIFbN++HWvWrMHq1atNEpiiHnvsMbz00ksIDw/HzZs3cfPmTZMEYO7cuRg1ahROnDiBwYMHIyoqythP5n7+xeXk5GDhwoX44osvcPr0afj6+mLq1KnYt28ffvzxR5w4cQIjR47EwIEDjZ//lClTkJ+fj127duHkyZNYuHAh3NzcLIqj+OcVFRWFP/74A1lZWcZ5tmzZgpycHDz88MMAgFdffRVr167FmjVrcOTIETRu3BiRkZHlHiul7R9RqSx+dCiRgxk3bpwYNmyYEEKILl26iIkTJwohhFi3bp0o+k9k9uzZok2bNibLfvjhhyZPMB43bpwIDg4WWq3W2NasWTNx//33G99rNBrh6uoqfvjhByGEEAkJCQKAydN+1Wq1aNCggVi4cKEQQoh33nlHDBgwwGTbV69eFQCMTxru1auXaNeu3T33NzAwULz33nsmbffdd594/vnnje/btGkjZs+eXe56evXqJXr06FFiPa+99poQQoh//vlHeHh4iLy8PJN5wsLCxP/+9z8hhGnfG7zwwguiV69eJtspvl+vv/66aNasmdDpdMa2ZcuWCTc3N2Pf3yu+JUuWiKZNm4qCgoJy99Og+Odv+Kw1Go2xbeTIkeKxxx4zex0GAMSbb75pfJ+VlSUAiE2bNgkhzPv8i1u1apUAII4dO2Zsu3z5spDJZOL69esm8/bt21fMmjVLCCFERESEmDNnTqnrtPY4VKvVol69euLrr782to0ZM8bYV1lZWUKhUIjvvvvOOL2goEAEBgaKRYsWCSHuPoU7NTW1zP0jKgvP7BAVsXDhQqxZswZnz561eh3h4eGQSu/+0/Lz8zM5YyCTyVC3bl0kJyebLNe1a1fj73K5HB07djTGcfz4cWzfvh1ubm7Gn+bNmwPQ19cYdOjQodzYMjIycOPGDXTv3t2kvXv37lbtc+vWrU3eBwQEGPfr+PHjyMrKQt26dU3iTkhIMInZHMX36+zZs+jatatJ/Ub37t2RlZVlcsasvPhGjhyJ3NxcNGrUCE8//TTWrVtnchnMHOHh4ZDJZKWu31JFY3V1dYWHh4dJX5rz+Rfn5ORkst6TJ09Cq9WiadOmJuvauXOncT3Tp0/Hu+++i+7du2P27Nk4ceKEcXlrj0O5XI5Ro0bhu+++AwBkZ2fjt99+Q1RUlHFZtVptclwqFAp06tSp3OOy+P4RlUVu7wCIqpOePXsiMjISs2bNwvjx402mSaVSCCFM2kqrz1AoFCbvJRJJqW2WFFNmZWVhyJAhWLhwYYlpAQEBxt9dXV3NXqctlLdfWVlZCAgIMKkHMfDy8gJgfp9au1/lxRcUFIS4uDj89ddfiI2NxfPPP4/3338fO3fuLLGcNeu3Zazmfv7FOTs7mySEWVlZkMlkOHz4sEmSBsB4qeqpp55CZGQkNmzYgK1bt2LBggVYsmQJpk2bVqHjMCoqCr169UJycjJiY2Ph7OyMgQMHlhm7OYrvH1FZmOwQFRMTE4O2bduiWbNmJu0+Pj5ITEyEEML4H6wtx8bZv38/evbsCQDQaDQ4fPgwpk6dCgBo37491q5di5CQEMjl1v+z9fDwQGBgIPbs2YNevXoZ2/fs2YNOnTpVbAeKad++PRITEyGXy02KuIvy8fHBqVOnTNqOHTt2z2SjRYsWWLt2rclnsWfPHri7u6NBgwZmx+js7IwhQ4ZgyJAhmDJlCpo3b46TJ0+iffv2Zq/DEk5OTtBqtRYvZ6vPv127dtBqtUhOTsb9999f5nxBQUF49tln8eyzz2LWrFlYuXIlpk2bVqE4unXrhqCgIPz000/YtGkTRo4cafycw8LCjDU+wcHBAPRJ78GDBzFjxgyr95fIgJexiIqJiIhAVFQUPv74Y5P23r1749atW1i0aBEuXLiAZcuWYdOmTTbb7rJly7Bu3TqcO3cOU6ZMQWpqKiZOnAhAXzSakpKCMWPG4ODBg7hw4QK2bNmCCRMmWPzl+corr2DhwoX46aefEBcXh5kzZ+LYsWN44YUXbLYvgP4Ot65du2L48OHYunUrLl26hL179+KNN97AoUOHAAB9+vTBoUOH8PXXXyM+Ph6zZ88ukfyU5vnnn8fVq1cxbdo0nDt3Dr/99htmz56N6Ohok0uI5Vm9ejW+/PJLnDp1ChcvXsS3334LZ2dn45dtZQgJCUFCQgKOHTuG27dvIz8/36zlbPX5N23aFFFRUXjyySfx66+/IiEhAf/++y8WLFiADRs2ANDfDbdlyxYkJCTgyJEj2L59O1q0aGGTOMaOHYsVK1YgNjbWeAkL0J8Jeu655/DKK69g8+bNOHPmDJ5++mnk5ORg0qRJZu8fUVmY7BCVYt68eSUuR7Ro0QKfffYZli1bhjZt2uDff/8t904lS8XExCAmJgZt2rTB7t278fvvv6NevXoAYDwbo9VqMWDAAERERGDGjBnw8vIy+8vdYPr06YiOjsZLL72EiIgIbN68Gb///juaNGlis30B9JdhNm7ciJ49e2LChAlo2rQpRo8ejcuXL8PPzw8AEBkZibfeeguvvvoq7rvvPmRmZuLJJ5+857rr16+PjRs34t9//0WbNm3w7LPPYtKkSXjzzTfNjs/LywsrV65E9+7d0bp1a/z111/4448/KnUsnREjRmDgwIF44IEH4OPjgx9++MGs5Wz5+a9atQpPPvkkXnrpJTRr1gzDhw/HwYMH0bBhQwD68WqmTJmCFi1aYODAgWjatCk+++wzm8QRFRWFM2fOoH79+iXqxmJiYjBixAg88cQTaN++Pc6fP48tW7agTp06Fu0fUWkkovgFcyIiIiIHwjM7RERE5NCY7BAREZFDY7JDREREDo3JDhERETk0JjtERETk0JjsEBERkUNjskNEREQOjckOEREROTQmO0REROTQmOwQERGRQ2OyQ0RERA6NyQ4RERE5tP8Hfl8Pg3QH3ucAAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "N_NEURONS_LIST = np.logspace(1, 4, 7, dtype=np.int64)\n", "X = rng.uniform(size=(50_000, 1))\n", "\n", "numpy_scores = np.zeros(len(N_NEURONS_LIST))\n", "jax_scores = np.zeros(len(N_NEURONS_LIST))\n", "\n", "print(\"Running on device: \", jax.numpy.ones(1).device)\n", "\n", "for i, n_neuron in enumerate(N_NEURONS_LIST):\n", " print(f\"{n_neuron} neurons:\")\n", " params = dict(\n", " units=n_neuron, \n", " rc_connectivity=5/n_neuron,\n", " input_connectivity=5/n_neuron, \n", " ridge=10, \n", " )\n", " model = rpy.ESN(**params)\n", " start = time.time()\n", " model.fit(X, X).run(X)\n", " stop = time.time()\n", " numpy_scores[i] = stop - start\n", " print(f\"\\tNumPy time: {numpy_scores[i]:.3}s\", end=\"\")\n", "\n", " model = rjx.ESN(**params)\n", " start = time.time()\n", " model.fit(X, X).run(X)\n", " stop = time.time()\n", " jax_scores[i] = stop - start\n", " print(f\"\\tJax time: {jax_scores[i]:.3}s\")\n", "\n", "plt.figure()\n", "plt.title(\"Running time on 50K timesteps\")\n", "plt.plot(N_NEURONS_LIST, numpy_scores, \"-+\", color=\"#4d77cf\", label=\"NumPy backend\")\n", "plt.plot(N_NEURONS_LIST, jax_scores, \"-+\", color=\"#9c27b0\", label=\"JAX backend\")\n", "plt.xscale(\"log\")\n", "plt.xlabel(\"Number of neurons in the reservoir\")\n", "plt.xticks(N_NEURONS_LIST, N_NEURONS_LIST)\n", "plt.ylabel(\"Execution time (s)\")\n", "plt.legend()\n", "plt.grid()\n", "plt.show()" ] }, { "cell_type": "markdown", "id": "de7cd1f7", "metadata": {}, "source": [ "We can see a major improvement in execution time between the numpy and the jax models. This will be dependent on your task, model, and of course your hardware." ] }, { "cell_type": "markdown", "id": "f875da5a", "metadata": {}, "source": [ "### Differentiation of ReservoirPy nodes and models\n", "\n", "With the Jax backend of ReservoirPy, you can now compute the gradient of any pure-jax nodes or models.\n", "\n", "#### Jacobian of a Node\n", "\n", "Let's compute the jacobian of the edge of stability reservoir:" ] }, { "cell_type": "code", "execution_count": 10, "id": "f77425d9", "metadata": {}, "outputs": [], "source": [ "# create a simple ES2N jax node\n", "node = rjx.nodes.ES2N(5, proximity=0.7, sr=0.3, input_connectivity=0.5)\n", "# warm it up\n", "X = rng.uniform(0, 1, (200, 2))\n", "_ = node.run(X)" ] }, { "cell_type": "markdown", "id": "9ddc6ce4", "metadata": {}, "source": [ "Every node have their `_step` method. This is a pure function, that takes two arguments:\n", "\n", "0. The previous node state: this represents the state of the node as a dict of 1-D arrays. Its `\"out\"` key holds the previous output of the node.\n", "\n", "1. The input. This is a 1-D array (a timestep).\n", "\n", "This method returns a dict that represents the new state of the node. The `\"out\"` key holds the output of the node.\n", "\n", "Knowing this, we can now compute the gradient out of the input or the previous node state." ] }, { "cell_type": "code", "execution_count": 11, "id": "839306e6", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Jacobian from the previous node state\n", "[[ 0.39231694 0.01568126 -0.23276064 -0.04635323 0.00888896]\n", " [-0.10896099 0.13722752 -0.04822905 -0.17233549 -0.165138 ]\n", " [ 0.08274485 -0.09067826 0.0278561 0.10745058 -0.25021824]\n", " [ 0.12227463 -0.11532653 0.13048898 -0.21138966 0.00597949]\n", " [-0.15159406 -0.22226101 -0.12527217 -0.04382525 -0.00235007]] (5, 5)\n", "\n", "Jacobian from the input\n", "[[ 0. 0. ]\n", " [ 0.3467805 0.3467805]\n", " [ 0. 0. ]\n", " [ 0. -0.6393939]\n", " [ 0. 0. ]] (5, 2)\n" ] } ], "source": [ "# observation point\n", "x = rng.uniform(0, 1, (2,))\n", "\n", "# Jacobian of ES2N with respect to the previous ES2N neurons state\n", "jac_from_state = jax.jacobian(node._step, argnums=0)(node.state, x)[\"out\"][\"out\"]\n", "\n", "# Jacobian of ES2N with respect to the input\n", "jac_from_input = jax.jacobian(node._step, argnums=1)(node.state, x)[\"out\"]\n", "\n", "\n", "print(\"Jacobian from the previous node state\")\n", "print(jac_from_state, jac_from_state.shape)\n", "print()\n", "print(\"Jacobian from the input\")\n", "print(jac_from_input, jac_from_input.shape)" ] }, { "cell_type": "markdown", "id": "808b3583", "metadata": {}, "source": [ "##### Jacobian of a Model\n", "\n", "The jacobian of a Model follows the same logic, though it is a bit more complex since we are handling more states and outputs at the same time.\n", "\n", "Let's create a simple echo state network." ] }, { "cell_type": "code", "execution_count": 12, "id": "e24bed11", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Model(Reservoir(units=10, input_dim=2), Ridge(ridge=0.001, input_dim=10, output_dim=1))" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "reservoir = rjx.nodes.Reservoir(10)\n", "readout = rjx.nodes.Ridge(1e-3)\n", "model = reservoir >> readout\n", "\n", "Y = rng.uniform(0, 1, (200, 1))\n", "model.fit(X, Y)" ] }, { "cell_type": "markdown", "id": "ade6e3d3", "metadata": {}, "source": [ "Model also have their `_step` purely-functional method. It takes two arguments:\n", "\n", "0. The previous model state: this is a tuple where the first element is a mapping of the feedback buffers (we don't have any here), and a mapping of the node states.\n", "\n", "1. The input. This is a mapping of 1-D arrays (timesteps).\n", "\n", "This method returns a dict that represents the new state of the model as described above." ] }, { "cell_type": "code", "execution_count": 13, "id": "bf69f276", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{Ridge(ridge=0.001, input_dim=10, output_dim=1): {'out': Array([0.4705091], dtype=float32)},\n", " Reservoir(units=10, input_dim=2): {'out': Array([-0.3220906 , 0.04102203, -0.01304624, 0. , 0. ,\n", " 0. , 0.23301846, 0. , 0.29908204, 0. ], dtype=float32)}}" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Illustration of the _step method\n", "new_buffers, new_states = model._step(\n", " (model.feedback_buffers, {n: n.state for n in model.nodes}),\n", " {reservoir: x}\n", ")\n", "new_states" ] }, { "cell_type": "markdown", "id": "7d8a1c61", "metadata": {}, "source": [ "Now that we have this in mind, we can compute its jacobian!" ] }, { "cell_type": "code", "execution_count": 14, "id": "6d162e5a", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Array([[-0.1528503, -0.0186974]], dtype=float32)" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Jacobian of ESN model with respect to the input\n", "jac_from_input = jax.jacobian(model._step, argnums=1)(\n", " (model.feedback_buffers, {n: n.state for n in model.nodes}),\n", " {reservoir: x}\n", ")\n", "\n", "# output of the readout with respect to the input\n", "# we want to get the jacobian of the [1][readout][\"out\"] output relative to the [reservoir] input\n", "jac_from_input[1][readout][\"out\"][reservoir]" ] } ], "metadata": { "kernelspec": { "display_name": "dev", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.0" } }, "nbformat": 4, "nbformat_minor": 5 }