{ "cells": [ { "cell_type": "markdown", "id": "0bc149cb-bdc6-4a0e-970c-5687adaa6a79", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "# 5. Multi-Layer Perceptron (MLP)" ] }, { "cell_type": "markdown", "id": "a492ffe0-9a3f-4712-ae47-9b346bd92910", "metadata": {}, "source": [ "## Introduction" ] }, { "cell_type": "markdown", "id": "ff8aecde-9f54-4905-a4d5-942406df1e95", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "Implémentation d'un réseau de neurones inspiré du papier de Bengio et al. de 2003 [_A Neural Probabilistic Language Model_](https://www.jmlr.org/papers/volume3/bengio03a/bengio03a.pdf).\n", "\n", "On cherche toujours à construire un modèle de langue au niveau des caractères. Dans le papier ils ont un vocabulaire de 17000 mots et construisent un modèle de langue par mots, mais nous pouvons utiliser la même approche pour construire un modèle de langues par caractères.\n", "\n", "Ils associent à chaque mot un vecteur dans un espace, par exemple à 30 dimensions. Ces vecteurs sont initialisés à des valeurs aléatoires et ensuite on fait apprendre au modèle, par rétropropagation et descente de gradient, la \"meilleure\" position du mot dans cet espace. Les mots similaires doivent être ensuite \"proches\" dans cet espace. On cherche à maximiser la _log-vraisemblance_.\n", "\n", "Généralisation sur des phrases comme:\n", "\n", "- \"The cat is walking in the bedroom\"\n", "- \"A dog was running in a room\"\n", "- \"The cat is running in a room\"\n", "- \"A dog is walking in a bedroom\"\n", "- \"The dog was walking in the room\"\n", "\n", "Le schéma du réseau de Bengio 2003 est présenté ci-dessous. Avec un vocabulaire de 70000 mots, le réseau de neurones se compose d'une couche d'entrée, avec si on a un contexte de 3 mots précédents pour prédire le 4ème, 3x30 soit 90 neurones. La table C permet de récupérer le vecteur d'embedding de chaque mot de dimension 30, cette table est de taille 70000. La couche cachée est d'une taille qui est choisi arbitrairement (un hyperparamètre), par exemple 100. Le but va être de choisir le bon paramètre pour cette couche cachée. Chaque neurone est complètement connecté à tous les neurones de la couche d'entrée. La fonction non-linéaire utilisée en sortie est la tangente hyperbolique. La couche de sortie a 70000 neurones et est complètement connecté aux neurones (par ex. 100) de la couche cachée. La sortie se compose de logits qui seront transformés en probabilités par une fonction softmax (log -> exp -> normalisation). Optimisation des paramètres (poids et biais) par rétropropagation en maximisant la log-vraissemblance.\n", "\n", "![Réseau de BengioY 2003](BengioY2003_network.png)\n", "\n", "Source: \n", "\n", "Dans la suite de ce notebook, nous allons construire un réseau de neurones similaire à ce lui présenté dans le papier, de type _Feed Forward Network_, permettant d'obtenir un modèle de langue au niveau caractères. Ce type de réseau est également appelé _Multi-Layer Perceptron_ même si les neurones utilisés ici ne sont pas similaires à ceux d'un perceptron." ] }, { "cell_type": "markdown", "id": "656e7e8c-ab0d-441a-87ce-da92689d7c3d", "metadata": {}, "source": [ "## Données sources" ] }, { "cell_type": "markdown", "id": "ac476840-86d4-478a-85be-b69d4fe76ed7", "metadata": {}, "source": [ "Nous reprenons les 7223 mots du code civil, amenant 41 caractères différents dans notre vocabulaire, incluant le caractère spécial '`.`' indiquant le début ou la fin d'un mot. Les entiers associés à chaque caractères sont les _tokens_ qui seront manipulés par notre réseau." ] }, { "cell_type": "code", "execution_count": 1, "id": "4bdbb915-9997-4a4c-9ace-341ba56c8e97", "metadata": {}, "outputs": [], "source": [ "import torch\n", "import torch.nn.functional as F\n", "import matplotlib.pyplot as plt\n", "%matplotlib inline" ] }, { "cell_type": "code", "execution_count": 2, "id": "e756a16f-f8b4-4977-aa68-7b9add577c71", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\"'\": 1, '-': 2, 'a': 3, 'b': 4, 'c': 5, 'd': 6, 'e': 7, 'f': 8, 'g': 9, 'h': 10, 'i': 11, 'j': 12, 'l': 13, 'm': 14, 'n': 15, 'o': 16, 'p': 17, 'q': 18, 'r': 19, 's': 20, 't': 21, 'u': 22, 'v': 23, 'w': 24, 'x': 25, 'y': 26, 'z': 27, 'à': 28, 'â': 29, 'ç': 30, 'è': 31, 'é': 32, 'ê': 33, 'ë': 34, 'î': 35, 'ï': 36, 'ô': 37, 'ù': 38, 'û': 39, 'œ': 40, '.': 0}\n", "nb_words=7223\n", "nb_chars=41\n" ] } ], "source": [ "words = open('civil_mots.txt', 'r').read().splitlines()\n", "nb_words = len(words)\n", "chars = sorted(list(set(''.join(words))))\n", "nb_chars = len(chars) + 1 # On ajoute 1 pour EOS\n", "ctoi = {c:i+1 for i,c in enumerate(chars)}\n", "ctoi['.'] = 0\n", "itoc = {i:s for s,i in ctoi.items()}\n", "print(ctoi)\n", "print(f\"{nb_words=}\")\n", "print(f\"{nb_chars=}\")" ] }, { "cell_type": "markdown", "id": "1a7525a5-cba9-4f9d-a1c3-59e42b0ee860", "metadata": {}, "source": [ "## Construction du jeu de données pour l'entraînement" ] }, { "cell_type": "markdown", "id": "21c8e5d7-b9b0-446d-8042-2039daa9c0d1", "metadata": {}, "source": [ "### Fonction de construction" ] }, { "cell_type": "markdown", "id": "1919f37c-fe97-4887-b427-e76d89543dc6", "metadata": {}, "source": [ "Nous créons une fonction `build_dataset` qui va permettre de construire le jeu d'entraînement à partir d'une liste de mots et d'une taille de contexte. Ce jeu d'entraînement sera composé des entrées `X` avec un nombre de caractères pour chaque entrée dépandant de la taille du contexte, et des réponses `Y` attendues (un caractère)." ] }, { "cell_type": "code", "execution_count": 3, "id": "493d5ad7-e4e0-411d-a27e-e683a2ebfc1c", "metadata": {}, "outputs": [], "source": [ "def build_dataset(words:list, context_size:int, verbose:bool=False):\n", " \"\"\"Build the dataset of the neural net for training.\n", "\n", " Parameters:\n", " words: list of words of our data corpus\n", " context_size: how many characters we take to predict the next one\n", " verbose: print or not the inputs and labels during construction\n", " Returns:\n", " X: inputs to the neural net\n", " Y: labels\n", " \"\"\"\n", " X, Y = [], []\n", " if verbose:\n", " nb_examples = 0\n", " nb_characters = 0\n", " nb_words = len(words)\n", " for w in words:\n", " if verbose:\n", " print(w)\n", " nb_characters += len(w)\n", " context = [0] * context_size\n", " for ch in w + '.':\n", " ix = ctoi[ch]\n", " X.append(context)\n", " Y.append(ix)\n", " if verbose:\n", " print(''.join(itoc[i] for i in context), '--->', itoc[ix])\n", " nb_examples += 1\n", " context = context[1:] + [ix] # crop and append\n", " X = torch.tensor(X)\n", " Y = torch.tensor(Y)\n", " if verbose:\n", " print(X.shape, Y.shape)\n", " print(f\"{nb_examples=}\")\n", " print(f\"{nb_characters=}\")\n", " print(f\"{nb_words=}\")\n", " return X, Y" ] }, { "cell_type": "code", "execution_count": 4, "id": "b40ff6b1-9ca1-40c4-adec-e9cc252314ef", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "acceptée\n", "... ---> a\n", "..a ---> c\n", ".ac ---> c\n", "acc ---> e\n", "cce ---> p\n", "cep ---> t\n", "ept ---> é\n", "pté ---> e\n", "tée ---> .\n", "acceptées\n", "... ---> a\n", "..a ---> c\n", ".ac ---> c\n", "acc ---> e\n", "cce ---> p\n", "cep ---> t\n", "ept ---> é\n", "pté ---> e\n", "tée ---> s\n", "ées ---> .\n", "accessible\n", "... ---> a\n", "..a ---> c\n", ".ac ---> c\n", "acc ---> e\n", "cce ---> s\n", "ces ---> s\n", "ess ---> i\n", "ssi ---> b\n", "sib ---> l\n", "ibl ---> e\n", "ble ---> .\n", "accession\n", "... ---> a\n", "..a ---> c\n", ".ac ---> c\n", "acc ---> e\n", "cce ---> s\n", "ces ---> s\n", "ess ---> i\n", "ssi ---> o\n", "sio ---> n\n", "ion ---> .\n", "accessoire\n", "... ---> a\n", "..a ---> c\n", ".ac ---> c\n", "acc ---> e\n", "cce ---> s\n", "ces ---> s\n", "ess ---> o\n", "sso ---> i\n", "soi ---> r\n", "oir ---> e\n", "ire ---> .\n", "torch.Size([51, 3]) torch.Size([51])\n", "nb_examples=51\n", "nb_characters=46\n", "nb_words=5\n" ] } ], "source": [ "X, Y = build_dataset(words[40:45], 3, verbose=True)" ] }, { "cell_type": "code", "execution_count": 5, "id": "6d932336-9964-420d-a6a2-6d32280aa51b", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "acceptée\n", ".......... ---> a\n", ".........a ---> c\n", "........ac ---> c\n", ".......acc ---> e\n", "......acce ---> p\n", ".....accep ---> t\n", "....accept ---> é\n", "...accepté ---> e\n", "..acceptée ---> .\n", "acceptées\n", ".......... ---> a\n", ".........a ---> c\n", "........ac ---> c\n", ".......acc ---> e\n", "......acce ---> p\n", ".....accep ---> t\n", "....accept ---> é\n", "...accepté ---> e\n", "..acceptée ---> s\n", ".acceptées ---> .\n", "accessible\n", ".......... ---> a\n", ".........a ---> c\n", "........ac ---> c\n", ".......acc ---> e\n", "......acce ---> s\n", ".....acces ---> s\n", "....access ---> i\n", "...accessi ---> b\n", "..accessib ---> l\n", ".accessibl ---> e\n", "accessible ---> .\n", "accession\n", ".......... ---> a\n", ".........a ---> c\n", "........ac ---> c\n", ".......acc ---> e\n", "......acce ---> s\n", ".....acces ---> s\n", "....access ---> i\n", "...accessi ---> o\n", "..accessio ---> n\n", ".accession ---> .\n", "accessoire\n", ".......... ---> a\n", ".........a ---> c\n", "........ac ---> c\n", ".......acc ---> e\n", "......acce ---> s\n", ".....acces ---> s\n", "....access ---> o\n", "...accesso ---> i\n", "..accessoi ---> r\n", ".accessoir ---> e\n", "accessoire ---> .\n", "torch.Size([51, 10]) torch.Size([51])\n", "nb_examples=51\n", "nb_characters=46\n", "nb_words=5\n" ] } ], "source": [ "X, Y = build_dataset(words[40:45], 10, verbose=True)" ] }, { "cell_type": "markdown", "id": "6a2ea3a5-3b3b-4e70-b6fb-09f6994a026b", "metadata": {}, "source": [ "### Construction du jeu d'entraînement complet" ] }, { "cell_type": "markdown", "id": "a16809b2-314a-4e72-b254-cac48277e479", "metadata": {}, "source": [ "Les mots du code civil générent un jeu d'entraînement avec les entrées `X` de dimension 2 de forme (67652, 3), soit 67652 contextes de 3 caractères différents et pour les labels `Y` 67652 caractères suivants." ] }, { "cell_type": "code", "execution_count": 6, "id": "979f56cd-1923-4278-ab54-78ca3e05e66e", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "X.shape = torch.Size([67652, 3])\n", "Y.shape = torch.Size([67652])\n", "tensor([[0, 0, 0],\n", " [0, 0, 3],\n", " [0, 0, 0],\n", " [0, 0, 3],\n", " [0, 3, 4]])\n", "tensor([3, 0, 3, 4, 3])\n" ] } ], "source": [ "context_size = 3\n", "X, Y = build_dataset(words, context_size)\n", "print(\"X.shape =\", X.shape)\n", "print(\"Y.shape =\", Y.shape)\n", "print(X[:5])\n", "print(Y[:5])" ] }, { "cell_type": "markdown", "id": "747be2de-7d27-4684-b5f5-837e08d8e135", "metadata": {}, "source": [ "Pour la suite de la discussion, nous allons uniquement prendre 5 mots, représentant 53 exemples, afin d'avoir un jeu de données `Xd` et `Yd` plus petit qui se prête mieux à l'affichage:" ] }, { "cell_type": "code", "execution_count": 7, "id": "0a912994-2203-441c-9590-a9c80de2c4de", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Xd.shape = torch.Size([53, 3])\n", "Yd.shape = torch.Size([53])\n", "tensor([[ 0, 0, 0],\n", " [ 0, 0, 3],\n", " [ 0, 3, 4],\n", " [ 3, 4, 3],\n", " [ 4, 3, 15],\n", " [ 3, 15, 6],\n", " [15, 6, 16],\n", " [ 6, 16, 15],\n", " [16, 15, 15],\n", " [15, 15, 32],\n", " [ 0, 0, 0],\n", " [ 0, 0, 3],\n", " [ 0, 3, 4],\n", " [ 3, 4, 3],\n", " [ 4, 3, 15],\n", " [ 3, 15, 6],\n", " [15, 6, 16],\n", " [ 6, 16, 15],\n", " [16, 15, 15],\n", " [15, 15, 32],\n", " [15, 32, 7],\n", " [ 0, 0, 0],\n", " [ 0, 0, 3],\n", " [ 0, 3, 4],\n", " [ 3, 4, 3],\n", " [ 4, 3, 15],\n", " [ 3, 15, 6],\n", " [15, 6, 16],\n", " [ 6, 16, 15],\n", " [16, 15, 15],\n", " [15, 15, 32],\n", " [15, 32, 7],\n", " [32, 7, 20],\n", " [ 0, 0, 0],\n", " [ 0, 0, 3],\n", " [ 0, 3, 4],\n", " [ 3, 4, 3],\n", " [ 4, 3, 15],\n", " [ 3, 15, 6],\n", " [15, 6, 16],\n", " [ 6, 16, 15],\n", " [16, 15, 15],\n", " [15, 15, 32],\n", " [15, 32, 20],\n", " [ 0, 0, 0],\n", " [ 0, 0, 3],\n", " [ 0, 3, 4],\n", " [ 3, 4, 3],\n", " [ 4, 3, 15],\n", " [ 3, 15, 6],\n", " [15, 6, 16],\n", " [ 6, 16, 15],\n", " [16, 15, 20]])\n", "tensor([ 3, 4, 3, 15, 6, 16, 15, 15, 32, 0, 3, 4, 3, 15, 6, 16, 15, 15,\n", " 32, 7, 0, 3, 4, 3, 15, 6, 16, 15, 15, 32, 7, 20, 0, 3, 4, 3,\n", " 15, 6, 16, 15, 15, 32, 20, 0, 3, 4, 3, 15, 6, 16, 15, 20, 0])\n" ] } ], "source": [ "context_size = 3\n", "Xd, Yd = build_dataset(words[5:10], context_size)\n", "print(\"Xd.shape =\", Xd.shape)\n", "print(\"Yd.shape =\", Yd.shape)\n", "print(Xd)\n", "print(Yd)" ] }, { "cell_type": "markdown", "id": "eb3311c7-ca4f-4a7c-a8c7-342606468614", "metadata": {}, "source": [ "Nous avons donc `Xd` qui représentent des suites de trois tokens pour lesquels nous devrions obtenir à la fin les `Yd` correspondants." ] }, { "cell_type": "markdown", "id": "a60c17fe-c9d4-41ba-bb11-b747aef6bb89", "metadata": {}, "source": [ "## Discussion: construction du réseau de neurones" ] }, { "cell_type": "markdown", "id": "b89c510f-4978-4729-8904-e8ecaefb28f7", "metadata": {}, "source": [ "### Création de la matrice d'embeddings\n", "\n", "Cette matrice est ce qui est appelé la _lookup table_ dans [Bengio et al. 2003](https://www.jmlr.org/papers/volume3/bengio03a/bengio03a.pdf). Elle sera de taille `nb_chars x e_dims`, où `nb_chars` est le nombre de caractères différents de notre dataset (41) et `e_dims` le nombre de dimensions (ici 2) que nous choisirons pour représenter chaque caractère dans ce nouvel espace. Nous initialisons ici les valeurs de cette table de manière aléatoire." ] }, { "cell_type": "code", "execution_count": 8, "id": "ac56497f-600c-4075-b3c7-32295624a038", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "tensor([[ 0.4806, 0.9659],\n", " [-0.9354, 0.8722],\n", " [ 1.9186, 1.1885],\n", " [ 0.1595, -0.9922],\n", " [-0.3260, 0.9572],\n", " [ 2.2385, -1.2704],\n", " [-0.2007, -1.0860],\n", " [ 1.1423, -0.2115],\n", " [-0.7590, -0.6401],\n", " [ 0.0339, -0.4439],\n", " [-0.3950, -0.5841],\n", " [-0.6752, -0.6698],\n", " [ 0.2760, 0.0204],\n", " [ 1.0637, 1.7657],\n", " [-0.4892, -0.0236],\n", " [-0.4793, 0.3979],\n", " [-0.2206, 0.3598],\n", " [ 1.2669, 1.5495],\n", " [-0.8553, 0.5060],\n", " [ 1.1250, -1.6645],\n", " [ 0.8648, -0.6060],\n", " [ 0.8861, 0.9204],\n", " [-1.0055, 1.5119],\n", " [-1.2406, 0.1779],\n", " [ 0.8390, -0.1560],\n", " [ 1.1340, 1.0290],\n", " [-0.5181, -0.3230],\n", " [ 0.4742, 0.2750],\n", " [-0.0866, 0.3084],\n", " [-0.1322, -0.6673],\n", " [-0.4096, 1.3299],\n", " [ 1.5092, -0.4951],\n", " [-0.0418, 1.0244],\n", " [-0.3153, -0.5744],\n", " [ 0.3304, -0.9862],\n", " [-1.9956, -1.8532],\n", " [ 0.2311, -0.9675],\n", " [-0.6461, -0.8295],\n", " [-0.7013, 1.7916],\n", " [-1.1629, -1.5063],\n", " [-0.6191, 0.1512]])\n" ] } ], "source": [ "e_dims = 2 # Dimensions des embeddings\n", "C = torch.randn((nb_chars, e_dims))\n", "print(C)" ] }, { "cell_type": "markdown", "id": "06274080-5b61-4a55-b8a0-f9dc983bc561", "metadata": {}, "source": [ "Pour utiliser cette \"_lookup table_\" sous la forme d'une matrice, on peut l'indexer avec le numéro du caractère dont on souhaite obtenir l'embedding, par exemple ici à partir du token `5` correspondant au caractère `c`:" ] }, { "cell_type": "code", "execution_count": 9, "id": "1eacb79a-9bc1-4d82-88c1-0fb527121700", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "tensor([ 2.2385, -1.2704])\n" ] } ], "source": [ "emb_char_c = C[5] # 5: token correspondant à 'c'\n", "print(emb_char_c)" ] }, { "cell_type": "markdown", "id": "59af65d3-9789-4dc5-9fae-d82494edd415", "metadata": {}, "source": [ "On peut également remarquer que l'on peut indexer un caractère à partir d'un vecteur \"one-hot\" correspondant au même caractère, en multipliant la matrice d'embeddings par ce vecteur:" ] }, { "cell_type": "code", "execution_count": 10, "id": "ee32c50a-56c0-4af1-a165-ea0d31e32a3d", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "c_one_hot=tensor([0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n", " 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,\n", " 0., 0., 0., 0., 0.])\n" ] } ], "source": [ "c_one_hot = F.one_hot(torch.tensor(5), num_classes=nb_chars).float()\n", "print(f\"{c_one_hot=}\")" ] }, { "cell_type": "code", "execution_count": 11, "id": "35bc3f3d-f31d-46df-9cdc-75e1c853ad6c", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor([ 2.2385, -1.2704])" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "c_one_hot @ C" ] }, { "cell_type": "markdown", "id": "f311087f-5484-4fb1-8a5a-c41bd6cefcb3", "metadata": {}, "source": [ "La première couche linéaire de ce réseau de neurones, composé de cette matrice C, va être activée avec les indices des caractères.\n", "\n", "L'indexation avec des tenseurs Pytorch est très flexible, ce qui permet de récupérer plusiers lignes, par exemple correspondant à la suite de lettres c, a, b (soit 5,3,4):" ] }, { "cell_type": "code", "execution_count": 12, "id": "91fb0caf-5c39-47ec-8cc3-ae88a418dd3a", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor([[ 2.2385, -1.2704],\n", " [ 0.1595, -0.9922],\n", " [-0.3260, 0.9572]])" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "C[[5,3,4]]" ] }, { "cell_type": "code", "execution_count": 13, "id": "caebec8d-f3cc-4b84-9d52-b2e47484d82e", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor([[ 2.2385, -1.2704],\n", " [ 0.1595, -0.9922],\n", " [-0.3260, 0.9572]])" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "C[[ctoi['c'], ctoi['a'], ctoi['b']]]" ] }, { "cell_type": "markdown", "id": "cfce8800-3de1-4320-9d52-e7de71b7e7fb", "metadata": {}, "source": [ "On peut également indexer avec un tenseur:" ] }, { "cell_type": "code", "execution_count": 14, "id": "2ddb6356-3314-4642-95db-1927f3c7116c", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor([[ 2.2385, -1.2704],\n", " [ 0.1595, -0.9922],\n", " [-0.3260, 0.9572]])" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "C[torch.tensor([5,3,4])]" ] }, { "cell_type": "markdown", "id": "427a7115-873e-4254-9ebf-b2b12cb70644", "metadata": {}, "source": [ "Ce tenseur peut être multi-dimensionnel, comme par exemple `Xd`:" ] }, { "cell_type": "code", "execution_count": 15, "id": "95d1d91e-5d91-42dd-b3d0-ccb83ee6ef96", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor([[[ 0.4806, 0.9659],\n", " [ 0.4806, 0.9659],\n", " [ 0.4806, 0.9659]],\n", "\n", " [[ 0.4806, 0.9659],\n", " [ 0.4806, 0.9659],\n", " [ 0.1595, -0.9922]],\n", "\n", " [[ 0.4806, 0.9659],\n", " [ 0.1595, -0.9922],\n", " [-0.3260, 0.9572]],\n", "\n", " [[ 0.1595, -0.9922],\n", " [-0.3260, 0.9572],\n", " [ 0.1595, -0.9922]],\n", "\n", " [[-0.3260, 0.9572],\n", " [ 0.1595, -0.9922],\n", " [-0.4793, 0.3979]],\n", "\n", " [[ 0.1595, -0.9922],\n", " [-0.4793, 0.3979],\n", " [-0.2007, -1.0860]],\n", "\n", " [[-0.4793, 0.3979],\n", " [-0.2007, -1.0860],\n", " [-0.2206, 0.3598]],\n", "\n", " [[-0.2007, -1.0860],\n", " [-0.2206, 0.3598],\n", " [-0.4793, 0.3979]],\n", "\n", " [[-0.2206, 0.3598],\n", " [-0.4793, 0.3979],\n", " [-0.4793, 0.3979]],\n", "\n", " [[-0.4793, 0.3979],\n", " [-0.4793, 0.3979],\n", " [-0.0418, 1.0244]],\n", "\n", " [[ 0.4806, 0.9659],\n", " [ 0.4806, 0.9659],\n", " [ 0.4806, 0.9659]],\n", "\n", " [[ 0.4806, 0.9659],\n", " [ 0.4806, 0.9659],\n", " [ 0.1595, -0.9922]],\n", "\n", " [[ 0.4806, 0.9659],\n", " [ 0.1595, -0.9922],\n", " [-0.3260, 0.9572]],\n", "\n", " [[ 0.1595, -0.9922],\n", " [-0.3260, 0.9572],\n", " [ 0.1595, -0.9922]],\n", "\n", " [[-0.3260, 0.9572],\n", " [ 0.1595, -0.9922],\n", " [-0.4793, 0.3979]],\n", "\n", " [[ 0.1595, -0.9922],\n", " [-0.4793, 0.3979],\n", " [-0.2007, -1.0860]],\n", "\n", " [[-0.4793, 0.3979],\n", " [-0.2007, -1.0860],\n", " [-0.2206, 0.3598]],\n", "\n", " [[-0.2007, -1.0860],\n", " [-0.2206, 0.3598],\n", " [-0.4793, 0.3979]],\n", "\n", " [[-0.2206, 0.3598],\n", " [-0.4793, 0.3979],\n", " [-0.4793, 0.3979]],\n", "\n", " [[-0.4793, 0.3979],\n", " [-0.4793, 0.3979],\n", " [-0.0418, 1.0244]],\n", "\n", " [[-0.4793, 0.3979],\n", " [-0.0418, 1.0244],\n", " [ 1.1423, -0.2115]],\n", "\n", " [[ 0.4806, 0.9659],\n", " [ 0.4806, 0.9659],\n", " [ 0.4806, 0.9659]],\n", "\n", " [[ 0.4806, 0.9659],\n", " [ 0.4806, 0.9659],\n", " [ 0.1595, -0.9922]],\n", "\n", " [[ 0.4806, 0.9659],\n", " [ 0.1595, -0.9922],\n", " [-0.3260, 0.9572]],\n", "\n", " [[ 0.1595, -0.9922],\n", " [-0.3260, 0.9572],\n", " [ 0.1595, -0.9922]],\n", "\n", " [[-0.3260, 0.9572],\n", " [ 0.1595, -0.9922],\n", " [-0.4793, 0.3979]],\n", "\n", " [[ 0.1595, -0.9922],\n", " [-0.4793, 0.3979],\n", " [-0.2007, -1.0860]],\n", "\n", " [[-0.4793, 0.3979],\n", " [-0.2007, -1.0860],\n", " [-0.2206, 0.3598]],\n", "\n", " [[-0.2007, -1.0860],\n", " [-0.2206, 0.3598],\n", " [-0.4793, 0.3979]],\n", "\n", " [[-0.2206, 0.3598],\n", " [-0.4793, 0.3979],\n", " [-0.4793, 0.3979]],\n", "\n", " [[-0.4793, 0.3979],\n", " [-0.4793, 0.3979],\n", " [-0.0418, 1.0244]],\n", "\n", " [[-0.4793, 0.3979],\n", " [-0.0418, 1.0244],\n", " [ 1.1423, -0.2115]],\n", "\n", " [[-0.0418, 1.0244],\n", " [ 1.1423, -0.2115],\n", " [ 0.8648, -0.6060]],\n", "\n", " [[ 0.4806, 0.9659],\n", " [ 0.4806, 0.9659],\n", " [ 0.4806, 0.9659]],\n", "\n", " [[ 0.4806, 0.9659],\n", " [ 0.4806, 0.9659],\n", " [ 0.1595, -0.9922]],\n", "\n", " [[ 0.4806, 0.9659],\n", " [ 0.1595, -0.9922],\n", " [-0.3260, 0.9572]],\n", "\n", " [[ 0.1595, -0.9922],\n", " [-0.3260, 0.9572],\n", " [ 0.1595, -0.9922]],\n", "\n", " [[-0.3260, 0.9572],\n", " [ 0.1595, -0.9922],\n", " [-0.4793, 0.3979]],\n", "\n", " [[ 0.1595, -0.9922],\n", " [-0.4793, 0.3979],\n", " [-0.2007, -1.0860]],\n", "\n", " [[-0.4793, 0.3979],\n", " [-0.2007, -1.0860],\n", " [-0.2206, 0.3598]],\n", "\n", " [[-0.2007, -1.0860],\n", " [-0.2206, 0.3598],\n", " [-0.4793, 0.3979]],\n", "\n", " [[-0.2206, 0.3598],\n", " [-0.4793, 0.3979],\n", " [-0.4793, 0.3979]],\n", "\n", " [[-0.4793, 0.3979],\n", " [-0.4793, 0.3979],\n", " [-0.0418, 1.0244]],\n", "\n", " [[-0.4793, 0.3979],\n", " [-0.0418, 1.0244],\n", " [ 0.8648, -0.6060]],\n", "\n", " [[ 0.4806, 0.9659],\n", " [ 0.4806, 0.9659],\n", " [ 0.4806, 0.9659]],\n", "\n", " [[ 0.4806, 0.9659],\n", " [ 0.4806, 0.9659],\n", " [ 0.1595, -0.9922]],\n", "\n", " [[ 0.4806, 0.9659],\n", " [ 0.1595, -0.9922],\n", " [-0.3260, 0.9572]],\n", "\n", " [[ 0.1595, -0.9922],\n", " [-0.3260, 0.9572],\n", " [ 0.1595, -0.9922]],\n", "\n", " [[-0.3260, 0.9572],\n", " [ 0.1595, -0.9922],\n", " [-0.4793, 0.3979]],\n", "\n", " [[ 0.1595, -0.9922],\n", " [-0.4793, 0.3979],\n", " [-0.2007, -1.0860]],\n", "\n", " [[-0.4793, 0.3979],\n", " [-0.2007, -1.0860],\n", " [-0.2206, 0.3598]],\n", "\n", " [[-0.2007, -1.0860],\n", " [-0.2206, 0.3598],\n", " [-0.4793, 0.3979]],\n", "\n", " [[-0.2206, 0.3598],\n", " [-0.4793, 0.3979],\n", " [ 0.8648, -0.6060]]])" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "C[Xd]" ] }, { "cell_type": "code", "execution_count": 16, "id": "5316d788-2b08-467b-9380-dc97cc20ed65", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "torch.Size([53, 3, 2])" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "C[Xd].shape" ] }, { "cell_type": "markdown", "id": "14b4361c-7d3b-4146-834b-4de41d88eff6", "metadata": {}, "source": [ "Ce qui nous permet de créer tous les embeddings sur notre jeu de données de \"démo\", un tenseur d'ordre 3 de dimensions $53\\times 3\\times 2$. " ] }, { "cell_type": "code", "execution_count": 17, "id": "ee9f0133-bcaa-4bf6-a72f-ec2a4e04574f", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "torch.Size([53, 3, 2])" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "emb = C[Xd]\n", "emb.shape" ] }, { "cell_type": "markdown", "id": "1bde58de-a2ad-4620-aee3-106adc259955", "metadata": {}, "source": [ "## Création de la couche cachée\n", "\n", "La couche cachée sera composée d'une couche avec `context_size * e_dims` entrées et un nombre de neurones `hidden_layer_size` à définir (prenons ici 100), avec pour chaque neurone un poids et un biais, poids et biais qui seront définis dans une matrice `W1` et un vecteur `b1`:" ] }, { "cell_type": "code", "execution_count": 18, "id": "f16ec2f1-4519-4b8d-b75d-41a4d68249ae", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(torch.Size([6, 100]), torch.Size([100]))" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "hidden_layer_size = 100\n", "W1 = torch.randn((context_size * e_dims, hidden_layer_size))\n", "b1 = torch.randn(hidden_layer_size)\n", "W1.shape, b1.shape" ] }, { "cell_type": "markdown", "id": "ab8fae7c-0140-49bd-815a-608239dd50fe", "metadata": {}, "source": [ "Nous souhaiterions donc maintenant multiplier les entrées par les poids et ajouter les biais, ce qui ne va pas être possibles car les tenseurs ne sont pas compatibles." ] }, { "cell_type": "code", "execution_count": 19, "id": "b755bc3b-7300-41be-a806-ab5b21756e48", "metadata": {}, "outputs": [], "source": [ "# emb @ W1 + b1 # Décommenter pour voir l'erreur" ] }, { "cell_type": "markdown", "id": "d5f9380a-77ce-4d6c-8b9c-75e762f8c92c", "metadata": {}, "source": [ "Il faudrait passer pour `emb` d'une forme `[53, 3, 2]` à une forme `[53, 6]`.\n", "\n", "On peut le faire avec PyTorch en concaténant en dimension 1 avec [`cat`](https://pytorch.org/docs/main/generated/torch.cat.html): " ] }, { "cell_type": "code", "execution_count": 20, "id": "c81fc199-8eb3-4388-9f67-c8fc9bf0a1a3", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "torch.Size([53, 6])" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "torch.cat([emb[:, 0, :], emb[:, 1, :], emb[:, 2, :]], 1).shape" ] }, { "cell_type": "markdown", "id": "4585c2a5-beca-457f-9e7f-6a04f7d14228", "metadata": {}, "source": [ "Mais cette approche n'est pas très propre car difficilement généralisable. On peut également utiliser [`unbind`](https://pytorch.org/docs/stable/generated/torch.unbind.html) qui est plus simple:" ] }, { "cell_type": "code", "execution_count": 21, "id": "3683449c-6570-4c07-9a8f-ebeda6323e1d", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "torch.Size([53, 6])" ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" } ], "source": [ "torch.cat(torch.unbind(emb, 1), 1).shape" ] }, { "cell_type": "markdown", "id": "bdd1bf4d-d286-4ac3-a4db-8fe3332e6f84", "metadata": {}, "source": [ "Mais il y a une approche plus efficace consistant à utiliser la méthode [`view()`](https://pytorch.org/docs/stable/generated/torch.Tensor.view.html) d'un tenseur, permettant de \"réorganiser\" les éléments d'un tenseur selon différentes formes et dimensions, à condition que le nombre d'éléments soit identique:" ] }, { "cell_type": "code", "execution_count": 22, "id": "b4089917-4b35-4824-978c-f8502e2a3ba3", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17])" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a = torch.arange(18)\n", "a" ] }, { "cell_type": "code", "execution_count": 23, "id": "a7cec345-d0be-4fb0-800b-ccd514dcc51f", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "torch.Size([18])" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a.shape" ] }, { "cell_type": "code", "execution_count": 24, "id": "0233b8a3-46fe-474c-b32f-e8c734bd5478", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor([[ 0, 1, 2, 3, 4, 5, 6, 7, 8],\n", " [ 9, 10, 11, 12, 13, 14, 15, 16, 17]])" ] }, "execution_count": 24, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a.view(2, 9)" ] }, { "cell_type": "code", "execution_count": 25, "id": "fed37cf8-1694-48cc-8193-9b3b34183a1b", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor([[ 0, 1],\n", " [ 2, 3],\n", " [ 4, 5],\n", " [ 6, 7],\n", " [ 8, 9],\n", " [10, 11],\n", " [12, 13],\n", " [14, 15],\n", " [16, 17]])" ] }, "execution_count": 25, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a.view(9, 2)" ] }, { "cell_type": "code", "execution_count": 26, "id": "187e666b-6086-477b-96cf-a6402cf84743", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor([[[ 0, 1],\n", " [ 2, 3],\n", " [ 4, 5]],\n", "\n", " [[ 6, 7],\n", " [ 8, 9],\n", " [10, 11]],\n", "\n", " [[12, 13],\n", " [14, 15],\n", " [16, 17]]])" ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a.view(3, 3, 2)" ] }, { "cell_type": "code", "execution_count": 27, "id": "f1ea6827-7a5e-48c3-8aed-990985ae3312", "metadata": {}, "outputs": [], "source": [ "# a.untyped_storage()" ] }, { "cell_type": "markdown", "id": "6897874f-d938-4935-bec4-64f8a499aaa6", "metadata": {}, "source": [ "Avec nos embeddings `emb`, il est donc possible d'utiliser `view` ainsi:" ] }, { "cell_type": "code", "execution_count": 28, "id": "24203d0a-fe41-48c7-9b64-1918d2dc223b", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor([[ 0.4806, 0.9659, 0.4806, 0.9659, 0.4806, 0.9659],\n", " [ 0.4806, 0.9659, 0.4806, 0.9659, 0.1595, -0.9922],\n", " [ 0.4806, 0.9659, 0.1595, -0.9922, -0.3260, 0.9572],\n", " [ 0.1595, -0.9922, -0.3260, 0.9572, 0.1595, -0.9922],\n", " [-0.3260, 0.9572, 0.1595, -0.9922, -0.4793, 0.3979],\n", " [ 0.1595, -0.9922, -0.4793, 0.3979, -0.2007, -1.0860],\n", " [-0.4793, 0.3979, -0.2007, -1.0860, -0.2206, 0.3598],\n", " [-0.2007, -1.0860, -0.2206, 0.3598, -0.4793, 0.3979],\n", " [-0.2206, 0.3598, -0.4793, 0.3979, -0.4793, 0.3979],\n", " [-0.4793, 0.3979, -0.4793, 0.3979, -0.0418, 1.0244],\n", " [ 0.4806, 0.9659, 0.4806, 0.9659, 0.4806, 0.9659],\n", " [ 0.4806, 0.9659, 0.4806, 0.9659, 0.1595, -0.9922],\n", " [ 0.4806, 0.9659, 0.1595, -0.9922, -0.3260, 0.9572],\n", " [ 0.1595, -0.9922, -0.3260, 0.9572, 0.1595, -0.9922],\n", " [-0.3260, 0.9572, 0.1595, -0.9922, -0.4793, 0.3979],\n", " [ 0.1595, -0.9922, -0.4793, 0.3979, -0.2007, -1.0860],\n", " [-0.4793, 0.3979, -0.2007, -1.0860, -0.2206, 0.3598],\n", " [-0.2007, -1.0860, -0.2206, 0.3598, -0.4793, 0.3979],\n", " [-0.2206, 0.3598, -0.4793, 0.3979, -0.4793, 0.3979],\n", " [-0.4793, 0.3979, -0.4793, 0.3979, -0.0418, 1.0244],\n", " [-0.4793, 0.3979, -0.0418, 1.0244, 1.1423, -0.2115],\n", " [ 0.4806, 0.9659, 0.4806, 0.9659, 0.4806, 0.9659],\n", " [ 0.4806, 0.9659, 0.4806, 0.9659, 0.1595, -0.9922],\n", " [ 0.4806, 0.9659, 0.1595, -0.9922, -0.3260, 0.9572],\n", " [ 0.1595, -0.9922, -0.3260, 0.9572, 0.1595, -0.9922],\n", " [-0.3260, 0.9572, 0.1595, -0.9922, -0.4793, 0.3979],\n", " [ 0.1595, -0.9922, -0.4793, 0.3979, -0.2007, -1.0860],\n", " [-0.4793, 0.3979, -0.2007, -1.0860, -0.2206, 0.3598],\n", " [-0.2007, -1.0860, -0.2206, 0.3598, -0.4793, 0.3979],\n", " [-0.2206, 0.3598, -0.4793, 0.3979, -0.4793, 0.3979],\n", " [-0.4793, 0.3979, -0.4793, 0.3979, -0.0418, 1.0244],\n", " [-0.4793, 0.3979, -0.0418, 1.0244, 1.1423, -0.2115],\n", " [-0.0418, 1.0244, 1.1423, -0.2115, 0.8648, -0.6060],\n", " [ 0.4806, 0.9659, 0.4806, 0.9659, 0.4806, 0.9659],\n", " [ 0.4806, 0.9659, 0.4806, 0.9659, 0.1595, -0.9922],\n", " [ 0.4806, 0.9659, 0.1595, -0.9922, -0.3260, 0.9572],\n", " [ 0.1595, -0.9922, -0.3260, 0.9572, 0.1595, -0.9922],\n", " [-0.3260, 0.9572, 0.1595, -0.9922, -0.4793, 0.3979],\n", " [ 0.1595, -0.9922, -0.4793, 0.3979, -0.2007, -1.0860],\n", " [-0.4793, 0.3979, -0.2007, -1.0860, -0.2206, 0.3598],\n", " [-0.2007, -1.0860, -0.2206, 0.3598, -0.4793, 0.3979],\n", " [-0.2206, 0.3598, -0.4793, 0.3979, -0.4793, 0.3979],\n", " [-0.4793, 0.3979, -0.4793, 0.3979, -0.0418, 1.0244],\n", " [-0.4793, 0.3979, -0.0418, 1.0244, 0.8648, -0.6060],\n", " [ 0.4806, 0.9659, 0.4806, 0.9659, 0.4806, 0.9659],\n", " [ 0.4806, 0.9659, 0.4806, 0.9659, 0.1595, -0.9922],\n", " [ 0.4806, 0.9659, 0.1595, -0.9922, -0.3260, 0.9572],\n", " [ 0.1595, -0.9922, -0.3260, 0.9572, 0.1595, -0.9922],\n", " [-0.3260, 0.9572, 0.1595, -0.9922, -0.4793, 0.3979],\n", " [ 0.1595, -0.9922, -0.4793, 0.3979, -0.2007, -1.0860],\n", " [-0.4793, 0.3979, -0.2007, -1.0860, -0.2206, 0.3598],\n", " [-0.2007, -1.0860, -0.2206, 0.3598, -0.4793, 0.3979],\n", " [-0.2206, 0.3598, -0.4793, 0.3979, 0.8648, -0.6060]])" ] }, "execution_count": 28, "metadata": {}, "output_type": "execute_result" } ], "source": [ "emb.view(53, 6)" ] }, { "cell_type": "markdown", "id": "f7c39a2c-e4fa-4dee-b204-77824b243b4d", "metadata": {}, "source": [ "ou avec la valeur spéciale `-1` qui permet de ne pas avoir à spécifier la taille du premier ordre du tenseur:" ] }, { "cell_type": "code", "execution_count": 29, "id": "3dcfcef4-92a1-4bd9-b877-98a5f9340bb8", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor([[ 0.4806, 0.9659, 0.4806, 0.9659, 0.4806, 0.9659],\n", " [ 0.4806, 0.9659, 0.4806, 0.9659, 0.1595, -0.9922],\n", " [ 0.4806, 0.9659, 0.1595, -0.9922, -0.3260, 0.9572],\n", " [ 0.1595, -0.9922, -0.3260, 0.9572, 0.1595, -0.9922],\n", " [-0.3260, 0.9572, 0.1595, -0.9922, -0.4793, 0.3979],\n", " [ 0.1595, -0.9922, -0.4793, 0.3979, -0.2007, -1.0860],\n", " [-0.4793, 0.3979, -0.2007, -1.0860, -0.2206, 0.3598],\n", " [-0.2007, -1.0860, -0.2206, 0.3598, -0.4793, 0.3979],\n", " [-0.2206, 0.3598, -0.4793, 0.3979, -0.4793, 0.3979],\n", " [-0.4793, 0.3979, -0.4793, 0.3979, -0.0418, 1.0244],\n", " [ 0.4806, 0.9659, 0.4806, 0.9659, 0.4806, 0.9659],\n", " [ 0.4806, 0.9659, 0.4806, 0.9659, 0.1595, -0.9922],\n", " [ 0.4806, 0.9659, 0.1595, -0.9922, -0.3260, 0.9572],\n", " [ 0.1595, -0.9922, -0.3260, 0.9572, 0.1595, -0.9922],\n", " [-0.3260, 0.9572, 0.1595, -0.9922, -0.4793, 0.3979],\n", " [ 0.1595, -0.9922, -0.4793, 0.3979, -0.2007, -1.0860],\n", " [-0.4793, 0.3979, -0.2007, -1.0860, -0.2206, 0.3598],\n", " [-0.2007, -1.0860, -0.2206, 0.3598, -0.4793, 0.3979],\n", " [-0.2206, 0.3598, -0.4793, 0.3979, -0.4793, 0.3979],\n", " [-0.4793, 0.3979, -0.4793, 0.3979, -0.0418, 1.0244],\n", " [-0.4793, 0.3979, -0.0418, 1.0244, 1.1423, -0.2115],\n", " [ 0.4806, 0.9659, 0.4806, 0.9659, 0.4806, 0.9659],\n", " [ 0.4806, 0.9659, 0.4806, 0.9659, 0.1595, -0.9922],\n", " [ 0.4806, 0.9659, 0.1595, -0.9922, -0.3260, 0.9572],\n", " [ 0.1595, -0.9922, -0.3260, 0.9572, 0.1595, -0.9922],\n", " [-0.3260, 0.9572, 0.1595, -0.9922, -0.4793, 0.3979],\n", " [ 0.1595, -0.9922, -0.4793, 0.3979, -0.2007, -1.0860],\n", " [-0.4793, 0.3979, -0.2007, -1.0860, -0.2206, 0.3598],\n", " [-0.2007, -1.0860, -0.2206, 0.3598, -0.4793, 0.3979],\n", " [-0.2206, 0.3598, -0.4793, 0.3979, -0.4793, 0.3979],\n", " [-0.4793, 0.3979, -0.4793, 0.3979, -0.0418, 1.0244],\n", " [-0.4793, 0.3979, -0.0418, 1.0244, 1.1423, -0.2115],\n", " [-0.0418, 1.0244, 1.1423, -0.2115, 0.8648, -0.6060],\n", " [ 0.4806, 0.9659, 0.4806, 0.9659, 0.4806, 0.9659],\n", " [ 0.4806, 0.9659, 0.4806, 0.9659, 0.1595, -0.9922],\n", " [ 0.4806, 0.9659, 0.1595, -0.9922, -0.3260, 0.9572],\n", " [ 0.1595, -0.9922, -0.3260, 0.9572, 0.1595, -0.9922],\n", " [-0.3260, 0.9572, 0.1595, -0.9922, -0.4793, 0.3979],\n", " [ 0.1595, -0.9922, -0.4793, 0.3979, -0.2007, -1.0860],\n", " [-0.4793, 0.3979, -0.2007, -1.0860, -0.2206, 0.3598],\n", " [-0.2007, -1.0860, -0.2206, 0.3598, -0.4793, 0.3979],\n", " [-0.2206, 0.3598, -0.4793, 0.3979, -0.4793, 0.3979],\n", " [-0.4793, 0.3979, -0.4793, 0.3979, -0.0418, 1.0244],\n", " [-0.4793, 0.3979, -0.0418, 1.0244, 0.8648, -0.6060],\n", " [ 0.4806, 0.9659, 0.4806, 0.9659, 0.4806, 0.9659],\n", " [ 0.4806, 0.9659, 0.4806, 0.9659, 0.1595, -0.9922],\n", " [ 0.4806, 0.9659, 0.1595, -0.9922, -0.3260, 0.9572],\n", " [ 0.1595, -0.9922, -0.3260, 0.9572, 0.1595, -0.9922],\n", " [-0.3260, 0.9572, 0.1595, -0.9922, -0.4793, 0.3979],\n", " [ 0.1595, -0.9922, -0.4793, 0.3979, -0.2007, -1.0860],\n", " [-0.4793, 0.3979, -0.2007, -1.0860, -0.2206, 0.3598],\n", " [-0.2007, -1.0860, -0.2206, 0.3598, -0.4793, 0.3979],\n", " [-0.2206, 0.3598, -0.4793, 0.3979, 0.8648, -0.6060]])" ] }, "execution_count": 29, "metadata": {}, "output_type": "execute_result" } ], "source": [ "emb.view(-1, 6)" ] }, { "cell_type": "markdown", "id": "b298a721-d430-4935-b290-c428a71d4472", "metadata": {}, "source": [ "De manière générique, nous pouvons donc écrire:" ] }, { "cell_type": "code", "execution_count": 30, "id": "245a3350-b808-4756-8f4a-e5397440354d", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor([[ 0.4806, 0.9659, 0.4806, 0.9659, 0.4806, 0.9659],\n", " [ 0.4806, 0.9659, 0.4806, 0.9659, 0.1595, -0.9922],\n", " [ 0.4806, 0.9659, 0.1595, -0.9922, -0.3260, 0.9572],\n", " [ 0.1595, -0.9922, -0.3260, 0.9572, 0.1595, -0.9922],\n", " [-0.3260, 0.9572, 0.1595, -0.9922, -0.4793, 0.3979],\n", " [ 0.1595, -0.9922, -0.4793, 0.3979, -0.2007, -1.0860],\n", " [-0.4793, 0.3979, -0.2007, -1.0860, -0.2206, 0.3598],\n", " [-0.2007, -1.0860, -0.2206, 0.3598, -0.4793, 0.3979],\n", " [-0.2206, 0.3598, -0.4793, 0.3979, -0.4793, 0.3979],\n", " [-0.4793, 0.3979, -0.4793, 0.3979, -0.0418, 1.0244],\n", " [ 0.4806, 0.9659, 0.4806, 0.9659, 0.4806, 0.9659],\n", " [ 0.4806, 0.9659, 0.4806, 0.9659, 0.1595, -0.9922],\n", " [ 0.4806, 0.9659, 0.1595, -0.9922, -0.3260, 0.9572],\n", " [ 0.1595, -0.9922, -0.3260, 0.9572, 0.1595, -0.9922],\n", " [-0.3260, 0.9572, 0.1595, -0.9922, -0.4793, 0.3979],\n", " [ 0.1595, -0.9922, -0.4793, 0.3979, -0.2007, -1.0860],\n", " [-0.4793, 0.3979, -0.2007, -1.0860, -0.2206, 0.3598],\n", " [-0.2007, -1.0860, -0.2206, 0.3598, -0.4793, 0.3979],\n", " [-0.2206, 0.3598, -0.4793, 0.3979, -0.4793, 0.3979],\n", " [-0.4793, 0.3979, -0.4793, 0.3979, -0.0418, 1.0244],\n", " [-0.4793, 0.3979, -0.0418, 1.0244, 1.1423, -0.2115],\n", " [ 0.4806, 0.9659, 0.4806, 0.9659, 0.4806, 0.9659],\n", " [ 0.4806, 0.9659, 0.4806, 0.9659, 0.1595, -0.9922],\n", " [ 0.4806, 0.9659, 0.1595, -0.9922, -0.3260, 0.9572],\n", " [ 0.1595, -0.9922, -0.3260, 0.9572, 0.1595, -0.9922],\n", " [-0.3260, 0.9572, 0.1595, -0.9922, -0.4793, 0.3979],\n", " [ 0.1595, -0.9922, -0.4793, 0.3979, -0.2007, -1.0860],\n", " [-0.4793, 0.3979, -0.2007, -1.0860, -0.2206, 0.3598],\n", " [-0.2007, -1.0860, -0.2206, 0.3598, -0.4793, 0.3979],\n", " [-0.2206, 0.3598, -0.4793, 0.3979, -0.4793, 0.3979],\n", " [-0.4793, 0.3979, -0.4793, 0.3979, -0.0418, 1.0244],\n", " [-0.4793, 0.3979, -0.0418, 1.0244, 1.1423, -0.2115],\n", " [-0.0418, 1.0244, 1.1423, -0.2115, 0.8648, -0.6060],\n", " [ 0.4806, 0.9659, 0.4806, 0.9659, 0.4806, 0.9659],\n", " [ 0.4806, 0.9659, 0.4806, 0.9659, 0.1595, -0.9922],\n", " [ 0.4806, 0.9659, 0.1595, -0.9922, -0.3260, 0.9572],\n", " [ 0.1595, -0.9922, -0.3260, 0.9572, 0.1595, -0.9922],\n", " [-0.3260, 0.9572, 0.1595, -0.9922, -0.4793, 0.3979],\n", " [ 0.1595, -0.9922, -0.4793, 0.3979, -0.2007, -1.0860],\n", " [-0.4793, 0.3979, -0.2007, -1.0860, -0.2206, 0.3598],\n", " [-0.2007, -1.0860, -0.2206, 0.3598, -0.4793, 0.3979],\n", " [-0.2206, 0.3598, -0.4793, 0.3979, -0.4793, 0.3979],\n", " [-0.4793, 0.3979, -0.4793, 0.3979, -0.0418, 1.0244],\n", " [-0.4793, 0.3979, -0.0418, 1.0244, 0.8648, -0.6060],\n", " [ 0.4806, 0.9659, 0.4806, 0.9659, 0.4806, 0.9659],\n", " [ 0.4806, 0.9659, 0.4806, 0.9659, 0.1595, -0.9922],\n", " [ 0.4806, 0.9659, 0.1595, -0.9922, -0.3260, 0.9572],\n", " [ 0.1595, -0.9922, -0.3260, 0.9572, 0.1595, -0.9922],\n", " [-0.3260, 0.9572, 0.1595, -0.9922, -0.4793, 0.3979],\n", " [ 0.1595, -0.9922, -0.4793, 0.3979, -0.2007, -1.0860],\n", " [-0.4793, 0.3979, -0.2007, -1.0860, -0.2206, 0.3598],\n", " [-0.2007, -1.0860, -0.2206, 0.3598, -0.4793, 0.3979],\n", " [-0.2206, 0.3598, -0.4793, 0.3979, 0.8648, -0.6060]])" ] }, "execution_count": 30, "metadata": {}, "output_type": "execute_result" } ], "source": [ "emb.view(-1, context_size * e_dims)" ] }, { "cell_type": "markdown", "id": "54b8daf0-fbe4-48ab-92ab-54fbc567633f", "metadata": {}, "source": [ "Nous pouvons maintenant implémenter la couche cachée `h` complètement:" ] }, { "cell_type": "code", "execution_count": 31, "id": "6f0a00e3-9625-4c2f-8d12-28137cd6ec59", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor([[ 0.4931, 0.9997, -0.4131, ..., -0.9919, 0.2336, 0.4416],\n", " [-0.7059, 0.9476, -0.9881, ..., -0.9353, 0.6110, -0.2495],\n", " [ 0.7830, 0.8349, 0.9091, ..., -0.7906, -0.6847, 0.9962],\n", " ...,\n", " [ 0.8753, 0.2654, -0.6717, ..., -0.1456, -0.2736, 0.8699],\n", " [ 0.7239, -0.0967, -0.9465, ..., -0.8542, 0.5216, -0.9400],\n", " [ 0.4366, 0.9808, -0.9875, ..., -0.1445, -0.8273, 0.1011]])" ] }, "execution_count": 31, "metadata": {}, "output_type": "execute_result" } ], "source": [ "h = torch.tanh(emb.view(-1, context_size*e_dims) @ W1 + b1)\n", "h" ] }, { "cell_type": "markdown", "id": "0ca0f7de-99d3-46b6-8e04-274daee62a56", "metadata": {}, "source": [ "### Couche de sortie" ] }, { "cell_type": "markdown", "id": "c08405f1-9558-4246-ad48-132eb2f01529", "metadata": {}, "source": [ "De manière similaire à la couche cachée la couche de sortie va se composer d'une matrice de poids `W2` et d'un vecteur de biais `b2`:" ] }, { "cell_type": "code", "execution_count": 32, "id": "67656527-ea8a-4b4d-b6fc-d0d7249dd8dd", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(torch.Size([100, 41]), torch.Size([41]))" ] }, "execution_count": 32, "metadata": {}, "output_type": "execute_result" } ], "source": [ "W2 = torch.randn((hidden_layer_size, nb_chars))\n", "b2 = torch.randn(nb_chars)\n", "W2.shape, b2.shape" ] }, { "cell_type": "markdown", "id": "4e70d9f1-16e2-4ad8-8ea3-48ed0667703b", "metadata": {}, "source": [ "Le calcul des _logits_ de sortie de la couche de sortie s'obtiennent en multipliant les valeurs de la couche cachée par les poids `W2` et en ajoutant les biais:" ] }, { "cell_type": "code", "execution_count": 33, "id": "c34f5423-14d3-4352-85aa-fd02237c7a3f", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "torch.Size([53, 41])" ] }, "execution_count": 33, "metadata": {}, "output_type": "execute_result" } ], "source": [ "logits = h @ W2 + b2\n", "logits.shape" ] }, { "cell_type": "markdown", "id": "1cf09b09-3c10-4e1a-b7f1-86d47c801adf", "metadata": {}, "source": [ "Pour obtenir des probabilités à partir des logits (interprétés comme des logs), on utilise notre fonction _softmax_:" ] }, { "cell_type": "code", "execution_count": 34, "id": "d63a0d2a-22b7-4136-bdf7-2694d7f8f4db", "metadata": {}, "outputs": [], "source": [ "counts = logits.exp()\n", "prob = counts / counts.sum(1, keepdims=True)" ] }, { "cell_type": "markdown", "id": "d0cac6ae-171f-4042-9f3c-5a742a0f9fe8", "metadata": {}, "source": [ "Nous pouvons vérifier que la somme de ces probabilités est également à 1:" ] }, { "cell_type": "code", "execution_count": 35, "id": "e4e56746-e4ad-4adb-9587-a93e7680ee8a", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor(1.0000)" ] }, "execution_count": 35, "metadata": {}, "output_type": "execute_result" } ], "source": [ "prob[0].sum()" ] }, { "cell_type": "markdown", "id": "2f5179a7-54b8-4e3d-8bbc-a4bc803913dc", "metadata": {}, "source": [ "À partir de ces probabilités, nous voulons maintenant obtenir la probabilité affectée à chacun des caractères attendus de `Yd`:" ] }, { "cell_type": "code", "execution_count": 36, "id": "2bf7a1a2-392c-4349-ae32-36e271df089d", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor([ 3, 4, 3, 15, 6, 16, 15, 15, 32, 0, 3, 4, 3, 15, 6, 16, 15, 15,\n", " 32, 7, 0, 3, 4, 3, 15, 6, 16, 15, 15, 32, 7, 20, 0, 3, 4, 3,\n", " 15, 6, 16, 15, 15, 32, 20, 0, 3, 4, 3, 15, 6, 16, 15, 20, 0])" ] }, "execution_count": 36, "metadata": {}, "output_type": "execute_result" } ], "source": [ "Yd" ] }, { "cell_type": "markdown", "id": "676f93ed-9341-41f7-affe-17f27a060c11", "metadata": {}, "source": [ "`prob` étant d'ordre 2, nous devons donc indexer par le \"numéro\" de l'exemple (`torch.arange(53)` car il y a 53 exemples dans `Yd`) et le numéro du token, soit:" ] }, { "cell_type": "code", "execution_count": 37, "id": "ba48af5e-c095-4b12-adbd-05850cb482eb", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor([1.3885e-09, 2.0777e-06, 9.6970e-03, 4.8020e-15, 6.1410e-06, 8.1274e-15,\n", " 1.1853e-01, 8.5158e-09, 2.0254e-05, 1.6990e-04, 1.3885e-09, 2.0777e-06,\n", " 9.6970e-03, 4.8020e-15, 6.1410e-06, 8.1274e-15, 1.1853e-01, 8.5158e-09,\n", " 2.0254e-05, 5.7130e-06, 1.3612e-06, 1.3885e-09, 2.0777e-06, 9.6970e-03,\n", " 4.8020e-15, 6.1410e-06, 8.1274e-15, 1.1853e-01, 8.5158e-09, 2.0254e-05,\n", " 5.7130e-06, 6.4163e-10, 2.2806e-05, 1.3885e-09, 2.0777e-06, 9.6970e-03,\n", " 4.8020e-15, 6.1410e-06, 8.1274e-15, 1.1853e-01, 8.5158e-09, 2.0254e-05,\n", " 1.5092e-14, 9.4455e-08, 1.3885e-09, 2.0777e-06, 9.6970e-03, 4.8020e-15,\n", " 6.1410e-06, 8.1274e-15, 1.1853e-01, 1.0905e-14, 1.0875e-09])" ] }, "execution_count": 37, "metadata": {}, "output_type": "execute_result" } ], "source": [ "prob[torch.arange(53), Yd]" ] }, { "cell_type": "markdown", "id": "c73109d9-efb7-492f-91c4-277e6c233994", "metadata": {}, "source": [ "Ou de manière générique en utilisant `size` pour obtenir la taille de `Yd`:" ] }, { "cell_type": "code", "execution_count": 38, "id": "fdbda2c4-12fe-43a9-a82e-aea5e203e0c5", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor([1.3885e-09, 2.0777e-06, 9.6970e-03, 4.8020e-15, 6.1410e-06, 8.1274e-15,\n", " 1.1853e-01, 8.5158e-09, 2.0254e-05, 1.6990e-04, 1.3885e-09, 2.0777e-06,\n", " 9.6970e-03, 4.8020e-15, 6.1410e-06, 8.1274e-15, 1.1853e-01, 8.5158e-09,\n", " 2.0254e-05, 5.7130e-06, 1.3612e-06, 1.3885e-09, 2.0777e-06, 9.6970e-03,\n", " 4.8020e-15, 6.1410e-06, 8.1274e-15, 1.1853e-01, 8.5158e-09, 2.0254e-05,\n", " 5.7130e-06, 6.4163e-10, 2.2806e-05, 1.3885e-09, 2.0777e-06, 9.6970e-03,\n", " 4.8020e-15, 6.1410e-06, 8.1274e-15, 1.1853e-01, 8.5158e-09, 2.0254e-05,\n", " 1.5092e-14, 9.4455e-08, 1.3885e-09, 2.0777e-06, 9.6970e-03, 4.8020e-15,\n", " 6.1410e-06, 8.1274e-15, 1.1853e-01, 1.0905e-14, 1.0875e-09])" ] }, "execution_count": 38, "metadata": {}, "output_type": "execute_result" } ], "source": [ "prob[torch.arange(Yd.size(0)), Yd]" ] }, { "cell_type": "markdown", "id": "a2736d77-eed7-4cb2-a7c6-9ca16d472587", "metadata": {}, "source": [ "### Fonction de perte: cross-entropy" ] }, { "cell_type": "markdown", "id": "488e5d76-5674-48d8-a23d-4f907f00462d", "metadata": {}, "source": [ "Nous pouvons maintenant estimer la \"qualité\" de notre modèle partiel (sur 53 exemples) avec notre fonction de perte:" ] }, { "cell_type": "code", "execution_count": 39, "id": "b8010307-d1d2-45aa-871a-0054c02ced66", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor(16.6951)" ] }, "execution_count": 39, "metadata": {}, "output_type": "execute_result" } ], "source": [ "loss = -prob[torch.arange(Yd.size(0)), Yd].log().mean()\n", "loss" ] }, { "cell_type": "markdown", "id": "af00ff39-85a6-46b8-a5ee-0deb57171c53", "metadata": {}, "source": [ "Jusqu'à ce point, le réseau calcule des \"logits\", qui sont passés dans un Softmax pour obtenir des probabilités. La \"Negative Log Likelihood\" est calculée manuellement.\n", "Le calcul manuel peut-être remplacé par la \"cross-entropy\":\n", "\n", "1. Plus efficace (opérations fusionnées, moins de tenseurs intermédiaires).\n", "2. Plus simple pour la rétropropagation (backward pass).\n", "3. **Stabilité numérique:** la \"cross-entropy\" gère mieux les très grands nombres (qui causeraient des `NaN` avec une exponentielle naïve) en soustrayant le maximum des logits avant le calcul.\n", "\n", "La methode [`cross_entropy`](https://pytorch.org/docs/stable/generated/torch.nn.functional.cross_entropy.html) de Pytorch permet de calculer plus efficacement le loss, tout en donnant le même résultat:" ] }, { "cell_type": "code", "execution_count": 40, "id": "be1860ba-0fc4-414d-8876-793d7bdea569", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor(16.6951)" ] }, "execution_count": 40, "metadata": {}, "output_type": "execute_result" } ], "source": [ "loss = F.cross_entropy(logits, Yd)\n", "loss" ] }, { "cell_type": "markdown", "id": "38cbe203-e858-4d82-bf60-7eca0cd589d1", "metadata": {}, "source": [ "Pour illustrer le fait d'utiliser des probabilités examinons ces deux exemples:" ] }, { "cell_type": "code", "execution_count": 41, "id": "83fe7bdc-a44e-4efa-a50f-190b5164db3b", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor([9.0466e-04, 3.3281e-04, 6.6846e-03, 9.9208e-01])" ] }, "execution_count": 41, "metadata": {}, "output_type": "execute_result" } ], "source": [ "logits = torch.tensor([-2, -3, 0, 5])\n", "counts = logits.exp()\n", "probs = counts / counts.sum()\n", "probs" ] }, { "cell_type": "code", "execution_count": 42, "id": "b18b4690-2299-4622-88e0-ec5bd77af328", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor([0., 0., 0., nan])" ] }, "execution_count": 42, "metadata": {}, "output_type": "execute_result" } ], "source": [ "logits = torch.tensor([-100, -3, 0, 100])\n", "counts = logits.exp()\n", "probs = counts / counts.sum()\n", "probs" ] }, { "cell_type": "markdown", "id": "c4cfe8e6-f93b-4f50-b8e0-2a5dbdb67c2b", "metadata": {}, "source": [ "## Réseau complet \"Feed Forward Netword\"" ] }, { "cell_type": "markdown", "id": "f6ae6a5c-18e7-484d-aa56-ae046635ad10", "metadata": {}, "source": [ "### Données" ] }, { "cell_type": "code", "execution_count": 43, "id": "fa2f8c94-53f3-4fef-a6bb-70072bd80fa2", "metadata": {}, "outputs": [], "source": [ "words = open('civil_mots.txt', 'r').read().splitlines()\n", "nb_words = len(words)\n", "chars = sorted(list(set(''.join(words))))\n", "EOS = '.'\n", "nb_chars = len(chars) + 1 # On ajoute 1 pour EOS\n", "ctoi = {c:i+1 for i,c in enumerate(chars)}\n", "ctoi[EOS] = 0\n", "itoc = {i:s for s,i in ctoi.items()}" ] }, { "cell_type": "markdown", "id": "60dd4eb5-67b8-4134-a5e2-657249560d74", "metadata": {}, "source": [ "### Jeux d'entraînement, de développement et de test" ] }, { "cell_type": "code", "execution_count": 44, "id": "677c6e27-9f82-4070-adce-5fbcf227bd63", "metadata": {}, "outputs": [], "source": [ "# 80%, 10%, 10%\n", "import random\n", "\n", "random.seed(42)\n", "random.shuffle(words)\n", "n1 = int(0.8 * len(words))\n", "n2 = int(0.9 * len(words))\n", "\n", "Xtr, Ytr = build_dataset(words[:n1], context_size=context_size)\n", "Xdev, Ydev = build_dataset(words[n1:n2], context_size=context_size)\n", "Xte, Yte = build_dataset(words[n2:], context_size=context_size)" ] }, { "cell_type": "markdown", "id": "25f92d01-5945-4fc4-b46a-6a7ae5392c22", "metadata": {}, "source": [ "### Hyperparamètres" ] }, { "cell_type": "code", "execution_count": 45, "id": "c248db23-c856-45c0-bad9-623f294a4d1d", "metadata": {}, "outputs": [], "source": [ "context_size = 3\n", "e_dims = 2 # Dimensions des embeddings\n", "hidden_layer_size = 100\n", "mini_batch_size = 32\n", "steps = 200000\n", "seed = 2147483647" ] }, { "cell_type": "markdown", "id": "d30086dc-427e-42d7-a72a-7bb9789f5219", "metadata": {}, "source": [ "### Architecture" ] }, { "cell_type": "code", "execution_count": 46, "id": "13d61b95-9f0d-4a78-b70b-173c13b29a10", "metadata": {}, "outputs": [], "source": [ "g = torch.Generator().manual_seed(seed) # for reproducibility\n", "C = torch.randn((nb_chars, e_dims), generator=g)\n", "W1 = torch.randn((context_size * e_dims, hidden_layer_size), generator=g)\n", "b1 = torch.randn(hidden_layer_size, generator=g)\n", "W2 = torch.randn((hidden_layer_size, nb_chars), generator=g)\n", "b2 = torch.randn(nb_chars, generator=g)\n", "parameters = [C, W1, b1, W2, b2]\n", "for p in parameters:\n", " p.requires_grad = True" ] }, { "cell_type": "code", "execution_count": 47, "id": "c03a4c5c-7511-4da6-ba08-53d3eea8ee58", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "4923" ] }, "execution_count": 47, "metadata": {}, "output_type": "execute_result" } ], "source": [ "sum(p.nelement() for p in parameters) # number of parameters in total" ] }, { "cell_type": "markdown", "id": "e842fda3-6f29-4963-b90b-f70cdc100186", "metadata": {}, "source": [ "## Entraînement" ] }, { "cell_type": "code", "execution_count": 48, "id": "0453e533-94c4-488a-bbdc-19a1aa92a593", "metadata": {}, "outputs": [], "source": [ "lossi = []\n", "stepi = []" ] }, { "cell_type": "code", "execution_count": 49, "id": "3511e928-3c1a-496a-ad60-ef5b0dd6bcc9", "metadata": {}, "outputs": [], "source": [ "for i in range(steps):\n", " # mini-batch construct\n", " ix = torch.randint(0, Xtr.shape[0], (mini_batch_size,))\n", " \n", " # forward pass\n", " emb = C[Xtr[ix]] # (mini_batch_size, context_size, e_dims)\n", " h = torch.tanh(emb.view(-1, context_size * e_dims) @ W1 + b1) # (mini_batch_size, hidden_layer_size)\n", " logits = h @ W2 + b2 # (mini_batch_size, nb_chars)\n", " loss = F.cross_entropy(logits, Ytr[ix])\n", " \n", " # backward pass\n", " for p in parameters:\n", " p.grad = None\n", " loss.backward()\n", " \n", " # update\n", " lr = 0.16 if i < 100000 else 0.016\n", " for p in parameters:\n", " p.data += -lr * p.grad\n", "\n", " # track stats\n", " stepi.append(i)\n", " lossi.append(loss.log10().item())" ] }, { "cell_type": "code", "execution_count": 50, "id": "c7e56eba-380f-4079-95d2-c457f3a8df77", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[]" ] }, "execution_count": 50, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAicAAAGdCAYAAADJ6dNTAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjgsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvwVt1zgAAAAlwSFlzAAAPYQAAD2EBqD+naQAASYxJREFUeJzt3Qd8VFX2wPETSkLvEFroXXoVEBGJFJG1/kV0AbErVqwootiwIq6LYsOyuwq6q7gLiCKCgKAUwUZRmiAdUUqAUDL/z7kwkzeTqckkcyfz+34+80lm8mbmvnmTeWfuPffcJJfL5RIAAABLFIl1AwAAAJwITgAAgFUITgAAgFUITgAAgFUITgAAgFUITgAAgFUITgAAgFUITgAAgFWKSRzIysqSbdu2SdmyZSUpKSnWzQEAAGHQOq8HDhyQmjVrSpEiRQpXcKKBSVpaWqybAQAAcmHLli1Su3btwhWcaI+Je+fKlSsX6+YAAIAw7N+/33QuuM/jhSo4cQ/laGBCcAIAQHyJNCWDhFgAAGAVghMAAGAVghMAAGAVghMAAGAVghMAAGAVghMAAGAVghMAAGAVghMAAGAVghMAAGAVghMAAGAVghMAAGAVghMAAGCVhA5OVm/fL68v2CDHT2TFuikAACCeViXOL/1fWGB+Fi2SJMO71491cwAAQKL3nLj9tG1/rJsAAABOITgBAABWITgBAABWITgBAABWITgRkbqVSsX6OAAAgFMSOjhpX6eC+dmketlYNwUAAJyS0MEJAACwD8EJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwSkIHJ0lJSbFuAgAA8JHQwYmbyxXrFgAAgFwHJ/Pnz5eBAwdKzZo1Tc/DtGnTgm7/4YcfyjnnnCNVq1aVcuXKSdeuXeXTTz+N9GkBAECCiDg4ycjIkDZt2sjEiRPDDmY0OJk5c6YsX75cevXqZYKbFStW5Ka9AACgkCsW6R369+9vLuGaMGGC1/UnnnhCPv74Y/nf//4n7dq1i/TpAQBAIRdxcJJXWVlZcuDAAalUqVLAbTIzM83Fbf/+/QXUOgAAkHAJsc8++6wcPHhQLr300oDbjBs3TsqXL++5pKWlFWgbAQBAggQn7777rowdO1bef/99qVatWsDtRo0aJfv27fNctmzZUpDNBAAAiTCsM2XKFLnmmmvkgw8+kPT09KDbpqSkmAsAAEg8BdJz8t5778nw4cPNzwEDBhTEUwIAgETpOdF8kXXr1nmub9y4UVauXGkSXOvUqWOGZLZu3SrvvPOOZyhn2LBh8sILL0iXLl1kx44d5vaSJUuafBIAAIA89ZwsW7bMTAF2TwMeOXKk+X3MmDHm+vbt22Xz5s2e7V999VU5fvy4jBgxQmrUqOG53HbbbZE+NQAASAAR95ycddZZ4gpS7/2tt97yuj5v3rzctQwAACQk1tYBAABWSejghDWJAQCwT0IHJwAAwD4EJwAAwCoEJ0bgBF8AAFCwCE4AAIBVCE4AAIBVCE4AAIBVCE4AAIBVCE4AAIBVCE4AAIBVCE4AAIBVCE4AAIBVCE4AAIBVEjo4SWLlPwAArJPQwQkAALAPwQkAALAKwQkAALAKwQkAALAKwYmIuFyxPgwAAMCN4AQAAFiF4AQAAFiF4AQAAFiF4AQAAFiF4AQAAFiF4AQAAFiF4AQAAFiF4AQAAFgloYOTJGFZYgAAbJPQwQkAALAPwQkAALAKwQkAALAKwQkAALAKwQkAALAKwYmIuGJ9FAAAgAfBCQAAsArBCQAAsArBCQAAsArBCQAAsEpCByfb9h02P49nkRILAIAtEjo4+e2Pk8HJ24s2xbopAAAgt8HJ/PnzZeDAgVKzZk1JSkqSadOmhbzPvHnzpH379pKSkiKNGjWSt956S2yy/Nc/Yt0EAACQ2+AkIyND2rRpIxMnTgxr+40bN8qAAQOkV69esnLlSrn99tvlmmuukU8//TTSpwYAAAmgWKR36N+/v7mEa9KkSVK/fn157rnnzPXmzZvLwoUL5fnnn5e+fftG+vQAAKCQy/eck8WLF0t6errXbRqU6O2BZGZmyv79+70uAAAgMeR7cLJjxw5JTU31uk2va8Bx+PDJhFRf48aNk/Lly3suaWlp+d1MAABgCStn64waNUr27dvnuWzZsiXWTQIAALbmnESqevXqsnPnTq/b9Hq5cuWkZMmSfu+js3r0AgAAEk++95x07dpV5syZ43Xb7Nmzze0AAAB5Dk4OHjxopgTrxT1VWH/fvHmzZ0hm6NChnu1vuOEG2bBhg9xzzz2yZs0aeemll+T999+XO+64I9KnBgAACSDi4GTZsmXSrl07c1EjR440v48ZM8Zc3759uydQUTqNeMaMGaa3ROuj6JTi119/nWnEAADArySXy2X9wjI6s0dn7WhyrOaqREu9+2Z4ft/05ICoPS4AAJBcn7+tnK0DAAASF8EJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACwCsEJAACI/+Bk4sSJUq9ePSlRooR06dJFlixZEnT7CRMmSNOmTaVkyZKSlpYmd9xxhxw5ciS3bQYAAIVYxMHJ1KlTZeTIkfLQQw/Jt99+K23atJG+ffvKrl27/G7/7rvvyn333We2X716tbzxxhvmMe6///5otB8AACR6cDJ+/Hi59tprZfjw4dKiRQuZNGmSlCpVSiZPnux3+0WLFkn37t3l8ssvN70tffr0kcGDB4fsbQEAAIkpouDk6NGjsnz5cklPT89+gCJFzPXFixf7vU+3bt3MfdzByIYNG2TmzJly7rnnBnyezMxM2b9/v9cFAAAkhmKRbLxnzx45ceKEpKamet2u19esWeP3Ptpjovc744wzxOVyyfHjx+WGG24IOqwzbtw4GTt2bCRNAwAAhUS+z9aZN2+ePPHEE/LSSy+ZHJUPP/xQZsyYIY8++mjA+4waNUr27dvnuWzZsiW/mwkAAOKx56RKlSpStGhR2blzp9fter169ep+7/Pggw/KkCFD5JprrjHXW7VqJRkZGXLdddfJAw88YIaFfKWkpJgLAABIPBH1nCQnJ0uHDh1kzpw5ntuysrLM9a5du/q9z6FDh3IEIBrgKB3mAQAAyHXPidJpxMOGDZOOHTtK586dTQ0T7QnR2Ttq6NChUqtWLZM3ogYOHGhm+LRr187URFm3bp3pTdHb3UEKAABAroOTQYMGye7du2XMmDGyY8cOadu2rcyaNcuTJLt582avnpLRo0dLUlKS+bl161apWrWqCUwef/zxSJ8aAAAkgCRXHIyt6FTi8uXLm+TYcuXKRe1x6903w/P7picHRO1xAQCA5Pr8zdo6AADAKgQnAADAKgQnAADAKgQnAADAKgQnAADAKgQnAADAKgQnAADAKgQnAADAKgQnAADAKgQnAADAKgQnAADAKgQnAADAKgQnAADAKgQnAADAKgQnAADAKgQnAADAKgQnAADAKgQnAADAKgQnAADAKgQnAADAKgQnAADAKgQnAADAKgQnAADAKgQnAADAKgQnAADAKgQnAADAKgQnAADAKgQnAADAKgQnAADAKgQnAADAKgQnAADAKgQnAADAKgQnAADAKgQnAADAKgQnAADAKgQnAADAKgQnAADAKgQnAADAKgQnAADAKgQnAADAKgQnAAAg/oOTiRMnSr169aREiRLSpUsXWbJkSdDt//zzTxkxYoTUqFFDUlJSpEmTJjJz5szcthkAABRixSK9w9SpU2XkyJEyadIkE5hMmDBB+vbtK2vXrpVq1arl2P7o0aNyzjnnmL/9+9//llq1asmvv/4qFSpUiNY+AACARA5Oxo8fL9dee60MHz7cXNcgZcaMGTJ58mS57777cmyvt+/du1cWLVokxYsXN7dprwsAAECeh3W0F2T58uWSnp6e/QBFipjrixcv9nuf//73v9K1a1czrJOamiotW7aUJ554Qk6cOBHweTIzM2X//v1eFwAAkBgiCk727NljggoNMpz0+o4dO/zeZ8OGDWY4R++neSYPPvigPPfcc/LYY48FfJ5x48ZJ+fLlPZe0tLRImgkAAOJYvs/WycrKMvkmr776qnTo0EEGDRokDzzwgBkOCmTUqFGyb98+z2XLli353UwAABCPOSdVqlSRokWLys6dO71u1+vVq1f3ex+doaO5Jno/t+bNm5ueFh0mSk5OznEfndGjFwAAkHgi6jnRQEJ7P+bMmePVM6LXNa/En+7du8u6devMdm4///yzCVr8BSYAACCxRTyso9OIX3vtNXn77bdl9erVcuONN0pGRoZn9s7QoUPNsIyb/l1n69x2220mKNGZPZoQqwmyNjmYeTzWTQAAALmZSqw5I7t375YxY8aYoZm2bdvKrFmzPEmymzdvNjN43DSZ9dNPP5U77rhDWrdubeqcaKBy7733WnUANu3JkJa1yse6GQAAJLwkl8vlsv1V0KnEOmtHk2PLlSsXtcetd98Mz+/TbzmD4AQAAAvO36ytc0pSUjQPBwAAyC2Ck1OShOgEAAAbEJy4XwheCQAArMAp+ZTvf9sX2yMBAAAMgpNT7vn395KVZX1uMAAAhR7BiUOfCfNjdyQAAIBBcOKwbtdB6f7kF/K/77Y5bwYAAAWI4MTH1j8Pyy3vrSjIYwAAABwITgAAgFUITgAAgFUITgAAgFUITgAAgFUITnJp454MeebTNfJHxtHoHhEAABIcwUkAd77/nXy8cqvnui7evGlPhvmpej07TybOXS//98rifD1A+w4fk3W7DuTrcwAAYBOCkwD+8+1vctuUlTL9+5M1Tx6fsVrOenaePP3pWk+A4q6Nkp9Of2KOpI+fLz9upbw+ACAxEJyE8MBHP8qxE1ny+sKN5vrL89bL24s2eW1zx9SVsnZH/vRuHD52wvxc8MuefHl8AABsQ3ASxrBK4wc+8bpt0pcbvK5/tGKrXPjSV1F7vsdnrMrRU7L81z+koC3ZuFfOe3GBfLs553Nv2XtI1uzYn+vH1v275u2l8vNOhqwAAN4ITnLBJTkXCDx09ITUu2+GTFmyWfLiiRmr5bUFG+W8Fxd63f756p3m8b/Z8LtEy5+HjsobCzfKrgNH/P790lcWy49b98v/TcqZV9Pj6bnSb8IC2X0gM+Dj6/DXiQCLKV700iL5fPUuufy1b/KwBwCAwojgJBd27g98Qr7vwx/Mz2Wb9sryX/d6naif/GSN/Gf5b0Efe9X27N6II6eGdJwGvfq1RIsORz06fZUMm7w06HaBAgy1eW9GwL9p4HHGU19I5vGc+3H0RJb5uedgpt+gSYfSUHhoT9vkhRvl8NGc7wUA8FUsxy3Is1Xb9sslp3obejerJiP7NJH9h4/LpC/Xm9su7lBbduw7IhPnrpOhXetK49SynvsePZ59Uj593Jywn/Nf3/wqGkMMOb1u2PeZu3a3+bnaERDl1a79R+TiSYvksk51ZPGpXp4Vm/+U0xtUDuv+v/1xSM54aq40qlZGPh/ZM1dt0Ondn63aIee2qiFlSxSXaMrIPC6lUwrvv01Wlkv2ZGRKtbIlIr7vi3N+kb2HjspDA0/zul0D0DOfmSuaR/7bH4dlzMAWUWwxgMKo8H7KxtBX67KTV+es2WUuL1/R3mubW99bIUs27ZV/fP2rNE0tK2t3HpBBHdPMT7c/Dx0L6/kOHT1uEnfVwNY1pEKpZImVF+b8Ilv2HpZnPl2bq/vPXrUzz7Oghr+1VFZu+VPm/7xHJvq87nmxdNNeM8SlAeCjF7SU/KbBbNkSxeSKLnXDCk6TJEku71InT89583vfyswfdsjIc5rIrb0bR3Tf52b/bH7+9fS60rBqGa9g0+3rKA5LAii8GNbJB3+fuy7kNj84El7dAcnUZVty9XzHTmQPu2Q6el5iwYbhGA1M1Iwftkf1cZ89FXBpQJnf9KSuw4DuoDOYfyzeZLa7/6Mf5GDm8Tw9rwYmavypQCM3Mo9lvwfmrtmVp/YASEwEJ/lAZ9z4SkqSArPtz8Py+oINMm/tLrnunWVmvN+G10TzbSZ8/rNXnRhfeXmZftq2T7bvOyzxYOf+I9Lzmbnyypfr/b4eGZnh5WZofs6DH//kd1jQpoTx7L8BQGgM6xQQ5/nn6VlrPPVLok2TVwf8bYH84RgS2nUgU6aN6O6ZwvvYjFVyb79mXvfTb91/7VJXWtQs5/dxr3prqUy+slPIcMJf3KEn0DZjP/Nc79awSsD2J+Uyivv19wwZ8DfvGU7RFs0TqwZpv/5+SMZ9ssbUzZl2c3evPI9gJ3g3Tbi++OX8rVAMRNvxE1lSrKj934u1zIF+njav4f8zEfnL/ndIIXHjv771/P7SvJOJsfmh25NfeAUm7iECt8te/Vq+3rBXLnxpkdc2736zWc792wKvDxCnL9bsklk/nuzyj9SyTd51UvZmBJ7tFC4NeJw1UpzDZKEE67kpKM6huG37jshLc9fL1j8Py/A3l8jCX/b4DfJ8veJTb8eWfXMH4Pf953t6SuBl0bo90vTBWQUyNJrX4ek+z8+X/i8syPNQKXKH4CROaff9JS8vkiFvfCM3v5sd+IQS6h9t/GdrzYlfT5S+bvjn8pCPn5dTo0451l6dYLNw5v+828woafvIbPPh8d2p/JJwz8nvLN4knR6fE/Xib9ounZq9L8wkZn9BxT3//s7MoPrrG9Gp/aKvk2+QmVuRBD26qQbgU5ZukQ27MyJ6HP2munj972ZWVGGjxyPR3fzeCnOMH5wWOpcqlpy5e/p5iIJHcBKHNJ+kyehPZNmvf5iy9qFK20fyZfpvX6wzJ/5nP/OfEPneks1edUvcsy/cibB5+eL+j8W/evUozPxhuxw4kn2y156doZOXeCUOX/nmkoieY8zHP5mprfqt/mR7XSbA0x6LkCdgx591Ww0QtVCdJspqu7RS8NOfrpHc0unl2Y+f64fxtE8L+fV8Zl6eA5T/frdNOjz2uakYHCl3PZtwvbFwgwx+7Wu54vXCVZxv7P9+kvaPzTZT7Qs7/f+ypQcvGgrRrvi14JfdMurD7637QkBwEocem7E6ou1/zzgqoz78wQwXhOt/351c8NCXPk7T0bM813XKsFau1RL/r85f7zdXwr0uUah/dt/E3Zv+9a30fu5L80Gnl+2nTt7O4SUdwtLVorMCPKj2rGgXsm+pffeXWM3H0eBOeyzcicy6fMBN/1oe9ANWpxRr8KQnbOfsLE1Gjob1u3M3ldrlU9BPe8A2/R64UF44dNr73oyjcvVbwYv15dXctbvkiZlrvGZcFRZvfrXJlAZ44yvv/4X8orOkvlhzclp+IPol48ufd/st9phbM77fLh0f+1xGW94zEkoBzl+IuSFvLJH3lmwxdbdsQkJsgtAeD73kJz2xnNe6RsjtAvVuv7045zi0Bg8P//cnmfVTdkCiH6hOulp05dL+a7ucPzF7zaPWtct7fteTn/Z8aIE4N3csossHqM5PzJE5d/aUciWKmy55rUvjpr1W/9extoTjl50HZP+R49KhbkUJ1y3vrZD88PHKrebn+W1reXrh1uw4IM9c0tpvMrIOpbn5HraRU1fKL7sOyoc3dZPiUUhwHP5mzuBHe+Si8diJROseaa0f9dPYvgGLBj708U9m6G1AqxqmHpAG49f942RQ/trQjmElp+uQx+rtB+T0BpXM9s+c6jn81zeb5fELW0V5zwovfc1zOxkgWrRAok34r0dUTf8+dG2RSBcx1KAl2JIB7t6hUL7/zTtpVofGnHk0WhfFmZOj6wZN/OLkt4nZq3fmurv3nOfny8UvL/IatvH17xDLGkSD7tttU1aai3s/tRdOn3vR+pzF0fT2do/ODvh4H67YahKRl0Yw3KOBkK7sHW4gpT1y01acDKgQHl3nyy3YrEANTJz1gLTnUIsg6ppXew6Gl2ehCaM6DKdDmqHE09IFBRkn6GuuvU2R9GwnAoITFLjJBdS1HSntivZNLv5s1U7z7f33MD+sg3HOmvL97MvIxQe3vw/QYAGTs/vetx6Kv/Hmuz74zuu6BjT7HTlAgTjboLPAfD01a43nG75Wtg2Uh6FBlLp96smfyD/apZ+boTR/Q63+aO9n8zGzzPBgvMnvnJNr31lmvlxFKxG+sCA4QcxpzootGfHzTq035LZxT4a0HfuZqQPjK9hnlhac096SQCs+r/BzIgjWrRtpgqEzsTj7MbJ/n7N6p2z+PTtY0u78cHo0NOfI17rdB2XcJ6sjzqlpMeZTU9n2ope9p7UXFH0Nvv+tYHNbdIkBG2nu2JV+htWi5a1FmySe2HqcckNnv+mXrmAryCvb8n7JOYEVdIaQrQL1avg7USvtCbjzVK/D07Oy1xiaunSLtK9TUV5fuCGitYO061wXZ9SFECuUKm6Kt+lUaM33CDZl2pczWfnuf5+creSvRyPUlGl/M6CcHp8ZeDq42xDHbJxYjHXr63/128vM75ueHCDxQKfgFi0S+KSpwwKayH5Tr4YF2q54pJWkteih8/Uc+f5KM5z19nDvYpPhFES02eDXvvZ8OYnmWmP5jeAEiCKd9aOJtP4Wb/xg+W9yZpOqnhkp/kra++NeNbrfhPlyPEStDB2K0YUJ9dtSDq7ondjTx38Z8O9a5C8ULTyXW9qL9Mj0VdIktawM7py7hQ51hlc80cRlDXTfu66LtE2r6DdIcQ8LHMsqmCUMxnz8Y1TeYlrdWWeVBascHU6woUFup3qVwpo6qzNUejSuIv+4uovn9g+/PZk3o0F/nUqlpLD5zTGsHA8Y1gHy0ec+ibS60nAgB44ErzMQKjBxz2S65p1lXicJLTXy2vwNcnkUaodoG/NrJlG4NPDSqbmBeq7CmbKsr5GT5tLo1NtoLVypSY5aA2fWj9ujMlVXE5e1ZowuV9Dr2XleBd00WBvhqEC9Nco9Ufpc/grIveOYXZeXuFdr8Vz+2jdmaY1IaJ6Lrk+lyz90HfeFmd7/7ebQyfa6vQpUH8p3BNV9Xade63s/VPK61hXSXi4bHT56Ql6at07W7cpZhNK22jQEJ0AB+mmbd72V/KLLDbi9Mn+9PD5zdURDScG4e3JsWlgzr1OW9Zv0VW8tk7/N+UWileSoNXBu+Oe3XoUC9eT5wEc/5GkfNu89ZIIpd57W7oOZUV+B23nC0uKH5zz/ZVgVbvOSqeEvONGTqQ5rPjY951ChzrTTIc6H/ps9rLjMMd0/2t5fusUMm/kmijtpUKLB1tnPeQeQNnCJyPOf/2x64NLHzxfbEZwAhZy7u9pm+m0urydRf0suhMu9DIK+VnpSWbpprxlm0zyOvFbYdQ5zacCiNUB6PPWFqaQaiOYtvfD5LwGH+nQmk+ZpfbPh93yZTaK52Tqz5kDmcTP9e/3uDNkZILk7mvQ4agCnM8N0lXGd4aPBsG8hx7w9R4Tbn/rpu2aZPzv2HzHvQw2aMo5Gt+LqvLW7TF6Ms2p2KL7B9gpHz9IPv+2zrrfEiZwTADHnTBzOjSdmrjbF88ac10KuOqO+19+0au/42T/LVd3rB1x12+m9pZvNLCK3u/o0kcu71JW1O7KLjTm/2X+2aod0bVjZTM+uXbGUWdwuFC3K95cXF8qiUb39/l2TdbWGzOzVO2T6LT1y/N1diFCHCZ+8uLXX374Jo+6MTuEORutuqEEd0zy3hTqPaRAXaWfBKp+eRA0ONZm8YdXSYdda8TfL5oNlW6RY0SS5sF3tyB8j6eTSGbYVZ7vyVI9fpVLJMvq8FmHdR9/3gQz8+8Jc52wVBIITANa7bcoKeeaSNpJcrEiO/AatYuuu6qtBim9wMuLdb00lYM0V6FQveJVerb3iDEzcicz6zV2Tmyde3l4GtK5hqv7qsgVadNCZXzBtRPewc3uCJQW7V9n+cWt0h9D0tZr54/Yc+xiIcx0rXT9q9h1n+p3yrjk2jR74JOL2OFdCv8+RQ6Q9Nf6Cn2IhqgXr8GX9KqU9s9Fqli9pFqEcdW4zaVY9dGCqdO9eP/V+cvZkOU/02rumL4PztdDXVqvuhiOcyscfLNti3s+Tr+wk7epU9Koto+85TXDVxF1nG7SEgb4Grw/raGYGOvkLLp1Vw23rQyE4AWC9j1duk871K8nlneuYNZec/u+VxUHv+/OO7OS/pZtyJkw6E2sDdd27Z11pgrMGJ5dMWuw3b8SZmBpuFdzkokVk7P9WycQr2kmHupUiSqbU2WHBTP9um1zZvb4nZ+KeUwte5obmLOlqvSWKFw37Ps/P/llKp4S/fTDu4Of6MxuYPJtAnMnOg149OY1Wh4iWjT4n4H2cwxu6JpXvEXj+819yBFVVy6aY2T7uwnIPfvyjVwK8Dk2VLVE8x3NpXoy+f+48p4nc0rux3/YczDzuCbD0PfXJ7Wdmt1Vc0nfCfK8csicvaiWXda7jKWFw0UuLJKVYkcimRFsWnRCcAIgLGgz4FpfTJQGcH9I6o0kXvevVrJoZXvl64145EWI8ItSaU87FHHXK6nXvLAuY0Bpp3ou7Cq7SmTirH+knf/siOkm56uH/rZJzW9WQauVK5CkwyU1Z96ajPzHBTLS9Mn9DxPdxDhH5vhumLt3sKeXve0zM9i5XjllcmoejF729/aklHnynH+sMImcNHe19eXTGas+iqs/N/llG9Grkt71PfeJdbmD699u8Amzfgmra6+S7Arjvax+qF06TqidKnCfETpw4UerVqyclSpSQLl26yJIl4S1bP2XKFNMFdcEFF+TmaQEkeF7KX/6+MMe0YF/uRe90eEUTAo8cy9sJ0hkQaXKrLmmQXzQBNNy1h9xCxQsDXlwonzoWzoyGUOXqVX4EJvnh3v/84LUAaCSv8X5HkOovUVUDG10VXROhtYfOd7V354KmTmscq6jr+kiavOoWqNKrbzHEeBdxz8nUqVNl5MiRMmnSJBOYTJgwQfr27Str166VatWqBbzfpk2b5K677pIePXImdwFAOPTbqlNWgF4RrUlRWOR1RoWezK7/R/YCl3mlSzo4F8yMN856QVrYMJRgr74z+dhfLo728jzp0wvitMZnWr4GJVe89o3XQqY61Ojs2UkUEfecjB8/Xq699loZPny4tGjRwgQppUqVksmTJwe8z4kTJ+SKK66QsWPHSoMGDfLaZgAIutLtuS9kJ1rGu0/C6KUoKJpUfPuU6BTh08qwseBcgsE32PXnvyu3+V2VXPnmP/kKFpiov51a9dyt34QFYa2wnggiCk6OHj0qy5cvl/T09OwHKFLEXF+8OHBS2iOPPGJ6Va6++uq8tRYAHAJVX/U32yNehToB5mbKbW7pLJbvHEMMefGXv38l8eCFOb94LUMRiK1VYRNiWGfPnj2mFyQ1NdXrdr2+Zo3/CHHhwoXyxhtvyMqV4S97npmZaS5u+/fHtiIlADs9+1ngOg6JwjkdN7/pkI4tlX4jpSuET/CZdRNNBb0/hV2+ztY5cOCADBkyRF577TWpUiX8RZ3GjRtnhoAAAIiGzo9nL8iJQhacaIBRtGhR2bnTe/xNr1evXj3H9uvXrzeJsAMHDvTclnVqxcxixYqZJNqGDXMu7z1q1CiTdOvsOUlLy65UCAAACq+IgpPk5GTp0KGDzJkzxzMdWIMNvX7zzTfn2L5Zs2byww/eK4eOHj3a9Ki88MILAQOOlJQUcwEAAIkn4mEd7dEYNmyYdOzYUTp37mymEmdkZJjZO2ro0KFSq1YtMzSjdVBatmzpdf8KFSqYn763x0KN8iVMKWAAABDHwcmgQYNk9+7dMmbMGNmxY4e0bdtWZs2a5UmS3bx5s5nBEw8sXpARAICEleSyec1kR85J+fLlZd++fVKuXHiLN4Wj17Pzopp9DgBAvNrkKLcf6/N3fHRx5JPzWteIdRMAAICPhA5Oru+Zc6YQAACJ6JjP4oGxlNDBSZkUFmUGAEB9tW6P2CKhgxMAAHCSTRmoBCcAAMAqBCcAAMAqBCcAAEAkyZ4XgeAEAABI5jFm6wAAAIscZSqxPZKL0nkEAIDLouk6nJkBAIBYFJsQnDSvUTbWxwAAgJhziT3RScL3nJQtUTzWxwAAgJhz2RObEJwkF0v4+AwAAMkiOLHHwwNPi3UTAACIuSyLuk4SvtugTuVSsT4GAADEnIvgBAAA2MRlT8cJPScAAEDIOQEAAHY5YVHXScLnnAAAALFqXIfgBAAACFOJAQCAVbLoOQEAADbJsmdUh2EdAAAg1DmxTa0KJT2/35HeJKZtAQAg0Yd1isW6ATb4+Obusnj979L3tOpmrZ3nP/851k0CAKBAncgSazBbR0SqlEmRgW1qehYBvK1341gfFwAAEnYhXHtaYpFO9SrFugkAABSoVrXKiy0ITgAAgLDwn+VqVigR6yYAAFCg7EmHpefErwZVyxT0cQAAIKZcFkUnDOsAAABxWdR3QnACAADEotiE4AQAAIhNsQnBCQAAEKsqxDKsE0U9GleJ5sMBAFBgLIpNCE4iMe6iVkH//o+ru0jXBpXzeEgAACh4FsUmrK0TytnNqsmF7WpJ27QKklaplBzPcsmD034MWFnvrr5N5eKXF+XLwQIAIL9QhC2OFEkSs+6OBiZqyOl1ZcatZ+TY7o0rO5qfHepWLPA2AgCQVwzrxLnTapaXO9KbeK43qFpaqpWlqiwAIH65LBrYKRbrBtiuWfVyfm+/Lb2xDOlaV95bslkubl+7wNsFAEBh7TkhOAlg+i1nyGerdsqNPRsGfPEqlU6WEb0a5dexAQAgIYOTXE0lnjhxotSrV09KlCghXbp0kSVLlgTc9rXXXpMePXpIxYoVzSU9PT3o9rZoWau8jDyniZRMLhrrpgAAkFAiDk6mTp0qI0eOlIceeki+/fZbadOmjfTt21d27drld/t58+bJ4MGDZe7cubJ48WJJS0uTPn36yNatW6Wwql6O/BMAQHxxiT2SXBHOHdKekk6dOsnf//53cz0rK8sEHLfccovcd999Ie9/4sQJ04Oi9x86dGhYz7l//34pX7687Nu3T8qV858DYpOjx7OkyehPYt0MAADC9sqQDtL3tOoSTbk9f0fUc3L06FFZvny5GZrxPECRIua69oqE49ChQ3Ls2DGpVKlSwG0yMzPNDjkv8SS5mP+XtU1aBc/vqeVSCrBFAADEj4iCkz179piej9TUVK/b9fqOHTvCeox7771Xatas6RXg+Bo3bpyJtNwX7ZmJdxVKFZePR3T3XE+SJElvXi2s+9YozzARACB/xX1CbG49+eSTMmXKFPnoo49MMm0go0aNMl1A7suWLVsk3iWd+jm4cx1PJdlXh5ws3BaK1lEBACA/xW2F2CpVqkjRokVl586dXrfr9erVg49TPfvssyY4+eyzz6R169ZBt01JSTFjU85LYfHEhS3l61G95ZIOtaWIlp8NoKOj0qz2sjh9dseZUqtCyYD3/fCmbnJlt3ry9lWdo9RqAEBh5xKJz+AkOTlZOnToIHPmzPHcpgmxer1r164B7/f000/Lo48+KrNmzZKOHcPrLSiskpKSpHqAYZpzW2UHeKVSigWs2le1TPB8lZRiReThv5wmPZtUzXN7AQCJwWVRdBLxsI5OI9baJW+//basXr1abrzxRsnIyJDhw4ebv+sMHB2WcXvqqafkwQcflMmTJ5vaKJqbopeDBw9KYfbdmD7yxZ09I7rP5Z3rhrVdSvHwD5v2oERTsSC9PQCA+OWyqO8k4uBk0KBBZohmzJgx0rZtW1m5cqXpEXEnyW7evFm2b9/u2f7ll182s3wuueQSqVGjhueij1GYlS9VXBpULRP29pP+2l6KOk78XRtUDrhtqeTwC/tqD8qGJ86VaKlRgeRcACiMsuK9fP3NN99sLoGKrjlt2rQpdy1LON49Etf0qC+VSydL14aVZcWWP+Wrdb+b29+9pkvEjxwstwUAALX3YKbYgrV1CjDXJJTaFbOTXIsXLSKXdkrz3N40tazUrVxKShSPbTl9m8YkAQDRs+n3Q5KQU4kTWbPqZUNuk1aplLw5vJNXPRR3YNO0elmvwCSMWCcs00Z0N/kxvvqe5l3LJlBwoisyr4/isBEAAAQn+eyT23rIFV3qyIRBbcPavlfTal6VZPNb27QKJj/mgxu6SsVSxQNOXw5UHO65S9uYXJlLO9bO55YCAPJTkWh9640CgpN81rxGOXn8wlZSLR8XAyxXIu+jc53qVZJ3rgqdz+J87zp7UZ68qHVU2uEUqxL/Q7uGN2sKAAqTIvbEJgQntiiZnPtckoX3ne11PZxeD3+rJ7eqXT7kfV64rF3ApNsL2tUKeL/c9KyEs7pztFeAXnTf2XL/uc2j+pgAEA+SCE7gNqp/M7mgbU3p0ahKRC/Kff2bmZ/DutaVciWyh2PC8dFN3aRH4yryztWdI36Ttg4SwARLln36kjby7YPnSONqZUJOlXbTHqdQpt96hkRTzQolTW7PxnHB82hm3d4jqs8LALFWrKg9gynM1omx63s2zNX9zmtd05zgK5VOjvi+7epUlH9cHfmUZF8ta4W3rECDKifXBtK2/uuaLvLf77aZ9p8+LrvSsD+BKuk6VSmTYh7zite/kYKcXdWsejlTiTfzeFZUnxcAYqVDnexlU2LNnjAJEatcJsVzEtVCa1q9VVc/bnSqdyKautSvZBJ7dYqzru1z9Rn15amLg6+R5I/m3lzTo4EJPM5vW9O0Nb25/5lBycWKyDVn1A/5mN0bVZF/BOkFCtffBvsfsgqkc/1KAf+mx+KNYdFdquHCIMNmAJBXxYraM65DcFJIaM7H2sf6y9IH0s1JPbdqnuqt6N+qhtftEy5r6xlmaZJaVh48r4UJjpyqlo0sgVXzVz4f2dOrJP7jF7b0/K7DVaPPa2ECrlB6NK4qY85rEdawiw7ZlC/p/ZjLR6fLX9rUjKj9vsNYGiAueaC3yVv55fH+0jtA0FUYVgyNlucHtYl1EwCc0qKmPYvsEpwUIjqlV3s2/Al3EcBPbj/TTCse2No7OAnHtT0aSF6d2fhkO8s6Zv6UykXhOR12Ccb5+IGGce5IbxJxgFitbAmTt+J+PJ1CXv/UsJZN2tfJ23T1aE0dv7AdU9ABW1Qra8/yJAQnCSLc3BTtUdBpxXpyfeGy7Nos4Xxpz8uMI2chOu15+Ob+3p7bHjk/uzclWK5LqCY+dXErWfVI37Cq9arz2kQeoPnSGUxz7zrLFNfrXC/wMFAgziG6aPab9GuZvQK2P86aN/4414ECgGgjOEkQOj1WZ+i8fEX7sO/Tq1m1qJwYrz0zsh4V7XlwLm6Y3iJVfhrb1wyVLB51trRLy13SVq0KpSJaNNFp0l875GnanRbXe/+Grjlub1EjeA9PUceTRHNUp2XN4NPGlzyQHlZVYwDIDwQnCULzQXSGjm8uSTC5+W6sSa5KAyEtwz/nzp5y2ak1ggIZdOrvwaYpl04pZoasapTPXn8omkLta+/m1aRJauBEY2cvUyRm3hZ8SvJpjl6iaPZW6IKSwehrbVNyHIDEwlRiRJUmuWqeRbhDJ+4eGg1inAsfBhPooQv6VOrsyTi/be5n0tzWu7G8MOcXOb1BJfl6w17P7Uvu7y0pxYpK1TIpZnioXMni8s2G32XbviN5avflXep4HZ8qZZJlz8Gjnuu9HT1mbk9f0toUvBs6eUmenhsAwkHPCaIuksDErWHVMuZEHE160ndOU3bmbwSqdBsLd5zTRFY/0k/6nZadB7Lm0X5m2rWuezTq3OZmGYRaFUrKVz7VgN10mrevR88/zTyu05MXtZLHL/DO4XHS/JhXhuQcwtLnPrNJVbn17EamtsytvRv7vf/1PcMbwtP7L7y3V1jbAkg8BCcIqHRyMTMcpGvmpEY4TdiWk/7rwzqaBNtPbz/Tq6hbh7oVTc9EMJUdScT5vSCWJhMP6lTHtOvuvk29VqD2DfweGtjC0wPi5lvb5qUr2suQrvXM4zoDBh1C8w0e/3ZZO7OvGszpzCJ/VSLdvUQj+zSVpQ/0Dji8Nqp/eKX/R57TRGpXLOUJfJB3eswLCy1V8G8/OVoQ04uZCBjWQdCpsXpi1xOTTWWNtbrsO4t/lbRKJcNOsPVH67To7CQ9V+uQia8KpZLNtOrkokVMvsdjF7SSS19ZLHf1iWyKcSDpzb2HTzSQ+M+N3ULeb3j3+nJR+9qm7Zd0qC0Lft4jfz29roz93yrz9xG9Gsq5jtwiZ2DlDExm3tpD9mYclW6Nqpj6OHq886NnTDcf0KqGTP9+uxQErVejVYgTjTNnS/83tuw9nG/PpT1svZ6dF/H9Ztx6hgz420K/swn1veimRR59Z4/9ceiYJLL/61DbzFzUz4l7/v29FHYEJwgqUN2UWNLKrFq8La/fuDXg0JOy+3d/dFq183l/fqx/norcOXs6/M0ACpe7iFz7OhXNRem+/LRtn6dWjJtW2f14xVY536fCrLPgkr/AJNwFJH3d26+ZPDVrjef6xnEDzM/p388Ied+yKcXkQOZxyWul33gMTj4feaakj58ftEKw9oL1m7Ag5GPp/0Z+Bie5dVqAWWKNqpaRJRnZ+VbISYdV3eUaOtWrKEs3/SGFmX1nHiDMk7tvXZWO9SKfYqyBRiTBhnPb9nVzvw5Fvcr+h0/yQofgzmpaLUegoT1EmquiQUN+eeDc5p4lBG4IM+/En5TiifuR1KhaWa/6Pr50uYhAxQW1q19zgdyinb+VaDrm4X9bh2ZDDVnl1ZTruppq1IVZ4n4SoNBpXbuCGRb5elTB/NPedFZDGT2guenFiVTbtOB1RmxIUnZqULV0yFo2uoSA+7m6nZqq3M5PJVrfirmaf+LWpX7o1aptck+/plE5wbmTg1PLBa7Q6QpQbUhzMy7tmGbylHTl7+/G9JF6lU/m80TDowESqM9pkVpghSLz+v6NVF6eLtTQbNPUvNcPKnqqGvXU606XworgBIWKfmsJZzXjaNCTgS5iGMlCi7PvONN8c4q0MF2sBcrbCWTi5e3Nfr42NOfih751by7uUFu+vPssWTnmHKlcxv8JqlqECdnX9qgvdSqV8ts1XrdyqVyt5u3PTWc1Cvg3LR5YIsyeIHdycKCZV4HotO+OjqFH3S+d4XVn36YyqGPw+kJ59eqQDl4n4mB1isKR16Tzt6/qbIo1RkNuhzTd/nNj1zznlyT6wqAEJ0ABapxa1iT7FfZu94qlk81+OocagqlbubRJQA4k0vPWAwNamIDH19vDO8ncO8+Sh/9ymtjKNxk0GE1m9kcXzXwql7M6zmoa3jpc2puhXwZ09tjrQzuaIcVgbjyrYYjHC/K3MNqjXxK0WGOgHiwNTJ84tXhpuNPdQw3RtE3z7hmc9NeTM6Y61K0kX9zZM9croVcrF/nsyOcHZReC1CKY2rMbzwhOAPhVwD3pYXMnQof6pqgnT9+Tkd4W7qwkf9WPtXcn3N4Qf0pFYf0p5+rZofbksQtaetXPCYcOVUZCZ49pL1EowXKeAg1ZReO9+OrQjqbmz4uXtYtouEuHG0MN0Uwb0d3r2PZrmT1LrkHVMrLgnl5S3KfSsu6rPrb2bEWywGpSUvjDXVoX6R7H662BWW56Y2KJ4ARAgcrrEkH6gT/9ljPkmTB6BrQWjG7rK9C5rmSQFbD1A197d8Lt8tfcGt9tNeHV+W1ZT36+QxGuXOaA+KNTzDWRNlzaZk3Mjdaq08F6ZnyH98KNP/Sk7m/FcFeAxad0qEtr/uhwlz9DTq+b88YIgqHujU7mSV3WqY7fhUx/HNtXPrypW45eGe3ZiiSXxpWHf5zq5VLM+mrBjsdF7e0aFmIqMQDrNUktKzv3Z5rfteejZS3v/IYKpYqbXA1/S75H8q072LaRnBy01kwfn96E7x7qI2VLFPdKiAw1bBBsGnm4QvVKOH1yaq0n59T6cJeViCQfadH6383Qw5SlW8K8V3Z73MNVuvaTBpOPTD9Z3yeanAtuBuI+dq8M6ShLN+6VbqeCFF86hOue7h+u89vWkolz14fcrkaY+XX6/xOI9uJoVWjbhpoJTgBYr89p1c2ltU9Q4nR337xPle7ZpKp88uOOoNv4nrd81yYK1BZ3UKELSAYKpCKV2+GOmuVL5FijSQMSd2ViZyB2VpOqcv+5zUxOx1VvLcvziVLzQvzN9Il0X0b0aiQHM497ghN/vRC+VWZ1+r9WQ9YeomW//hFwPx67MHjvlPaEND81rbtMSjGvFdwjdcvZjeTFL9aZ33Xo5b7+zcz0/3Boz4zm+1QsffK9pQHxZ6t2yuWdc/biBFrIU3v3bAtMFMEJgAIVrAci0PmpV9OqXjNa8suTF7c20597NqkmfScELojmm3egRd/+PHRMXp2/IeT2ehJ9PEhiZiQ9NOHMOtKgqE3t8nI8yyU/bdsfcDtdRTxQe687s6FkZWU3LNVPwqaeWH/ZedBMI7/mncBBTG4EClwCDeW4OWcyKQ2+dGhNA7H6o2bm2L5lrXIy/ZbgK4WrSHtCnHxr1dzZp6knONEAItzAxM2Z76M9IIePnZBSyTlP7dprF0/IOQEQdJrseREk7UV7mEE9fXHrPAUmkZzs9USuJ+Km1cvmOAEHare2TV+r/i1zJp4+d2kb81O/DUfT84PayODOdcxSDqFocKEB1P9uzpl74+Q7VOZLh9OWj043a1L5O/lpUcExA1uElRzrt51BEj10fa9ICyU2DFCbR9vp7GWJ9P2YW5pMrfVstFiiL3dzujtmX6WFuTyH9+Mk+T02br883j9uFtyk5wSAX/1aVpfFo86W1CgMP+RFoNoneeGvOJyvs5ulyntLNucIcnQGzMj3vzPd8d6PWdH0PtRy5GjoGkerHukb9ISRGxe2q20ukZy0ojH7KtJv9Xk19i+nyc87D5gE0gsmfmWKzYXy35u7y6R5683Cn/mdaBoJTaYONF1eC0eu2rbfaxr3P67qIi/M+UU+WrHVXC8bQYAWbDkS30C/WY28F4XLDwQnAAIKtPqwbSI9wegHdKjF63RKbdPUMvLwqQUV3XTRxbObVfN7omnjU/dCRRqY6HRXnS6ti1Gu3p49DFOsSPQ6uquVy5lzEmv+gqdh3ep5ftdFSIPNbnEP8eiwyYTL/NcPCUSL/O06kCm9m6UG7b05r03NfClup5WBfasD16tS2tQu0cRbHY7zXTMrEoFet/euPd3K9dOUna0CYD13GfrcdD9H8iEaSQXeSNt/Z5Bv15q4eWV3/wXRghWMyysddtACcjN8pkAHWpwyNzQA0hWt86s3QRebjIQuqpmUi/dGNMra665Ov/UMeeGytibJNlh+j9bN8ReA5qe/nl5Xxl3UKtf1eVSrAEN2UXxLRR09JwByRXseNu7JCHuWRqQnPn383w9mmuqxeaHTjAO5+exG8tzsn8U27kUh7+rTRJ797OeAlURzy3WqqNy/l/8m+UFzbAa0riGXv/aNSdAMRIOweWt3y6BOafLRtyeHL2JBZ07p9N3C5vORPWXTngwT/MUbghMAuea7iF+0Hzsaj69DOI9f2NKUdPf3zXvW7T1k7H9XyV19w8tRKEg3n93Y9N7odNVoc6499GKUgx8NrjQHZ2CbGvL+st+kcYDeLw08h3UrHZPF/RJBo2pl8q3nMb8RnAAo9K7o4qcK6Cmao/BeGKu7hpq2ml/yIzBxBwZvDe8klUunSKs8LtoXiK5hpNN5NUfHVqWCVAVG7BCcAIh7zarbOePAdoEW62sQYBpupDQZOJwZNrGgvWkffrvVDO3BPgQnAOKWDsm8s/hXufXsxvn+XDHqOImJK7vXk/1HjkmvECsNx3tvWrAetURQPx+HZfOK4ARA3NIhGd+VhxFaqCRjLWcejeUAIpGblBPnQo1V8qkGy19PryP//Hqz3NW3qRQW8+/uJQcyj5kp5bYiOAGABDHlutNl9qqdctNZDaUw0OnV3z/cR1xZJ0vT54dHz28pt/Vu4reya7yqUzn/l4LIK4ITAAhDYRjVOb1BZXOxUW4n6/ibhRVNOouoMAUm8YIibAAKlHupeX9a1PBeFM0miZRzEgvB1tZB4qHnBECB0PLjWrQt2Df3izvUloyjx6WTz2qyABJLrnpOJk6cKPXq1ZMSJUpIly5dZMmSJUG3/+CDD6RZs2Zm+1atWsnMmTmXqgZQuNWsUNJr1dVAOQTDu9cPuUIuCh9qsCFPwcnUqVNl5MiR8tBDD8m3334rbdq0kb59+8quXbv8br9o0SIZPHiwXH311bJixQq54IILzOXHH3+M9KkBAEACiDg4GT9+vFx77bUyfPhwadGihUyaNElKlSolkydP9rv9Cy+8IP369ZO7775bmjdvLo8++qi0b99e/v73v0ej/QBQIE6raW8+TGFAxglyHZwcPXpUli9fLunp6dkPUKSIub548WK/99Hbndsr7WkJtL3KzMyU/fv3e10AIBY+u+NMmTCorfRuXngLkgFxHZzs2bNHTpw4IampqV636/UdO3b4vY/eHsn2aty4cVK+fHnPJS3NzvLHAAq/Jqll5YJ2tViYDkj0qcSjRo2Sffv2eS5btmyJdZMAAPmIVYmR66nEVapUkaJFi8rOnTu9btfr1atX93sfvT2S7VVKSoq5AAASQ88mVc1U80qlk2PdFMRbz0lycrJ06NBB5syZ47ktKyvLXO/atavf++jtzu3V7NmzA24PAEg89/ZrJo9e0FKm33JGrJuCeCzCptOIhw0bJh07dpTOnTvLhAkTJCMjw8zeUUOHDpVatWqZvBF12223Sc+ePeW5556TAQMGyJQpU2TZsmXy6quvRn9vAABxqWRyURlyemKvEow8BCeDBg2S3bt3y5gxY0xSa9u2bWXWrFmepNfNmzebGTxu3bp1k3fffVdGjx4t999/vzRu3FimTZsmLVu2jPSpAQBAAkhyuexfMUKnEuusHU2OLVeOWgMAAMSD3J6/rZytAwAAEhfBCQAAsArBCQAAsArBCQAAsArBCQAAsArBCQAAsArBCQAAsArBCQAAsArBCQAAsArBCQAAsArBCQAAiO+F/2LBvfyP1ugHAADxwX3ejnQZv7gITg4cOGB+pqWlxbopAAAgF+dxXQCwUK1KnJWVJdu2bZOyZctKUlJSVCM6DXi2bNlSaFc7Luz7yP7FP45hfCvsxy8R9nF/Pu6fhhgamNSsWVOKFClSuHpOdIdq166db4+vB6MwvuESaR/Zv/jHMYxvhf34JcI+lsun/Yukx8SNhFgAAGAVghMAAGCVhA5OUlJS5KGHHjI/C6vCvo/sX/zjGMa3wn78EmEfUyzcv7hIiAUAAIkjoXtOAACAfQhOAACAVQhOAACAVQhOAACAVRI6OJk4caLUq1dPSpQoIV26dJElS5bEukkybtw46dSpk6mGW61aNbngggtk7dq1XtucddZZplKu83LDDTd4bbN582YZMGCAlCpVyjzO3XffLcePH/faZt68edK+fXuTod2oUSN566238v01evjhh3O0vVmzZp6/HzlyREaMGCGVK1eWMmXKyMUXXyw7d+6Mi31z08f03Ue96H7F4/GbP3++DBw40FR41LZOmzbN6++aUz9mzBipUaOGlCxZUtLT0+WXX37x2mbv3r1yxRVXmAJPFSpUkKuvvloOHjzotc33338vPXr0MG3VapVPP/10jrZ88MEH5v2i27Rq1UpmzpwZcVsi2b9jx47Jvffea56rdOnSZpuhQ4eaitWhjvmTTz5pxf6F2kd15ZVX5mh/v379CsUxVP7+H/XyzDPPxMUxHBfGecGmz85w2hKSK0FNmTLFlZyc7Jo8ebLrp59+cl177bWuChUquHbu3BnTdvXt29f15ptvun788UfXypUrXeeee66rTp06roMHD3q26dmzp2nv9u3bPZd9+/Z5/n78+HFXy5YtXenp6a4VK1a4Zs6c6apSpYpr1KhRnm02bNjgKlWqlGvkyJGuVatWuV588UVX0aJFXbNmzcrX1+ihhx5ynXbaaV5t3717t+fvN9xwgystLc01Z84c17Jly1ynn366q1u3bnGxb267du3y2r/Zs2frjDjX3Llz4/L46fM/8MADrg8//NDsx0cffeT19yeffNJVvnx517Rp01zfffed6y9/+Yurfv36rsOHD3u26devn6tNmzaur7/+2rVgwQJXo0aNXIMHD/b8Xfc/NTXVdcUVV5j3/nvvvecqWbKk65VXXvFs89VXX5l9fPrpp80+jx492lW8eHHXDz/8EFFbItm/P//80xyHqVOnutasWeNavHixq3Pnzq4OHTp4PUbdunVdjzzyiNcxdf7PxnL/wjmGw4YNM8fI2f69e/d6bROvx1A590sv+j+RlJTkWr9+fVwcw75hnBds+uwM1ZZwJGxwoh8wI0aM8Fw/ceKEq2bNmq5x48a5bKInOv1n+/LLLz236cnttttuC3gffdMVKVLEtWPHDs9tL7/8sqtcuXKuzMxMc/2ee+4xQYLToEGDzD9Bfr5GGpzoB5w/eiLQf+QPPvjAc9vq1avN/utJwfZ9C0SPVcOGDV1ZWVlxf/x8P/h1n6pXr+565plnvI5jSkqK+fBW+iGn91u6dKlnm08++cScHLZu3Wquv/TSS66KFSt69k/de++9rqZNm3quX3rppa4BAwZ4tadLly6u66+/Puy2RLp//ixZssRs9+uvv3qd2J5//vmA97Fl/wLtowYn559/fsD7FLZjqPt69tlne90WT8dwl895wabPznDaEo6EHNY5evSoLF++3HSnOdfv0euLFy8Wm+zbt8/8rFSpktft//rXv6RKlSrSsmVLGTVqlBw6dMjzN90H7U5MTU313Na3b1+zuNNPP/3k2ca5/+5t3Pufn6+RdmFq92uDBg1MN7F2NSp9Pu1Gdz6ndo/WqVPH85y275svfa5//vOfctVVV3ktWhnPx89p48aNsmPHDq/n0XU0tKvXecx0GKBjx46ebXR7bc8333zj2ebMM8+U5ORkr/3Rrus//vgjrH0Opy3R+p/UY6n75KRDANqN3a5dOzNc4Owuj4f90+587epv2rSp3HjjjfL77797tb+wHEMdXpgxY4YZlvIVL8dwn895wabPznDaUmgW/ou2PXv2yIkTJ7wOktLra9asEZtWY7799tule/fu5iTmdvnll0vdunXNCV7HQHVMXP9BPvzwQ/N3ffP72zf334Jto2/Uw4cPm3+2/HiN9J9QxzD1A3D79u0yduxYM4b7448/mjbpP77vh74+Z6h227Bv/ujY959//mnG9AvD8fPlbo+/53G2VU96TsWKFTMfrM5t6tevn+Mx3H+rWLFiwH12PkaotuSVjqXr8Ro8eLDXAmm33nqrGafXfVq0aJEJOPX9PX78+LjYP80vueiii0wb169fL/fff7/079/fnEyKFi1aqI7h22+/bXI3dH+d4uUYZvk5L9j02RlOW8KRkMFJvNCEIj1pL1y40Ov26667zvO7RsKaWNW7d2/zodKwYUOxmX7gubVu3doEK3qifv/9901yWGHzxhtvmH3WQKQwHL9Ept8GL730UpPQ+PLLL3v9beTIkV7va/1wvv76600io00lwQO57LLLvN6Tug/6XtTeFH1vFiaTJ082PbaazBmPx3BEgPNCYZOQwzrana7fBnyzh/V69erVxQY333yzTJ8+XebOnSu1a9cOuq2e4NW6devMT90Hf/vm/luwbfTboAYJBfUaaXTdpEkT03Z9XO021J6GQM8ZT/v266+/yueffy7XXHNNoT1+7scK9jz6c9euXV5/1+5ynf0RjePq/HuotuQ1MNFjOnv27JDLyusx1X3ctGlTXOyfLx1y1feQ8z0Z78dQLViwwPRShvqftPUY3hzgvGDTZ2c4bQlHQgYnGhF36NBB5syZ49VVpte7du0a07bptzJ9A3700UfyxRdf5OhG9GflypXmp34DV7oPP/zwg9eHifsDtUWLFp5tnPvv3sa9/wX1GulURO0x0Lbr8xUvXtzrOfWDRHNS3M8ZT/v25ptvmq5wnbpXWI+fvj/1A8f5PNoFrHkIzmOmH1Q6Fu2m721tjzsw0210OqgGAc790eE/7S4PZ5/DaUteAhPNldJgU3MSQtFjqmPx7qEQm/fPn99++83knDjfk/F8DJ09mfp/0aZNm7g6hq4Q5wWbPjvDaUtYXAlKp0NphvRbb71lMtGvu+46Mx3KmckcCzfeeKOZZjZv3jyvKW2HDh0yf1+3bp2Z7qbTszZu3Oj6+OOPXQ0aNHCdeeaZOaaM9enTx0w702lgVatW9Ttl7O677zaZ1BMnTvQ7ZSzar9Gdd95p9k3brtPudFqbTmfT7HP3FDSdIvfFF1+Yfezatau5xMO+OWkGu+6HZvM7xePxO3DggJl6qBf9yBg/frz53T1bRadG6uPqvnz//fdmJoS/qcTt2rVzffPNN66FCxe6Gjdu7DUNVTP8dZrmkCFDzHRJbbvun+80zWLFirmeffZZs88688vfNM1QbYlk/44ePWqmetauXdscC+f/pHuGw6JFi8wsD/27Tk395z//aY7X0KFDrdi/UPuof7vrrrvMTAp9T37++eeu9u3bm2N05MiRuD+GzqnA2h6doeLL9mN4Y4jzgm2fnaHaEo6EDU6UzuHWF1DnbOv0KJ2/H2v6j+XvonPc1ebNm82JrFKlSuYNorUG9I3krJOhNm3a5Orfv7+Zh68nfw0Kjh075rWN1t1o27at2X89QbqfIz9fI52WVqNGDfN4tWrVMtf1hO2m/6A33XSTmbKn/yQXXnih+SeMh31z+vTTT81xW7t2rdft8Xj89Hn8vSd1+ql7euSDDz5oPrh1n3r37p1jv3///XdzIitTpoyZujh8+HBzQnHSmg9nnHGGeQx9b+iHuK/333/f1aRJE7M/OuVxxowZXn8Ppy2R7J+erAP9T7rr1ixfvtxMF9WTR4kSJVzNmzd3PfHEE14n9ljuX6h91BOcnrD0RKUnUp1Sq7UrfIPYeD2GbhpE6P+TBhm+bD+GEuK8YNtnZzhtCSXp1I4DAABYISFzTgAAgL0ITgAAgFUITgAAgFUITgAAgFUITgAAgFUITgAAgFUITgAAgFUITgAAgFUITgAAgFUITgAAgFUITgAAgFUITgAAgNjk/wFBwHHqOoJiYAAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.plot(stepi, lossi)" ] }, { "cell_type": "code", "execution_count": 51, "id": "6a918c14-8394-4759-86bc-0014cb6bf4a5", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor(1.8583, grad_fn=)" ] }, "execution_count": 51, "metadata": {}, "output_type": "execute_result" } ], "source": [ "emb = C[Xtr] # (batch_size, context_size , e_dims)\n", "h = torch.tanh(emb.view(-1, context_size * e_dims) @ W1 + b1) # (batch_size, hidden_layer_size)\n", "logits = h @ W2 + b2 # (batch_size, nb_chars)\n", "loss = F.cross_entropy(logits, Ytr)\n", "loss" ] }, { "cell_type": "code", "execution_count": 52, "id": "fb5e0fa1-86ef-4d72-926f-a3f6bbbcff8d", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor(1.9103, grad_fn=)" ] }, "execution_count": 52, "metadata": {}, "output_type": "execute_result" } ], "source": [ "emb = C[Xdev] # (batch_size, context_size , e_dims)\n", "h = torch.tanh(emb.view(-1, context_size * e_dims) @ W1 + b1) # (batch_size, hidden_layer_size)\n", "logits = h @ W2 + b2 # (batch_size, nb_chars)\n", "loss = F.cross_entropy(logits, Ydev)\n", "loss" ] }, { "cell_type": "code", "execution_count": 53, "id": "0f813077-760c-45b9-82d2-9f42bc391eb9", "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAp4AAAKTCAYAAACw6AhNAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjgsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvwVt1zgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAYHhJREFUeJzt3Qd8VfX5x/EnG8JMGMoIJICKiBAFgrhtBa1atSJK1VattbZ1FP1XsVatqyraCnW0Sqtoh0WBKmrRQq0IKHvI3gTCJgGSkED2//UcvDHj7vG7597zefeVxtycc3Jyci/3e37j+SXU1dXVCQAAABBhiZH+AQAAAADBEwAAAMbQ4gkAAAAjCJ4AAAAwguAJAAAAIwieAAAAMILgCQAAACOSxcZqa2tl9+7d0qZNG0lISIj26QAAAKAJLQlfWloqXbt2lcTExNgNnho6s7Kyon0aAAAA8KGgoEC6d+8eu8FTWzpdv0jbtm1DPl5VVZXMnDlTRowYISkpKWE4Q+fhGnId7YTnI9fRTng+ch2d+lwsKSmxGgpduS1mg6ere11DZ7iCZ3p6unUsgifXMJp4LnId7YTnI9fRTng+xu419GdYJJOLAAAAYATBEwAAAEYQPAEAAGAEwRMAAABGEDwBAABgBMETAAAARhA8AQAAYATBEwAAAEYQPAEAAGAEwRMAAABGEDwBAABgBMETAAAARhA8AQAAYATBEwAAAEYQPAGEXW1tHVcVANBMcvOHACAwq3cVy5QlBbIo/6Bs3n9EqmrqJCUpQfp0bi152ZkyanCW9O/WjssKAA5H8AQQtPzCMnlg2kpZtO2gJCUmSE2Dlk4Nn+v2lMrGfUfkrfnbJS8nU54bOUCyO7biigOAQ9HVDiAo01fskhHj58jS7YesrxuGzoZcj+t2ur3uBwBwJlo8AQRMw+OYySskkJGcGkBrpM7aT12V240rDwAOQ4sngIBsKyyT+6es9Bo6fzdqgEz8wSC339P9dH/tpgcAOAvBE0BAxk5bKTV13ts6H/9grfxyylcev6/769hQAICzEDwB+G3VzmJrIpGn8ZwupRXVUnKs2uP3dX89js6GBwA4B8ETgN+mLi2Q5MQEn9t562p30VnwWoIJAOAcBE8AftM6ndVhKg6vrZ6L84/PiAcAOAPBE4DftDh8OG3aX8rVBwAHIXgC8HsZTC0KH056PJbXBADnIHgC8O8fi8QEaxnMcNLj6XEBAM5A8ATgN117PZxO6tyGqw8ADkLwBOC3vOxMazZ6OOhxhmRncPUBwEEIngD8Nmpwls8anv7S4+jxAADOQfAE4Lf+3dpJXo7vVs/UpEQpq6zx+H3dX4+jxwMAOAfBE0BAnhs5QJISEjwGSh0HembPDNm0z3OpJN1fjwMAcBaCJ4CAZHdsJc+PGiDuoucpJ7SRD+86VzbuOyJ/X7jd7f66n+6vxwEAOEtytE8AQOy5Kreb9fn+KSulpq6uftzn2j0lcuqjn3hsDdWWTg2drv0BAM5CiyeAoGh4nHnv+TKo5/GZ6Z7GfboeH9wzw9qe0AkAzkWLJ4CgaXf5u3cMk9W7imXKkgJr7XVdBlNXJNLi8FqnU0sm6ex1JhIBAAieAEKmobJhsNRlMFmRCADQFF3tAMKO0AkAcIfgCQAAACMIngAAADCC4AkAAAAjCJ4AAAAwguAJAAAAIwieAAAAMILgCQAAACMIngAAADCC4AkAAAAjCJ4AAAAwguAJAAAAIwieAAAAMILgCQAAACMIngAAADCC4AkAAAAjCJ4AAAAwguAJAAAAIwieAAAAMILgCQAAACMIngAAADCC4AkAAAAjCJ4AAAAwguAJAAAAIwieAAAAMILgCQAAACMIngAAADCC4AkAAAAjCJ4AAAAwguAJAAAAIwieAAAAMILgCQAAACMIngAAADCC4AkAAAAjCJ4AAAAwguAJAAAAIwieAAAAMILgCQAAACMIngAAADCC4AkAAAAjCJ4AAAAwguAJAAAAIwieAAAAMILgCQAAACMIngAAADCC4AkAAAAjCJ4AAAAwguAJAAAAIwieAAAAMILgCQAAACMIngAAADCC4AkAAAAjCJ4AAAAwguAJAAAAIwieAAAAMILgCQAAACMIngAAADCC4AkAAAAjCJ4AAAAwguAJAAAAIwieAAAAiI/guWvXLrnpppukQ4cO0rJlSzn99NNlyZIlkf6xAAAAsJnkSB780KFDcs4558hFF10kH3/8sXTq1Ek2bdokGRkZkfyxAAAAcFrwHDdunGRlZcmkSZPqH8vJyYnkjwQAAIATg+cHH3wgl1xyiYwaNUo+//xz6datm/z85z+X22+/3e32FRUV1odLSUmJ9bmqqsr6CJXrGOE4llNxDbmOdsLzketoJzwfuY5OfS5WBfBzEurq6uoidSItWrSwPt93331W+Fy8eLH84he/kFdffVVuvvnmZts/9thj8vjjjzd7/O2335b09PRInSYAAACCVF5eLjfccIMUFxdL27Ztoxc8U1NTZfDgwfLll1/WP3bPPfdYAXT+/Pl+tXhqV31hYaHPX8TfRD5r1iwZPny4pKSkhHw8J+Iach3thOcj19FOeD5yHZ36XCwpKZGOHTv6FTwj2tXepUsX6devX6PHTj31VJk2bZrb7dPS0qyPpvSihfPChft4TsQ15DraCc9HrqOd8HzkOjrtuZgSwM+IaDklndG+YcOGRo9t3LhRevbsGckfCwAAABuKaPC89957ZcGCBfL000/L5s2brbGaEydOlDvvvDOSPxYAAABOC55DhgyR9957T/75z39K//795cknn5QJEybIjTfeGMkfCwAAABuK6BhPdcUVV1gfAAAAcDbWagcAAIARBE8AAAAYQfAEAACAEQRPAAAAGEHwBAAAgBEETwAAABhB8AQAAIARBE8AAAAYQfAEAACAEQRPAAAAGEHwBAAAgBEETwAAABhB8AQAAIARBE8AAAAYQfAEAACAEQRPAAAAGEHwBAAAgBEETwAAABhB8AQAAIARBE8AAAAYQfAEAACAEQRPAAAAGEHwBAAAgBEETwAAABhB8AQAAIARBE8AAAAYQfAEAACAEQRPAAAAGEHwBAAAgBEETwAAABhB8AQAAIARBE8AAAAYQfAEAACAEQRPAAAAGEHwBAAAgBEETwAAABhB8AQAAIARBE8AAAAYQfAEAACAEQRPAAAAGEHwBAAAgBEETwAAABhB8AQAAIARBE8AAAAYQfAEAACAEQRPAAAAGEHwBAAAgBEETwAAABhB8AQAAIARBE8AAAAYQfAEAACAEQRPAAAAGEHwBAAAgBEETwAAABhB8AQAAIARBE8AAAAYQfAEAACAEQRPAAAAGEHwBAAAgBEETwAAABhB8AQAAIARBE8AAAAYQfAEAACAEQRPAAAAGEHwBAAAgBEETwAAABhB8AQAAIARBE8AAAAYQfAEAACAEQRPAAAAGEHwBAAAgBEETwAAABhB8AQAAIARBE8AAAAYQfAEAACAEQRPAAAAGEHwBAAAgBEETwAAABhB8AQAAIARBE8AAAAYQfAEAACAEQTPJmpr68xceQAAAIdJFodbvatYpiwpkEX5B2Xz/iNSVVMnKUkJ0qdza8nLzpRRg7Okf7d20T5NAACAmOfY4JlfWCYPTFspi7YdlKTEBKlp0NKp4XPdnlLZuO+IvDV/u+TlZMpzIwdIdsdWUT1nAACAWObIrvYZq/bIiPFzZOn2Q9bXDUNnQ67HdTvdfvqKXUbPEwAAIJ44ssVz7LSVUlmT4Pf2GkBrpE7GTF5hfX1VbrcInh0AAEB8clSL5/aicutzsNOHdL/7p6y0uukBAAAQGEcFz0c/WB3yMWrq6qyxoQAAAAiMY4Lnqp3F9WM6Q6Hd7johSWfDAwAAwH+OGeM5dWmBJCd6HtfZMiVJnvpef7n0tBOlrKJaJs7dKhefeoKs3V0iT3y0ttG2OgteSzBRZgkAAMB/jgmeWqez2ktx+IcuO1WG5mTK7X9dIkVHKuX+S0+R07q2tYKnu1bPxfmht54CAAA4iWO62rU4vCfpqUly3ZDu8vSMdfLlliLZsK9U/u/dryQ50fPl2bS/NEJnCgAAEJ8SnbIMphaF96Rnh3RJS06SFTsO1z9WfLRKthZ6Dqt6PJbXBAAA8J8jgmdiYoK1DGY46fH0uAAAAPCPI4Kn0rXXvdX3rKyuldwe7esfa9syWXK8LJF5Uuc2YT9HAACAeOaYyUV52ZmyvdD9uMzyyhp5d0mBNcHoUHmVFB2pkPsvOUU8zUXSWe1DsjMie8IAAABxxjHBc9TgLJm8KN/j93VikU4yev3mwVY5pT/P3SZtWqS43VZntevxAAAA4D/HBE+tuTmop7ZSFnps9bzv3a+sD5dv9e3strVTj0MNTwAAgMA4ZoynevLK/iEfIykhQZ4bOSAs5wMAAOAkjgqePTqkW5+DnYuu+z0/aoBke5l0BAAAAId3tTc0buQAuX/aGqmpq7PGa3oyeuKC+u51benU0HlVbjeDZwoAABA/HNXi6XLZ6V1k5r3nfz3m83iwdMf1+OCeGdb2hE4AAIDgObLFU2l3+bt3DJPVu4plypICa+11XQZTVyTS4vBap1NLJunsdSYSAQAAhM6xwdNFQ2XDYKnLYLIiEQAAQPg5sqvdG0InAABAZBA8AQAAYATBEwAAAPEVPJ999llJSEiQMWPGmPqRAAAAcFrwXLx4sbz22msyYAAr/gAAADhVxIPnkSNH5MYbb5Q///nPkpFxvG4mAAAAnCfi5ZTuvPNOufzyy+Xiiy+Wp556yuu2FRUV1odLSUmJ9bmqqsr6CJXrGOE4llNxDbmOdsLzketoJzwfuY5OfS5WBfBzEurq6jyvGRmiyZMny29/+1urq71FixZy4YUXSm5urkyYMMHt9o899pg8/vjjzR5/++23JT39+DrrAAAAsI/y8nK54YYbpLi4WNq2bRud4FlQUCCDBw+WWbNm1Y/t9BU83bV4ZmVlSWFhoc9fxN9EruczfPhwSUlJCfl4TsQ15DraCc9HrqOd8HzkOjr1uVhSUiIdO3b0K3hGrKt96dKlsn//fjnzzDPrH6upqZE5c+bIyy+/bAXMpKSkRvukpaVZH03pRQvnhQv38ZyIa8h1tBOej1xHO+H5yHV02nMxJYCfEbHg+e1vf1tWrVrV6LFbb71V+vbtK2PHjm0WOgEAABDfIhY827RpI/3792/0WKtWraRDhw7NHgcAAED8Y+UiAAAAxEc5pYZmz55t8scBAADARmjxBAAAgBEETwAAABhB8AQAAIARBE8AAAAYQfAEAACAEQRPAAAAGEHwBAAAgBEETwAAABhB8AQAAIARBE8AAAAYQfAEAACAEQRPAAAAGEHwBAAAgBEETwAAABhB8AQAAIARBE8AAAAYQfAEAACAEQRPAAAAGEHwBAAAgBEETwAAABhB8AQAAIARBE8AAAAYQfAEAACAEQRPAAAAGEHwBAAAgBEETwAAABhB8AQAAIARBE8AAAAYQfAEAACAEQRPAAAAGEHwBAAAgBEETwAAABhB8AQAAIARBE8AAAAYQfAEAACAEQRPAAAAGEHwBAAAgBEETwAAABhB8AQAAIARBE8AAAAYQfAEAACAEQRPAAAAGEHwBAAAgBEETwAAABhB8AQAAIARBE8AAAAYQfAEAACAEQRPAAAAGEHwBAAAgBEETwAAABhB8AQAAIARBE8AAAAYQfAEAACAEQRPAAAAGEHwBAAAgBEETwAAABhB8AQAAIARBE8AAAAYQfAEAACAEQRPAAAAGEHwBAAAgBEETwAAABhB8AQAAIARBE8AAAAYQfAEAACAEQRPAAAAGEHwBAAAgBEETwAAABhB8AQAAIARBE8AAAAYQfAEAACAEQRPAAAAGEHwBAAAgBEETwAAABhB8AQAAIARBE8AAAAYQfAEAACAEQRPAAAAGEHwBAAAgBEETwAAABhB8AQAAIARBE8AAAAYQfAEAACAEQRPALCB2tq6aJ8CAERccuR/BACgqdW7imXKkgJZlH9QNu8/IlU1dZKSlCB9OreWvOxMGTU4S/p3a8eFQ0zdPCUmJkT7NGBzBE8AMCi/sEwemLZSFm07KEmJCVLToKVTw+e6PaWycd8ReWv+dsnLyZTnRg6Q7I6t+BvBdrh5QjAIngBgyPQVu+T+KSulpu542GwYOhtyPb50+yEZMX6OPD9qgFyV242/E2yBmyeEgjGeAGAodI6ZvEIqa2o9Bs6mdDvdXvfT/YFo0+eh3gzpTVEgN088f+FC8ASACNtWWGa1dAY7fUj30/21pQmIFm6eEA4ETwCIsLHTvuleD5bur2NDgWjg5gnhQvAEgAhatbPYmkjkb/e6J7q/HkcndACmcfOEcGFyEQBE0NSlBZKcmCDVHoJnalKi/OqyvvLdgV2lTVqyrNxVLE9+tFZW7mweMHUWvJZgoswSonHzFKqGN088h52LFk8AMc/Oxde1Tqen0Kk0dH6nfxf55btfyeUvzZPtRWXy1x/lSbuWKW7fuBfnH5/UAZi+eXInIUHk5xf2lrkPXCTrn7xUPv7FefKd/id6PJbr5gnORYsngJitH7h8e5Hc1lNkwBMzpVYSbVl8XYvDe9IyJUluHNpTfjnlK5m98YD12IPTVsm8sZ3k+iFZMnHO1mb7bNpfGtHzBQK5efr5hX3ke2d0k1+/t0q2FZXJ0JwOMuH6XDlYtkgWumkl5eYJBE8AMVs/MDmhVqSnfYuva0usnpcnPTukS2pyYn1pGqVv8F/tPGyFaHf0eKwQAzvcPOkwkTsv6i03/WWhLNtx2Hqs4OBOGZydITcM7eE2eCpunpyN4AkgZouvJyfZu/i6Lh+oy2B6C5+B0uOxLCHscPOkN07pqcnyt9uGNno8JSlR1u72PAmOmydnI3gCiJn6gYHENw2gNVJn7aeiFT615VJbYt3ZXlQuFdU1Mqhnhuw6fNR6TMfSDejeTt6Yl+92n5M6t4no+QL+3jy1SjseIX705mLZW3Ks0fcqq2s9XkhunpyN4AnAEfUDB3ZvH5Vudx1zqt3/7sopHa2qkX8s2CEPXXaqFB+tssLnTy/oZY39fGfJjmbb6/CCIdkZhs4c8H7ztGlfqVRU1UjX9i09dqu7w82TsxE8ATimfuC7dwwT03Sik4459WTcJ+utmcEvXDdQWn9dTumHbyySkqPVzbbV8KrHA+xw81RWWSMT526VR67oJzrpXSsutGmRLIOzM+XIsSqZtqz5Mq/cPIHgCcC24qF+oP48neikY07dtXpWVNfK4x+utT680Tds7ZK3y2x9OIe3m6ffz9woB8sqrdntWZnpUnKsStbsKpZXZm9xuz03TyB4Aoip4us6k/bB73xdcL1FspSVHJbTd6+T7QePya++01cuPKWztc+yHYfksQ/WSH5RedSLr+vsep3opGNOg5WUkGAdBzDN183TpC/yrQ9fuHmCooA8gJiqH6gF168Y0EUemLpSvvfKXCkpKZFJtw6VqT8dZs2m1dIuV7/yhew+fFT+/uOh0iIlMer1A3Vsqc6ud1+C2zfdT/ePZmkoOJve9OjNTyi4eYIieAKImfqBroLrT89YL59t2C9bDxyRlStXSuGRCklLTpJfTF4ua/eUyNbCMvn1+6ulRUqSfLvvCbaoH6iz6ieMzrVabLXlxx+6nW6v+0VrVj6guHlCuNDVDiBm6ge6Cq4vzm887nP5jkNyVW53WfXYJY0e1+Cp+9ilfqCGR51d37AIvruuS9fjg3tmyLgoF8EHXFw3P656uu6eu+6ey9rSGc16urAXgicAWwq0+Lq2fv74r0uaPX64vMpW9QM1ROrseteyn9r9ry2x+nvq+WmpGS2ZZKdlPwEXbp4QKoIngJipH+iu4Lo6o0eGnNi2pRw5Vi1FZZUxUT9QQ2XDYMkymIgV3DwhFARPADFTP7BhwXUt27KvuFwGDOgp7VumSn5Rmfzl5sHy3CcbpOBQuXRp10KG9ztR3lm8Q7YcKLN9/cBot8QCgeLmCbYLns8884z861//kvXr10vLli3l7LPPlnHjxskpp5wSyR8LII7rBz778Xrr8/PXDrTKKR0pPiQ3vzFfth+skF+OOMWaiNM+PUX2l1TIgq1F9V3t1A8EIoubJ0Q9eH7++edy5513ypAhQ6S6uloeeughGTFihKxdu1ZatWKwPIDA6wdW1tTKEx+ttT7Skurkubwa2bA3SSpqEuSR6autj6aoHwgADgien3zySaOv33zzTencubMsXbpUzj///Ej+aABxguLrABA/jI7xLC4utj5nZma6/X5FRYX14aKFoVVVVZX1ESrXMcJxLKfiGnIdTevWLlWeH3matWZ70/ntaYl1jT67oyMnx408zToOr333eF2HB9eR6+jU52JVAD8noa6uLvg13AJQW1srV155pRw+fFjmzZvndpvHHntMHn/88WaPv/3225Ke/k0tPgAAANhDeXm53HDDDVYDY9u2be0RPH/2s5/Jxx9/bIXO7t27+93imZWVJYWFhT5/EX8T+axZs2T48OGSkpIS8vGciGvIdYymHUXl8sgHq60xnzpuM1lq5cnBtfLIkkSpqE1oVnz9iSv7S48GBeThHq/r8OA6ch2d+lwsKSmRjh07+hU8jXS133XXXfLRRx/JnDlzPIZOlZaWZn00pRctnBcu3MdzIq4h1zEaep/YTt7+yTn1xddXbNcVjA5ZobNWEim+HiJe1+HBdeQ6Ou25mBLAz4ho8NTG1Lvvvlvee+89mT17tuTk5ETyxwFwWP1AvaufMWOGrHx0hKSlpUb7tAAA0QyeWkpJx2dOnz5d2rRpI3v37rUeb9eunVXXEwDCgfqBABAbEiN58D/96U9Wf/+FF14oXbp0qf945513IvljAQAAYEMR72oHAAAAIt7iCQAAALgQPAEAAGAEwRMAAABGEDwBAABgBMETAAAARhA8AQAAYATBEwAAAEYQPAEAAGAEwRMAAABGEDwBAABgBMETAAAARhA8AQAAYATBEwAAAEYQPAEAAGAEwRMAAABGEDwBAABgBMETAAAARhA8AQAAYATBEwAAAEYQPAEAAGAEwRMAAABGEDwBAABgBMETAAAARhA8AcCQ2to6rjUAR0uO9gkAQLxavatYpiwpkEX5B2Xz/iNSVVMnKUkJ0qdza8nLzpRRg7Okf7d20T5NADCG4AkAYZZfWCYPTFspi7YdlKTEBKlp0NKp4XPdnlLZuO+IvDV/u+TlZMpzIwdIdsdW/B0AxD262gEgjKav2CUjxs+RpdsPWV83DJ0NuR7X7XR73Q8A4h0tngAQJhoex0xeIYGM5NQAWiN11n7qqtxu/D0AxC1aPAEgDLYVlsn9U1YGFDob0v10f+2mB4B4RfAEgDAYO22l1NSFNmtd99exoQAQr+hqB4AQrdpZbE0k8uSCkzvJXd/qI6ec0MbqWl+245A8/uFa2XGwvNF2+j09js6GZ7Y7gHhEiycAhGjq0gJJTkzw+P2WqUnyl7nb5Lsvz5Mb/7JQdF7Raz8YJAludtFZ8FqCCQDiES2eABAirdNZ7aU4/Cer9zb6+oGpX8nyR0fISZ1bW2WVmrZ6Ls4/PiMeAOINwRMAQqTF4b3J7pAu9w0/WXKzMiSjVYokft3U2bV9y2bBU23aX8rfBEBcIngCQIjLYGpReG9ev3mI7Dp8VB7810rZV1Ih2is/674LJDXJ/WgnPZ4eN9FL9z0AxCLGeAJAKP+IJiZYy2B60j49RXp3bi0v/W+TfLmlSLYcOCLtWqZ4PaYej9AJmKc3fIgsWjwBIES69roug+lO8dEqOVhWKd/P6yH7Syus7vWxl/b1eryTOrfhbwIYoBUkdDKfjtPWITPa26A3fvqazsvOlFGDs6gwEWYETwAIkb5B6VhNd8tjamnPu/+5TB777mkyc8z5srWwTB77YI28c8cwt8fSWe1DsjP4mwARpAs1aM1cLV+mr7mGr10Nn3ojqa/pt+Zvl7ycTHlu5ADJ7tiKv0kYEDwBIETaKqJvUJ58sblIho+f0+ix7Af/7XZbfQPU4wGI3NK2ukqYa8EHdzeMDR9fuv2QjBg/R54fNYAlbcOAMZ4AECIt9q6tItpyEgrdX49j9+LxjINDLIfOMZNXSGVNrcfA2ZRup9vrfro/QkOLJwCEgXbFaatITdCrtYskJSRYx7EbxsEhHmwrLLNaOj29Qh+9op/069pWRk9cICt/M0Ke+GitTF26s/77up/uP7B7e7rdQ0DwBIAw0PFf2hWnrSLBRE9tK9X97TSOjHFwiCdjp33Tve7O72dukOSvS5xd9LvZcqSiutk2ur+ODX3Xwxht+EZXOwCEyVW53WTC6FyrPqe/3e66nW6v++n+dqFditqCq+PblKtb8sKTO8k1Z3bzOA6OrkjY0aqdxdZEIm/d62WVNVYVClVUVikV1bXNttH99TjaC4Dg0OIJAGGk4VG74jzNmHVxPT64Z4aMs9mMWdc4uKZnfWLbFvLId/vJ0coa2V5UXh9Klf4uOsxA91N2CtHA1KUFkpyY4HFpW11M7GcX9LbKnnVqk2Z1y7/46Sb5uMlyt67XrpZgsvtYbLsieAJAmGmI1K4419hIXXtdl8F01QjUOp1aMsmONQK9jYN78ur+8sj7q2VP8TH5w+hcGfXq/GatQoyDgx1pnU5PoVP9/MI+8r0zusmv31sl24rKZGhOB5lwfa4cLFskC7cdbLSt3mTpaxrBIXgCQIRoqGwYLGNhGUxv4+Bu/+uS+v++8uUvPB4j3OPgYuG6hSLefz870OLwnuhQlzsv6i03/WWhLNtx2Hqs4OBOGZydITcM7dEseCq9kURwCJ4AYIjdw8Xa3SXW8AB3Lji5k9z1rT5yygltrBafZTsOyeMfrpUdB8u9joMLpkU33mfRx/vvZ8dgr9fYk54d0iU9NVn+dtvQRo+nJCXK2t3ux3Lq8bhhCA7BEwBgeX/FLo/j4FqmJslf5m6T9XtLpFVqstw7/GR57QeD5LIX51qrM4VjHFy8z6KP99/Pzjd8Guw9hc9Wacej0I/eXCx7S441+l6lmwlGSo9n9xtJu2JWOwDAopOFPI2D+2T1XvnPmr3WpKK1e0rkgalfyald2spJnVu73T7QcXCeZtG7O24szqKP99/P7rQ12ZNN+0qloqpGurZvaT2/G37oeGZ3dJw2gkOLJwDAsvWA53Fw2R3S5b7hJ0tuVoZktEqRRJ0GLGK9WWsrnTv+joPzNIte6YSPQ+WVMnvDgUaPx9Isem+/nyex9PvFAh3CoM9Td4FfyyhNnLtVHrmin2gjpt4wtWmRLIOzM+XIsSqZtqxx+NfWap0ciOAQPAEAlirrTdl99+HrNw+RXYePyoP/Win7SiqsN+hZ911gTczwxJ9xcL5Wk1m+45D84/az5MdvLba6opuy+yx6X7+fL3b//WKFjpvVIQye/H7mRjlYVmnNbs/KTJeSY1WyZlexvDJ7S7NtNbzq8RAcgicAwJKSmCAVNc0vRvv0FOndubUVOl3d51p/1Bd/xsH5Wk0mv6hc7vrHMnnhuuPlm2JtNRlfv58/7Pz7xQoda6zjZnUIg6dhDpO+yLc+vNHWzkE9M5j8FQLGeAIALL06uR8Hp6u5aGuQFtfWGcDDeneQh6/o5/Oq+RoH589qMmp5wWH5zh/mug2ddl5Nxt/fzxe7/n6xRidrJX09RCRYur8eB8EjeAIALNqS426pT22wu/ufy+T0bu1k5pjz5dEr+skzM9Z5vWr+jINzrSbjiWaEn1/YW+Y+cJGsf/JS+WTMeXLlwK4ef57OorcTb7/fvLEXyY/OyW702Ix7zpUxF58UM79frNGhCs+PGuBhMIlvup/uz5CH0NDVDgCon8gzab77cPPF5iIZPn5Oo8eyH/y3xyvnzzg4f1aT0aD5q3+tkvyisuMlhq4dYE02mrupMGKryYSrPqOv3y8QrJYTHq5JWjpuVocw+NMaraFfWzo1dDLJK3QETwCARcsj+RoH5w9/x8H5s5rM9ycukK92Hu9i3nlolwzumSnXD8lqFjxDWU0mUgXdvf1+wWC1nPDQ8KiTtTzVVHVxPa7jmcdRUzVsCJ4AgHo6fk3rR2opn0iOg/N3NZnpd53b7Hsrdx5f1jDU1WQiWdDd1+8XDFbLCR/9O+pkLddNh7aWa7B33XTo+GQdKsIqUuFH8AQANBsHF2jdyUDHwfm7msy54/4nOw8d9etnB7KajNbWdHW3BlLQ3d/uVl+/X22tjmFtfK7JXkpTKVbLCT9tyW7Yms0ymJHH5CIAcCB9g/VEg9WE0blWd7e7yUbu6Ha6ve7n7zi4Ph5m0TdcTeacPh3FX/6uJuMq6F5ZU+s2cJ7Zo32ziT+6nW6v+/m7mpC31XIOllVIpzZp9V+3TkuWrIx0r8djtZzIYxnMyKPFEwAcwNs4xrOy20uugXFw7s7BE9dqMr/6Tl+prTteTkjD2dCcTOt77ywuCGo1GV8F3VulJslz1w6UA6UVUnqsWqYs3Rl0QXdvq+V8uaVIrh3UXT5dt09KjlbLfSNO9lrvk9VyEC8IngAQx/wZx7i9sFRyB4vcPGmRPH1Nbn2gCtc4OG/n4I1rNZmfnt9bfnv16WFZTcZXQfeHLjtVXpuzRf6zeq/88ydnyZxNB6yVmoIp6O5ttZw/zt5irZDz+i1DrID7wswNkpXR0uOxWC0H8YLgCQBxKtBxjCsKDrsdxxjKODh/z8GTcK4m4yro7s2v319d/9+XvzjPZ0F3bz/T22o5Wgz/7n8ub/RY0zXBXVgtB/GEMZ4AEId8jWPs1bGV3PWtPpKWnBjwOMZAQqe3cwgXf1eT8VWwXseo/ua7/WTJwxfLhicvlSk/HSYDurcLqaA7q+UAjRE8ASDO+BrHqNnr99cNlEE9MuTei09u9n3XOEbtIo/UOajJPznLWgUpFIGsJuOroPuvLusr3+nfRX757ldy+UvzZHtRmfz1R3nSrmVK0AXdWS0HaIzgCQBxxtc4xp+c30sWbC2SH/91ieT2aC+nd2/fbBvXOMZInYO6429L5fczNwR1/GBm0Xsr6N4yJUluHNpTnp6xTmZvPGBt++C0VXKsqtYqWB9KQXdTVQKAWMAYTwCII/6MY3z18631/z164gJJS6oT6RrcOMZgz0EVH62SQAW7mow/BetTkxOt8Zgu2jr61c7DHssiBVLQndVygOMIngAQR1zjGD11KWvN8p9d0Fu+n9fDqiOpXeJ/+myTLkjpcRxjoMHT1zk07Gpfu7tEnvhorc9jhrqajK+C7sEItKA7q+UABE8AiCu+xjH+/MI+8r0zusmv31sl24rKZGhOB3l+VK4sXqirAx0OahxjoOcQqFNPbCMfjzk/5ONoy6WWj3Jne1G5VFTXWLPjdx0+vlKShmedXPTGvPywFnRntZzgFzdA7KPFEwDiiLdxjDpm8M6LestNf1koy3YcD5kFB3fK0JwMGZatK/WsCHoco7/nEIzNB8JzPG8F3Y9W1cg/Fuyw6njqEAANnz+9oJc19vOdJTsiWtDd6avlNF1YIFFq5bk8kZF/+lLO6NmB9dLjDMETAOKEP+MY01OT5W+3DW30eEpSopSVFoc8jtGfcwhGoOcQTEF3Ne6T9dZQhBeuG2itkrRyV7H88I1F1spCTVHQPXSeFhZISzr+/Q37SmXN3jLrb6b1UJ8LYEwv7IvgCQBxwtc4xlZpx//J/9Gbi2VvybH6x1MT6+SX/avCMo4xEmMp9cev3VMS8LjOQAq6q4rqWnn8w7XWhze+CrrTVRyehQV+OCxHlu44bLXO69/M3eIGiD0ETwCII97GMW7aVyoVVTXStX1LWdhg1rnOaj92rEYjVVjGMXo7h2BoJrnipXlhafXS/TXA1HitMBpYwXpXV/Hy7UVyW0+RAU/MlFpJtK6Ddu8HMxkqnrkWFvD2F+jdu7d0aXWivDV/R30w1b+Z7qcIn7GLOp4AEEc06HiqFVlWWSMT526VR67oJyPP7CY9MtPltK5t5aazsiUrKyts4xi9nUMoXK1e3lZVMlnQXbuKr3ttvhWK/75wh9U17KItvhq+9XH9vm4XSkH+eOHPwgJn9siwno8///tia9WrcC9ugOgieAJAHNHWNW/LU/5+5kZ56X+brNnt/73vAnnrR3ly4Smdpby8PGzjGH2dgy8ahn98Xo7b8/FnSU8TBd3152sIdtX99PT7uh4PR2iOB/4sLLBsxyGZPXu2lB5rPrY2HIsbILroageAOOJrHKOa9EW+9dGwq/25vMZd7b7GMYZ6Dq7i9U3p2vG/GzVQDpZVSllFtfxzUfP10F2tXgO7tw+62z2Ugu7+dBU3RVex74UFXDVmb8jrIV3apckpeWXyh083ycer94ZtcQNEHy2eABBndPyhjkMMRdNxjKbOYczFJ8nUpTut5TRvOqunnNA2LWKtXq6C7h/dfa7cNLSH9OvS1poYpfSzfq2P6/ffuWOYtb0/XcXeOLmr2LWwgCfaCn/Nmd3lN9NXyf/+9z9584utMuH6XBmak+lxcQPEHlo8ASDOuMYxBtoq524co+lzGPfJN2u3X/7iPI/bhbPVK5CC7v50FfviCs0aep3E28ICDWvMrtl1SK7MrJH3lu+U3J6ZcsPQHo0mw4WyuAGij+AJAHHINevXVbLGnzGX1jhGSQxbyZpAz2He2IvkjXnb5I0GwwBm3HOuzFy7Tyb8d1PYlvT0xVPo9HcNel+c2lXsbWGBpjVmtZbnxZcerzG7drf7GrPBLG6A6CN4AkCcCmQcozojq7389prcsBbpbnoO4WS61cvfNej9EanQbFe+FhZoWGP24JGjMnZgrYz7KlEqaxOksrrxzPZwLywAswieABDHXOMYXbUmNahpS5G+aes4Rq3TOTRbw882efPWPElJSYnoOVz58jyrLme4mGz1Cuca9E7rKva1sEDDGrMrdhRJWVmN7DiYJBU1nkNloIsbwB4IngDgAN7GMVZVVcmMGdsifg46WSecodN0q5evNei/O7CbXH756VYXsbpl0iKv4dJpXcXeFhZoWGM2NUkkPf2A9OuaJgOyOsiRY1UybdmusCxugOgjeAKAA0WjpchXq1dtrZbUaXxeyUnei6+YavXyZw36/63bK7NnF9V3Ee8t/mZZUnec1lWsCwts3HfE41hfrTGrZbTuuKCPZHcYIGcOq7JayV+ZvSVsixsg+gieAABbtHodLKuQTm2+KZ/UOi1ZsjLSvR7PVKuXP2vQa6tdWVmlzy5ip3YV68ICb83f7nUbrS/79oJtVl3ZBxZ5vo7BLm6A6KOOJwDAGG/LaX65pUiuOaOb1ZJ1yglt5PfXDfRaush0q5eG5nByWlexa2GBUJdT1f31OE6ZmBVvCJ4AAGO8Laf5x9lbrHqNr98yRN64dYjMXLNXdhR5LrRuutUrnGvQO7Wr2A6LGyC66GoHgBgUq2MDvS2neaSiWu7+5/JGj7mbVBLqkp6R7Cr2l1O7iu2wuAGii+AJADHAVQ5JS/ro7GpXOSTt/tWWOA0xsdL1qK1VI8bPkZqgF56MTquXv2vQ+xKN0BwPixvo3zxcixsgeuhqBwAb0zW9r3ttvlzx0jz5+8Id1sQc1wQX/axf6+P6fd0uFtYAd7V6BdteG81WL7qKw0PD48x7z7cCuPI0hMH1+OCeGdb2hM7YR4snANjU9BW76luFlKeWIdfj2hKnLYmx0CoUq61edBVHfnED+bolvO8Jba212mOpNR++ETwBwKah09s4uNFDsuTAkQr5dN3++sc0vGn3te6nYiF8+rukpz6urV7jRkZ/fF+shuZYWdygoqJSPvnkY5n6s2ERWUkL0UXwBACb2VZYZoUaT3HmwlM6yffzeki7linWeM/tReWNvq/76f4a6qId0sKxpKfO/rZbq5e70OxqqbNzaI4FsThpDv4jeAKAzYyd9k33elNt0pLlgUtOkdveWiJZmeny7DUD5Ia/LJCmm+v+Goo01MX6kp6xEppXbD8oIseXyLRzaAaiieAJADayamex1YLmSWlFtVz24jzrv/cUH5Pv/3mB2+20lU2Po6EoFkOP3UOnu9B8fM37GbLy0RGSlpYa7dMCbIlZ7QBgI1OXFkiyH6Fr8k/Okkev6Od1G+3m1ZY4mBVLoRkwjeAJADaidTqrQ6gR2bTVU8dMAoBdEDwBwEZ0slA4HS9PAwD2QPAEAJvQCTWu4vDhosfT4wKAHRA8AcBGYwN1NnQ46fEYcwjALgieAGAjuvZ6OGlJH8AkWtjhDeWUAMBG8rIzZeO+I36thuOLzmrXOpJAJLnqmOrEOB2j7Cr+rzdR+nymjikaIngCgI3om/Rb87eH5VgaXvV4QCTkF5Z5XO5Uw+e6PaXWTZQ+n/NyMuU5Vm4CXe0AYC9aiFzfpI8vwejZ6IkL5ImP1nr8vu6vx4nF4vGwv+krdsmI8XNk6fbj5bo8tdC7HtftdHvdD87GGE8AsBltGUpKCG2Ske6vxwHCTcPjmMkrpLKmtj5Yatf6zy7oLb08rEWv2+n2uh/h09kIngBgwzXAnx81QIKNnrqf7q/HAcJpW2GZ3D9lpTRt39Su9e0Hy+SVG8+UtGTP0UL30/21mx7ORPAEABu6KrebTBidK6lJiT673V10O91e99P9I4mZy840dtpKqalz360+Y9Ve+deyXfL4Vad5PYbur2ND4UxMLgIAm9LwOLB7e48TOFxcjw/umSHjIjSBg5nLWLWz2HoeevPnuVt9Xih9rupx9DnFGGTnIXgCgI1piHz3jmH1wU/XXtdlMF0la7ROp5ZMilTJGmYuw2Xq0gJJTkyQajc3Pxec3Enu+lYfOeWENlawXLbjkDz+4VrZcbDc7QXUmyV9PhM8nYfgCQAxQN+gG75Ja1d3pFck0kkgOh7P1bXq78xlHV8a6a5+mKd1Ot2FTtUyNUn+MnebrN9bIq1Sk+Xe4SfLaz8YJJe9OFfc9czrc0ZvouA8RsZ4vvLKK5KdnS0tWrSQoUOHyqJFi0z8WACIWyZCZ9OZy74wczm+aXF4Tz5ZvVf+s2avbC8ql7V7SuSBqV/JqV3aykleVuLSlns4T8RbPN955x2577775NVXX7VC54QJE+SSSy6RDRs2SOfOnSP94wEAYZq57C/XzGUdn8rM+vigLew6vMOT7A7pct/wkyU3K0MyWqVI4tflwLq2b2kVkXdHj2ei5R4OC54vvPCC3H777XLrrbdaX2sA/fe//y1vvPGGPPjgg422raiosD5cSkpKrM9VVVXWR6hcxwjHsZyKa8h1tBOej5G5jr/+1wpJTqyVhKCjp47hq5WH/rVC3ro1T5wi3p+PrVNEqjy0fr9xyxDZdbhcHnn/K9lfWiGaJf/9iwslPSVB0pLc75Oik+JqqqWmxlnX0QTT1zCQn5NQV+ehLkIYVFZWSnp6ukydOlWuvvrq+sdvvvlmOXz4sEyfPr3R9o899pg8/vjjzY7z9ttvW8cBAETPOeecYzUI1NTUSM+ePaW2tlby8/OtHiw4V0pKilx22WUyd+5cOXjw+Kz3zMxMOe+882ThwoWyd+/eaJ8iIqy8vFxuuOEGKS4ulrZt20avxbOwsND6B+qEE05o9Lh+vX79+mbb/+pXv7K65V30H7isrCwZMWKEz1/E30Q+a9YsGT58uPVCAdcwWnguch3t+nx8ftZmeXdJgdtJJH89PUH6dc2SSV9slY8+/EJys9rLsyNzZdyXh+XLLYVuZy5fPzhLHrrsVHGCeH9dPz1jnbyzpKDZmN+EhFoZdkGlHGydIy8vqrK61/9v0PG/+VubEuXTdUnNjqUtoqOH9HD73Ij362iC6Wvo6qGOuVntaWlp1kdTetHCeeHCfTwn4hpyHe2E52P4ruOC/MNSZvWaNR93p3Fj3Z5SeWHWZuvrjfvL5YazciSvV0f5bGNR8wPWiCzML3bcv7fx+nwcObinTJpf4Pa5cdc/l8lj3z1NPrrnAtlaWCaPfbBG3rljmFTXJkhFjfsxnB3atPR6neL1Oppk6hoG8jMiGjw7duwoSUlJsm/fvkaP69cnnnhiJH80ACDMM5eVlstp6EDpMenQunmDgQszl+OHlvPKy8m0ymY1bfX8YnORDB8/p9Fj2Q/+2+vxXvx0s1wxoCsT0BwmouWUUlNTZdCgQfLpp5/WP6ZjgvTrYcOGRfJHAwDCPHNZVTf5vs4S8DYp2TVzGfHhuZEDJOnrGeuhYulMZ4p4V7uO2dTJRIMHD5a8vDyrnFJZWVn9LHcAgD1oWRtdDclX+AyEHo9yOfFDy2Pd8+2T5HczPU8o+07/E+UXF58k2R1aydHKGlmzu0Ru/+sSOVrVePo6S2c6U8SD5/XXXy8HDhyQRx991JrZlpubK5988kmzCUcAgOjr07m1NY4zXHRJT8QXHV6hrdzuGrI7tUmTF79/hjz78XqroHyr1GQZkpMpnhpJWTrTeYxMLrrrrrusDwCAveVlZ1oFv/1drcgbDRW6jjzib+lMT0+Pzm3SJCUp0VrJaNfho9ZjG/Z5vpFh6czwWb2rWKYsKbD+PgWFpfLbwSJnPDFTsjq2sV7XowZnNVp2N1psNasdABBd+ub01vztbr83euKCZo/95G9LvYYKPR6cMwFt3Z4SmbepUD4Zc57M2VgoczcdkBmr90jJ0WqP+zABLTT5hWXywLSVsmjbQetmT193rqL9WvBfezD0ZlJf1zo5TMfpRnNFMSNrtQMAYmvmsr6BhUL31+PYoYUF5iagaUvoTa8vlFsmLZbN+0vl5rOz5X//d6F0z2jpcR8moAVv+opdMmL8HKvSgPLUU+F6XLfT7XW/aCF4AgDCPnNZ99fjID4noPmiAWf8fzfJ5S/OlaqaWrnkNM8lFJmAFhwNj2Mmr5DKmlq/h8bodrq97het8EnwBAA0ot1wz48a4KZMuH90P90/mt15iOwENE90NaufX9hbTu/WTrq2ayGX9j9RMlulyhYv3fNMQAvctsIyuX/KSmtRh2Dofrq/dtObxhhPAEAzV+V2sz7rm5PWW/SnRUW717WlU0Ona384awJa6bFqGZqTKT86N0fapCXLzsNH5bf/XiezNx5weywmoAVn7LTjr8tw1FF99w6zddUJngAAtzQ8DuzevtnEhaZcjw/umSHjojxxAdGdgLblwBG5edJiv4/FBLTArdpZbL0eQxWtOqoETwCARxoitUXEVaplcf4haxayTgjRsXnaTaolk+xSqsXUBBsnF8X3tnRmIPSGZVDPDMc8b8Jl6tICSU5MkOowlTzT1zXBEwBgK/rG1PDNyUnhq2F9RC0l5ArdOtbRTvURTdKJYzo7uiboUYZMQAuWPg/DETqjVUeVFk8AQMCcEDrd1Ud00fDZtD7iM1f1a7R/PIdz1wQ0nR0dTARiAlpk6qgGw3QdVYInACBmRSrcaakZ18Qqf+sjXvXKPGu1mJF/+lLW7SuL+5ZRJqDZr45qMFx1VE3dJBE8AQAxw0S3t6s+YiBv7xpAqxLq6peIrKpJ8NgyGu2VY8KJCWjRqaNaFcbwabqOKsETABB33d7BhrtQ6yN60nTlmHgqOcUENLP6dG5tPd+9ufGsbDn7rBNEFi30eTzTdVQJngAAWwum2zvYcBeO+oje6DnqhBxtUVXxEj5DmYAWz2NhTddRdclIT5VWrXzfeEWjjirBEwBgW8F2ewcT7sJVHzGQlWO0Tmq8dLs35SlM+houMfKMLsbPNV7qqLq8/L+N0uPIOo2WYrc6qgRPAEBcLwvob7jzVh+xe0ZLmTf2W80eX7C1SEZPXNDs8d+NGiA7Dx2VCf/dZLuVY+w+XGLyonx5Lk9kR1G59D4xfiZjhUv/GK+jylrtAABbCrTbW1vNPIW7UOsj7j58VIY89d/6j8v+MFcOllXKwhBaSBuuHOOElmsd/qBhyZ/hEurqV76w9kNzOoZZl6cNhe6vxzGNFk8AgO340+09+SdnyYa9pVZYufqMbtZ/f//PC4JeFtBbfUTNQweOVFj/nZacKBN/OEiW7TgkE/67UWJt5ZhYGC6hKmtr43IsrNPrqNLiCQCwHVe3ty8jB3WXyppaufZPX8qv31vlNdyFqz7ic9cOkFZpyfKLfy6XUOchRWPlmFgcLqHd9GhMw/iE0bmSmpRoPcf9odvp9rpftMI8LZ4AgJhdFlADybMfrw853PlbH/Gub/WR80/qJFe98oWUVdZ43O6XU/zr3o/GyjEmhaNKgNPGwsZ7HVWCJwAgZpcFXOXn+Eh/wp2v+oiX9j9R7vnWSXLLpEWy42C5xOrKMaaEq0pAIMMlnCi7YysrlLuqBehN1o7CEut7KYkJ0rtzW6tkkl1WziJ4AgAiJphAFUi391EvrY6Bhjtv9RFPPqG1vHDdQHn18y2yad8R6dQ6zXpcu/mLj1ZJLK0cY4q3KgGZrVLlP2POk0lf5MsfZ2+xHjuzR4Y1bvf2t7To+X7HjYUNZx3VqqoqmTFjhix/dISkpKSInRA8AQC2WtIyWssCequPOKB7e0lPTZZ7vn2S9eGrnFIgTK8cY4fhEloR4P6pK2XiDwbL3E2FsvXAERl//UD56/x865pe09FZY2GdhOAJALDdkpb+LAsY7nDnrT7i1KU7rY9wi8bKMXYZLjF7wwGZvHiHNdFFu+XLK2vkuU82WDOunTYW1kmY1Q4AMFqj0bWkpbcajdo66u9M3XCGu3DURwxENFaOMcHf4RK//fc6qzv+stO7WKWBdOiCr+ESiG0ETwBAyDUaNTD4u4qKbqfb636ewqeGMV/H0y7uJz5aG9Zw56qPaCJ6aiDWFtZ4HLfoGi7hS88O6XJC2xai9xjdM1s6ciys0xA8AQARr9Gok0YevaKf3zUaXd3eobZ6BhPugqmPGEsrx5iiwyV8BckJ1+fKRyt3ywuzNsqz1wyQDq1SHTcW1mkIngDCiq4w5whnjUa7LQuo4XPmvedba1lbxwlzAI3myjGm+Bou8csRp0ibFiny2Adr5U+fb7FuZLQ4v9PGwjoNk4sARH0WM2KPiRqN4VgWcNy1pwcd7tzVR9QJLq7nuLbAdWidKvO3FH19fr7PUgOUhmH9veJ9GUhvVQLO6pUpPzo3R74/cYEcqai2Hrvv3RUy4xfnyffzeorIVkeMhXUigicAW8xiRvzUaGyZkiRPfa+/XHraiVJWUS0T5zYOEU15q9HoCmfaJa+to/6MI3W1sSUmJMi973wlD0xdGdKNUMP6iKppPVDXa+GrHUVef0e7rBxjircqAQu2HpSTfv1xo8d2HjoqAx6bKWlJdXJGXuNrpy3P3MDGB7raAdhiFjNiy7zNhR5rND502akyNCdTbv/rEvnB64vkrF4d5LSubT0ey1eNxkC7veu+/nANA3DdCP194Q654qV5ct1r80Na+7vpBBdXy+iUr5d07HtC2/qJNfq5X5e2ctPQHvLR3efKO3cMc0TotMNwCdgTLZ4AgprFHEjXpwaLGqmz9lPx3sXohOfAlgPug1t6apJcN6S73PvOCvlyy/EWwP979ytZ8Ktvez2mrxqN3rq9NeO5MrC352XTG6Fwd3ef2qWtbFsuMvVnw6zVYuJxGcxAhWO4RLyPhXUagieAiMxidsc1i3lg9/a8kcT4jYe38jhpyUmyYsfh+sd0ScmthUfCsl55027v95btlPve/arZc7J7RkuZN/Zbctkf5taPJzx33GdGb4ScHjpDGS6hUhMT5blRA7lRjTN0tQOwzSxmxPeNR7hrNOr5jJ22yu357D58VIY89V/ZsK9U3l60Q658+Ytm23gr54Tw8ne4RMPH37/zHEJnHCJ4AghoFrO/rRX+zGJG/N14bC8ql8rqWsnt0b7+sbYtkyXHR1dpMDUavZ2PPk0PHKmwnm/HqmqttcHd4UbIHNdwCR3rqmNedeyru7GwrrGyPTqkGzw7mEJXO4CQZzErnT/wk/N6yffzekiX9i2k8EilvL1wh7zy2eaAZjEjtssn6Xrb7y4psCYYHSqvkqIjFXL/JafUj8EMV41Gf89HC9ev3V3icYUjb+WcEBm+qgRUVVVZY2URnwieAPyidTo9hU419pK+MjovS578aK018aNzmzTp7WHlEl+zmBHbNx5d27eU6tpaeevWIdb4zj/P3WYVCvckmBqNvs4nENwIRRdjYZ2F4AnAL1oc3pNWqUly6znZ8ugHa2TasuMlk3YcLJclX5dbCmYWM+LjxuOdxQXW9yfOcV/LM9gajb7OJxDcCAHmEDwB+KRdYTrr2BMtzp2WkiRfbC70+2r6O4sZ8XnjEWqNRm/nEwxuhAAzmFwEwPc/FIkJ9ZMA3NHJGyZmMSN+bjxcgqnR6Ot8guG6EQIQWQRPAH7RcOFJflGZHK2skXP6dPT7agYzixnxc+Oh/jA6N6hyOb7OJxjcCAFmEDwB+EXXufZUe6+iulZe/XyL/Oo7feWaM7tJj8x0OSOrvVznYcJIMLOYEV83Hn06tQqpRqO38wkGN0KAGYzxBOAXnXX81vztHr//4v82WZM97ht+snRu00L2lx6zyimFaxYzon/jsXHfEbd1XBveeFTV1MqS/EPSoVWqnHRCG6u0krsbj0BCaqDnEyhuhABzCJ4A/KKzjvNyMq11rt292Wsdb63Z6a5uZzhmMSN2bjy6Z6RbKwf9fcH2iN14+Dofl9ETF/jchhshwBy62gH4TWcf6yzkUAQ7ixn2uPHwNNzCdeOh66E/98l6mfDfTfLH2Vuabaf763FCvfHwdT7+Ctf5APAPwROA33T2sc5CDvatPiHIWcyIjRsPnaDTu1Mrqa2rk+H9Okf8xoMbISD2EDwBBEQnhEwYnSupSYl+tzbpdrq97hfKhBLY+8ajRXKSTPnp2fLzi/rIJ6v3hnTj4U9pI26EgNjDGE8AAdPwOLB7e3lg2kprnWsNlu7GfboeH9wzQ8aNpKWzqVgsoO+6cbh/ykqpqatr9HcvraiWM5+c5fZ5oC2dGjo93XjoWulTlhRYKxJpcXitq6ktqDp7XScS6ZhOd93h3s7HE3/OB0BkEDwBBEVbm969Y1h9YNBlEnX1F1dg0PI0WjLJU2BwomDDVTzfeOQXlnk8jl6fdXtKrdnrOpFIx2I+5+Y43AgBsYPgCSAkGpQahqVYbMWLtHCEq3i88Zi+Yld9S6Xy1FrpelwrKowYP8dtS2W83Ajx+kG8I3gCCCtCZ+TClR3DTLA3HnpdxkxeIYFU4dRrVCN11n7K3fWJtRuheGkFj2V2f47EG4InAERIpMKVncOMP2/g2wrLrDAebOl33U/31+5+Xy3Ddg0U8dgKHisI+9FF8ASACPAVrib/5CxZu7tEnvhobcjhKtbCzNhp37QAB0v3199Vu9djjR1awZ0oVl4f8Y5ySgBg83Dlb5jRcKIhJZAwo/uZtGpnsfXG7+n8WqUmyYTrc2XtE5fIooe+Lbedm2OF9Eev6NdoO91fj6OtV7HYCl5ZU+v3cp+6nW6v+5n+e8WLWHl9OAHBEwAMhyt/+RuuQg0zM1btEVOmLi2QZC/d3w9f0U8GZ2fIj99aIje9vlCGZGfKaV3but1WW610SEGsCNcQA225g/8I+/ZC8AQAw+EqEL7Clacw466V0B3d7+H3VospOu5U13T31No58szu8tt/r5MvtxRZ3Z73T/nK40IFGp519nqsMN0KDsK+HRE8AcBguAqUr3AVljATdBtc4HSykyc9OqRLanKifFVwuFFR+q0HPLfwacmkWGC6FRzHEfbth8lFAGAwXAXDU7hyhZlQucLQuj0lMqBHB4lk2RqdxBFOerxYKIfjagVveEOirdLr95Za5z9yUHeprK6V38/cINNX7JYnrjpNvnN6FyksrZDHPlgjszceaNYKTpkl78L5+nCFfa556GjxBIAYCVeBduknJIg8+J2+suLR4bL419+WMRef5PXnvLc8shMpNBxqWSdPdhSVW+FrQFb7+sfapCVLjpeZxXo8u4dOb63gI8/sJgfLK+Wql+fJW/Pz5amr+8sfbzzTmtxyxYtzZe6mQnnh+lxpkZIYs0MMosXT6+OaM7vJ8keGS2pS4wg08QeD5IXrBsbFeGI7I3gCgMFwFQxP4cpXl762oh2trJGrX/lCnvl4vdzzrZPk3D4dPW6/bPs3XdyRorVEPSmrrJFpy3bKQ985VYb16iAndW4t464dILV1daL/c0dXJIrlVnAt4fPy/zZLflG5/PGzzVJRXWsF0cmLC6zHXvx0k2S2SpVTT2wbk0MMosnT6+PfK/dYQfLifp3rH+vQKlUu6ttZpizZ6fZYhP3wIXgCgMFwFQxP4cpXl/76PaXyh083WQHmX8t2ycpdxXJOH89d6VsORD7MaAF7T5OF1FMfrZVlOw7J67cMln/8eKjV8rdl/xGpqKpttq0eR5fBjOVW8PV7S77Zrk7kUHmlbNj7zd/hwJEK63OH1ql+tYLD9+tDw70OZxg1KKv+savP6Ca7Dx+V+VuLPF5Cwn54EDwBwHC4CoSncOVPl37DUKMOlB6TDq3TPG5fVRtamPFnX101ydsEG231HPPOCun36H8k7+lP5e2FO6RXp9ZWeG5Kj6PHi+VW8Go3f8PqmuYhO1HHTcTgEINo8fX6mLx4h5x3Ukc5oe3x18O1g7rL1KXuWztdCPvhweQiAAgzDUO6+kk4eApXrjDj7c21aajRye/eskpKYmBhJpilB/VrXRVGWzLdBVCt2dm7U2tZUXBY2rRIll98+/i41Flr9zYL5IN6ZsTMZA+9JtqtHi6xMsQgWny9PtbsLrH+Hlq+a86mA3LyCW3kR28u9npMwn54EDwBIMx8hSs1euICn8fxFa7CHWZ6d2pjZOlB/VpXhfFUxun283pJr06tpKqmVlbtKpZRr86XQ+VVjbZJSkiwjhMrNIjrNQm1nFIsDTGINl+vj3cW75Bbz82RE9q2kC82F8qe4mNej0fYDw+62gEgAjQUaTgKha9wFc4ufXVmz29mk0dy6UENobrueIKHlqjvvjxPTvvNfyT3iVnyg9cXyYZ9jcOD7qf7x9I62r6GGAQiVoYYRJuv14eO8+zSroWMzsuSd33MWCfshw/BEwAiwFu48oc/4SqcYUZ974xuxpYevCq3m0wYnWuVtPE3POt2ur3up/vHYit4w99VW72f+Ghto+3OHfeZvPFFfqPHsh/8t8xcu8/6b91fjxMrQwyiydfrQxcn+Hj1XimvqJGZa45fX08I++FD8ASACIl0uHIXZryFmp/8ban8ckrz5RZd+5/axf2a6JFaZ1x/v5n3nm8NJ2h4Hp7Ob3DPDGv7WAudJlvB4d/rw+XEti3k/RW7rJsjj9ecsB9WBE8AiKBIh6uwhBk/2mUjtfSgtui+e8cw+ejuc+WmoT2kX5e29TPA9bN+rY/r99+5Y1hMda9HoxUc/r0+2rZMlktOO0HO6tVB/uZjIiBhP7yYXAQAEeYKV65Z4LrqjNYEdM0C10kLOlnE3Sxwf46tYUS7soOJhfqW/NT3+osULI/q0oP6dcPHYmEZzGC4bii09VeDuD9DFvSmRMOP/p1jtbU3Wjy9Pmbcc560bZkiz368XrY2aIVvirAffgRPADAkUuEq1DBz2WmdZYaX4OlunXH1/bwsGXPxyXLWM59apZpc/vzDQdYs9Aemuu/W92ed8XgMnQ3/XgO7t/dYGcDF9bi2go9rUhkAgV3vpq8PHUvrDWE/cuhqB4AoCWe4imSXvselB1ftkfbpKdbyli7tWqbI+Sd3kvc9rPvO0oPOG2JgB04bT2xntHgCQJyIVJe+p6UHS45Wy+cbDlhvzl9uOb7U4GWnnyiHyqpYetBPThliEO9DXuA/gicAxJlwhhlfSw/qjOBnrxkgj7y/2poZfHVuN/lw5e5GXe+elh4kYDXHNYk8wn500dUOAHEulDDjbZ1x9em6/dYMjIv6draKcQ/JzvTYze7C0oPBrXOPyCDsm0WLJwAg6KUHK6pr5T+r98rVZ3SV7A7p1gxhXX3IG5YeDG6d+4ZoMUasIngCAEJaZ1y729+4eYic3LmNvNdgdSJ3nL70YLDr3IcaVAG7IHgCALzSUKNByBOdWHT4aJX07ty60bKY7jh56UG9Nq6SPv6ucz98/OfSIzNdthwoCyioAnbFGE8AQEhLD2qOGvr0p9aa4gUHj3o8jpOXHnS3zv2NQ3vI0JxMj/vodhouNXS6vva0nSuojhg/x2f4B6KJ4AkA8Il1xoPnaZ37r3Yelheuz5Wu7VqE5RmoAVSDrQZcwifsiuAJAPCJdcaD52md+9W7SuTh91bL+OtzPbYmB0N/kgZdHU8K2A3BEwDgFy0UP2F0rqQmJfodlHQ73V73c+IqMK517j11k3+2Yb9cP3GBX8ucBkKDrk5iAuyG4AkA8BtLDwbGtc59U/PGXiQ/Oie70WMz7jlXxlx8UrNtv9W3s6z8zQhxHUaX08x/9nIZe+kp9ds8O/J0q+XURYOsBl6dDQ/YCbPaAQABYelB/3la5z4Qi7cdlFZpyXJa13ayalexDO2VKUVHKuSsXh3qtxma00Fe/XxLs9ZmLcHkxMlcsC+CJwAgKCw9GPw694EoraiWtbtLrKCpwVM/vz5vm/zi4pMkPTVJ2rRIlpyOrWTh1qJG+2mrp65HDtgJXe0AgPC8oYRxgkw88LXOfSAWbiuSs3odL72ky5L+Z81e2bL/iPXf2tq5t/iY5BeVN9tv0373K04B0UKLJwAAEeBa595d+KytFUlIaBzUk5M8twUt2Fok1w3OssZ3VtfUWrU9F2w9aIXRdi1TrGDqjv5slteEndDiCQBAhOiSlu4cLKuQTm3S6r9unZYsWRnpXseK6jjP287NkYXbDtaHUe12H9qrg/Xf7mjwpSUadkLwBAAgQnQddXelp3SZ0WvO6GatW3/KCW3k99cNdFvr06XkaLWs31siV+V2rQ+ZGkB1wlHvTq1l4dbjYbSpkzq3CeNvA4SO4Ak4lHa/AYgsXZfeXY3OP87eYgXH128ZIm/cOkRmrtkrO4q8F3zXcKnd8a7gWXy0SjbvL5X9Jcdkq5ti8Rp4NdgCdsIYT8AhtJ6fllbRLjudaatjv7QbTrsCtVVG3yApuwJEZp17XUe9YQA9UlEtd/9zeaNtpy3zvsb6Ex+ttT4auuzFeR6315+nr2vATgieQJzTZfN0BRMtJq0tIA3f/DR8rttTKhv3HZG35m+33iB1TW6t0wggPPQ1NWL8HKlptlp75OhrfVDPDG4mYTt0tQNxbPqKXdYbnra2KE/L8rke1+10e90PgD3WuQ9GUkKCFXgBu6HFE4hTGh7HTF4RUBuLBlBtldH9lBPX1gYiwfVaun/KSmsSUbjXZm9IA64GXXouYEe0eAJxaFthmfUG1/CtbfJPzpJHr+jn1/66n+6v3fQAorPOfe9OrSQlMcHjdu72S01KlAmjc7lphG3R4gnEobHTjreqhEL317Gh794xLGznBThdoOvcexuj7eJ6fHDPDBnHGG3YHMETiDOrdhZbb1Kh0jcyPY6+QTLbHYjOOveBBlXA7gieQJyZurRAkhMTpNpDy8jjV54m3zuzm1TX1MnfF2yXF2Zt9Hgs3V7f7HhDAyLL1+pC/gZVwO4Y4wnEGa3T6S50qpGDulstmVe//IU8/uEa+fF5OTJ6iOc6f7qttrAAsBdCJ2IVLZ5AnNHi8J7sOXy0vgC1rnTS98Q21trPkxcXeNxHu/UAAAgHWjyBOKLdbzr2y5PlBYcbfb1sx2FrDJm3Hjs9HstrAgDCgeAJxFn3m044CCc9Ht16AIBwIHgCcUbXXvckN6t9o6/PyGpvlWvxVstaZ80CABAOBE8gzuRlZ3osON21fUt5+PJTpVfHVnLlwK5y89nZMumLfI/H0uNoqRYAAMKByUVAnNF6fm/N3+72e/9atlNapCTJ+3edY43b1ND59qIdXme16/EAAAgHgicQZ7TWX15OpizdfqjRKiejJy6o/++H31/t8zja2qlL+1HDEwAQLnS1A3HouZEDJCkhtElGur8eBwCAcCF4AnFISyQ9P2qABBs9dT/dX48T6ygFBQD2QVc7EKeuyu1mfb5/ykqpqatr1O3urXtdWzo1dLr2jzWuNa11BSctpu9a01pn++vEK9a0BoA4a/HMz8+X2267TXJycqRly5bSu3dv+c1vfiOVlZWR+HEAPNDwOPPe862xmsrTbHfX44N7Zljbx2Lo1LJQ1702X654aZ78feEOWbentL6Yvn7Wr/Vx/b5up9sDAOKgxXP9+vVSW1srr732mvTp00dWr14tt99+u5SVlcnvfve7SPxIAB5od/m7dwyrbwnUtdd1GUxXS6DW6dSSSbHcEjhj1R65f9oaq2VXeWrddT2uE69GjJ8T0y27AMI7JIeFMmI4eF566aXWh0uvXr1kw4YN8qc//clr8KyoqLA+XEpKSqzPVVVV1keoXMcIx7GcimsYu9fxlM7p8vBlp3j9hzbWXhuu8330va8kQRIkOUEkOcmfPY8H0LFTlovU1shlp3cRJ+N1zXV02vNx3Z4SeW/5LusmdOuBI1JVWycpiQnSq1Nrq4foe2d0k1O7tJVYVWX4PSaQn5NQV/d1E0GEPfzww/LJJ5/IkiVLPG7z2GOPyeOPP97s8bffflvS09MjfIYA4tHAgQOla9eukpqaKp999ln9DS0AIDzKy8vlhhtukOLiYmnbtm30g+fmzZtl0KBBVmundrkH0uKZlZUlhYWFPn8RfxP5rFmzZPjw4ZKSkhLy8ZyIa8h1tJPb3lwgIzsflEeWJEpFbfPxq+ef3EleuXGI/PD1+VJwsFwOlVc264bX8a26lOhbt+aJU8Xi69qOLVaxeB3tKFLXUYfkPPzeaqmRACZbSoI89b3+MdcrUmX4uah5rWPHjn4Fz4C62h988EEZN26c123WrVsnffv2rf96165dVrf7qFGjvIZOlZaWZn00pRctnBcu3MdzIq4h1zHaVu0sloX5xTKys1ihs6KmefDs0r6V7C89Jgu2HW7waJPtakS+3HpYNuwvj9kxrk56XeuksAemrZRF2w5aweCbAKHPAZGVu4/Imr1lMml+gbWQgtaiNV0WLBauYywI53WcvmKXjHl31deDbJQfxeZq9P/qrP0kMSkmx4OnGHouBvIzAgqe//d//ye33HKL1210PKfL7t275aKLLpKzzz5bJk6cGMiPAgCvpi4tkGQPs/TV70YNkGsHHV/uM//Zy2XnoXI5d9xnbrfVAKMTr5wePO1Ow4OrPJhiEhn8sa2wzHreBNu9q/vp/gO7t4+L2sbRFlDw7NSpk/XhD23p1NCpXeyTJk2SxERq1QMIH63TWe2lu+zxD9bK9qJy+X5eD7nq5S/qw4o7GmB0tj/sy2qxmrwioPCgf1ftVtX9VCy2WCF0Y6d9c7MSLN1fW9q1QghCE5E0qKHzwgsvlB49eljjOg8cOCB79+61PgAgHLQ4vDelFdVSVlEttXV1cuBIhRws815HWEtMIb5brKjd6swhOTosw58xnd7o/nocLUsHG5ZT0gGtOqFIP7p3797oe4Ym0QOIY1oKSuuQpvlVOsk/ejxq+cVmi9Xkn5wla3eXyBMfrfW4DS1Wzh6S4653RJ83G/Yev+H83pndpLqmTv6+YLu8MGuj22MxJMfGLZ46DlQDprsPAAiV1h/V4vfhpMejgLT90GKFSA7JGTmou9WaefXLX8jjH66RH5+XI6OHHB8b3hRDcsKDtdoBxCRde33r/vDV5NQVnBBbLVauSWRn9epgffzo3BzrsXPH/U92HjrabFtarJzH15CcPYeP1reUby0sk74ntpHbzs2RyYsL3G7PkJzQETwBxKS87EzZXhiecZkaSHTZUMTmJLKcjq2tLtPxX3eRFpV9Uw+6IVqsnDkkx5vlBQ1LrYks23FYfnxeL9GCGe6edgzJCR1TzQHEJF1bPtQJAy56HD0eYnMSWVVNrRyrqrEmkemHt6cFLVbOwZAceyJ4AohJWnNTV6jx5o0v8j3W7mzY2qmFxqnhGZstVoFytVjBOUNyvNFVyxo6I6u9Vf3A01OEITmhI3gCiFlPXtk/5GMkJSRYq9vAfmixQjiG5OjNpSdd27eUhy8/VXp1bCVXDuwqN5+dLZO+yHe7LUNywoPgCSBm9eiQbn0Odn677vf8KPNLKiJ8LVaqsrrW74oEtFg5i68hOf9atlNapCTJ+3edI09cdZoVOt9etMPttgzJCQ8mFwGIeeNGDpD7p62xajX6M+5TWy60pVNDJ6vZ2L/FauO+I17/rjqDXbtMu2e0tBYNOHy0StxV76PFynl0CI0OpVm6/ZDb55DW7nziozXy8PurvR5Hnzs6tIchOaGjxRNAzLvs9C4y897z68d8eupacz0+uGeGtT2hMz4mkf157lZr3Oasey+Q5Y+OkG7tW7rdjhYrZ9KhNHqjGQqG5IQPLZ4A4oJ2l+s6yrqk3ZQlBdba6zqDWSeTaHF47WLVkkkaZGi1iJ8WK9eSmtf86Uuvx6HFytn/NmjvxpjJK4JadpUhOeFF8AQQd0GlYbBkGcz4aLEaMX6O1AS9WjstVk7n6t24f8rx5Vf1Jmb0xAVe92FITmTQ1Q4grrEMZvy0WDGJDKGGT4bkRB8tngCAmGyx8oUWKzTFkJzoI3gCAGImfA7s3l4emLZSFm07aAVLdwHU9bhOItOKB5TLQlMMyYkegicAIGbQYoVIYEiOOQRPAEDMocUKiE1MLgIAxDxarIDYQPAEAACAEQRPAAAAGEHwBAAAgBEETwAAABhB8AQAAIARBE8AAAAYQfAEAACAEQRPAAAAGEHwBAAAgBEETwAAABhB8AQAAIARBE8AAAAYQfAEAACAEQRPAAAAGJEsNlZXV2d9LikpCcvxqqqqpLy83DpeSkpKWI7pNFxDrqOd8HzkOtoJz0euo1OfiyVf5zRXbovZ4FlaWmp9zsrKivapAAAAwEdua9eunbdNJKHOn3gaJbW1tbJ7925p06aNJCQkhCWRa4gtKCiQtm3bhuUcnYZryHW0E56PXEc74fnIdXTqc7Gurs4KnV27dpXExMTYbfHUk+/evXvYj6t/BIIn19AOeC5yHe2E5yPX0U54PsbWNfTV0unC5CIAAAAYQfAEAACAEY4KnmlpafKb3/zG+gyuIc/F2MdrmutoJzwfuY52kWbjvGPryUUAAACIH45q8QQAAED0EDwBAABgBMETAAAARhA8AQAAYATBEwAAAEY4NnhmZ2dby3A2/Hj22WejfVoxq6KiQnJzc63ruGLFimifTsy58sorpUePHtKiRQvp0qWL/OAHP7CWi4V/8vPz5bbbbpOcnBxp2bKl9O7d2yolUllZySUM0G9/+1s5++yzJT09Xdq3b8/189Mrr7xiva/oa3jo0KGyaNEirl2A5syZI9/97netZRf1veT999/nGgbomWeekSFDhlhLjXfu3Fmuvvpq2bBhg9iJY4OneuKJJ2TPnj31H3fffXe0TylmPfDAA9Y/FgjORRddJO+++671D8S0adNky5Ytcu2113I5/bR+/Xqpra2V1157TdasWSPjx4+XV199VR566CGuYYA0rI8aNUp+9rOfce389M4778h9991n3ewsW7ZMBg4cKJdccons37+faxiAsrIy69ppiEdwPv/8c7nzzjtlwYIFMmvWLKmqqpIRI0ZY19Y26hyqZ8+edePHj4/2acSFGTNm1PXt27duzZo1WhO2bvny5dE+pZg3ffr0uoSEhLrKyspon0rMeu655+pycnKifRoxa9KkSXXt2rWL9mnEhLy8vLo777yz/uuampq6rl271j3zzDNRPa9Ypu8l7733XrRPI+bt37/fupaff/55nV04usVTu9Y7dOggZ5xxhjz//PNSXV0d7VOKOfv27ZPbb79d/va3v1ldcwjdwYMH5R//+IfV3ZmSksIlDVJxcbFkZmZy/RDxFuKlS5fKxRdfXP9YYmKi9fX8+fO5+oj6v4PKTv8WOjZ43nPPPTJ58mT57LPP5I477pCnn37a6i6G//Sm9JZbbpGf/vSnMnjwYC5diMaOHSutWrWyboZ27Ngh06dP55oGafPmzfLSSy9Zr20gkgoLC6WmpkZOOOGERo/r13v37uXiI2pqa2tlzJgxcs4550j//v1t85eIq+D54IMPNpsw1PRDx4IpHY9z4YUXyoABA6zg9Pvf/956o9JJMk7n73XU61VaWiq/+tWvon3KMf98VPfff78sX75cZs6cKUlJSfLDH/7QCvdOFug1VLt27ZJLL73UGqeorfEI7joCiG133nmnrF692mpks5O4Wqv9wIEDUlRU5HWbXr16SWpqarPHdUKC3hHoP76nnHKKOJm/1/G6666TDz/80HrTctE7fw1NN954o7z11lviZKE8H3fu3ClZWVny5ZdfyrBhw8SpAr2GWglAbyjPOussefPNN60uTwT3XNTrp60lhw8f5hL66GrXYUZTp061ZhC73Hzzzda1o+ciOPq+8t577zW6pvDfXXfdZT33tFKAVvuwk2SJI506dbI+gqElgPRNSssPOJ2/1/HFF1+Up556qv5rfdPXmZw6w1PLiThdKM9H7SJRTm+BD+QaakunVgcYNGiQTJo0idAZ5HVEYDSs63Pu008/rQ9J+vrVr/XNHzCprq7OqtCjoX327Nm2C51xFzz9pQO+Fy5caL1Jaa0r/free++Vm266STIyMqJ9ejFD60421Lp1a+uz1lDs3r17lM4q9uhzcfHixXLuuedazz8tpfTII49Y19HJrZ2B0NCpLZ09e/aU3/3ud1YLn8uJJ54Y1XOLNTq+WCe46WftwXDV5e3Tp0/9axyN6dAtbeHUse55eXkyYcIEq3zNrbfeyqUKwJEjR6zx2S7btm2znn86Mabp+w08d6+//fbbVmun5hvXOON27dpZNY5toc6Bli5dWjd06FCrVEiLFi3qTj311Lqnn3667tixY9E+tZi2bds2yikFYeXKlXUXXXRRXWZmZl1aWlpddnZ23U9/+tO6nTt3hv+PFMelf/SfM3cfCMzNN9/s9jp+9tlnXEovXnrppboePXrUpaamWuWVFixYwPUKkD7H3D339DkJ/3j6d1D/jbSLuBrjCQAAAPti5D0AAACMIHgCAADACIInAAAAjCB4AgAAwAiCJwAAAIwgeAIAAMAIgicAAACMIHgCAADACIInAAAAjCB4AgAAwAiCJwAAAMSE/wfHbhnRp5w1hQAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# visualize dimensions 0 and 1 of the embedding matrix C for all characters\n", "plt.figure(figsize=(8,8))\n", "plt.scatter(C[:,0].data, C[:,1].data, s=200)\n", "for i in range(C.shape[0]):\n", " plt.text(C[i,0].item(), C[i,1].item(), itoc[i], ha=\"center\", va=\"center\", color='white')\n", "plt.grid('minor')" ] }, { "cell_type": "markdown", "id": "2b79491e-9cdf-4993-acb5-c083d0b375dd", "metadata": {}, "source": [ "## Utilisation du modèle: génération de mots" ] }, { "cell_type": "code", "execution_count": 54, "id": "70ccd180-a067-447b-b465-8dfdc6d46d3a", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "torch.Size([1, 3, 2])" ] }, "execution_count": 54, "metadata": {}, "output_type": "execute_result" } ], "source": [ "context = [0] * context_size\n", "C[torch.tensor([context])].shape" ] }, { "cell_type": "code", "execution_count": 55, "id": "99f39562-e15c-48ca-b5e9-a626538bb16c", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "écraine\n", "mécicée\n", "mutes\n", "reure\n", "moliaux\n", "achis\n", "rectues\n", "partépépéries\n", "dev\n", "plêquêtre\n", "accude\n", "prement\n", "angentrapterission\n", "prange-prodétaluser\n", "édomilatectivecouraclementéroies\n", "lersé\n", "sai\n", "craie\n", "hre\n", "decables\n" ] } ], "source": [ "# sample from the model\n", "g = torch.Generator().manual_seed(seed)\n", "\n", "for _ in range(20):\n", " out = []\n", " context = [0] * context_size # initialize with all ...\n", " while True:\n", " emb = C[torch.tensor([context])] # (1, context_size, e_dims)\n", " h = torch.tanh(emb.view(1, -1) @ W1 + b1)\n", " logits = h @ W2 + b2\n", " probs = F.softmax(logits, dim=1)\n", " ix = torch.multinomial(probs, num_samples=1, generator=g).item()\n", " context = context[1:] + [ix]\n", " if ix == 0:\n", " break\n", " out.append(ix)\n", " print(''.join(itoc[i] for i in out))" ] }, { "cell_type": "markdown", "id": "05380400-31df-4b5d-8140-7eebf24479c0", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "## Exercice\n", "\n", "Modifier les hyperparamètres de l'entraînement pour battre le score courant de test." ] }, { "cell_type": "code", "execution_count": 56, "id": "9b7b0c71-04ca-4c13-a41b-6c64e4fb27ec", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor(1.9492, grad_fn=)" ] }, "execution_count": 56, "metadata": {}, "output_type": "execute_result" } ], "source": [ "emb = C[Xte] # (batch_size, context_size , e_dims)\n", "h = torch.tanh(emb.view(-1, context_size * e_dims) @ W1 + b1) # (batch_size, hidden_layer_size)\n", "logits = h @ W2 + b2 # (batch_size, nb_chars)\n", "loss = F.cross_entropy(logits, Yte)\n", "loss" ] }, { "cell_type": "markdown", "id": "30cda010-9116-45e5-af30-898690687ff8", "metadata": {}, "source": [ "## Post-cours: détermination d'un \"bon\" lr" ] }, { "cell_type": "code", "execution_count": 57, "id": "db95386d-f96f-4af2-bb45-d6c99cd7cdef", "metadata": {}, "outputs": [], "source": [ "g = torch.Generator().manual_seed(seed) # for reproducibility\n", "C = torch.randn((nb_chars, e_dims), generator=g)\n", "W1 = torch.randn((context_size * e_dims, hidden_layer_size), generator=g)\n", "b1 = torch.randn(hidden_layer_size, generator=g)\n", "W2 = torch.randn((hidden_layer_size, nb_chars), generator=g)\n", "b2 = torch.randn(nb_chars, generator=g)\n", "parameters = [C, W1, b1, W2, b2]\n", "for p in parameters:\n", " p.requires_grad = True\n", "\n", "lre = torch.linspace(-3, 0, 1000)\n", "lrs = 10**lre\n", "lossi = []\n", "stepi = []\n", "lri = []\n", "lrei = []\n", "\n", "for i in range(1000):\n", " # mini-batch construct\n", " ix = torch.randint(0, Xtr.shape[0], (mini_batch_size,))\n", " \n", " # forward pass\n", " emb = C[Xtr[ix]] # (mini_batch_size, context_size, e_dims)\n", " h = torch.tanh(emb.view(-1, context_size * e_dims) @ W1 + b1) # (mini_batch_size, hidden_layer_size)\n", " logits = h @ W2 + b2 # (mini_batch_size, nb_chars)\n", " loss = F.cross_entropy(logits, Ytr[ix])\n", " \n", " # backward pass\n", " for p in parameters:\n", " p.grad = None\n", " loss.backward()\n", " \n", " # update\n", " lr = lrs[i]\n", " for p in parameters:\n", " p.data += -lr * p.grad\n", "\n", " # track stats\n", " lrei.append(lre[i])\n", " lri.append(lr)\n", " stepi.append(i)\n", " lossi.append(loss.log10().item())" ] }, { "cell_type": "code", "execution_count": 58, "id": "73e19fbc-1e3d-482e-8242-4d2a1bff367f", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[]" ] }, "execution_count": 58, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiMAAAGdCAYAAADAAnMpAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjgsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvwVt1zgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAdgxJREFUeJztnQl8FPXZx58QICGBhDPhPgS5b1QEtN7FarH2tNV6tdpDfdtKj7dq1dZa8W3V2lqs9da2nq21rVDUYlFRlAqiqAgi95FwEwiQELLv55mdZ/Y//52Zndns7sxmf9/PZ0l22WN2ZrP/3zzP73meolgsFiMAAAAAgJBoE9YLAwAAAAAwECMAAAAACBWIEQAAAACECsQIAAAAAEIFYgQAAAAAoQIxAgAAAIBQgRgBAAAAQKhAjAAAAAAgVNpSHtDc3ExbtmyhTp06UVFRUdibAwAAAAAfcF/Vffv2Ue/evalNmzb5LUZYiPTr1y/szQAAAABAGmzcuJH69u2b32KEIyLyZioqKsLeHAAAAAD4oK6uzggmyDqe12JEUjMsRCBGAAAAgPwilcUCBlYAAAAAhArECAAAAABCBWIEAAAAAKECMQIAAACAUIEYAQAAAECoQIwAAAAAIFQgRgAAAAAQKhAjAAAAAAgViBEAAAAAhArECAAAAABCBWIEAAAAAKECMQIAAACAUCloMbJx1wH6w8sfU92hw2FvCgAAAFCw5MXU3mxxzu8W0u4Dh+nDmn306/PGh705AAAAQEFS0JERFiLM6x/vCHtTAAAAgIKloMUIAAAAAMIHYgQAAAAAoQIxAgAAAIBQgRghouZYuAcBAAAAKGQgRoho+76GsI8DAAAAULBAjAAAAAAgVCBGAAAAABAqECMAAAAACBWIEQAAAACECsQIAAAAAEIFYgQAAAAAoQIxAgAAAIBQgRgBAAAAQKhAjAAAAAAgVCBGAAAAABAqECMAAAAACBWIEQAAAACECsQIAAAAAEIFYsRk275D4R4JAAAAoECBGDH51h+XhHskAAAAgAIFYsRk6YY94R4JAAAAoECBGAEAAABAqECMAAAAACBUIEYAAAAAECoQIwAAAAAIFYgRAAAAAIQKxAgAAAAAQgViBAAAAAChAjECAAAAgFCBGAEAAABAqECMAAAAACBUIEYAAAAAECoQIwAAAAAIFYgRAAAAAIQKxAgAAAAAQgViBAAAAAChAjGiMO+9mvCOBAAAAFCgQIwo/GXJpvCOBAAAAFCgQIwAAAAAIFQgRgAAAAAQKhAjNmJhHQcAAACgYIEYAQAAAECoQIwAAAAAIFQgRhRiyNIAAAAA0Rcjr7zyCs2YMYN69+5NRUVF9Oyzz3re/5lnnqEzzjiDevToQRUVFTRlyhR6/vnnW7LNAAAAAChkMVJfX0/jxo2j2bNn+xYvLEbmzp1LS5YsoVNOOcUQM2+//TZFjc17Doa9CQAAAEDB0TboAz71qU8ZF7/ceeedtuu33HIL/f3vf6d//vOfNGHCBIoSO+sbw94EAAAAoOAILEZaSnNzM+3bt4+6du3qep+GhgbjItTV1eVk27bva6A9Bxqpc1n7nLweAAAAAEIwsN522220f/9++tKXvuR6n1mzZlFlZaV16devX8627//mrczZawEAAAAgx2Lkscceo5/97Gf01FNPUVVVlev9rrnmGtq7d6912bhxY862cVvdoZy9FgAAAABymKZ54okn6LLLLqOnn36aTj/9dM/7lpSUGJcwQHUvAAAA0AojI48//jhdeumlxs+zzz47Fy8JAAAAgNYaGWG/x+rVq63ra9eupWXLlhmG1P79+xspls2bN9Ojjz5qpWYuvvhi+s1vfkOTJ0+mmpoa4/YOHToYfhAAAAAAFDaBIyNvvfWWUZIrZbkzZ840fr/hhhuM61u3bqUNGzZY97/33nupqamJrrzySurVq5d1+e53v0tRJIY2rAAAAEC0IyMnn3yy54L98MMP264vWLAgvS0DAAAAQEGA2TQaMLACAAAAuQViBAAAAAChAjECAAAAgFCBGAEAAABAqECMaKCYBgAAAMgtECMAAAAACBWIEY0DjU3hHAkAAACgQIEY0Xh/S104RwIAAAAoUCBGNOAZAQAAAHILxIhGDG3PAAAAgJwCMaKByAgAAACQWwpajEwfVR32JgAAAAAFT0GLkd98OT55WAWzaQAAAIDcUtBipLRdcdJtXhOJAQAAAJB5ClqMAAAAACB8IEY0EBgBAAAAcgvEiAaSNAAAAEBugRjRONIMOQIAAADkEogRAAAAAIQKxIgDW/ceRFUNAAAAkCMgRhyYMusl+vFfl+fqGAAAAAAFDcSIC0++tTG3RwIAAAAoUCBGAAAAABAqECMAAAAACBWIEQAAAACECsQIAAAAAEIFYgQAAAAAoQIxAgAAAIBQKXgxcv9Fx4R7BAAAAIACp+DFSGm74rCPAQAAAFDQFLwYAQAAAEC4FLwYKSoK+QgAAAAABQ7ESNhHAAAAAChwCl6MAAAAACBcIEYAAAAAECoQIwAAAAAIFYgRAAAAAIQKxAgcrAAAAECoQIwAAAAAIFQgRjyIxWK5OxIAAABAgVLwYqTII0/T0NSc04MBAAAAFCIFL0YAAAAAEC4FL0bQDh4AAAAIF4iRkA8AAAAAUOgUvBgBAAAAQLhAjHgw592tuTsSAAAAQIFS8GKkyMM08v2n38npwQAAAAAKkYIXIwAAAAAIl4IXI6P7VIR8CAAAAIDCpuDFSFn7tmEfAwAAAKCgKXgxAgAAAIBwgRgBAAAAQKhAjAAAAAAgVCBGAAAAABAqECMAAAAACBWIESL6w4WTwj0KAAAAQAEDMUJEvSpLwz4OAAAAQMECMUJEReTeEn5XfWMujwcAAABQcECMEFGMYq47aFd9Qy6PBwAAAFBwQIwAAAAAIFQgRgAAAAAQKhAjKXH3kwAAAACg5UCMpDCwAgAAACBiYuSVV16hGTNmUO/evamoqIieffbZlI9ZsGABTZw4kUpKSmjIkCH08MMPU74YWAEAAAAQMTFSX19P48aNo9mzZ/u6/9q1a+nss8+mU045hZYtW0bf+9736LLLLqPnn38+ne0FAAAAQCujbdAHfOpTnzIufrnnnnto0KBBdPvttxvXR4wYQQsXLqRf//rXNH369KAvDwAAAIBWRtY9I4sWLaLTTz/ddhuLEL7djYaGBqqrq7NdAAAAANA6yboYqampoerqatttfJ0FxsGDBx0fM2vWLKqsrLQu/fr1y+o2wsAKAAAAhEckq2muueYa2rt3r3XZuHFjaAbW255fSfsbmrL6+gAAAEAhE9gzEpSePXtSbW2t7Ta+XlFRQR06dHB8DFfd8CUKzHu/hnr8q4R+fu7osDcFAAAAaJVkPTIyZcoUmj9/vu22F1980bg9X/hgKzwrAAAAQGTEyP79+40SXb5I6S7/vmHDBivFctFFF1n3/9a3vkVr1qyhH/3oR/Thhx/S3XffTU899RRdffXVFBXgGQEAAADySIy89dZbNGHCBOPCzJw50/j9hhtuMK5v3brVEiYMl/XOmTPHiIZwfxIu8b3//vtR1gsAAACA9DwjJ598MsVi7oZPp+6q/Ji3336bogo6sAIAAADhEclqGgAAAAAUDhAjPsAYPQAAACB7QIz4MLBu29dAu+sbs3gYAAAAgMIFYsQHG3YdoAk/fzH7RwMAAAAoQCBGAAAAABAqECOopgEAAABCBWIkAF4lzQAAAABID4iRAB1YoUUAAACAzAMxEgDERQAAID/ZVncI0e0IAzESAKRpAAAg//jrkk103C3z6U9vrA97U4ALECMwsAIAQKtmhTl5fUXNvrA3BbgAMRKAX8xdQU1HmoM8BAAAQMgcOHwk/rOhKexNAS5AjAQwsD702jp6eskmX/cFAAAQDUSE1DfGRQmIHhAjAdmy52B2jgQAAICscMAUIQcaERmJKhAjAAAACkSMIDISVSBGOE2DsbwAANBqkYjIgQaIkagCMUJEI3pV0DEDutDZY3qFfTwAAABkGImI1CNNE1kgRoiouE0R/eXbU2n2BRPDPh4AAAAyDNI00QdiBAAAQGFERlDaG1kgRgAAABSEZ6ShqRm9oiIKxEhAmjEtDwAA8obm5hgdNJueqQ3QQLSAGAnIocPowAoAAPnCoaYjtonrB1HeG0kgRgKCwAgAAOQPem8R+EaiCcRIQGKkSGwAAACRRu8tgsZn0QRiBAAAQKvlwGF7C3hERqIJxEhAkKYBAID8QY+EIDISTSBGAAAAFEyaBl1YownESEBiCI0AAEDeoE/qxXyaaAIxAgAAoIDSNHZxAqIBxAgAAIDCKe1Fn5FIAjESEBT2AgBAHqdpEBmJJBAjGudP7u+5wx5dtD6bxwMAAEBWm56hHXwUgRjRGNStPJwjAQAAIOPo1TOIjEQTiBENdFgFAIDWg8yiqShtG3nPyN0LVtMdL66iQiR+dAAAAIBWnKbp3qmE6g410YGGaFbT7K5vpF/OW2n8fvGUAdStYwkVEoiMaKCNCAAAZId5722lyx99i/YeOJyzXSxpmR7m4h7VyMiHNfus32vqDlGhATECAAAgJ9z/6lp68YNaeuWj7aFERtS0TdRYVZsQI9v2NVChATGSZulu3aHD9PKq7dR0pDnzRwUAAFoh+80Uyb5DuUuVSMfVRGSkKfKRke11hSdG4BlJgzteWElzlm+lj7fXG9fX3HIWtWlTlOljAwAArVKM7G84nPOpvd07to9fj2hp78qaOuv3WqRpgB/PyG9fWm0JEWb+h9uw4wAAwGfKZH8IkZHuEY6MxGIxWlW737qONA2gU4dXpeWCBgAA4DNNk8OKFhFAPUzPCF+P2sDTzXsOWvuG2bYPBtaCZ1jPToH3AXqTAACAN4ePNFNjU3PuIyNmJEQiI0eaY9RgbkdUWKn4RZhaF88Iv5crH1tK977yMbU2YGDNABET2QAAEDlUr4YaBchVZKSb6RlRb4sKK81Kmv5dy4yf212qaR5fvJHmvLuVbn9hFTU0Res9tBSIkQwALQIAAN6oXo1ciRGOxDQ1x7+hO5W2o9J2bSLZEn6lGRk58ejulhjRU0kcWXrg1TXG7xzZeW9zwvDaGoAYAQAAkHXqFQGSq9JeVXSUtS+m8vZtoxkZqYmLkROGxMVI45Fm2qM1huOIyJa9CS/JkvW7qDUBMZIBkKYBAABv1M6nuYqMiOhoX9yG2hW3oQ7ti+PbEqGW8Bzx+Hh7vJJmdJ9K6lLWzvi9VjGxcpTknpfjPpE+nTsYP/+7bje1JiBGAAAggjz134301rrWc/arCoD9OY6MiAgpj2BkZO2Oejp8JEYdS9pS3y4dqKpTqXH7NsXE+upHO4ymaBzd+cVnRxu38WcjalVBLQFiJANc+7flmXgaAAAwWL1tP/3or+/S1U8ta51iJMeRkXJTjJSVRC8yIp1Xh1Z3pKKiIqqqKEnqNfIHs3rmy8f2p6mDuxvel90HDtv6XeU7ECMAABAxdu6PL0Sbdx80SlFbo4G1OQfvq96s4IlyZGSVKUaGmW0lJDIiXVg37DxAr63eScVtiuhrJwyk9m3b0Ph+nY3/a02RM4gRAACIGLJY8nq9s751zCkRYWBdz0FFy0GzFXyZKUI4zZGr1w4aGRlWbYoRMzIi5b3LNu0xfo7pU0l9u8RLf48d2LXV+UYgRjJEa8rdAQDCRT1zV70D+YyeGslFqkb2o4iQ8pK2kZtPI9N6h/WsMH5WmZ1ipQvr+1v2Gj9H9Y7/PzNpQBfj51tKRQ0bYfN5HYIYAQCAHPD3ZZtpxVZ/vSHUM3e3Blhh8t7mvfTa6h1pV9Nk08TKzcAkxSGiQ8SI/IxKmoanvm/afcD4fVD3cuNndYXdwPq+2U+EK22EiQO6UFER0fqdB2hb3SEjXXP8LfPpu0/kr8cIYiRD5LEgBQBkmQ9r6oyFYuZT7/i6/4EIzynhs+9LHlpMFz7wprEQphsZydZ8mpv++QEdP2s+Ldu4x6qmKTMjIgkxEo00DZtUORXXrrjIiojITy7t5X3tFBmpKG1Hw81Iyj0vr6GLHlxMO+sb6fWPgwnEKAExAgAAWWbrnviiXbP3YOAogtuckrCoqTtEO/Y3GovoR9sSk2ZToQuAbEVG/r2i1jg5XPjRdms/lrWTyEjbSHlGtuyJfx56VpZSmzZFxu9qaS83OeOqmbZtimio6SkRjh0YT9U8+NraxDTkCFUJBQViJEMgMAJA/sPNp/61fGvGc+97D8a7adYdavL13OrCHbXIyMfbEuWk63Ymfl+zfT/VHbJ3DVXZr/k0srFwcnpGxBsLpYNS2mtGRsrN0t6oeEZ4Wi/TuzLeyEw1sHLL90Uf7zR+H1LVkUpNQSUcY5pYmYn949U1hw43G96RfARiBAAATH749Dv07T8vpRVb7VNUMyVGuEzXj18hygbWNTsS0RD2LIiH5NTbX6YTbn3J6BR66PARz9RTtiIjyzfFUxrMqtr9VgRESnujFhnZarZ37212VWVYdFSUxrfzPx9uS/KLCCcN7UGDe5TTp0b3pEe+dpx1e5R6qAQBYsSBxdedFnhH5rOLGQAQh0PizK76xqyIEf13N9Qzd7X5VRT4WEnNrNsRj4z81+x3wZGfW//1IZ3121eTIh9ynftlZMsz8u7mvbYolwgeSdNYkZGIGFglTdO7czw1I1SZJtZXPtqe5BcRKju0o/nfP5l+/9VJxhDAkrZtcjr3J9NAjDjAOTueZQAAKCyamuMh7sYjmV2sVAHilcrIh2oateunREZk0NvUwd2oW3l7WrO9nua8u8X2OBEAYtDMRmSEIzTqxN6VZtlswsDaNlLRg4QY6WC7XfaRCAunyIhOp9JoRX2CghU3QyAuAkD+Y2oRamzKjmeEqTuYerFQz9ydxsmHiQx1Y9bvqjc6qUrjrvMn96fLTjzK+P2vSzfbHicCQEpX9zekFmVB4H30rpmmkSiBlFKXRbQD62bT2NxbEyOyjxgu4R3RKzkyoiO+mFzN/ck0ECMZwilHCgCIHo8uWkc3P/eBo4FSIiOZNgHaxYiPNI1ydivj5N/dtMcq8wwLFhTic+BFkg2TXF0jjbuG9+xE507obfzf4rW7aOOuA4nHmu+p2jRoZtrAGq/yaTDSQKcMqzJu4wF0qhgR70g2xAh7Zf75jj0alIqte5MNrGpkhBnUrdwYopcKuU+2SqazDcRIhrj5uRWZeioAQBbhPhT3L1xLJ/9qgTGQTkXmwGRVjPhI0+iLJUcevnjPIvryvW8YjbLCnDDLcCpmQNd4a/KFq3cY28szUwZ2K6delR1o2uDuxv/97e3NSe3ge5pn/Zn2NkhUhEtgx/S1pzUkPZPwjDRlfL+wV+a6AENTWdixyHTyjPRQxMgoHykaVYxEJQUVFIiRDPHXpZsy9VQAgCzBKYUmU3DwWfRVjy3NiRipC2hg1ReUZ9/ebJR68gLu5/HZTtEM7tGR+neLdwx94f0a4+eQHh2prem1+9zEPsbPZ5ZuMtInfJHIiJgzMx0ZkUqaMX0qknpyyNTecsszktnIyAYzAsQGXr+fHYmKsNejU2k71zSNk3nVS4ykk6Z5Z+MeozFfmBF+iBEAQMHAKQ894sCiRBChwubHKHhGOpfFF6m5y7da/7f7QGYrffwIODFaSiXN4KpyGtgtHhl55aMdVopGmD6qp5EaWbfzAC3dsIcOHj5idamWyEimvQ3LTfPqmL6d6eiqjrb/S5T2xn/y9mRyGrLsH79pONUv0kfzi+hpmtG9fUZGTANrOiJv5lPL6Mw7X6Ul63fnlxiZPXs2DRw4kEpLS2ny5Mm0ePFiz/vfeeedNGzYMOrQoQP169ePrr76ajp0KFqNfAAAhSVGBpiL6VLlC1gWqEbTa5Ap6tJM0wwwow+qD2BXfW4iIxzN+PcHtUaZ7tRbX6K7F6y2Kmk4MiLbJsJtmCJG2Ez5yZHVxu/zV9TaIhGSgshkZIS3VcTI2D6V1K9rmWVitadpEt4LFiTZECN+I1dbXSpp1OhRWpGRNParNIpTIzKRFyNPPvkkzZw5k2688UZaunQpjRs3jqZPn07btsWbs+g89thj9OMf/9i4/4oVK+iBBx4wnuPaa6+lKHPi0fGcJwAg+vBixGd1u1P0B1EjHscP6mb8XLJhd1bTNPxcanv3IAbWQaZgUslVZOTav71Hlz36llUpc+eLH9Fis5/IUT0SkRFhqCJGGKkA4YVa0k6cLqno0C7jnhHuZMq9YbhtOosiNrGyYBIkIsICxWxzktSEraWvL3CqJoiA6VWZLADYj3Pa8Cr68rH9qEt5+6ymaVi8iIDhtvR5I0buuOMOuvzyy+nSSy+lkSNH0j333ENlZWX04IMPOt7/9ddfp2nTptH5559vRFM++clP0le+8pWU0ZSwue2L4wLdPzqFdwAUHu9s2kuf//3r9IOn3/ElRngw2SRztsfb6/cki5EMpml08ZEqMsLbKFUgA81JriqpBJcX3F5cunqm4pVV8YZbFx4/gD4xtIcRVZKeJ2pkRFDTNOrCxtU34hfhfh8tOYN3QzrmHl3dyWqbPrQ6WYwUFRVZUZJMVtSkExlxK+tleE7NA5ccS7d+fiz5Jd39WmNWR/Hj/VTtREKMNDY20pIlS+j0009PPEGbNsb1RYsWOT5m6tSpxmNEfKxZs4bmzp1LZ511luvrNDQ0UF1dne2Sa1iN/nvmJ6hf1+QPihP8JfbmmvgcAQBAbpEx7B+YfSVSiRFuajixf1yMvLNpj3X7kVjmIyP64pTKM6JWeshYeZVdaUZGWARd/NBiuvzRt2w+GTckAvP1EwbRL84dTR3MRZ73Xd8uZcZ3I5fwMty+XLwgAlfVSMmtpGk4MiLNuXjRzFT/FBFJqv+ChYmgpmdEmGRSDG0xhUWgNI1pYHXyjKRDup4RnucTdlQksBjZsWMHHTlyhKqr47lAga/X1MQd1TocEbnpppvohBNOoHbt2tHgwYPp5JNP9kzTzJo1iyorK60L+0zCYEhVJ3r1R6f6vv95975Bn7v7NSPPCgDIHdzvQhY+L/OpeEa4DPWo7uWGQZSrVFjEsFFT1saGLIqRVIuVnLHzoq+eNR8zIC6epBw0KK+v3mnsGzbpqi3dnWhoOmJtR5ey9oYH4/ufHGpclzRISdtiqz8Gj7PnqIOKiBM+87bSNMrZN5/AyXFrKTtNcdW9YyKlISZW3izVPyKelUwNIOTPjQiLIAZWt+6r6VLewsiILiZbXTXNggUL6JZbbqG7777b8Jg888wzNGfOHPr5z3/u+phrrrmG9u7da102btxI+QI7xznPCgDIHWJGZDGhLgyukRH2DrQpogn9OlsmVqmkYQ5nsAPr3oBpGomMlJUUGyZbTil171hipEpaMjfnZTPtovYL4Z4l3KWUF1QVETzsr5BIxqXTBhnp6199MZE6GNi9LMm86jR9dpO58Ja3b2tEJkS37MtQF1aJ9HRTxAj35+DX4coUVSiJR0OatwmpojT6PlJfW9JqfiMjRoWS+fpOnpF06JSmZ4QFfNjm1cBipHv37lRcXEy1tfYzf77es2dPx8dcf/31dOGFF9Jll11GY8aMoc9+9rOGOOHoR7P0XtYoKSmhiooK2yUf4Q+32wcYAJA5GpTKiM273cUIL4wiRpgJZqqGZ5o0K4tRkDQNd0b9+zJ763MVWZyk82iqM+dESqOtMSfr8cuPp8cvn2yd0afjGeHvIvGAqGKEu4Z+6jev0pNvbXRM0XQua2+INoajIV+Y1NeIggiTTRPwtCHJhn/2bnBzNEYiMdx0jIWBarbkCMGyjQnfTjrsMPdJt/JESSynPx68+Fi698JjbPeVdIREBJh579XQxJ+/6OqnYa/N8Ovn0UOvrfU0r/qNjOysbzSEMWukTKVHOpa2MDJSmdh3kRcj7du3p0mTJtH8+fOt21hQ8PUpU6Y4PubAgQOGr0SFBQ0TpXkLma6s4ff21QfepE/ftTCj9ewAgGQOKmbETR5iRPWMMF3MPh6Hmo7YIyM+xQifbFz2yFv03SeWWcPidGRx6telzCrT9TpJ0cfeHzOwq+F/4HRJutU0a3bU2xZNESOvrd5pm+Ei7DbLh6XPiRv/c+oQWnztaXTmaOeTUVlopVmaDKyzzuIbmuiiBxfTubNfs3w/LUnTqJER5pThVTTOjH7pXhY1MvLCBzXGxGY3UckihFN8ar8XJ7+I39Jtid5VdyqldhkaylqebppGPCP5FBlhuKz3vvvuo0ceecQo1f32t79N9fX1RnUNc9FFFxlpFmHGjBn0+9//np544glau3Ytvfjii0a0hG8XURJ1vnr8gLQex3/onIteu8M7PwsAaBksJgSvRS3hGYl/9xSbJ0pNR2J0RAm1683R3Hh/Sx1tM82Tm/cc8IyM9O0SXwT5HGy/RztyEVbSNVToakYZeNEMyssrt9vMmyxG+IRJZt2IiVHYYwoeEUBucJRD7YmhIwscT/FlOpqVLHIW/+HWfVZLfj1tEoSd++Pb26Nj6rN71csibNp10NZSXmXfocO0wIwq8fbqJ9H6cfeTpkn4RTInADqlKUZqI5KmCVzHc95559H27dvphhtuMEyr48ePp3nz5lmm1g0bNtgiIT/5yU+MDyz/3Lx5M/Xo0cMQIr/4xS8y+04AAAXLwcaEeBB/QirPCMN9KRiOXkoljXq/VLy8alvSgqgjixOnWdhIyamivQcOU4XWAlyQniRSgipIFCcdz8grH8UXU06zPLpoPa3feYA27jpo9cSQpleCCB55zXSRyIhEZdgHw0ia5t8rEil/p5kqfBxe/Wg7HTeoa1LLdD3twXTzIUYSnpHE52SjKWA5gsTHq9LshcK8+EGt9XngqBa/F64m0iMjvStLDR+IHzEiZb29MmReVQUe70cWTLqhOHWaJs8iI8xVV11F69evN0pw33zzTaMLq2pYffjhh63rbdu2NRqerV69mg4ePGiIFe7g2rmzPXQWZfwdUjt5koECIG/hWSTXPLPcMBCq3TT9pGlKzNA4+yAYTtHIxN4gaRrVFOomEmRx4gVOGn55hfKlGZcMdROk+RU/X5BheTxv5A2z7cCXj+1vCDGO/HBqQtimRUZUz0hL0M2ZIkI6msLiVbOVvNu8mKeXbKSvP/IW3f7CKtfX4H0h26unaZyQRVcEGH8mJFWhzrgRnnvXnprh6IiKCC1p8uZWus0CQWa/SGQkU2W9apqGzbTijUoF7zsx/+alGAEAgLB58LW19PjiDfTcO1t8G1gbj8Tv165tXIS0LRYx0mzzdqnVEW6wKODquVT9P2xixDx79eo14hYZ6aycrQcZlreqdp9RQstm0hG9OlnTdtVFllNNqo8lkaZpaWTEvtjKe5KUgioixSuj8t7mOqsXjBu83/nkjwMBqdJK8W1KDOrjFAwLA/XkUX0tjmBxZIYZY07P5YFyKiIshvfq5Hlsfjt/tWGC/eSvX7aEIEdTMkW58nnxm6rZvr+B+LBzhLC7Yv4NA4gRH/gNd6kgMAJAdpESWE41qIsan+W6RQ50A6sVGWHPSEAD6+urd9ges8slTSPCo8JnZOSglPZqnhGeiCvpgyAm1m1mBKBPF25SVmQ1UlMrWDgypIopSdNkPjJiT9OoOLVnF7/d6tr9rgUPkh7rWtbeOp5esCCS/cgpCknRqBNshec/qDGE6bDqTvTpsb2M21ZoRuUtWmTESYywD+j+hWuM31fV7jdSZJnsMcLwexefkd/yXknRcPmzVE2FBcQIACAvEWHBzbnU0ecsENzMkJ6ekeZgnhFJ0cjC5idNI/f1Kv90i4zYfSP+IyN89quaOwf1SO7qqptY/RpYU6GbIuU9ib9BRZ3fI6zbccDyargdUxEjflI0Tr1GJK0nlUOqiVWiRyxEhpti40Ol8ogFsQg3K01z6HBStdQ/391izOLp37WMfnf+BPrSMX3pcxP6WL1jMkXQ8l7LvBpyioaBGPFBOnrxuXe3pPEoAIBfJJXCC4I+gdXNN5LoM6JV0zQHj4zIGTIPNPOfpmmXMs3i5hlRfSNBIiPSKl36lHDnWRWZXiwRlGwYWHVfg2NkREvTsBFT9XJ85NI1dme9dF/1n2ZQe41s3BUXPKcNrzaavPFr8iLNPprXVsc9LWeP7UUjzMZuXImU8H7Et48bw4n/w6la6rE3Nxg/v3xcP/r02N70yy+MozvOG2/N0ckUQct7ReCFXdbLQIxkCe47AADIHhK9YOOj3lbcrbxXBIykaVwjIz7EyD5TUAw22467RUbqbAZW0zPiEUb3ioxwKiJo4zNpe86heGZQ98QAOZ68K9Nt1chIpgysLDqkg6sqsNTbJKKgG1jX7YyXAwsf1Tr3cdlhRUYCiBFz8eXFeKMpXIf17EhDzXk2nKr5zfyPjM/ExP6d6ageHQ0xx+XV/DH5qHZ/khGVhYW0nWevifDBljojJcaftS9Oyu5ok04Bu7BGpfsqAzECAMhLRDDwGbWcqYofYpHL0Eo9TWOvpgnWDl4ExUBzeq2TGOHFjFMMemTEK00jfUZ0z4gqDna1IDKiDt8b2bvCEilqea+0g+9S3rLIiO4b0SMjvPtPMlMVemmvpGhUI65nwzMzahQoMlJ30BKu3JRubN+4SfXZZZvpif/Gu9L+6Mzhxk/228hk4hWmiVWfLyNpODXy9dji9cbP6aN6WscgW3SU8l6PPjYqtREp62UgRnyQhn8VAJBlJJWiekYumNzf+PncO1sdB6FJNY2cwSYiI3o1TerIiJhQJc3BngDda6KKDr8GVllInMRIV1McBImM6GKEh8nJGfSo3pVW07Jac3+x3yFTnhH9rLvcjPZIA7fRfSotsSKD+XTzqmyrW5pmh8OQvCCeETGTcu+QsX3jLSfmLq8xPg+nj6ii44+Kt7xXZ/BIt129eZl+fFlgPft2PGV/vvnZzCbl5v7lz2I+dV9lIEZyIEbQcwSAbBpYE54RXjg4rM5Rkz8tWh8oMhIkTcPiR56Lz6jleXQvh5whc5UDt/1OREbcF4sDMpvGwVeR8Iz4N7Bu08QIn+FLamRC/87WzBzpNcILmeyKVO3gg0dG4gKLjZszzxhKvzh3jCW69LP5tWZk5GTTk+NWUZMwsAbxjHSw/B8iZvp17UDjldbxfEj/14yKCCPMuTxS3rvJJTIiInTO8q2Gf4PTYVMUUZMtOgY2sEajxwgDMeKDorQsrACAbCLRC9UzwvNcvn7CUcbvf3pzg63Kxqm0V/qMsBAJMptGzjz5RIX9D27dUVXzqvoz7chIQM8IL94SGeGhewJP3/39BRONBZLno6iiRQQVv36JafTNVK+RcvPMnf0V3zntaBrTt9ISXSLC9MjIqcN7GBGsfZqhVR+SF8TAKgKJO9FK9IWPDUc+RKied2w/YyaQivQSWWG2hdebl0kfGTnuPESR+dSYXjkpne1k7kunbrY6vP1W91VERgAAID1EWHBURCIjvMhNH1Vt+AdYGOgD4BKzadokzaZRp/amKu3lZlnifeBFRtIZbmJEwvdiYFUNju6ekbYt9ozwAi4VROpi3b9bmbFAcpRE0ihiYN2dwRSNm2dERUSXfja/zhQKbCodaPpcuEeH3yF5XuiRAOnBwtErTvWxN+Tq04cmPe7oqk5GxISP8yOvrzPMqc6Rkfh7kRQQR0ZyQXmJ/zSN2p8HkREAAEiTRrMyhoWBiIcO7YqN5mBi0tTHuydKe5OraViQ+O3AKuZVSbuIB0JmpLiJERlxL6mB4J6R9jaDaSokKsJnzDIFWEfSNHxf3g/y3JlI0agLXbviImu/q1iRESVNw2JNhB0bhIdWd3StqJE0TZAOorw/1EGE/cyutMyNM0bRvO99wnEAIO9DEUY//ecHxueABYhU4egGVikbVp8/KmmaWlN88jZnusQ4HZCm8QOyNCBivLlmJ20wzxwLlUZzUq/qnyht18Z2piph9FRpmqCeEfEESImqnJXvTpGmqTIXfhYtbqkgL8+IGFj9DsuT3iE9zNd1gr0WrMn47XOUIdOREfbUeD1fuRkBUpuerTXLelko8X4YUhVf7KWkVuB0hJzdB4mMcBREjQbINvph6uBuVmrjh9OH0UvfP8k6viI6+bizEVj63XCzs1zQKUCaJko9RtKa2gvSZ+7yrfR/8z6k2edPNFzkAKQDn22dd+8bNLJXBc397okFuxMleqGKiFLT48Bhd7UpVeoOrM22qb0sFLwmn0oYXBYfWWjdIiOyWEnLct5mjo700ma38GtKZEQ9c9fTNDIsj6NAQbqvOsHbw+ZWNjPyZXeGIyNDqjrS9Z8e6ZqqEFOr2g5e/CJSNi2RkVXb9jlGRViEOkWSvOB9//H2uOjpa35e/HDtWSPoS8f0M0zAnNZRUT1BXJ3EopY/Y/pxzhblAZqeSVlvFLqvMoiM5DAwcsWflxqGqW/+cUmGnhEUIlKy6lS6WkjokQUWGGISlMiI3onV1TNiREYSz8e6RBU5OmJAFcOi9LjYZXYDte6niRHePilBlRSKnkaSl3VKq/CwPNFHfipq9LJeN1TfSCbLeoWvnzCIThtR7fh/4o05cPiI1UZdKmmOMlvXs1fDqaJmh7m/Of0VdIaYLTISIHLB28slwLoQ0SMjErlkYexnZk4m6BjAM5Io6w13QJ4AMeKDTH+QdIc/AEFoMCtH/MxPaa00a9Uv4hcR+ph9H9zSNHqfkfigPPtreKVqxMCqe0Z2azNjRLSIGFGrWtT264Laa8PJwMqREDGiSiWEF06VNE7I//PZfCJNk5nISCokMsIaQ1IuXHLLiPeHfzpV1Fh+kTSaianGWi7rzQRqu/8Npl8kVykaW9OzhgBiJEdRm1RAjPhAbXoDQNiICdNPy/LWitN7F78I06dzfAHYstfbM1Jsawdvf06vLqxSLSGeEen/wXNSXvqw1hqip6dpGOl4KqW0KrKI8HtxOwmy/DDae3NComepIyOJLqyZmtjrFxaREtSQFNU6U4xImoYjWU4VNVJJ0z1A91WnyAg3PMsEap8RaTOfqecOEhkJkqaJimcEYsQHTuG4lpC60TQA7jSYxs1UFR+ZhEPj723ea712FMWIGhmRjphcGaKeJeppGisy0tycFGnxFRnRqmTe31xHX3v4LfrGo28ZEVBHMSJNxhzSbBIZKXeIiqSK+mQiTbNNTdNkoBW8Hzi9Ui6pGtO8K3Np1Nb1R1clV9SIRyeIeVWPjHAEyGlwXzokqmmarEqanEZGSvzPpklERpCmAQC0IDKiD3fLJv96r4Y+fddCuuPFVVl9HZ52/Zclm1Le77BDikotT+xUynNg2iYt2m4dWHk3qqW9xmt4iBEp7ZXIiKRpZA4NHyMWAk5ipIfWZExFIgNuZbhMbzOsnlkxEv//lbX7rFRTriIjjNqFlQWveB7U9JI0IFMrauT9Bem+Kkzo14V6V5YaU3QzhTUIUUnTZCoFFKi0t7HJsVutU2lvFIbkMaimASBPPSPMm2t30uurd9JVpw7Jaq8AOcvLZjkxRzC+98Qyo/nYmaN7ep6tOkWF9PfP6Yy6mn1GrxFZyJJKe00Dq/p/idcI7hnRK1n0PiO2NI2DZ0RKg70qWXpZZcupPSPSz0Re042pg7sb++TtDXuMMt9MG1h9VYHsazAiQ9LnhIWiOt1XIiNqRY0VGUkjTcOptdd+fGpg46sXIjo5qiYRnDAiI7FYPMrmVB7ObN170Jh2zG89l2kkL5Cm8YlTmV26pFKsAHihpkpmzf2Qfvef1bTwox1Z3WmSwhCDYTZYvW2/8Tr8UmqZpxNO5l01TaO26FYbn+lNz4rNPiPx/3NuHe/tGWnnmtLYwZGRA+6eke0OaRo/hlMrTZPCM8Klv7JYp4qMcDXJN0+Kt9GXYFuuDKx6F1bpocKvr7ZQl8ZiakWN5RlJIzLCZFKIWB15i+zRs1yKkQ7tiq3X9/KNLFgZ9zTxLB71sxkmECM+cerGB0AYyILKyBe316yTTCBRAmlVni0x4vQe0zGw2nuNHEzpGXF6TS/PiFXaa4bleYbL8Ud1NULykwZ0sdIwkraR+6nCwKm0V1I3XpEMt4ZuOixEeM3mCIPMtPHiipOHWAIu12macsUzItEhPTLD/pFiraLGqqZJU4xkGhY3ahRMZt7k8vU7+jCxLli5zfh5yrD4EMIoADHik0zqZ8RFQEtQF01JF6RavDMlRrJZlq6OiE9llHWMjGjRy8Sifci1tFetWNH3oZdBWDwNEhlhHr/8eHrp+ydb6QQuT5UgqN3AWmqlcfQoqR+Ph7wvFi5e0Rt5Lu5r4mdIG++/G2eMtCIV0s0zl+W97BmRuTtSoSQYFTVm4zSuqOG03hqzOVqQpmXZRj3WHHHKdPQlFfKZdDOx8mdGIqknD+tBUQGeEQDyjAZFEMjZT7Z714i5M7tpmoQXIJW4cvJzSPdVfdHerDQ+k8e1L47ft1hZKIJ4RvSmZ0x80Fq8mynz8fb9VsRGnX4r3VBZ7HAZreo3kQobr8gI+yN4YebtZROiW8Muv+ZVlTNGVhsTfbntfC6mzApl1uTeJuvz7RTN4VQNd01lPwbfj/fhgG5lxiWaYiT3IqncFHZukZG31u8yWu+zSB3dOzqdwBEZyUE0Y/G6XS14NAB21IVa8vuHFFNrNpAoQbY9Iy1K02iREUk5bNp9IElwtGsbX2h5wZU1Nyky4rIN3HBNvujVyIggKQMRI3qYnoWE+DH08l4/AoJFD1eBOA0CVFlj9uoI0oqcn/sLk/rSqcOdu6Vm25PHi+Qus5pHj4zYy3v303+UVEOuow9eiKk5134Rv11YxS9y0tCqnArOVECM5IDr/vaeVUbFwL8KgnDb8ytp5lPLrJC+00Kd7f4f3IeDOdiYHdHDkR0phdQrhnyX9mqRkaPMHhVb9h4yxIPatVWqadSKmiQDq0tkhD0L8jesVnsIIiSkFb2TZ8CtC6t4RqT81w2J+nBVhBtvb9htmRSjjnSb5dSLdICVoYAqUhXFJcj/+TC+qJ4yPDq+B/14hyJGStt5dmEVv0iUUjQMxIhPWqof/fQEAECHBQhXyzyzdDMt37zXVXhkPzKSXc/Imu31VpTHl2fEqelZe/vXGZ9ZS7qDw/rqY9RR9jK5V0/TuPkxxKfDz+FUTi2RESe/SHLjswbbsfZjYHXzw+hwmS4zIQ/EiDUsz4iMuM/GkYqadzftMUysnAKbPKgrRQnVrNw3lMhIsfGTBTgbzs/53UK66MHFRpk5RwnZb8MBkU8cDTFChZ7mQWkv8ItqopTUgFPUINueETVNk43P72ozpeE7TeMjMsIM6xlfvFaxx6DJWYyIidWpmuZPb6ynN9fsdCzrVcPxKnqKxUmMyH3UNA2Xgsr7SuXzsPwwLic53EmV/4+zF2PzQow4RUaSxcjA7mXG8RLhOm1w96z210kHtZomzDTN/oYmev79Gnp30156ZdV2+vK9b9Bfl2w2/m9i/y5UmcPSbT8gMhIibIJTWxsDoKN6NMREesghapD1NI0ZVeCOr9loQ79a+ztINQTQaRucupbKmfTKmv2257SnaZzFyAdb6ugnz75HM596x6XhmbP/X6byOi1OXmka6TvCqZ9UC6x4Rtwirm9vjEdFhlV3ylir82xSLqW9amTEQYywEVgqaqKYolHFJwtBtVQ6V3QsMatpGpro2WVx8cGs2FpHv/73qsjuN4iREJk26yU649evGCFHAJxQIx7yu1NkJJXHoqWoi382TKxqWa8/A2vyNpQ4LOC8GEtkxOoxUtzGZngsFs+I9r4k6sA/1eNgtYJ36R/Bi7+UDqeKjHB5r+A3RWPzjLikaawUTf/oR0X0dvDSZ8StN8rRVfFjGtVFVSJm1Z1KQ4nadDTTNOt31tOrZgnvQ5ccaxNGJw2NVoqGgRgJAflal4ZIYsQCQEdtMibueKeF2ilakknUMtdspISkkka6qKaK9DhN1NU7sDJDeyYMj/pcmlSREWmopadDUkVGWOioaRZnA6spRmyRkdTdV/02PhPzKs9fyQckTcNNz3Z5pGmYodUdLaEZRuQhFTIFd3BVYshfLulofi5f/KDWiGSO69fZEG1Pf2sKjetbSSce3Z1G9a6gqBH9+B0ABYwahUg0OMu9gVWdaJvpLqwsdLhBGDOydwUtWb87ZaSnwRRHHIUQL43egVUtBeWFXiradDGS8IwccZzrIrN5BvfoaA1B8/KMiInVq5pGTK476hvS6gsiU4n5hIbTveq2cEqNfQL5GBnhfS6fZac0DXPO+D703PKtdMUpgymKnDSsB/10xkg64ejuobx+RzNNI9HMc8f3tgTs3686gaIKIiM5wub5007qIlQiDyKGjJRPFRlpyGFkJNNpGg4ns9jhXhPSvMqrFbuxPeY+UBd6p8gIn3FL46n3zGok1S/iVU3Dg8SEjUrjtET3VfdzOVVQOIkWqbhQe0EESdNwKawM09NTNRwF4mPEHVRFQEUd8bVsMiM9fIzc5oENqepodLr9zPg+FEXaFbehS6YNoiFKOimMyiQR2pmcSpxNIEZyxN8VI5EO+o4AN9SUyD7PaprclPZmQ4xIioYXGcmxp+wzYm6POt3WSYyovhFLjLhGRuyvuUuJWmxSeqAk5tJ4R0YEp8iICBSJskgFTJCOqfK8+xsOO/pFODwfpaZWfvqMiCDkwYNRamSWT3RSRPIJQ7oH6sAbJhAjOeLRRetz9VKg1XpGvNI0uWkHb7xWhtM03E2TGVzV0TJ++p1No/aicDKwqhU1722p8/SM6JERte+JpFxskRGPKhWbZ8ShhFKEDAsgOXZiZpUeJKkQ8aU3ottodpyV950PqGfzbj1GQLA0DfPZCdGMHjkBMRICGJQH/KJGIaS/haOBNdt9RlTPSKYjI2aPEa6SkBkufmfTVPqIjEivhw07DzimaaxqGo/XlAWeEY+KmC6d6KGU9zpFRljIyIm/iBsp8+3R0d+EcCllPtBo77QpnTe90khRjYwIbuZVkJpu5meP/x541lC+ADHik199cVxGd7xMTQTAtxjxmNCb7am90mdE36ZMRkaODhAZEQNrZ2WhdzKwMtVmdYNV2utaTeP+mmxg1YWfU18Tv2kaTp8kZogcbllkRDseMq01n8RIUmQEYiRt2Cd002dG0T0XTvIUzFEDYsQnkwZ0MSZyZgI+s/rqA29m5LlAAXlGxMDqIAZy6hnJYJqGSw9loBx7RkQopGx6Zpb2crpDPB9uX7z64u7mGbGG6Dn8nfN03cSE5GZP8cOkKu21+UYONRlCaM+Bw7apvn4rUPTjIduZD83OVGGlWkTceowAf1w0ZWAke4l4ATESZGfBUAUCsnzTXvr1i6vSTqM4e0acmp7l0DOSwdfiPhn8flgg9OtapkRG/KVpOOXyw+nD6OsnDLIiIDr67WpDMqc+I26NqmT6r7x/p/bzQk+zQypXhLg9n/hG2MQqlTv8flRTrhfyvGrFld80UtRgs2q5kqpBZKTwyJ9PaysmBhdJq+X/5n1IC1fvoOE9O9GnxvRqYZ+RJiNdovb8yFWa5rA5tVffppYi5azVFSVGhKLEZzWN2sDsWyd595vgs2wWHE4Te52qafgsXS255ftzimfjroM0vGeF1WDOq7tm3y5ldM2nhluixAlpmsbpN7WSxm8ViRUZOewSGcmjNI28H9n2rhGbmwKyT359WgHIM6TRlvxsqRhx67/Bt3PKQxbWbEZG9OqNliClrZ07xMPyfj0jamQkFezP4N4dW/Y6Nz2TPiO8/5y8IEdXd6T3t9RZvhFJ05R4pGmYb6YQSYnISBOVtG10nGvjx/SZlKbxUe0TRYxIjilOERkpPJCmCQCqYFoXPJBLzkizBXsNjNcyfwZFLaNlYSJnjk4EbXzGE2P/5/G3k6bS5rLPyJ6D8UVYUhNqmoarRKQCxs3A6tfHVaWkapINrPbrevpFWmdLRY1lYG3h3JGEZ+Qw7TTNq6rxNRUSmUmOjBzJuzSNGulhUE1TeECMgIIkFovRxJ+/SMfdMt8qhczGa+wx52zI8K+g6H4AdV5KS02s1z/7Hv3znS103r1vBBqUl0nPiJg2JUogYoTTMFf+eSl94lf/sVrF27bHStP4EwScBhKSOrBq0aRSLTIysleFrddIwsDaQjFidmGNe0YabGWZQRbvZM/I4bwzsOriCX1GCg+IkTyAzxBn3LWQbn9hZdib0mqQkLze0CqTcMdU8SnI8C+mZu8hQ6j4QT/rlfklmYiMSOfTwKW9aVTTsFH11NsW0EOvrXUUI1Kiq/YZkf4j63Ymi5HGgJER1cTqVk0jdFDSL7ygdzcrY/aaKSUxC5dmMDIiBtZuASIjiaZnTbbjJGIpn0p7GbX9OyIjhQfESITyNPPeq6GZTy1LOvP869LNtHzzXrrrpdXZ3YACQjWBNmepH/+e+kRqRiIjPBbg+Fnz6Wf//MDXcxwKIEaCRkbUaEe2m569sWYnrdlRT3Pe3Wq7XRb45DTNEaWUudndM6IJi3TEiHhGBDX9wiZTiUDIsZD371Xa6wfVM7LT/HwESdOIt0U9HvVmiiYv0zSIjBQ0ECMRQNbCb/1pCT2zdDM9qJ09+j2LBulFRrIlRtRoCPtTmFvmrjB+Pvz6Ol/PoS/86iRZnbvmf0TvbIzPJfGD6gMJct90xIhEQHQDrqSxLAOrucBzZESMmE4RH+kz4sfAqg+fS46M2K+rBlYWDGoJrVrN5FXa6we1mmaHKTKDGFhFNNmGKZopGhZ1PLAtHyMj/L68GsqB1kl+fVoLpAR3xz67LyDfvlTyLjKSparY3YoYkd8DrP+OKZF1LoZO5pm3N9NnZr9mRCFmzV3huyLFj3BT9Vo6npE9Voqj2fF2aevevrjY8lHIMXJ6vYYWREZKUnlGFJHBqRS1auWQUkLdcs9Ios/ITnMoX7dy/5ERPWLDiME531I0jOxnpGgKE6xyEUBvK6BXZ/o9+9PhiAp3t2x26EtR6OQiMqKaVnfXHzaOR9DXOmgu3oO6lxs/392UOvLB3qI/vLKGFn3sXSWTqsupm2hJxzMiJby6QLLSNOIZMSMjUoXkln4SA6tfoa6KEf0xxR4GVhYMaj8PdeHXm6cFRe3AKsbk7p38R0ZkO9XIiJix8y1Fo7aE54m9oPCAGIkA+vqkj/1ul+aX3m/nr6bTbn+ZZv0rnhoACZqUcIhTE7FMoC6onJ7gs1ZVBPlBFr8J/TsbP7nfRSrEN+BVBhzEM6Lvn/TSNI2Ozdn2mvtIWqY7LfBOEZ6EgdWvGHFP0+iREZtnpENbJR3SZB0Pfg797zTdahoWrZLSCxQZcSjtFZ9NvlXSqAIKlTSFCcRIRJA+A06REjUyolY1pOLX/15l/LzvVbsHJcpw74tlAXwP6aKKAr8RgnQXYDU6EjRKJVGICf27+H6MRDJSznfx+VmSKERLxIhVidLknKbpbM4ikWoaFcfISMA0DYsduW/qahp7mkb8C7wdiVbwLf/qlMgIm1f5hIT/7rsE6Dzq1PQsH+fSqF1rmaPMKCAoLPLvExsi2fKRPrJoHf1m/keuM3Daty2ynRG2bcUekimzXjKEwj+vOoHG9K3M2uuoHUWDGDm94H4YPD9mbN/ONtOqwGe/RwKnaeILzXjzOQVeP910jbyfVC3i/UaE1FbweiO24J6RxGNZmFkGVq2axvZ6DuLH6jPi82+BW6xzdIRbuqeMjLikaYz3YUZyMmGwFM+IwBGBIH/bHdq3caimyV8xctbontTl68fR+H72zzooDFrvqpZHyBecoEd/1VB0qpkd+Y5ELF75aHtWX0ddiFUxsmb7fvrJs8utoWhBOOW2BXTO716zHqsfVw7HB03TyELDzbD6dulg3e612Ej6JVPD81Thpm5TSyMj+xubLEElaRqnSIdTZKQxYGSEqe5U6jKbRuvAqpX2qoZWKcFtqXnV6RgGqaRhOpiREVs1jaRp8tDAykLsxKN7UCczYgQKC4iRFsBtor/5iaMo0xSRXY2okRK32SStjZaMqedeHKkqPo4oZ/tqOuPPb26gP72xgf6yZFPar790w56kahpZyFQDa6qUGwsX2TZOHYwwO4Eyp4+sdnwMC1m/kZGcihFTmLEIlPctt3G/DlncHSMjTp4Ry8Dq37fxleP609i+lTR1SHfPPiNq/xCOXrA3RG4TU3JLy3olPaTOjwniF1HTSbwvROTmc5oGFDb4xLaAOd850TDXceVCJtEjI2pkv7VHRlo6/4SHmZ34y/9Q/65ldNWpQwxxce+Fk5LGyKuREVXgcZolHTGk9oKp2XvQlqbh/gn1jUeM96RGRvi2yg5tfO0DTgtcMLk/bd17kC6ZOsiYAsw9aZwWqEyLEV0Ac6SCUyx+DZxGW3wzMiLPx2fBie6riYgA386LtLqfGjIUGfn8pL7GRSeVZ0Ru4/ctpuSWNjyznr9DO6NTb9BW8IyaPuLPCguQfE7TgMIGkZEA5KpAVv+SV/ubNB7J3FyQKKPP2/DLgpXbjJ8bdh2gH/3lXaMJ2K3/+tDzbF+NjEhKIGgESs2+1OxtsIX0RQhx2271fqlm4qiCiCMGJw+rouf+50T6wqS+rmWlnKKR9+bVZyRIIz2pPFIXaadohRssupzEhd59VdDfm9NrSSoq3bL3INU0qllUol0lGUjT6P1AgnRfNbahbRvL7M6VPgwiIyBfgRjJcEolE+gGVnXdaM3NWNUFUp23EQSnUk+eBaOjLo5OQ+D01EQqVN9Jbd0hI3IgIf0eZvdP3UMiC4gb8v+8OLIBU8UtIsBmU2kI5hVF06MmXuJE9gUvzLIZqcqGvaqK5LVlYq9u5NTFiO594WMnxy8TDQE9+4xIZMS8bVcGPSP6ew/qGeHPhAinQ43Nee8ZAYUNxEgL0ctwM4H+lOoyMf/DbUlf7q0FNRqRbprGqRrBKULgZmCV1w1aYaOKm5q6Q0aLb3kNGV+v9h3R54g4sXjtLuPnQIdSR3URVs/sWVNIpMcruqOLCa+eI2oZrXgceJ6KXyQCoh8PfUiem9DSDazqsQmSpnFDFzRqZESMtXKb/O1lorRXFTtBh+QJVg+Uw0153/QMFDYQIxFET9OopkdOOXzu96/7e56i/Jpvo56tp5umcTI0Onkn3PqMNKSZplEjKRyJkSmsHIaXKbBqR1Y/aZp/vVdj/DxzVM+k/1MX4bu+MoEumTow6T5ekZEDmhDyEl8iVNq1aZNoYW56a/wgRlVdXLinaYo9xaR6bLIRGWEvBgs8vlna1GcvMpJ+mkbdLknpWe3gIUZAngExEgCnBb0oF9EW7WXXbE8eqZ4q3TP2Zy8YU4GjTGNGxIhTZKTZswOruriJP8EtTcOVIE6fA/X5OE0jA+14gZHSUb26hr0UbvBiv/CjHcbvZ43p6fk++3Uto2vOGp50Hy/PSL2WImIx4tYkTapfuOpEzuQlHeAH1byqbleix0h7b8+IJqrU7QxSTeNG0myadsX0f58fS7d8doz1fsUsmnEDqy0yEixNo26XiBGkaUC+AjHSQvRcflY8I2laZ9Xn4S8pngocZdRFRmaZZAKntJZb0zPxjDhFCtgH8pX73jD6iegLvRpp4fTMB2bb9m7l7a3FTveMeEVG3lq3yxBJPJPm6OpOSf+vLtjGhFatV0aqahrdr3LbCytp/E0v0OsfxwWQymHzvbVt08YyXAY5PslpGtMzorWCt96PttDrZdpW2qi4TUb+/vTICF/nqpsvH9c/KR0iglJN5WTMMxKwtFfdDkkvishEmgbkGxAjAfjFZ8dQLuDvxmueWU5n/eZVY9FLO8OSjbBNjsSIfjbtFycRwSkT/Xa3NE3CM5K801/+aDv9d91uY3IulxDbXldrZvb3ZZutCaSy2Mn8EbfohIp4Mvp0TjQ6c4uMcFqDU3v6Gb6XGNH9KtxbhaNR59/3ZlKERCIjPCMpnTSNLsIkfZRoBa95Roq9q2nS6THihb7fih0EjqRDRFhlLE2jGE3TiYzIdkkkcb8ZsUKaBuQbECMB4KZJuVjvOaLx+OIN9MHWOhr2k3me4Xzv56G8Qk2XpDsvxs2IuW1fYvaPl4FVUgJOoubPb6x3XWCPaK/7zqa9lilRxIgeofGKjIhQUXtJqPBzci8JXjclWqE37/JM02ivrUYH/qi8T3VftGuTSNN4GVh5wVZn8LgZWN16YuieETcDaybMq04dWPVIiXoc5MQgU6W9Iu44wpFONMOKjGieEVTTgHwDYiQg+llbVqpptCd9+LXkQXf3vbKG/mP21HB9njwLjagCJGjb9FRdTXXzqFsHVq80zZtmdYuTGNHntwhqmkYXSl7VNGIw9VqgfvPl8XTbF8ZRl/L2jn4ZLwOrl8CVXi2CbLfhGTENl26RkVW1+2jCTS/QVY8vtW7ba5bwWtsl1T7mT90joqdp9NJeeXwmzKtOkRFd1DmlZTLtGUknKmIblnf4iCHy5FghTQPyjbT+ombPnk0DBw6k0tJSmjx5Mi1evNjz/nv27KErr7ySevXqRSUlJTR06FCaO3duutvc6tG/CvUzS+YXc1fQpQ/9N+8jI9y6/cw7X6EHF661pRVUQ2gQ9HSJm7ixd2D112dEFS26GVWen9Myatt2XmSczrTlLPbKPy+l219YaVxnv8b1z75n+DlSRUaY00ZU2zqK6ouzV0WQ7hlR94/u0ZBjwc+fMLA6i5G31u02GrvNXV5Dr6za7hkZkeOtR0KSm541Ozc8y1hkxE+axi4KM9EOnhnRq5MhhsalORxO0kWcppEUDVOubS8AUSfwX/OTTz5JM2fOpBtvvJGWLl1K48aNo+nTp9O2bc5n6Y2NjXTGGWfQunXr6C9/+QutXLmS7rvvPurTpw+1BrJhYNWfMl3PiNO2ceTgyseW0v2vZraFfbrc8eJK+rBmH9303Ae2xT5NLeIaGdHFjb3pWbN1myx0Tgu5KmD0BVbECy9s3CFV4DSNfuYtvPrRdpqzfCvd9dJq4zr7NThF8sjr662wu5cY0dE9FJ6REY+ojO41sUp7ixXPiI8+IzfP+cA4Hm6eETneuqjQxQnfT037qAbWTKBHQpzEY3JkJDNiZEC3clp83en02y9PSOvxVjXN4SNWiobHD7gJYACiSuC/5jvuuIMuv/xyuvTSS2nkyJF0zz33UFlZGT344IOO9+fbd+3aRc8++yxNmzbNiKicdNJJhogB/vAzdv6mf35Af3j5Y9ttTjrp3ytqac67W+nmOSsisfu370tEGFQBkG5kxK0kV0+ROLWDVyMC+uvzYqgKGD0yIvdn4XHu+N7W7fGFwf5nxrcx6iKtlgvv3N+gREb8n+EmpWk8PCNe3V/dqlf4vVnVNIfs286DBTlFozarW1W732zSd9jmDRGxI9unixGniIcqkNxETLroCzdXDenoojBTaRrd5Jx+aW9TQoygxwjIQwL9RXGUY8mSJXT66acnnqBNG+P6okWLHB/zj3/8g6ZMmWKkaaqrq2n06NF0yy230JECmbGSDrr2UJueOfFR7T568LW1NEubwaKXCLekf0e2UBdFW2Qkll6jNrfUhC5SnCIj6iJ8uCnm6QnRz/YlasJn2RwNufas4XT6iCqaNqR70pm3lLKqCyynq4SqihLFM1LcAjHivwOrl2FU9p2aplFLe5es300/ePoduvaZ5UlC5p6XP6aVtfuM34dUdTS364i3Z8S8ri746vPKMc6WZ8RBi1hVK5mOjLQU2Q4jMoJW8CCPCfTXvGPHDkNEsKhQ4es1Nc4NtdasWWOkZ/hx7BO5/vrr6fbbb6ebb77Z9XUaGhqorq7OdikkdPGRKkigLsCq8dLpZCtq4VvVSOnVu8Mvrs3KtJ2oigvZZ+pZvW5g1Z83SYyIydNcyb7xicF0/8XHGouFvs8l1aG+X44iqCJSRKPuVQiUpvHqM+KZpnGJjNgMrAkxs3lPfEoxN3qT9NLUwd2Mn29v2GMcx2lDuhn+CKcut25pGp7mK0JB3gunff60aL1jSXC66JErp8iInqbJVJ+RliKREcMzgu6rII/JejVNc3MzVVVV0b333kuTJk2i8847j6677jojvePGrFmzqLKy0rr069ePooI+1CvsyAhHD9TcuSyofLs+C8XtizZMDihn6En9LdIRI83BIyOy0KkRAT0SkiRGDrqnaXR0Q6ScZatiQaIHDC/oVvOqAJ4R/djqVSh+e5wkRUaUoXROBlZpkc4Lonz+2JA5qnfCyHvZCUdZIsNK0xx2MbCaEREuT5Uzf46McGn0d59cZqR+OHryvdOHUlb6jHiU9urbGDbyWeL9gzQNyGcC/UV1796diouLqba21nY7X+/ZM7llNcMVNFw9w48TRowYYURSOO3jxDXXXEN79+61Lhs3bqSo8PClx9LIXhX06NeOy11kxEOM8KK6VZlKK+HsFz6wH6PIRkY8xEg6kRHdG1JlTszVRYpTB9YgaZrd9e4GVj+zTxj1sK6q2Wdr8iWRkUCeEX3arWcH1gCREaXJWKWDgVXEyEFFjHDk4PMT40beo3qU00lDe1jpF6uaxiUyIuKaPSbymFdX7zA637LfiXfn786fSJMGdKFM4NSBNV/SNNagPCUyovdtAaDViZH27dsb0Y358+fbIh98nX0hTrBpdfXq1cb9hFWrVhkihZ/PCS7/raiosF2iwug+lTT3uyfSJ4b2yNpr6GtwbZ29YZcKh8gvejBRWi1nm/NX1KY8C5TFnhdjvaNoS+HhaH9dssnmTVi6Ybcxt8UtTaP7PfwYd1NV0/QyO5jqIsWpA6uXgVWPjOjVNPJ8Tj0q9NucQvz2yEizIkb8L3rttdfhiIaboFu/s94zMqL6ddR28BIZYdEh+03ECEdbJE3D7/Grxw+gH39qOP3+gklGh9iEGIk/v2VE1bwfEnVgs6ws+txwjiN9g3uU0+OXH09njLSnijPqGXHQ60nVNBkq7W0pagdWeEZAPhM41shlvVya+8gjj9CKFSvo29/+NtXX1xvVNcxFF11kRDYE/n+upvnud79riJA5c+YYBlY2tAJnHljov+yW25OryILq1mKiWFmwJCJw8YOL6cRf/odeW508lyRduHz4+0+/Q//713eN6+9t3kufu/t1mnxLQsh6GViN9+Ax1t4N3eshbbG9+oyIUFHTE/q26M+bXE1j94x4nWk7+UC4EsUeGZHKiPTTNG6dbLla52Nz2KJbdEGiKiweJZXGokrt7CmpGhEjvAusdunti42Ix7dOGkzDenaydS1lwawKTz3lIQs/ixH5v827476Uy048iiYfFfejZIq2ihjiY+VUEq9HqDJZTdMSRKzy371bR1sA8oHAf1Hs+bjtttvohhtuoPHjx9OyZcto3rx5lql1w4YNtHXrVuv+7Pd4/vnn6b///S+NHTuWvvOd7xjC5Mc//jG1FtJtWOSGjJ/3w2/nf2S7LguqW2pH7T0hC+jrH+80fj702jrr//iLTR/97gafDV/95DKauzxx3BeawobD6jL4zQk1YtGQAc+I3vRMohLJs2mavdM0mhDSH89nomo6w5ps63Bard/WwWEhU9Mmh9gz0tDyNI1bee9b6+MC9uiqjlYaSx9Yx8di0+4DNPXWl+j+hWutCIa0oVdNrCJGmJ3m707RHzVNox5rPTLCUY9Th1fRBZMHWBGIfeZCyx1tM40qFt3SmNnqM9JS1KZnso8gRkA+kpaEvuqqq4yLEwsWLEi6jVM4b7zxBrVWZp8/gX71/Er6+7ItOX/tDVp6RYaKuYXn1cm9hhegxN6DhM+EqytKjQmuvCC//7PpKfsWPLJoHf3t7c3GZd2tZ9P7W+JzWVQ4TJ8K/Sw+VUmznzSNRAuavDqwSpqmySNNY96fKzj47J83jX9WdSpOKu1NVa2RSmCokZFATc8c9jE3k5sxrjedMqzKuk2E4bGDutq6dvIixtEOfitsfuXoifo5kvfGw904/SblvTYxsr/Bhxhpth1rvbS3b5cyevCSYx0jEOm2TfdCFYtuDeqi6hmx2sEjTQPynGjEGvMc/vL8TZodFDONlabxsZA7zVP546L1RoMviQysMcP5XsgCJJz924VJ93HqeaL3EdE9I2lV0ygRjR+dOcxaXHSRYvOMSGmvEp3g92/zTShdP8vNBUAtj01U0yT/SekLXKqFTK1KaUnTM+aZpZvpZ/943zG1d+zALrbH8MIv28aCQR+mJ+9NKsr2mUJGTVmJMOnQPnlb1Goa1S/i1cVYr7TpWq6o52xERly2JZtNz1qC2oFVSqy7Z2EfAZBtovEXBTKGlabxsZA7TbjlL2ZVFMQoZmvM9dX736R/KemY+GPcP0ayEKtiRBZ5PXrj5RlZ+NEOWrZxT+r3ZD7ndWeNoCtOHmKdzevCRn3vVprGI02kNv6Ss2S1PNbqM1Lsv5rGDTZqig4K5BlxeG1m0+6D1r5mscr+HeaYAV1tlSwsRCRKoZaK6n1MrMZnHEVptpeQS7rJSXCJ/0NN0+hREZ2cREaU/aZ6quzbEVEDq7ldfKzeMf8+Mp02BiAXQIy0MsQj4CfF4TTHJUmMKE/z8+c+MLwg3/5zYiKrV2hbni/+U3ldc2HUBULS8LYYR2iaadu+Q/TVB96kc2e/lrIrq7wnWcTkzD+5z0iyWVXvy6H6RKzIR3GR1ftD9XlY1TS+PCPeC9mu+nikifVbkEXPbVYL72epYqrZe8i4zvunb5cOtgoc3q5EX4/kyIjsS2l8xn6lBau2OaYEPdM0bGD12dJdFQEshsSQnOs0DX+OZft5m/2kHXOBCGPen1yZxqk26XQLQD4B23UrQzwAHgNbXU2ZlhixtWVPLDQSBnZ6jBvy5a6G4vmsmBc2/fX14W0ra+ronLsW0qg+idJu7m9R6dF5U55TFk55/aQmZg7VNGqaxuo10t5+H34+SR2oYkQiMk77Ql+4dP+Bzi7TwMwLepBFz6s9OkdHenfuQDWmKOlZUWocE3uaJhEZYVGblKYRMWJGRnjA4dcefsvx9Zzeo5qmcZtL4yVGeIZLNgZTqpE9p3SiGtHibS/N0EycTKCLvnH9KiPXSwgAP0Tnr6oVwB6FsPnxM8uNCIOfyMjqbfXGXBuVtpoYUX/XF2unNuR65EK+GNUveYlA6NEKPTJy1WNvGxUCb6xJVOLsNKMGbliiQcSIpGn89BnRm305VNzw4i2pEy55fX31DqOnyxGrZXpqz0iqNI30XgniF4m/tvsitHlP3OgsERI2KRvvp62zZ4QjI/s1cSjH+bQR1Sk7w5a1S952Vei4zaXxStN0y5IXwk9kRF34o2JedRIjE/plphEcALkGYiSDsEfha9MGUdi8+EGtL/MnV9ac8etXbLdxma8asVBLMNXZLW5nlroPRcSImhaR50wVGXF6D1I6qrJh5wG64P43aMHKbVY6RRZOP9U0Ijr0Nui2NI3NM5KYXHv+/W/S1x95i2rNQXeO7eADpmmEIH4R2TY3Nu06aKVpmF6VpUmpHe4DYvUCcYqMmPvy7LG96P2bzqSxfStdX6/UycBqeUb8p2lUA2s2/CJJBlYPQSfRniiJEbWZHDNxAPwiID+BGMkwXcuzP7smFRw+f2XV9rQe+/Kq7cYUVkcx4hIZURdgPbogQsWpn4jeE8RrrL1b5Q5z+4sr6bXVO+mSh/6rpFP0yIi7OZaDORwp0SfOLtuwh+5esNpYONVhcRIVUNvwy9Rd52oaramXS1RBjzYEHcamD8rT0zSMpGmqRYwoC1kHm4HVyTNSlNSN2A1nz0ii6VnCwOr9HtWGaJymyQZtfVTT2MVItL421UjbeERGQJ4Cz0iGyUZOOyhuDcb8olat2NI0ymK9eO0uw3fQv1uZYbQU9AXd8mzYoi2SptEiIx7zUrwawqmaRhY5ESGWgdUjMsJ8849L6I018eZvghh1ecGUSEK7Nm2s9MkWxUMjnUdbEhnp3qmE6ncm+sak6u8SKDJipmkkMsLHLv6YIpc0TaKJltvzj/EQI6U+m56l9IwoYiVbYsRP0zM19RSlyIh8nnbTYRrYrSxr+wiAbBMtid8K6N+1LOxNsM7S00W1fagdPNXIyJf+sIi+/eclyRNwtVSHfLmrng25T1JXU23xc2KnKUa4ffp3n3ib1mzfT/27xufPiOlVTT84iaH4Ntuvc8M3vZRVfU5bZMRMn6iG3t1m+sgpzK97OTgV4qRZu3e0eyKCNDxLJUaknbpqYNUfwwu/2pgs2cBa5EuMsMBx2hYnA2tqz0ix6/7JFGrkymuqtRUZiUhZr75dE/rDLwLyF0RGMszZY3rRuh31tHr7/lA6sjLrlLPrdFB7i6hpGr0p2ftb6pKiDPo0WFnA1McmPCP26ISbGFARA+sX71lkRCN4+N5ZY3pZ/y++D3ldESNes2lSwYIq0UdEjYwcSmr25dQFVT/b5m3ixVrvq9Jd80QEFyNeBtaDRk+Q2r32NI1eTaNGRtw8I8LQ6vjMGR23yEE6nhE1JZK1yIiy37yqlySipc/SCRv5PE7oD78IyF+i9VfVCuAvs/857WiaNqQ75Su2tIeLaVVFjTrI8DTPyIiVptE9I0d8R0YkLbJx18GkaIzdM2L3rDyzdBNNnTWfliq+GF9iRIyxbYoskaBGRsRY69QATk/dsABw6gnSLSkykpk0Db8+v38eHbDNjJr1cvKMtC+2ylZZMOjiUBdVbkLCLQ0lURAWhpLyc+uNknhM9tM0fqtp5LgH9fJkm8+M703DqjvRmaN6hr0pAKQNIiNZwqtfQdQxZta4DK/T+dFf3rGJCGkRrhsCbZ4RSdN49P5wQ0SIitM2WtU0moF15lPvUFD4PTQq1TSyKKmRDdkup+iE/lngRd3pfty/gxdniSIFHQrnVFYsZ8zcAv7pJRuNfczrbQ9T+Niqadq2sc7645ERuzjUhSZz74WT6Km3NtLgHh3pD6+s8TToqsJC5tpI9Y6fyIgeOcoUqsjyjIxEsJpGJhnzBYB8BmIkS+SvFCHr7NmPGHnqrU2261zu6vRFrwoPPuNevmmv43j7dBq1OU2mlSgBG04ZXoT9VOs4wdGOJptnxP3PxskAqXst3DwVHUs4TZIQI706J7wwflC7qapMH9XTECMPm1OZ2XshwiUpTWMKBq6K0tM0att34ZOjehqXP76x3rrNLXKgRlJEtKaKjNibnkWlzwgCygBkGvxVZYk8DowY5b3CHS+uStmCXUXGyieJEe5mavK9J5fRjN8tpPvMM2kdrwXBWYw4pGnMhdlKEzXHaOn61LNtHLenOOEZUSMjzvdNnaaJR0aS78ciR118e5uplJamac4c3dNWrdRTeV57NQ33GUm0bJfZO8cN6mq0YZ8xLuHNSdp2ZZ+4RQ7UiJCI1lT+i1x4RrgCLjG2wP2zx9VO8e3AIDoAMg0iI1kin8WIUzltD/OLOBUSfhesqbkOE4Lnf7jNNRwuZ84jelXQiq1xo2z8eZKFkZNnxIqMKGmaZRv9+UR4QVINr5xmkcgOvx8vL4dzaW+bZM+Ig9+C54qoKY6gkREnIcSChqdKc4OydzfFB+RNUAap2QfltbEiIzyJV3bBw5cea+wDr/SEKtC8PBWcqjl8pMkSrSkjI+b28HGsKM3e15Ucc68+I185rr/x3rjpGwAgsyAykiWK8jpRY+e02xfQf1Y6Cwcd3TMifVecIhpuqIvZhccPSDlp2ClNI6JANbA6pRmc6KQtevxUajWNVyt0xzSNY2SkyFGMqNGN4JGRIttoAu4G/MQ3phjXf3L2SDprTE/6zZfH0/WfHmndTxUDRtMzMxIhRmHedHWAnhvSlTb+u5cYaWP7nKSKjMhzZWsuTfLnxf01Kju0o4unDsxaiTEAhQwiI1miNUVG+Cz20of+6/O+9gVfZuQ4iQg3DE+G6VvhiMyvvjCWfv3iKtrCE2d9pmn0QXkcmZEhgqlgUbBHES58xiyvywt+mYdnxElk6AKFfSxuaRo1shR00VOfc0DXcmM8gcCpFr4kPcYWGUmIDqkOKm/f1pcI8BsZkee3DKwpIiPc5XXGuN402WHbM4mfNA0AIHtAjACrCZY0xGoJeppG0h3pRkY4dfDFY/rRwO7lRm8RxzSNlxgxxQFvh58+JkwnYyrtQdvzy4LMJcPekREfnhE3A2upXQQFHVOvPqfXGb7bY9Q0jbTd99sFVj1m/iIjh331GeHtu+srEyjbyDHyStMAALIHxEiWyNaE0WxxdHVHozfKX5faq2OCoqdpRIzoPUW8UBczOZN266Tq1gsl0fRM0jTNSe3N/aZpHn49XoXCtGtb5LnY+omM8Htx8kqUt2+b1FguCOprp/JiOD2G/RxWmkYiIz6H9fmNjIj42HvQTNNEpJupiEhERgAIB3hGssS0Id3o8hPDn+DrF16s1UWYGymlg56mkUhGkEVWXdjkTNqaMeMgapxKhKWkN2FgjTn2yXDCyyjJz1vuYWAt9tuBtW1RoA6qQaMcXq3hVdR27CyyRByIiOSUlR9UU6+XWJPPGBtk/URGcoUfzwgAIHtE45ugFcJh/evOThgFow4v1mo64YfTh1ldOluSpuEW5PHn9y9G1LNlKzJiLhJ+Snt5UeeUhy0ywmkan54Rz2oZwzOS2L7PTexj/38fYsSttFeNpqUjTFqepklERgTfaRofpb1MZYf2NrGTajZNrpBjlM/NCgHIZ6LxTQDSirxkEl6sVT8GVy+kc5YoJZvynS7P6dfAyotweyVqIB4GNd2SqpqmqlNpwpColPb69Yxw1YT79sVbufOQuD6dO9D1Z4/UmmY5eUbaJAlVVQR8/4yh9Njlk6myrB393+fHGAv0Q5cc52tbba+jHC+/kRH7oLyEZ0TwHxnxl6bpXGbft5GJjGizjAAAuQWekTwl02dw+mLNC4rXBNNUkRF+PLeJ5zPg11fvoIWrd/h6PC/06nuTM3XxQDgZWGU4nt6cSk3X8HbofhYxbSY93qOKhRcrFhN/v3KakXriKEBFh3bWoDwnAeeUulE9HccP7kbHDoxXi5x3bH/6/MS+rq3dvVCf079nxDsy4leM8POwkGTR2cGjXLdzh2iKkUQ1TTS2B4BCA395wErTqGkMXnDTOUuUlImcHbMIuOKxpb4fz4uTKjesyIji/UgVGZG5K+rj2FfiFBnpWJIcBenmMQNFRAJXukg6Qo2kOEdGvP0h+v+nI0SSIiMOnhQ3DwcLDp6DY/QT0SIjftM0anrLyzOiR0aiYmC1qmnwjQhAKOBPL0/JdAOoQd3LU05pVeGz95vPHZ3SqMiRDLVc1Y8YUf0lcqZueUaam+kRpbrFuE0TKGq3WFn0nQbsMSN6dUq6zSsy4jT/RTW8Ou0zpxLddMymgTwjPs/wWVD9/app9MwVU02BlZ5nRE3VeHlGOpe1j2hkRKpporE9ABQa+MvLUzKV2j59RLUhLG46d1RS6avXIsmv37uzu8FVogWNDt1RveAzZTX6YVXTmIsE91C78R/vez6HzGJRF2Wp3lAjEhwJcPI3eE2HdYpacJomqPFUbTaWqQU5nTQNwxN3B3Qrd5yiy8P7/CIRkSCekagYWBEZASBcovFNAAKTqbjI8Ud1pdu/NM4wfX7n1HjHzi9O6psyMsK+Dq+zb1mg9cF5qeCFmc20gtVszMciv+AHJ9NDlxxLJw3tYd0m70GeUvVAsAnYSXCl8ox4iRG/fSpUsZAp02Q6aRodNrGqTOzfxfdjB3QtM372M3860dmspoleZMTelwYAkFvwlxchvnJcv5y/prp4njaimt689jT6v8+PTXmWz9/ZXgJBXaCDwIu0UxlwqlQGvw/u0nrK8CrPx3HJ702fGUXDe3aim88d4/geVAOsn+1I5Rlxfp7glS+pn7PlqR81MjKgWxlNGey/auvO8ybQP66aZgw3zLfIiBwPND0DIByi8U0ADMwxLhaXTB2Ydc+IXpVTXVFqeRy8vpj18tRU5bHs4+CyVX+ekWSTaqrogduipouNTiXt6KIpA2ne9z5BPStLHcWDV7t3J/FSYbSPd/9/J9R9l6nogKSy9N/TjYxcMLl/oM8ZlyaP7dvZ+z4doilGMJsGgHCJxjcBsFAXbK+JpplK03jNP/E6yy9KIRD0RYcNraqx1DtN0+y4WHiti26Lmr4oSzO0xOvZn/Rn54zyXICd9klFh7aBUy52s2lmjiZ/XvgMny+ppuF6eWLOHtuLjhvY1RBtmSapz0hxVKpp0A4egDBBn5GIIf0mUi1SmYqMeA0Gky6ZbhEVr8iIGi2QSgu3hYf/j3uSiKg4aP6eFIlp08a1rbxbiag0PRP06a+quHj0a8fRJxS/iRNOqStbmsZnekSNhqhm1pbAVSy/+fKElBUtqZh9/kTKFuzZYWFpdWBNUzRlzzOCpmcAhEE0vgmAhZoa8YxMeHxnBvk+PW2E3WOh0qRFKNTn5d/VlMRnJ/TxjIyUtWvr6kFR26+zZ8Rtwq9XCsStt0U7ZaPH9q2k75x2tOtzquJqWHVyya9+H8c0je/ISPChdn44a0wv4xJVWFSqjc8y+d5bghw3tIMHIByi8U0ALM+I+mWYbvDDzxcqmwzfueGThkfEjUbNu9GrsoOybfZqGm6a5SVGWCy4RQDUAX2GZ8QlIuO10Pd3qeBQIxXnHdsvSUzYvRuJ53/qm1Mc+6g4CSL1vRaHmKbJF9hbIiAyAgBgIEZCgvt76MQoZhMg+gA4laI0fSACr4XqouCEXtXSt4sqRuxn9920cljVR5FI0zh/3NRSXL3pmYpXWoibtqWamXLW6F6+q1p436jbJRQ57PkgfUbk+Mpr8fVCq+CIZGREqmkwtReAUIBnJCS+8Ymj6N8raj2jGk7eiUylaRw8oknoVS19WIysTWynuoj6iYw4VY0cN6grjexdoaVpXCIjHgvFUT2cxQh7J/713RONbe2ibaPxnGoFirYwOm3vIYcmbvbIiE/PiPla7IPJdDfdqNPF7MLKxyTd1vdZ68BaYMcCgKgAMRISbt95fsWIVyrGT5rGzwxdvapFFhGmSFu8uW24OnTOqYRTvT9HWf582WQjTTTvvZrE/dpxmsbFM9ImeGSE8ep74dXvw+msveHwEc928KlSLkVaUzK/HVtbExKRi0pUhKk2K72qKlJXfAEAMk90vg2AgbqWHXRY+ATWG26aQz+7c/JTxPSmJg7oRlLVqMmpIHXhZbGhvq4+g4SjHWqkgRcibkHOkQv77cU0Y2xv4/eRmojw6sfhJUa8UM/MdWHg5HHhxmpeaZqgfUYyVUmTT0gX1qh0X2W+d8ZQo5rq0+ZnDwCQW6LzbVBgOGkBvk0N2X/luP6ecz6e/MYUx/SELlL+98zhvl4/VZqmXJlTwq9hW8i1Nu6dtAFrLGxsFSRqaavWAOwH04fRb78ywYicqHhFHbxauKfbtVTd3nF9K41W88N7JkdZ+HFnjupJ4/p1NtrqeyFRK3mtQmw/Lr1GotLwTEqOuaw7U91wAQDBwF9e5NI0id85ZLzsxjOcH09Fht/ipe+fnPR/fgyRzb4iI7oYaav1GVFTHIneEcb/a9vAwqZE6TNiFyN2kcLRknPG9U7yeHj5C9Ltq+E3TcMiRG81r3LPhZPo2Sumptz3ctzluZ2mABeKGIlSZAQAEC74Nsgy3ECqT+cOSV++RT79Hm7NvLzKafTn4CqddDwjundDrUzhFIp6Vs+Lq1eTNCMy0tbZ8KruF6+z5Wz7K3QPgxql8lOhFMSIWshpGvETRSkyAgAIFxhYswy31uZLc3PMGGM/6eZ/O7YlZ2KakPAKXniZVP0sir4iI1ppMYeyn/ufE+idTXvo02N7pYxa/M+pQ+iul1Zbj1UjD2preFUEeJka3dI0LTnDVneDl98jU9W3Uhos0QG1zLVQkH41uq8IAFC4QIzkCMPwqSy0bOK7+4KJxgL7jT8uSTQ9s62r7oLBa23kludzlm+1rg/s5mDu9BEaUT0gIihG96k0LjqDHF7j+58cZrz2o2+spx+eOcwmJlRvhc3A6iEs3NI0asSmJXj5BTLdmGxU7wr61RfG0qjeyfuytXPswC5GQ7mJ/buEvSkAgIgAMZJDuAT0mAFdjJRFVacSx7bdfiMjbvz83NE0Y2wvw9jKF06lOIkHP5GR0b0raOmGPcbvkwZ0MS46b157Gh06fMS1gdrnJ/U1LjpqZMTvBFvXdvItmMPi5/n9pml8UZSIXn3xmH5UiPB7/+rxA8LeDABAhIAYyfGX8NPfmmL97oRNjHg+V/JtF08ZQBeaX/IclfDCw95h8bvzJxpplq9NG0hHu8xq8Won74VabmwTIx7RCbfIRWkLIiNqibNXegszSwAAIHtAjOSYVH4O9b+DekaCGCidTK06vTt3oFmfG+P7OaePqqbn36+lM0Ymt7oXfnTmMFq9bb+t1XqJ3zRNm8ynafwGnzLVsr3wamcAACA1ECMRggWCPTISzDMS5Ow9nRRQKn71xXF0xsha+uQodzFyxclDPCMe6VTT8ETgbKPO5WkJ6DYOAADJoLYuShhTe33etyjYQtdV69mRDTHCHVq/MKmvrVOrH2w9R3waWMcoPhiee5MtHrj4GLpk6kCjAV0mcBq0BwAAhQ7ESMRoiYHVS8jwsLi7vjIhkIE1V6gRD6+lup3yBs8c3TMzaZoUu+G0EdX003NGoTMnAABkEYiRiOHXM6KeYX/1+Hjb+EunDXK9PxtNZ4xLzN2IkBbxvdDbW7cn3r9Xy/xUnHh095w24EKaBgAAkoFnJELETBPqwG5ltLO+kQZXlfuKgtx87hj66YxRgcax+zGw5gq1gsbLhCuNwkSY8DyaHfsb6FMOJdJ+4Sqhf888iXqkOdsmKNyrBQAAgB18M0YQXhy59NarsmRYT3upbRAh4re0N1f47eGhduxkMfL8906kj7btN5q8tYQhVR0p29x/0TE0618r6Nfnjc/6awEAQL4BMRJBUgmLmWcMpYunDmzRa6j9NaKEV+Sga7kaGSmibh1LjEs+cPrIauMCAAAgGYiRPKNTSVv6zmlHt/h5oqZFbvj0SFq3s54m9u/sep8uWmQEAABA6wBiJEK4RSsuO2EQ3b9wbfxKhipDo1RNw3ztBHfzrVN5MsQIAAC0HnB6GSHc5MFPPj3S+j1TXSqiJUX80QViBAAAWiUQI3lGkJbvXri1Vo8yXZU0DUpkAQCg9ZB/K1IrZqjLMDqVli7C3PisuqKE/nDhJMo3KjokDKz1DU2hbgsAAIDMAc9IBPjHVdPopQ+30WUnpvZNtDQuwo3P1OZn+YQ6rG4/xAgAALQaIEYiwNi+nY2LF/27ltGGXQeM9uSFDJtYd9U30tTB3cLeFAAAABmiKBbVhhMKdXV1VFlZSXv37qWKigoqRGrrDtEL79fQZyf2LegunvsOHaYd+xtpUHf37rQAAADya/0u3FUtz+DZMhdOaVmjs9ZAp9J2xgUAAEDrAQZWAAAAAIQKxAgAAAAA8k+MzJ49mwYOHEilpaU0efJkWrx4sa/HPfHEE0afjHPPPTedlwUAAABAKySwGHnyySdp5syZdOONN9LSpUtp3LhxNH36dNq2bZvn49atW0c/+MEP6MQTT2zJ9gIAAACg0MXIHXfcQZdffjldeumlNHLkSLrnnnuorKyMHnzwQdfHHDlyhC644AL62c9+RkcddVRLtxkAAAAAhSpGGhsbacmSJXT66acnnqBNG+P6okWLXB930003UVVVFX3961/39ToNDQ1GOZB6AQAAAEDrJJAY2bFjhxHlqK62N97i6zU1NY6PWbhwIT3wwAN03333+X6dWbNmGXXJcunXr1+QzQQAAABAHpHVapp9+/bRhRdeaAiR7t27+37cNddcYzRIkcvGjRuzuZkAAAAACJFATc9YUBQXF1Ntba3tdr7es2fPpPt//PHHhnF1xowZ1m3Nzc3xF27bllauXEmDBw9OelxJSYlxAQAAAEDrJ1BkpH379jRp0iSaP3++TVzw9SlTpiTdf/jw4bR8+XJatmyZdTnnnHPolFNOMX5H+gUAAAAAgdvBc1nvxRdfTMcccwwdd9xxdOedd1J9fb1RXcNcdNFF1KdPH8P3wX1IRo8ebXt8587xgXD67QAAAAAoTAKLkfPOO4+2b99ON9xwg2FaHT9+PM2bN88ytW7YsMGosAEAAAAA8AOm9gIAAAAgK7Sqqb2xWMz4iX4jAAAAQP4g67as43ktRrhEmIHhFQAAAMg/eB3nCElep2m4YmfLli3UqVMnY9BeJhUbCxzuY+IVPgLYz/kAPs/Y160NfKbzfz+zxGAh0rt3b08/aV5ERvgN9O3bN2vPzzsfYiT7YD/nBuzn3IF9jf3cmqjI0lroFRERUPYCAAAAgFCBGAEAAABAqBS0GOGW8zfeeCNaz2M/twrweca+bm3gM104+zkvDKwAAAAAaL0UdGQEAAAAAOEDMQIAAACAUIEYAQAAAECoQIwAAAAAIFRavRiZPXs2DRw4kEpLS2ny5Mm0ePFiz/s//fTTNHz4cOP+Y8aMoblz5+ZsWwtlP99333104oknUpcuXYzL6aefnvK4gOD7WeWJJ54wuhefe+652JVZ+Ewze/bsoSuvvJJ69eplVCUMHToU3x9Z2M933nknDRs2jDp06GB0Db366qvp0KFD+Fx78Morr9CMGTOMLqj8PfDss89SKhYsWEATJ040PstDhgyhhx9+mLJKrBXzxBNPxNq3bx978MEHY++//37s8ssvj3Xu3DlWW1vreP/XXnstVlxcHPvlL38Z++CDD2I/+clPYu3atYstX74859vemvfz+eefH5s9e3bs7bffjq1YsSJ2ySWXxCorK2ObNm3K+ba35v0srF27NtanT5/YiSeeGPvMZz6Ts+0tpH3d0NAQO+aYY2JnnXVWbOHChcY+X7BgQWzZsmU53/bWvJ///Oc/x0pKSoyfvI+ff/75WK9evWJXX311zrc9n5g7d27suuuuiz3zzDNcPRv729/+5nn/NWvWxMrKymIzZ8401sK77rrLWBvnzZuXtW1s1WLkuOOOi1155ZXW9SNHjsR69+4dmzVrluP9v/SlL8XOPvts222TJ0+OffOb38z6thbSftZpamqKderUKfbII49kcSsLcz/zvp06dWrs/vvvj1188cUQI1na17///e9jRx11VKyxsdH/AQWB9zPf99RTT7XdxgvmtGnTsDd94keM/OhHP4qNGjXKdtt5550Xmz59eixbtNo0TWNjIy1ZssRIAagzbvj6okWLHB/Dt6v3Z6ZPn+56f5DeftY5cOAAHT58mLp27YpdmsHPM3PTTTdRVVUVff3rX8e+zeK+/sc//kFTpkwx0jTV1dU0evRouuWWW+jIkSPY7xncz1OnTjUeI6mcNWvWGKmws846C/s5g4SxFubFoLx02LFjh/FFwF8MKnz9ww8/dHxMTU2N4/35dpC5/azzv//7v0YuU//wg5bt54ULF9IDDzxAy5Ytw67M8r7mRfGll16iCy64wFgcV69eTVdccYUhsrmzJcjMfj7//PONx51wwgnGNNimpib61re+Rddeey12cQZxWwt5uu/BgwcNv06mabWREZAf3HrrrYa58m9/+5thYAOZgUd2X3jhhYZZuHv37titWaa5udmIQN177700adIkOu+88+i6666je+65B/s+g7CpkiNOd999Ny1dupSeeeYZmjNnDv385z/Hfs5zWm1khL+Ai4uLqba21nY7X+/Zs6fjY/j2IPcH6e1n4bbbbjPEyL///W8aO3YsdmcGP88ff/wxrVu3znDQqwsm07ZtW1q5ciUNHjwY+zwD+5rhCpp27doZjxNGjBhhnGFyOqJ9+/bY1xnYz9dff70hsi+77DLjOlc81tfX0ze+8Q1D/HGaB7Qct7WwoqIiK1ERptUeOf7j5zOU+fPn276M+Trndp3g29X7My+++KLr/UF6+5n55S9/aZzNzJs3j4455hjsygx/nrk8ffny5UaKRi7nnHMOnXLKKcbvXBIJMrOvmWnTphmpGRF8zKpVqwyRAiGSuf3M/jJdcIgAxJi1zBHKWhhr5WVjXAb28MMPG+VJ3/jGN4yysZqaGuP/L7zwwtiPf/xjW2lv27ZtY7fddptRcnrjjTeitDcL+/nWW281yvn+8pe/xLZu3Wpd9u3bl/kPQQHvZx1U02RvX2/YsMGoCLvqqqtiK1eujD333HOxqqqq2M0339yCI976Cbqf+TuZ9/Pjjz9ulJ++8MILscGDBxuVkMAd/m7lVgp84WX/jjvuMH5fv3698f+8j3lf66W9P/zhD421kFsxoLS3hXB9dP/+/Y3Fj8vI3njjDev/TjrpJOMLWuWpp56KDR061Lg/lzbNmTOnpZtQEATZzwMGDDD+IPQLf9GAzO1nHYiR7H2mmddff91oBcCLK5f5/uIXvzBKq0Hm9vPhw4djP/3pTw0BUlpaGuvXr1/siiuuiO3evRu72YP//Oc/jt+5sm/5J+9r/THjx483jgt/nh966KFYNinif7IXdwEAAAAAKFDPCAAAAADyA4gRAAAAAIQKxAgAAAAAQgViBAAAAAChAjECAAAAgFCBGAEAAABAqECMAAAAACBUIEYAAAAAECoQIwAAAAAIFYgRAAAAAIQKxAgAAAAAQgViBAAAAAAUJv8P9oNzHK8+tgoAAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.plot(lri, lossi)" ] }, { "cell_type": "code", "execution_count": 59, "id": "57606454-7964-4491-be49-2f6b7edf7a53", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[]" ] }, "execution_count": 59, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiMAAAGdCAYAAADAAnMpAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjgsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvwVt1zgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAbzdJREFUeJztnQd4FNXax9/0RhJqQu+9N+mKCIi9dy8oKjb8LFjRK9jxWtCrF8XeBVGxgqCiCEjvTXoJBEInCellv+dMmM2Z2TN1Z/v/9zyb7M5O29nZOf95a5TL5XIRAAAAAECAiA7UhgEAAAAAGBAjAAAAAAgoECMAAAAACCgQIwAAAAAIKBAjAAAAAAgoECMAAAAACCgQIwAAAAAIKBAjAAAAAAgosRQCVFZW0oEDByg1NZWioqICvTsAAAAAMAGrq5qfn08NGzak6Ojo0BYjTIg0adIk0LsBAAAAABvs27ePGjduHNpihFlE5A+TlpYW6N0BAAAAgAny8vIkY4I8joe0GJFdM0yIQIwAAAAAoYVRiAUCWAEAAAAQUCBGAAAAABBQIEYAAAAAEFAgRgAAAAAQUCBGAAAAABBQIEYAAAAAEFAgRgAAAAAQUCBGAAAAABBQIEYAAAAAEFAgRgAAAAAQUCBGAAAAABBQIEYAAAAAEFAgRhxg+vIsWrLzmBOrAgAAACKOkOjaG8ysyTpBj83cID3f8+KFgd4dAAAAIOSAZcRLsk8WOfNNAAAAABEKxIiXxEZHOfNNAAAAABEKxIiXxETjEAIAAADegJHUQctIZaXL29UBAAAAEQfEiJfEcGKkrLKSgonC0vJA7wIAAABgCMSIk2KkIngsI8/8tJk6TphLq/YeD/SuAAAAALpAjDgpRsqDxzLy4d+7pf8vz90a6F0BAAAAdIEY0WDH4Xx64Ku1tOvIKd0DyOfSlFUEjxiRiY5Ctg8AAIDgBkXPNLj2naV0rKCUVu09QQseGaJ5APmY1dIgFCPQIgAAAIIdWEY0YEKEkXW8UPPguVwuKueCVlnMyMHcImk6AAAAAMwBMeIFN76/jEZ+sNz9evqKLOo/6Q96+qfNFCyUllfS0VMlgd4NAAAAQBOIEZsw68diVXO8d/7aJf3/ePEemvrXTlPrOZJfIrmCfMWKPSeo93O/06G8Yp9tAwAAAPAGiBGbMRdG8SEv/rLF1Bdw5kt/0JVvL6bVWb4TJAx0FQYAABCsQIwYEKdR7p25P5yguKxqPct3+7YeCAJZAQAABCsQIxbqiPCUOCBGikor3M9rJceRL4mCGgEAABCkQIwYEBsTZdsyYtSrJoeL44iPjVaUcd9/QjuLxw6oNgIAACBYQZ0RowPkhWWkwuWiaB0ZwNKA3esrq6QVe47TZ0v20s/rD0j1S34fN5haZ9QgJ0DxMwAAAMEKxIhNzFhGKipdFBej/T4vaIrLKujqqUsU7/+55bBjYgReGgAAAMEK3DQGnCgskywWakrKq+M9tGAptft0iqbxbpxigbiptFk8TVR0DW4aAAAAwQrEiAke/nqdLTfNqZJy3UZ1fEgJs4yI3Dx2KBfEqvgrgPWPLYfog0VVTfoAAAAAM8BNYzOjxmxqr8i6cbKwlH5cd4ASuKBVluJbIyFWEjAydqvKixr2+ctNc8vHK6X/3ZvUpF7NavlnowAAAEIaiBETbo64mGhbbhrGyj2exczum76W/tp2RDFNVLHVKBtHi7Jyz+X83S7nMCq+AgAAMAncNET0946jNOSV+bRo+1Ghm8MbywhL31VXV1ULES1sahFhdVi78Sd2QcAsAAAAs0CMnG54t/toAb0xb7t0UN6er7RSbDqQ52EtsVL0bOkuZQ8bs8gCYsfhU/TNqv2muwHznYSrp1UvO37merpsyt+OVZEFAAAAvCHixQg/wMfFVllAJv+2zeNAtRg/W+GaYXVBzPLSnK00fuYG02JCLUaGTf6LHvp6nRRnwpi7KYc2H8iz5Kap4ATKtOX7aO2+k7TApIXGHsjfAQAAYI6IFyO8hYMN8FnHtFNxWd0PmRKDRnlqpi3P8ujya4TatcK6+67ff5Lu+GwVXfDGwqp5Kl1SPRMjN41od/NLyiztDwAAAOALIl6M8Cm1rKbIWS//qXmwyiqqB/28IusD+ZyNOZbmV8eMMG2y7dCp6vcrXXTJlEV04RsLFcGuomwa2TLCW2cKSswF4QIAAAC+JOKzaeSuuWbIL65Ouz1wsrqUu1lExdOMLCO8a8hFLkV5+sP5JbQxu8pdk1tURrVS4nXEiPzfpeiB4yS80EEAKwAAALNEtGWEDcx7jhWYnp8N+LzLxCpbD+Ur1mEEs3aM+XRV9WsXUTQnRo4VlAgHfz3LCG/dcdoywltyEDECAADALBFtGbngvwslgWCWP7cepn4ta0upvltyzC8nwwwHqy2IGDa480GmbPkvlu51v87Jre76y1s8fv+nOrZFnU3Dx5N8umQPzVyznz686Qxqk5lK3qKOXQEAAADMENGWkYy0BEvzL999nC5/azGt259reVtyrZKFp2uZ2AlgZUGwy3ZXu3qyOVeRXDqeVXdVpybzQqGcEyMsRmbf8SJ64vuN7mmzNxyk695dQodsFC3j95dtb+wXq+m9BbssrwcAAEBkEdFiJDMt0dZyi3eYFxQyIzplumuOmI2n+GTxHt33s09Ui5Hx326Q/h89VSqcVxYjvJtG/R7j7i9W09Jdx+mZnzd7zHckv0Sqx2JGjDDBNmvDQXp+9j+014IrDAAAQOQR0WKkvk0x8ovFrBhG64wqN8jmg6yAmrlljLweLIBVZt7ptGOtMvWym0YUTxIXEyUUHmrOeP53qVKtVql3XtTwBdVY0TgAAABAi4gWI5np9sSITLygZ40WdWtUZbo4iSgYVisotVJXjHh+Dt6do+YfVbyMnEXDF37lRZG/S9EDAAAILSJbjKRaixlRk5IQY3reOinebcuMGGGWifziMgPLiMuUqGLrWrnnuNS3h8HXMWHig8WW7DicT3d8tlIqLc/mL+BShXnLCAJbAQAA6BHR2TR1angrRmKlIFAz1D5dA8RJ1OnFzOrB10LhKS6vkESBWcsIq0x71dQl0vN3RvaiwW3rud9jDQXfX7RbMf+/3l9GS7gePMWcGIFhBAAAgB4RbRmp46VAqJFgXsvVS4133PWjpkqMiMXRO3/toqGvzqfnZnkGpu44Ul3VVeZUSbWoYeXn+bL5G7I9s4l4IcIo4qwkVi0jx06VmGrixywzThduAwAA4H8iW4x4GceRHG/eTdO8Toqp+dKS4mzvD3PB6BVV23OsUMqUUcO6Ai/cfkRTjKhjQMxkAxWV2YsZYb2Bej33u1TiXo9lu47RsMkL6LzX9ecDAAAQ/ES0GLFi2dBy05ghITaaYk1aPKzEoYiCTkUFz8zw87qDitcFKjFSXFptqRAJGjV/7zhmy03zy8aq/dh+2NNaw/Pz+qr5so5rNzYEAAAQGkS0GInibvEfO7+95eVTE82JkXkPDja9zprJ9q01rLoq6zzsBOpA18Iy++4QuSCbWuzMWn/QQ/TIgbZGoPcNAACEDxEtRhhT/9WL7hzcim4/s6XlZVPizYmRxrWSTc3XqGYSNaqZ6JWAKOPzay3AmvDp4U0fG+am+W3zITr75T9pTVZV0O3D36yjsV+upgdnrFPMKwqwFYHeNwAAED5EvBg5r3N9ySrCN6Bz2k1jlvuHtVFYa6zCMlq8yVzRCxpVWzCsUNXwb6UUszL64xXStNkbqgrHzdmUQ2/M207nvb6A8orLqFyQeizCm+MEAAAguIh4MWIVvnhZmkk3jcysewcZ9q+J9mKQ5XvVWIW5TNr++xfN973JWuE9L6cEqceTf9smNR78bMle25YdAAAAoQvEiEXqcrVJrGa+dGqYTt2b1DQQIxQQCkorfOam4VN7RfEjvIvGrGUEAABA+AAxYpF+LetoihFeqGihN9Qyq4g3lhFf8vs/h2wvy6f2sqfFXNovD/vsemXoeYL0MAEAAPCHGFmwYAFdfPHF1LBhQ8lv//333+vOP3PmTBo+fDjVq1eP0tLSqH///jR37lwKNf54cLDkZmlVr7peSFpitRg5p32GMGtGLVDkPi5alpHOjdIpGLHTHFCrzsiLv2wRzhclyKaZszGH5gmEUBRCWAEAIHLFSEFBAXXr1o2mTJliWrwwMTJ79mxatWoVDRkyRBIza9asoVCiae1kyc3C35Knc5aRjg3SFK9l5o1TChS9AFMmRkb1b0bjz29P397VX9MScH2fpob7+8ylnTTfu3lAc0p1OPhWjzVZJxWv524SCxsWRMy7aVgBtzs/X0W3frJS05oCAAAg9LE8Ip1//vnSwyyvv/664vULL7xAP/zwA/3000/Uo0cPChWYUGDwbgS+zohWNk56slKg6FUjZW4K1ifmjsGtpNf3Dm0jZZqwQFkW06EXb6FmVP/mNLRDJn2+dC+9PX+nx/tLHx9KXyzbSy/MFlspfGlVOZxfojkvH8DKZ/cUllZQYlx1QTi4aQAAIHzwe8xIZWUl5efnU+3atTXnKSkpoby8PMUj0MippHwwZmxMtQAxG3hqpU3LuOFt6bu7B9AP9wxSrN/sQMzqllzWvZFgH1xSWnKyyTopTqPVq6YqZsTlIQDV5egZvgoZYWnI42eup8+W7vXRFgAAAARcjLzyyit06tQpuuaaazTnmTRpEqWnp7sfTZo0oWCBj2mI4VSB2cBTvZgR0Xs9mtaiFnVTFOu3MhCnJcVqfga9fZHJSPWus7EVmPYo5ywjLF5EpqRMGdjqK8vIX9uO0LTl++jJ7zf6ZgMAAAACK0a+/PJLevrpp2nGjBmUkZGhOd/48eMpNzfX/di3bx8FC10bpwtdM2YtI3aLktnNsknlgmxlZFeTmV1hbiN/wT4iX4b+8e82uJ8XqywjvuJkUalftgMAAKAav9npp0+fTrfddht9/fXXNGzYMN15ExISpEcwMqBVXXp3ZC9qnVFDaa0QiAVRUTS9suv6ab/Vz63okhRBZ2HZMsJcEkbwrhKrDGhVhxbvrG6YZ4Ream+Rqg4Kf7yZhcepiqyouQYAAGEqRqZNm0a33HKLJEguvPBCCnXO7VRf+r+P6xgrslxc0bOxVzEjPHbK1TNEg7Qcs2FmV/i4GKskcQGnZtFqlFekyqbh94ot4sVumg4wBgAA4Bss2+BZvMfatWulB2P37t3S86ysLLeLZdSoUQrXDHv96quvUt++fSknJ0d6MPdLsPH0JZ2ofpr5RnW81UAUmyHqBGx3sFPGjHg38rotIyZ2JZb7jLcNamFpO4kCq4zRZ9QKbvVI7eUOAR9n4i3QIgAAEAJiZOXKlVJKrpyWO27cOOn5hAkTpNcHDx50CxPGu+++S+Xl5TR27Fhq0KCB+3HfffdRsHHTgOa0ZPw5VCeluv+MWTHC6pCo4VNR3diOGSHHYDVRpF0xMfLGREfrfx4dEmOtzb8hO5dWn+7qq6aotNIvrhVYRgAAIATcNGeffbbuIPbxxx8rXs+fP59CCSuxB/ysTWp5ihEnBztvYjfU3HZmlYXDZdEykhhnTbsmxVub/7s12ZrvebppolSWEesuIRFWarkAAABwBvSmMYDV+WDWEhG8paBhzSRTB1zPNaI3DvIiSUsv1TORhsvmSThtsdALpq2REEsPDm+rEEFWLSN2YkZMixHuGDDXzob9ufTUj5voZKF32TB2Y3oAAADYJzBVr4IcNmAfKyh11/nQgvWm+eSWPpQQG03xseZ0nZ4A0IOJgyOnK5dmasS1mDGe8PNoDbxsW2snDKfYmGhasP2IbTHiZFpwiU45+J1HTtGVby+pmq+8kiZd0cX2dsy4rgAAADgLLCMC/ndDT+rdrBZ9eksfwwM4uG09RSdfI3qeFje8+8OMmHjz+h7UrE6y9P/WQS3oyp6Naeq/einm0QpsbZNRQ1iojYkOEQ+d21YSItL8XlhGnHQtsXLwG7NzqbC03MPddf27y9zP9xwt8Go7fLozhAkAAPgHiBEBrIbIN3cNoLPa1vNJxs7YIa1ozv1nKaZ3a5xOZ7fTLgTHuvn+9fAQurhbQ0kUvHpNNzqvc1WKsYzW2M+sNzIt61ULk6t6NaaBrT2FVAxn0YhVBLBG+6VQm1Y8yUVvLqLr31vmIRpKudokoqwmK/DWIq3MHgAAAM4CMeJnaibH08Mj2kuCR2ZYh0yp/4xZV4/V4FsWz8K6AF/UtQG9fHVX93Qmar64rZ/H/HGcquGtG3wMyLwHB9Pogc39JkZ2n7Z4rNtX1QFYozaa5DrzBt7iAi0CAAD+AWIkKHDmDlxv7O/VrLbkfmqQ7hlo++WYvorXvABRZtNUi5H4mGipjsrka7rRwyPaKToYV6+HfIZWVlJakjUxwlwxfNArv97pK6pT1AEAAPgOiJEgwKmYSbuGiFac60ZddZV/zrtpWHAqy8phVWbHDmlN6yacK9gf33SzO5hbRB8v3iN8TySK9Jj82zbq/sxvNH/rYQ9ryIQfNnm3owAAAEwBMeIwrG9NzeSqLBuzOBWZYLcyqzqYlo8T4Z/LKcHSdFX9dVG5eicDWHmuOp0544Rr6M0/dkj/7/+qqqIwip4BAID/QWqvD/rWDO+Y6TOrgB52x345c8b9mnfTaFhJ+Kwcp/fHiOyTRbqWjk4N02hoh0xL6zxZWCb9R2YvAAD4H1hGfIBVIeJUCqndgNE4lZWDt2go+u9wwaFJJvrOiPanc6OqUvS1TZbct8Otn6w0PS+rEcNjppMxAAAAZ4FlJIAMal2XFu04SqP662elmMamJUJdnIx/zVtJaiTG0rd3DdCsOXLLwBb04d+7dcXI5Gu6U90aCZRfXEaDX/Zvq4DcwjKKiYlS1FdJjo+RCqUZdQ0GAADgO2AZCSAfjz6DFj4yhIa0164vYoQ83jeqmUSXd28kPW9fP9WrmBEtywhzzfRqVkt6iJhwcUdF12ORm4YJHWYV8VU8iRa5RWXU7ZlfacRrCxTT+XRlZqFCbREAAPA/sIwEEBar0UTQ7dcKP44dRP+dt01Ks21WJ4U6NEij3s21S9ibcSvx4kRLmGjBWxZE88uxJnxgrD9YuuuYO97k65X76OreTaTXiZy7qaC0gsqcbAEMAADAFLCMhDhdGqfT+zedQa0zUiWrw7COmVJhNaucz1Vz5QNaedFgToxU6sbOMDcJIzkhxqfWEbWFgy8T//A362n7oXwP4cVcR+UVcNMAAIC/gRgBEref1dKUm8aIccPbSv+v79NE1zLCgmGHW8x4scKp4qoeNjJ7jxcqXm/JqRIjvPjIKyqncq3SrgAAAHwGxAjQrbrKPxfVElEzsl8z+uvhs+n5y7oIY0Z478w957T22dHPK65K1ZUpLlV2/ZXrifBuGWYZKVNZVNAsDwAAfA/ECKg6EaLsx4nwMNcMi11hwkWUTcNbV1jzv5//bxCNObOF49/CpgO5itcVqvRptxgpr56eX1xOFSo3jRwDA1ECAAC+A2IEVJ0InEjgU3u9iesQiRF14CoTJN4G8Yr4ddMh3RgS2RvDx7gUCgJYmRvnVEk5nfPqXzTxh42O7ycAAACIEWAUJ+KNGIk2Oc0H1WrzS5QxI5ValhHOElJcVuERwMrEyXer90tdgz9Zstfx/QQAAAAxAk7D1z1TpPZ6IRREIkOUq+KLrBp1IKraMiJXWi3j5isur1BYSqrW46JSQYYNc9tMX55FG7OV7iCnOJJfQiXlyjgXAAAIV+CmAR7CQdGDRlUq3lsxksg125NhFVnN0KdFbdPbXrD9qGTpkFEXVpVf85aQ4rJKhaWk6v1KYYn4XzcfosdmbqCL3lxETpN1rJDOeP53Ov/1hY6vGwAAghGIESDIpqk+LeK8KE7Gr/PfF3agufefRfGqXjCMc0xWoGV9ZIZx6cCNayVpzsssIee9voCe+nETHS8o9RAULKCVWTdKectIWYWHBYVl16iDXxm+sogwfv+nKt5lF1cbBQAAwhmIEaCbTWMmnVfz5IpSBqq20yhTb9ZNw+I87jq7uh7KK1d3051/z7FC+njxHrrr81XClF218Cgpq1C4bWTLiKhEvNqC4iQiwQYAAOEMrnpAt4uvum+NFdKT4h2NC2HhHLxLR9SsT8T6/bnCGBJ1U7zi8krPANaKKguKmgoflo2HGAEARBoQI8ADXjjILhTWiM8qfVvUlsrMZ6QmUNsMa837tCwjrIbJUxd3pNeu7WZaKDVIT/SweLy/cLeiW687m0YdwFrJLCPkV8sIc0fJiOJVAAAg3ECjPOCR+srHjLAaIMufGCqVb7cKc/G8dWNPYqv2xt0j0+h0jMjNA6uKpG3JyTO1HIu9UMdfsIZ5szcc9BAjapGx+UAebcg+6bFOtWjxlRgpKquglAT8TAEA4Q2uckCCvwFXu1QyUhNtHyVWkdWJMiIXdmlAT1zQQTHN286/2043y+OzadTxIeNmrBMu68uGevznghgBAEQCECNAomHNKsGRFBejiBkJFqbc2NNjmrf7qRYeVZYRfYsHix9hAksdb+IrK1WRqqcOAACEIxAjQCIhNoY2PzNCyqphg20oEMtXarOB2iXDAlj3qbr7qmFpwnVqJPi0uy8vRvhaKQAAEK4ggBW4SY6PNZ2h4mtY518j4ryMQ1FnxCzYdoQKDCwRvZ77XSpKpk4VVrPryCl64rsNtP9EoY39IoWbBgAAwh1YRkDIWj28tYzMWLnf1nKzNhw0tIxcPXUJHSsopdVZJ+mX+860tH6+yBpr3gcAAOEOLCMgKDFj9ODL1ntD7ZR4S0G2e44WSI3z9GBChPHPQXMZPzx8Oi8sIwCASACWERCUmGnQ502pep7UxFgpnfZgbrGp+b9auY98CR9YW1Lmu9gUAAAIFiBGQFBiJojWKcuIVDzNRyXY7VSe5d00fDArAACEKxAjIHTdNA4UUmPESbEnzlkgvlm13ysxwrtpfJlCDAAAwQJiRkBQYmYQ560nbTJqUIcGaTR6YHP3tDPb1DUtRqoEiTM889Mm9/PS8kp6d8FO+5YRiBEAQAQAMQKCEqu1TpLiY6SslcfOb++eNrJfM1PLMnePk2JE3ejuhdlbpP8sJfiVuVvp2KkS3eVhGQEARBpw04CgxKp3Qw765INazQoMJ9w0P6zNluq0DO+Yqbndq99ZTIfySmh9di59eksfzXXxrhlYRgAAkQDECAhKWCVYK+QWlVUtx6kY82IkisilnDc5PsZ0jY/DecV03/S10vPdky7Q3C4TIozFO46azqaRhQmbxj5aqFTHBQAAK8BNA4ISq4GfRznXB4sVaVk3hXo0rWm6MV1crHJ7/VrWMbUs280ThVVCSBYNRlk+RkGpfAYNix8pLC2nAS/OozGfrjK1TwAAEGrAMgKCEqsGANZxV4a5QKzEfYosGUYN82TYdnjxwIRGvJfxJ/ymKyoq6a+tRySryqG8Q16tFwAAghVYRkBQig+rbhrluqIky4pZ4wpz06g7AHdulG56ewpLRqXL62BYpWXEujADAIBQA2IEBAVq8WGmAitj/OnsmdsGtfB4z2x8hTq199Hz2tM9Q1rT2CGt6OYB1anCpmI8KozdNFbWV9XMD2oEABDewE0DggJmxeDDRc1aA24/qyUN7ZBBLevWsL1tJh5cnC4f2LoOpSTE0sMj2tP3a7INl1cGnFYKLSMuVSXV3MIySk+OM1yfSW8RAACENLCMgKBAbcUwa9Vg87XOSFVk0Zip/aGu5BqnkYVjxuVSxnwpnJAQxYyoG969NLeq9oix26cSbhoAQNgDMQKCgqS4GJ+uv6tODAizPvCuFV64mHG57Dte6H5+00crKL+4OrtGpuOEuYrXe49VLyOz7VA+XffuElqy85hi3/SY/Ns2Ou/1BcJtAgBAqAAxAoKCD28+w6fr17Nw5BWXKd7nLRtmMmMe/Hqd+/k/B/No3f5cw2VYl2A1d36+ipbuOk4r955QWkZ01vPGvO20JSefPl+aZbhNAAAIViBGQFDQq1ktmjamn8/Wr+emOZJfomi6lxBnvYqrVfht8PuhhtUZMeOyYj1wAAAgVEEAKwgaujVJl1Jym9VOdnzdeqKCiQC+TklaYnVgqbeZMVokxMaY6kKsLpDGysMbxccAAECoATECggbW22XT0yOEg7K3xKsqrPKkJMRQQWm5+3UiF7/iM8uIwFITK9gWEx9RKktJtMBx4yILVd4AACDIgJsGBBVMCIgGZW/Riv1oWS+FJl/TnYo0+tD4ygghEiN8Ro+mZUSVIlw93cGdAwAAPwPLCIgItCwcP94ziGokxGo2xfPVGC9ytWhaRrhZpRpoIjRECgAAhAKwjICgRa5+enG3hl6vK04jgFWu9Mqa0fkTUe8brZgRhRiBZQQAEIZAjICg5fELOtCXt/Wll6/q6jM3jTzQF2hZRjiDw0PntnU/r50S79X+lJV7WjJEwbJq8cFiRoT7iZgRAEAIAzECghaWjjugdV1FQKld+rWsI5zOsncYDwyrEhrXndFE8X5mWoL7+T3ntHGsSJvYMuL5c2S9bh7+er3CbSMCMSMAgFAGMSMgrFnw8BDafDCXRnTKpDvOaknvLNglbNB3YdcG1LPZOZSZmqh4v3GtZJr6r16UnqTsI1NSLrakmKVEJEYElpH9J4roWEGpoehAyAgAIJSBGAFhTdM6ydKDMf6CDnQgt5h+WnfA/T4fptEgPUm4jvM61/eYxtclsUNZubmYkVKVaOGb6IncNHuPFdDafSfp4q4NUY8EABAyQIyAiCItMdZWQz6jxndW+XXzIfp1Uw6d26m+bjaNOrVX3f1XJie3WHoMfnl+1XIVLrqyV2Ov9hEAAPwFxAiIKJxKgNWyUFjh9s9WSe6j2ikJNOmKLhQncNO4TAaw/rD2gPSQWbHnOMQIACBkQAAriCiCLbZi7qZDNG15FpVXVFKMRgCrlgjSCmYFAIBQA2IERBS3DmpBwUhhWYWwAqs6tfevbUfo65X76Ie12ZpWEgAACDXgpgERReuMGtS7WS1aufcEBRMPzlhH87YcNowZeeK7je7nvZvX9su+AQCAr4FlBEQcSfH2a4S0qJsi/U+Mc/an89vmQ8LpzH2jRWGJf6vGAgCAr7B8RV2wYAFdfPHF1LBhQykT4fvvvzdcZv78+dSzZ09KSEig1q1b08cff2x3fwHwGm+8Gx/c1JvO71yfvrlzgF++iT3HCjXfkwu2AQBAxImRgoIC6tatG02ZMsXU/Lt376YLL7yQhgwZQmvXrqX777+fbrvtNpo7d66d/QUgoLSsV4Pe/lcv6twoPeDfhNqFAwAAERMzcv7550sPs0ydOpVatGhBr776qvS6Q4cOtGjRInrttddoxIgRVjcPgNeESx8XUUl5AAAIRXweM7JkyRIaNmyYYhoTIWy6FiUlJZSXl6d4ABAKPHdZZ49pHRuk+WRbZaq0XwAACFV8LkZycnIoMzNTMY29ZgKjqKhIuMykSZMoPT3d/WjSRNm8DABv8EVGbLfG6bTj+fNpUOu6fovteGH2Pz5ZLwAA+JugzKYZP3485ebmuh/79u0L9C6BMMIXYuTyHo2kcu4i4RFtU4ywrsV6LN99XPd9Vr31ONdkT80vGw7SB4t229o3AAAIqToj9evXp0OHlGmL7HVaWholJYkbk7GsG/YAIFRiRuQeN3GC/jKCKu+mSIqLoVJBQz2zvDR3K709fye9dm03uryHZ5+au75YLf3v37IOdWzoG1cSAAAEhWWkf//+NG/ePMW03377TZoOQLgg99uLFSgPu26aZC/qoTCYEGE8/dNm3fn0rCcAABCUYuTUqVNSii57yKm77HlWVpbbxTJq1Cj3/HfeeSft2rWLHnnkEdqyZQu99dZbNGPGDHrggQec/BwABNRNI8uNWJGbxmZnYG+Ks1nB5u4BAEDgxMjKlSupR48e0oMxbtw46fmECROk1wcPHnQLEwZL6501a5ZkDWH1SViK7/vvv4+0XhAwXD4c0VnciFNixFvLiFnxBS0CAAi5mJGzzz7bo605j6i6KltmzZo11vcOgBBBzzJit6EdixmxvT8GCkPxG4YaAQAEmKDMpgHAp/jCTXN6QFcHsN7QtylV2qyUmhTvTHy5SJigeCsAIJiAGAHAAaJOmxd4w8hHN59BL1zeRdMyUrdGAk24qKPmOpO9sIxUcGpDtPnyykqv3UgAAOAUECMg4vBlOXg5xZffjpZl5MqejeiWQS18EsA6Y+V+3fc5LWLKS8O6B+84fErXRQsAAHaBGAERh0+yaQQjutw6hreMPHtpp+r9CGA2Db9PvIDS4t7pa2jY5L/oawORAwAAdoAYAcABMlI9i/RVnh7w+X52I/s3dz83sjIkGFRgNYtoO7wbx4yXZvaGHOn/1AVVtUsAAMBJIEZAxOGUYaR3s1rS/4Gt69A57TM83u/cKF36375+qng/DHZEVM3VKXgxYgl4aQAAoVgOHoBwZfrt/aTqpRlpiYrpK/89jE4WllGjmlXtDiZe3JHSk+Loyp6eJdn1iLNbR16FyA1jV4zI1h4AAHASiBEQcTgVhMkKnKmFiJwlwx4yNZPj6alLOnnuh9H6o31nGeFFhZXDASkCAPAFcNOAiCNYBlQjEWDUtdepmBEr1g4YRgAIDXILy6i4rIJCBYgRAIIUUTVXp9iQnet+bqUoG9w0AAQ/JwpKqdszv9LAF/+gUAFiBEQcoXJ3L+pzI6JLo3Rqm1nD0rrv+GyV+7mV8JFQOXYARDKr9p6Q/h8LoY7cECMg4nCFSPG1eAsBrN5UUXXC2vHViix6F2m/AACbIIAVgAARYyAgYkwGsLLVmClc5oQY0Zr30W83SP/P79yAmtROtr0vAIDIBJYREHkE2NfwwLC21KR2Et15ditHYkbYXN6El/CHY92+k3TV24tpddYJw3lFMSenSsrt7wgAIGKBGAHAz9w3rA0tfOQcd/rvS1d1Fc4XY1ZhREXpumlcBtk1fGbNZW/9TSv3nqBbPl6hsS5BZg4CSQAAXgIxAiIOs4Gh/uKa3k3oSUH33liNmBGRRjHSLev3n6TBL/9JszccpLIKl2HNkbyiMuF6eN0hixrb1VwBAOA0wXVVBsAPvHhFF2qYnkiTrugSNMdb5JLRsoyox342l1HMyENfr6O9xwrp7i9WU3F5hcf6mLDgaxJkpCbqbrukvIKGv7aA7p22BmIEAOA1CGAFEUebzFRaPH4oBRPRAuFhOmaEBbDqvU9ExWXV3fpKuOcMJkQe/XY9zeA68tZNjddYW5UaWbT9KO04fEp6PHd5Z8W+AACAVWAZASBoLSMms2kMUnuZfEhJqL7vUFdlZNYOXogw6qQkiNflElRwhZsGgLBvf+FrIEYACAJEHXrNW0b0A1gZKfEx7ufMxcLznzlbBPuj5SJySdaQ27miaYfzS9zPQ+S6B0DE4AqR3yTECABBQFJctViwmk1TFTOiP08yZxnJK1am32YdLzR9Z8We3fzRcsX75762wP0c5eIBCC4qQ0SNQIwAEAQkc5YLOzEjRpaROG5dG/ZX96UxA++SYde1/SeKNOcNkeseABFDZYj8JhHACkAQkOiVZSSKjMJLCkurXTMLtx81vV+l5ZWKOyujuyyk+QIQXFSGyB0CLCMABAFJIsuIg71pCrmg1d1HT5lYYxQ9/PU66vLUXF1LiBoUQAMguHCFhhaBZQSA4I0ZMXmvYOSmcREVlVbHieTkFpta7derqjJsPvp7t3tavireRA0yawAILipDRI3AMgJAkIoRJ3vT8G6aAu650xczuGkACC4qIUYAAGZJjI+2HzNiIoCVFyNm12lHYMBNA0BwUamscRi0wDICQBBaRhLjoi1YRqKotKJS984oV6PXjBl0Vh2yFz4AwpmoqNCzjCCbBoAgFCNf3NbPkmXkcF514TE1Vt0yDP76ZclNEyIXPgAihcoQ+U3CMgJAkHUS/vSWPtSrWS2KNVsOPsqzqqq38IXOrLhpEMAKQOBxKW4mKCSAGAEgyKhbo6ovTEyMBTdNeaXP7qYsiZEQuQsDIJyp5Ksmh8hvEm4aAIKEz27tI9X06NgwzXIF1hKHxUiFTTfNuBnrpPiUl6/qSlf3buLoPgEAzMHfP8AyAgCwxJlt6tH1fZq6X5uNGWFYtYz897rupt0tViwjcqDsw9+st7Q/AADn4K0hoWKthJsGgCCFt4wYCZMSKykvRJQQG0MN0xM13+ezc1A7BIBQtoy4FO8dLyilB75aS0t2HqNgAmIEgCCFFyAJsdo/1agoczEjvJ5JiIvWNd/ybh9kyAAQyjEjpOC5nzfTd2uy6fr3llIwATECQJDCZ9PoihGT60uJrw4RS4hhYkRbjZRwvWxgGQEgeNmYnUu3fryCth3Kd0/Ta265+1gBBSMIYAUgJCwjrA6JuHCZQfFVN8kJMZRfUm7KMsJbWkLF5wxAJHLZlL+pvNJFG7JzafkTwwxTe4M1/R6WEQBCIGaEiQdHLSOxMfqWEU6MOJ02bDYQdsG2I7DKAGAAEyKMw/klpiwjwep2hRgBIEiJ5sRIomQZ0Y4ZMUNSfPU64mMN3DScACkusydG5mzMkYLlxn65mj5fulcR6f/rphzaq2Muvu7dpTTqw+X04aLdtPtoAd0/fY3CDA0A0IY3fqjrjFiMdfcbcNMAEAI4bxmJ9ghs4+EruhZz8SNWuPPzVe7ns9YfpH/1ayY9n7/tCN3+WdV7e168ULjsPwfzpP/fr82mL5btpT3HCun3fw7TxqdH2NoXACKJSoVlRPUe3DQAALt0qF9VCM1qjxt1zIhZNw3vmnG6oNqqPScszc+ECOPU6XgXAID9OiNw0wAALLPmyeG0+LFzqFZKvOY8zEvz0egzqGW9FPr8tr6a8/GunnhDywjvpnG47w2Z91lHm43OBQCI64xUaltG/v39BgoW4KYBIIhhIqSW4VxRNKRdhvTQg+91w9w0ZrNk+MA4J7ASPwctAoB1zAawfr40i+4f1tbdDyuQIIAVgBDH7IDNZ+cYBbD6EitbhV0EAG8DWJXvqesGlfONqAIIxAgAIY56wO50utGenpuGCRN/x7GV2Qnjj4qy1KMHAED6MSOqH77Tbli7QIwAEOLWD/V7P4wdSF/f2d9jvu5Na9LZ7aqa8bF0YH+3Fu/29K9Sqq8lNw1zL8FXA4Al+LgQQzHCZc4FEsSMABDiXNmzseJ1bEw0pSfFeczH4kQ+Ht3H/drflpHC0gr6fk22pWWYUUSqih8c10sA/EJRaQVNW55FQztkULM6KV42ytN+z5s6Qk4DywgAIcyc+8+k4R0zTWWhxMUof+6BiBlxWcymYRYc3jLCCqUN+s8f9MniPT7aQwACz2u/b6Nnft5MwycvcKBRnkv3dw83DQDAa9rXTxNWYBWFWajFyJvX9wjMN8BdC0d+sIwW7zyqOWuUqhLtMz9tpv0nimjij5t8vZcABIxlu45J/0ttlkvV600TrDEjcNMAEIYw0277+qm0JSdfs/PvRV0b0tD2mdRhwhy/7Rdz07C4EZmF249Kjx5Na1LfFnXosfPbK+ZnOosPYLV7cQYglIj2MmhbL7VXXYEVbhoAgGn4S9M1vatiRFggqhZsAJ9975l0TvsMTcuIul+NHl0apdO5AneQVVhn0eyTRR7T12SdpKl/7fSYHkVKNw0AkUC0l+e8suiZfgXWorLgqGwMywgAIcazl3WmK3o2pp5NaxneXSVyPW3451a5f1gb+mPLYfI7UgArxAiIDP7YcohSE+MMBbjUbHLzIerYQJzGf6KwVCeAVTnhga/W0eU9lEHwgQBiBIAQg/WV6deyjul5Rc+twnre8EXT/EX2iSJYRkBEcOBkEd3y8Urpeb+WtXXn/W3zIbrjdLNJUZPLdxfscr8OVHFDqyCbBoAwho8T0ev8a0RifAzFSDm2Ykb1r+rI6zTMpZOTV+yTdQMQTBziznOjQn9Ldx3XfO/YqWqriEiMBKs2gRgBIIxhZd+1AlgtW0a43jZq0hI965r4AlHmEAChButAPXP1fsotKnNPczkUM6LOllGLjyDVIhAjAIQCw04Hj4qKmZm2jHjhpqlTI173bs1fGsHfVWMB8AUPzlhL42aso7FfrLa1vEtHUqgzzkLFTYOYEQBCABasOvf+s6hBzURLy/GuFW8sI3VTEnRjN2CxAMA8czcdkv4v2iGuseMy0A9676vrhnhUWg5SbQIxAkCI0K5+quVleP2QGGffMsIyWvQsI/6KbeWzBAAIV1xeKAZ13RCPOiMCJcMsjoG+oUDMCABhDH/d4eNHzDCkXT3Fa71sGlYPxB9szM7zy3YACORv1eWF9aJEZRkx49r0d58qEbCMABAhGEXoq3n0/PZULzXBXYMg2g+WESt3aLM3HJS6ECfH4zIGQoP84jJ67ud/LC1TUFJOKQmxpgWGuguvLDQ+/nu3VMNEZBlh02L8dEOhBSwjAAAhDWsm0UtXdaP+reoIo/QVFxKH1Ai7TpoNUr37i9X06LcbHNkuAP7gx3UH6KuV+4TvuRTN7aqnP/DVWs95Lbpp9p8opKd+2kwPfr1OaAUJhiBXiBEAwhhvfM8pKouDnhhxClaq2spmflp3wJe7A4CjxOnU6innTnz+d8sqrVpBFMDKpxCL2HwgjzZm50pWmEAB+yYAQKJmchy1qleDmtRKogu7NvRw6/hFjFS6qDIq8HdpAPgCvcKDFbwYMfgJnCwsM20ZYRYXo/Xd/tkqOpJfQt/eNYB6NdNvMxFUlpEpU6ZQ8+bNKTExkfr27UvLly/Xnf/111+ndu3aUVJSEjVp0oQeeOABKi5GVUUAgonmdVKki9Hr1/Wg4YKmePydm6/qfzBzsT9EDwDBRoXJ8565XJi7x7xlhFkb9ddddro2SSBaPtgWI1999RWNGzeOJk6cSKtXr6Zu3brRiBEj6PBhcROtL7/8kh577DFp/n/++Yc++OADaR2PP/64E/sPANChvY10YC30LmhOuZzZBVlP9AAQiuw+WkC7jpzS/Q1VKNw02rw137O7tW4Aa6U4W6ZPi+r+N2XllbaC3APqppk8eTKNGTOGRo8eLb2eOnUqzZo1iz788ENJdKhZvHgxDRw4kG644QbpNbOoXH/99bRs2TIn9h8AoMOl3RpJvSp6N9dvvGUm9be8QkeMOPQtsAun3TiXYKiVAICa0vJKGvLKfOn585d31jxA5bxiEPwE1mSdoB2HTxnGdYgCWNVWF6Y5vrytL7V+4hfpddnp37Zey4egsoyUlpbSqlWraNiwYdUriI6WXi9ZskS4zIABA6RlZFfOrl27aPbs2XTBBRdobqekpITy8vIUDwCAdViWy21ntqTuTWpqzvOfK7tQk9pJ9MLlXXTXVcGUgq8tIy57lpF7p62h8/+7ULrwAxBM8JaK46omdlq/rwrBD+rytxbTw9+sp+W7tZvkqdcjixG1GzU2JlrR/6Y0CNw0liwjR48epYqKCsrMVPqT2estW7YIl2EWEbbcoEGDpANSXl5Od955p66bZtKkSfT0009b2TUAgE2uPaOp9DCCFwlntqkr/V+4/ajXWTs87A7u399bT9eVfehLdx2js9rW83jv08V76H839KT66dbK6QPgLVE6fWO0fl+lOqL6YK5+vKVazLPXan0fFx0l7Cel15nb1/h8y/Pnz6cXXniB3nrrLSnGZObMmZJb59lnn9VcZvz48ZSbm+t+7NsnzssGAPgP3tT72a196ZPRfRy3jDATtNy3ww6iCyyzmqzce4KenbXZu50DwAa8oaJER2RUcL8vOaCU0axOsqWg7gqVO5W5V9VBrSw2hLk01b+XkLGM1K1bl2JiYujQIeXFgr2uX7++cJknn3ySRo4cSbfddpv0ukuXLlRQUEC33347PfHEE5KbR01CQoL0AAAED7VT4hWv+QuZUzEj/EXYDnpl6fMMai0wKitdjhVwA0DtclGXapemlVfQ50uz6HhBidAy0rhWEp0yWf9DFAA+8cdNHvPFxVSNu8xVw+9fIANYLVlG4uPjqVevXjRv3jz3tMrKSul1//79hcsUFhZ6CA4maBhoBw5A6HDX2a3o3I6ZNOWGntJrRbCoQ6YRvTtHM+hdS4128T9zttAZz/9OOQZmcACswGfQqINLGW/9uZOe/XkzTflzp9Cdw4JLWRl5j/P1yi703GXKgNiP/t5tyooiB6qqfy8hldrL0nrfe+89+uSTT6RU3bvuukuydMjZNaNGjZLcLDIXX3wxvf322zR9+nTavXs3/fbbb5K1hE2XRQkAIPhhfS3eHdWbLuzawGfb0POpm0JPjBjYb96ev5OOFZTS//7c7t0+AKCytmml3TL+2nZE10LIxIXIMsKy3zJSlR6EmauzTQWAx542EKizz0Iqtffaa6+lI0eO0IQJEygnJ4e6d+9Oc+bMcQe1ZmVlKSwh//73v6UPzP5nZ2dTvXr1JCHy/PPPO/tJAAABwyk3jZ1sGP5iz2cIqDFrvPFXB2IQGfDaoKjUU4wUlnoKDf53wMRFfnG5MNhUdrfIbD6YZypIW9syEh1a5eDvuece6aEVsKrYQGysVPCMPQAA4WuKZibjf3+/0e9ihPd5Q0aAYONrrjFekSBmpKDEc5pc90NO1RW5aZhLRVQX5I8t4gKk6mVF4j0mVOqMAACAFv/q1ywgAay8jzzKAcuIU2nKADBe/W2bdctIBWcZqXAJBQtzqagtI2bZeaRAKEZCKmYEAADUOJXaa0eM8AGCegVYITJAoBFaRgQCpYwXI5Uu4e8iJoqJEe0TPsGgorLo96Ln5vQ1ECMAAK9xqp2MnWwa3jLyw9ps4Z2mFRAzAvwpRkSuSZdLla4raMXAXCp6MR6JcdoJIjf1r7JiwjICAABOBbBy12lWr0ErbkVkvTmcX0wvzdlC+44XVs8HNw3wESI3jRHllZVUJmjFEGvgpkmM037vmjOaSP95rwwzigSyxg4sIwAAr3FqAOcD9+xk0zC+W5Ntetl7vlwjdUG95h1xby0AfG0ZMaK8QsMyEm3kptG2jKQlxnlYRgIZLyLtS0C3DgAIDxxy05QK6jAYoW4qphW/IposNx0z6vcBgBOYqQKsRi9mhDW8sxMzkp4c5xHwHcgaIwyIEQBA8NQZsRjAyq6lasuIFqj4DEIxtqpCUOKdwdJ69awZeuKiRnxVVQ9+lkDWGGFAjAAAvMapgd5qzEiURrt1AMKF8opK6SEqesaqsNoRI3JsCO+mgWUEABDyOKUHrIoRdsO4Nuukx/R1+zynQbOAUKRcctO4xEXPbFpGZJSWEbhpAAAhjnNumqo1NaqZZHqZu75Y7THt0il/294HiBYQTBbDcslNI7KM6MeMmKkZwmfPwDICAAh5nLaMpCXF0UejzyAngTMHhOLvokKrzkh0lK6AMGPpQDYNACCscCq1t7Siwn0hZdkCvr5LFW0igEUoQYA5UVBKf249rCik5w18dWC7VFS6hIHdTIjE61lGODHSuJbY0sjrlUD2pWEggBUAEHSWEXahDVRpajOf5ZFv1tHVUxc7NmiB4IC590Z/tII+XbLHkfU5dXoUl4mLnrEA1jeu70F1ayR4vM+L+Zl3DzBhGUE2DQAASMiBeuxCq742Pn95Z8tH6bfNh9zPnZQNM1bupxV7TtCqvSccXCsINFmnK/H+siEnaCwjjBJBsTTZRXNJt4Z07RmNNd9nZKQmkghe7yNmBAAQ8jhlxJCrp0YLLCNxFu/cWP2RMZ+udL8WjQtRQTLYgPDEqdOjUFBGnrdkiKyIZkq7I2YEABAW3D+sjZT5cvfZrR1drxQzorqYsiJPVlD72SEbgGmigiuWqlBgGeG1uUiMmPm5BFOdkaoybAAAYIP7h7Wl+4a2UZSVdgJRzIjVi2Wx+gIOKwZwCNYZmrl02tdP80vMSGFJuWXLiF7arwy/GOqMAABCGqeFiDtmRLVavQ6lIkpMFFCzs+9my8+D0MXorLjojUV03usL6a9tR4Tvz1ixj+ZszHHMjVd42k3DV1zlxblIp48e0JwapifSrYNaaK4XlhEAANBBVEPB6p1biSoDweywYDQfys+HP8t2H5ea02kJ4F1HC6T/P649QIPb1lO8l3WskB75dr30fN2Ecx2zxDDqpsTTgdNNHfnfgyg+pG5qAv392Dm6gpt39ZixpPgSpPYCAHxCRqpnuqE3bhqrMSNr9vkm0wXpvJHBtOVZtmrXHMwtcj5mpLTKMtK4VrKGZUQQMxIdZWj545dLjo+hQAIxAgBwnM6N0ui/1/WwvTzzh3vGjFi7XN03fa3iNT9uyI3HRJdqI8mD0JPIYNeRKuuHHiI3TBEXq+SUcC1yi5Ek024aLUsi/7PixUpSHMQIACDMyExNpF7NalHrjBqOuWnivIz2l+9S31+4i9o9OYeW7z6uMZ8+cNMAvXOFD5x26lwpOO2mad8gVVjUTGQZ0XIxKZernp4EywgAIBxhwXa/3n+WF24az2lO8Nysf6Q71ke/XW+rPgrcNEBGpDWctIwknA5YlVfTrE4KTb6mG711Y09FnIjoPE5JECfL8ssFk5sGqb0AAJ9hpvCSphjxqDMS7ejAwQaKKMkp4zlgrNxznBLjYqhzo3TBepBNEwmY+Z5Fbhq+dLuowZ0VkuNjFFlhTOCP6KRfbVWmhoYY0bSMwE0DAABE6UlxigumRwVWLxt5qccNrbTLk4WldNXUJXTRm4uEAxJ/twtdEh78czCPbnhvqfNuGi8tI8nxSkGRonptFMAqnlcjZkRj3f4ClhEAQFBQMzmOcovKpOdxsZ5de71107hM1gs5kl9SPY/Ls5IlHwfgVLYECCw3fbicDnPfO8PMNysSq7ybptxLMZKSoHSdaLlSrPw0eIujnDKst25/gWwaAEBQUCs5XhF8p06e8barqHrgkMYJg4t4eaVn4TR+kuBtEIKohYhZRJax4lLnLCNJKmuFlmCwUryPF/Ubs/OqtwU3DQAAKN00khhRXWCdbp2h5aapiiPRFhv8clrZEnnFZbTtUL4TuwmCGKPUXpGYtUKKSnxoBaVasRqKXDqM/OIqq2SggGUEABAUpCnEiGdqr9Nl5zVvWrnNiAYT/m5XS9AMevEPOve1BbRu30nvdxREbDZNcrwP3DQav6NTJZ7N+PwJxAgAwG+0zaxBn9/aV/heelKswjKivmY6bRlhbhujomdGlhGtuJO84ipf/B9bDnu/o8DniM4tM8HJ4gBWLpvGSzGSqHKdqANarQj1+mmJ0v8h7ZTl62VG9W9GgQQBrAAAv9Amowb9+sBgzffTEpVuGnUAq7eWEY/UXhOjjZFlBDVHwgNmLbDT1E4YwOpQzEhcTJSiMR6rOaKdIROlWO7FK7p6zDPz7gFS875rzmgiXEfDmtXVXQMBxAgAwC9omYfNumm8tYyoM1+0rBr8+CIaTPhJaOAbTuem8rs2kykl+v7laqne1hmJjY52Fz3Tixdh8CV4FjwyhBqkJwnFxi0aHXwT4wLvJAn8HgAAIgIjw4baMqK2hBiJGSPUN7HstWiVvMWEf75w+xG6+aPltP9EoXua0d00En9DA7unlsgyUlhi3TKSKBADrDGk2Qqp/Hxqi6IZtNw//iTwewAAiAiMxERSfLSi2qraMuKtS8SjzoiGkOC3w9/ZjvxgufR/w/5cx/YJhLYYMbSMmMymSYqLUcSaiDLKeCuJHnbq8QS6xggDlhEAQFBc8Pk6B/HSXaG4jbpTaMWMKAJUBfMcKyjVfR+Eh1C2G8DKn6dmxWpCrKcYULsqzYoMK/V45I/dt0UdCjQQIwAAx7hvaBtJVDx2fnvPi42BGkngxIiozki91ATHi57xNUVE1hCjbAhejJwqKaef1h2Q/oPQwq4LUOSmKeC+f9NiJE7gpolWZpTp7SO/G1ZqA869/ywaO6QVTbykIwUauGkAAI7xwPC2dO/QNsK7OKMbu0Tu7lDtpundrJbXYmTnkQK6bMrfyoFEcIFXFDUzGEwqOMv6QzPW0ZxNOXRep/pe7SfwP3aDo0WWMd4yYja1N0HggmGWEUUsiM5O8sG2ViwjbTNT6eERnjcOgQCWEQCAo2hdNI1Sc/kgvio3TfX8vZrVcmTf1nJFyLTGCStihJ+XCRH+vwTcOEED+y7/98d2WrX3uKlz01xvGvVrl6LfCy9G/ndDD831xAvECBPk/E9JT4zwoSne9nAKFBAjAAC/YGQJT/Rw04ibezkFExKifeIHEEMxggDWkGHGyn30yq/b6Mq3l3i8Z/f0UosRdu7wp0TFaZXQMD1RV1DHxUTTS1d1pXvPae2eFhsdpTjv9cQ8vxuhKkbgpgEA+AWRz5sVQmMXz2cv66wQI+yukL/4+uL6qmW04AWGoZvm9EqyjlWn+4LgRK9XkO2YEZX9RF1XRHbjsXM5ni8GItj+Nb2b0ImCUnrjjx3C1F5192jFfnAnc4hqEVhGAAB+utgILpJ9WtSmOfefRWc0r61w0zB/uXJZ/11h+Swb4wDWqv/3Tl8jfB+5Nr7l2KkSOpxf7PV6hG4al3I7905bQ4t3HFXMoz49ylSpvLlFVc3nWBiHyBWj/m3EcOc9C64266bh99XpHk7+Am4aAIBfMIwZ4QJY1eLDVxdYUbowP56YcdPsO16oiEUB/oEd+17P/U59np+nKMFuByNrwjM/b6Yf1x2gG95fpnxD7aZRWUae/Xnz6fVHSa4Y7e1HeRQsY0/513qCPBxSzCFGAAB+QdiUjpvIu2nUsRjqweLMNnXJV1jpPcPef/XXrT7bl5zcYho++S/6ZPEen20jVOGtEIfyjK0j/HjNjqvStSE6O6vf36vhhuNFwJQ/d9CQV+YL52NrN3LTqK0fTIBHmc6mCX0gRgAA/rnYGNUZ4czYaveIfIf4f+e0psa1kug/V3bVvbh7Qw43sFnJphEhepsNghN+2EifL91ruC//nbeNth8+RRN/3GQ4byRjdTDuN2kePT/rH9OWEVE9EfX3//LcrW63jOjc1wvCjo6uDlrl98l0am8YqBGIEQCAfy42Blcb/mKtro4qv/fgue1o4SNDpKZf6yaeS7cMFDf+cgqjzr5sMEqyWEr77x3H6NMle+nf32803j6ydTThvxotsaDH+4t267oBpy3f536u9TWY3aqRlzFaYBmRBIzJomdw0wAAgEmsBKG2y0zVXFYeOJgIEFWudBI5NVP7faV7yQzHC6vLyRvBl8gH2gOwt4YBdcC0zNgvVntsi8esBjI696NOv8+LoiiVQNez3oSBYQSWEQCAfzAThLpk/Dk05/4zJcsHj9aFWDR90aNDyCmMWsBLlhEdwSBqQ2+lNkmigw3MWKBtcZmz/X0CCW818tZNwVLJRczacFDq0qxpGTG5YaNTPyZKvIx5N03oyxG4aQAA/rnYiC64qrDWBulJ1L5+mmBZ8YVY1C69bg3vysZbMX8zYWHVemG2RLg6w8gb1u8/SWe+9Cdd8MZCCheserBK+dr96nXprCy/uFxzsDe7C0aWkWjB+1EW3DRhoEUgRgAAfrrYeJGeq7WoyNrCu01YTxtvWJN1Uspk0RqMKizEjBwvKJUGPSuWEX7d3tz9sgZ+jF1HCiiYYJ9p+e7jlFdcZmtZ7pXh/CVllbYEItuMZoE805YRc24aHisBrE61SwgkqMAKAAicZSTKOyGjNf2stvVozd4TNLRDJq3ce4Ls8s6CXbrWFiYs+G7DWqzJOkGXv7WYhrbPkPbJLIlchlFJeaXl+JRAFI2zwvdrs+mBr9ZR64wa9Pu4wZaWNavpmGtqya5jlFtUaitQmLnaKryOGbH+fhQresbHjOispHOjdPr2rgHUsGYihSoQIwAAv+BN4TKtQFWt7N5PRp8hDd7frckmX5YSl0t3GyHXCZm35TCd3T5DcXevd1x48cEKtNkVI8FalfPHtVUWmx2HT1lelrdKFJVW0tM/baLhHTJpQGtlDRo2nc+MEVGuE6jMNqNlATEriIwOf1SUOPtMUYHVYCWhbh1BzAgAwC/YGQ7vOrsVdWmUTlf0aCxep8YFmk1nA7dTQzBf/8EK8himSFvmYheMBjPeosF3g7VKsPYr0SuRrubAySKFi4sXCO8s2Ekf/b3Hs0KqKkVXC51wEl0LiFnXmfw9sh5MonMpSnCmVpWDN+emCQdgGQEA+AU7roJHz2tPj57n7DrtILdzt1v2g99PPkGH3ZHHRMeYvPu3nwkTrG6aeJMBuj+szab7pq+Vnl/fpylNuqKLQiDs1ImFMfO9GaVwK9KIBQLESJTIonlkv2bUvXFNuvh/i5TLk0sjmyb4v0OngGUEAOCfi40Prjaii7gvYINRrBcfgDex8wOfwRioGESLvEjLDdabarNVdCf/ts39fNryLCkOhBcI5TqmjZR443tuOYD18h6NDN00vO6QY03KDFLA+eNvJU4q2mSdkXAAYgQA4BdELhVvr69GVnKnbiYLSsqllu5WkXdP4abhG/EZlZPnxJZXxViD9K7arJtGHS9xqqRcEXSqlw2TnGBsfZHXJbI+sO+AF428MJGf68WcMIyOvsvlXZ2RcABiBADgF3xxKbWSJusNG7JzvTKT8+MIbxmpMCyqVv08r6iMft98iErLK73afjDB9yPSc3WoM0lOSbU/ql+X6VhGkg0sIywWR+7enJYUq5HaKxaF8nNjy4iyG68ZotV1RoL1S3QIxIwAAPyCVmEnbzDSIqLAQDss3H7Uq+Vj7FpGuPdv+2SlVLhrzJkt6IkLO9o+9vI6gyHDhreMsM+WoBFDoraMsEJkvHDQS81NNqgDk32iSPqfmhhLaYlxwnmUFU14i0yloRjyECMmz8lotWUkCL4vXwLLCADAPxcbH1xLDYtOBcn1mx9UeJO+USM8/uPJFUSnr9hnY/vVz294bxld9tZiv1mVzPaE0QvQVVsF8kvKFEJUZJmQS9+nGFhG9p8WI41rJQvfd+mUnpe/SqO2AbyOEGkKl3gpuGkAAMBpfJENEOw9Odju/WfOFinoUoZ3sxiWmxe8b0dE8FYQVgBs3b6T7kE4kPAfr0BHjKjjXJmbRhHAqorZYAGvXZ/6ldbuO2kYM3Igt+o4NNIoGMaECH/IWQox/54Zy4iRGNG0jHCfG9k0AADgBazqKOOmAc0dP45BcHOvy9p9J+jt+TulAmwy/HM58JJZBUa8toCe+nGT4eczcu2I0ArMDDS8xaFIp46KyE3DC1F17M0b87ZLlqT3FuyiWsnxuvsgi0NWl0Z0RJjo4bfFxKV7u+4AVgsxIwJznctUACuFNYgZAQD4lPdG9abcojKqlaI/KNjBzsDsT7JPFuk2bJOtHKx3zNZD+dJj++F8unlACxreMVNo+TFKBzbrIgsGIce7V/SCQD0CWEuYZYRbD3dQ+M7EDdITDfveyC6WuJhoapNRQ8MyotGbqLJKqOTkFvsogDUqYgJYw1xrAQACDbuI+kKImHFzBPryze7g1ZRxlhHZMsCLqr93HKMxn66Unos+HpuXNd17ee4W2n20wLZlxGyTN1+iyCzSUUfq/WfZL/z+88se5IQBO++MRJds1WBBxhd2aUBPXqQMDmbb0VoH2+77C3fT9e8tNe+mIZtiJCrQZ3MQipEpU6ZQ8+bNKTExkfr27UvLly/Xnf/kyZM0duxYatCgASUkJFDbtm1p9uzZdvcZAABM1hmpvoD3b1mHBrau49cjd7KwTNcyYmTZEQkGNgA++u16mvLnTrpUVclTC2HQZAC1yK+bcmjY5L+kmA4z+6N207DjwgsQ3qrCC5wSVXE0NZe/9bfb7cKCaZlwvnVQC8mi4t5WpU5vmkoXPT/7HzKCPw/FmsLlfjaiU1UjxVsGtbDUmybixMhXX31F48aNo4kTJ9Lq1aupW7duNGLECDp8+LBw/tLSUho+fDjt2bOHvvnmG9q6dSu999571KiRZ6U7AEBk4e311Uow57Tb+9FFXRuSr7hCUL1TBB/AaphNozF92a5j0v88geVFhDiN1yX1exn14XKav1V8/fYVt3+2SmqOt25/rnuanmhQF79l37vW7PwhLWbHWucQr8k6aVhUjAlGrW0xq4qZYmTKWfTnf/vGXrT6yeFS47uoCHLTWI4ZmTx5Mo0ZM4ZGjx4tvZ46dSrNmjWLPvzwQ3rsscc85mfTjx8/TosXL6a4uKocbmZVAQAAb7F6c+/LKpaJBvUs7IgRrQFaa6nJv26V/o87t51iuqh4LNv0v7/fSAu2HZEee168kAKJnpVI7aKo1O2k6zJtGeHRKvfPhI/WOsyW6DeKGXFxq2eio/ZptyZ/vsIyorJyrFq1ioYNG8YduGjp9ZIlS4Rfwo8//kj9+/eX3DSZmZnUuXNneuGFF6iiwn6fBQAAMDOYO9V918l189k0VuqMKN/wnMSChN/4Y4f0OFlYamgZYds+kl9CwYJemrZaRFYYxHHwx9rsKcLXPFGvz9v4GkVvGpvLxYR5hKelj3f06FFJRDBRwcNe5+TkCJfZtWuX5J5hy7E4kSeffJJeffVVeu655zS3U1JSQnl5eYoHAABY7pbqR8uI2UZ6rPy4eTFi3jLCzyuXN5cRfWy27WDqd6J3KNT7yT6rptWId9NwlpGJF3fUjRmK0fj+9ISPWVimjoyVyrfREeSm8bnWqqyspIyMDHr33XepV69edO2119ITTzwhuXe0mDRpEqWnp7sfTZo08fVuAgACQFKcOdeG7XLwquu3Lwdfs3euvFAwDmA1L1LUFgGt2AhlJ+LgGeD0hJnaRSGn1Bqthx0HeTY2sOuVYteyjLDteFtcr2vjmu7n4ugdMfzHhpuGo27duhQTE0OHDh1SHDD2un79+sKDyTJoWPYMW06mQ4cOkiWFuX1EjB8/nnJzc92Pffuslz8GAAQvEy7qSN2a1KQ7Brfyaj1Wzee+TI80u24+zsAoANdMkGbVfC7afDBPaH3ZfiifZq7Jth186S/0vssoYcyI8Xp4ywj7qHpfkWYAq5RNQ17BW2SsnIIx3D4htZcjPj5esm7MmzdPYflgr1lciPBLGDiQduzYIc0ns23bNkmksPWJYOm/aWlpigcAIHxgaYs/jB1I6UnixmRmqVMjwdL8Ph17o2xYRnRGuX3HC+nTJdWlx/Wqp367OptGfrBc2OeFz1jhYduOFVgDmJAxKuJlBpYtM3vDQUko5ReX0eIdR3U/r25qrzqbhrlOKk0EsEoxIy63oNEb0HlXCg+zXlmNTeJhm+zSKN2gAqtLuCzcNDqwtF6WmvvJJ5/QP//8Q3fddRcVFBS4s2tGjRolWTZk2Pssm+a+++6TRAjLvGEBrCygFQAAvOGOs1rSBV3EVlkRUcFgGVGJkT+3HKbxMzd4zDfi9QV0rEBsPVaPXR8u2q14zfd50Rq0WeVRUZzEWS/Np36T5tH+E4XkDayOyN1frKYF24/Sv95fRje8v4w+XiwWV4ZuGnUAa6VL08WlSO2VLCOcm8aGZcRs+nitZLGwZoLbuM6IGTcNhTWWY0ZYzMcrr7xCEyZMoO7du9PatWtpzpw57qDWrKwsOnjwoHt+Fu8xd+5cWrFiBXXt2pXuvfdeSZiI0oABAMAKKQmx9NaNvah9/VTh++oLv5ZgcEKjmLW6KAJYXS4a/fEKjfm0Mw5dqnRelknDw/d50eqbwqwVon0+eqoqw2bBtqPuacxSYjQos9L3Y79YTav2nlBM37D/pNs6882q/aYsGsapvTp1RjxiRqrdNHqCUSt+xmz6Lm9Z6da42hLiTVxOjKI3TXirEVu9ae655x7pIWL+/Pke05gLZ+lS/XK5AABgl3dH9qZXft1KdwxuqZiuNolrXc/ZZJefLCP8WGzb/M8txlJ51RSUGAfJssJjejz+3Qa6undjWrj9CN3y8Uq6sGsDmnJDT835x321lpbtPk6zNhykOfefKcwCYcKguFw8uFsJ/9Gr/cEfUw/LiM46edHAr/qdv3ZaFiMDWtd1CzD1eWFF+EbzMSNhLkbCPHMZABAJNK2TTG9c34M6Nay+I7UiGOQiU95gxwVkt36FUcddhfXFoL29UdwH6zrMmLW+2uItIut4tVvnvNcXCu/u2ectKRPvj5V4Er0A1jLuDWYVqo4Z0d19itWIGdlzzJy7io+/4XWD2jIiOk9cGuvkF40zmToeqoT3pwMARDTq675oQGJCpDMXYGgXOzeucsdYqxhpGN7FY8X4UqYSLsw1ILInsODaG99fSn9yZeS1hB7vXmD7ouU2YqKBxan8sDbbQ5jIr+ucFo1VtT+0YmEqFceXt4zo4W2aM28Z4QWY2qIhTO11idfJC5fkBO/S4MPSTQMAAKGIaEC6c3BLWr77uE/W7TvLiIX0YQvbUNcnkT6R4GM9/M06WrrruNRhWC4jrxXTwB+Xqg64Ls1tn/3yfEmsnCoppxv7NvNwNSXERhvW/uCb5rGmhO6YkWj946ZlGTELL2Z4EeFpGTG/zhhu5pT48B6uYRkBAEQMWoLBiRoOdm6s7XpQjIpw8d2CtSwRIliMBQ8bzEWfa9/xIo9pWmKEn852W2t//m/aGvd787ceEQalyoJBKtGucezKuTeYlaS6zoj+F6RV9Mws8aeFknpb6uOiH7mihF9PssneR6EKxAgAIGwZ2Lqu9L9ZnWRdwaAeMC7q2sAvMSP8wGkFI3nBUmh/23zIUpDs5gN5tPtogUdTP9HgqRYt+pYRc/VBtFwtvGVEFgx6jfJ419eJwjLJemPm+/E2W4W3gCh7yti3jERFKTPHwpnw/nQAgIimbo0EWv/Uue6y86IBKT4m2sOv37VxOv1sELAZUDeNicXGfLpScqGYFSMXvFEddCqzdp9nGXlNMaJldTJpGeGR52HbYRYT2VIix2XoZdOo417c+xGlb1Ey21vITMwI/5nVtVyslIOPhmUEAADCg7TEOPdAwd+lPnRuW+rZtCZdc0YTD7++HWER5ccAVrMwITL5t222l3/6p83Cz1Wsii3RSz1VZ9NUmLAGyQLql40H3RYehRjRaV73+bIs4XRfB7Bqu2lUMwo249KqwBodOZYRuGkAABEDP97cc04bmnn3QEqOj/W4q/dlpVbe3eDr/jDLdle5KLxBZIHgrS1MLJSUV2g2ClRbRszEycgiTS0g5PRZ9raWZWSdhjXH6DCLSuM74qaJsh8zUs6JVcSMAABAmKBVT0R9V29HH1hxucj7UVJW4VU3VmbZ0WP/Cc9AU6etN8wd9N3qbGFpebWgYBYAM3EybJ7//bGdPl+6VzFd6aYhSzCBqZtN46Wbhs/G0espY+XrLuLcYUw0hzPh/ekAAICjZb0a9Oylnah2irLBnloQ2HHTWAn/YNs/lFdCT/202avvh3U+Xp0ltgQw5m7MIW/hi4hpsXDHUc3eKbwLQgo8NWEZYYGnr/zq6V6qDmDVTu3VwtBN4202DSdG+E15pPbaFIIxYV6BFWIEABBRjOzf3GNajGog8vV1v3aKd92KZRJPB+ZqMW9LdVEyu/AxHswiIYoNaVwriY7kVfW0UcNrhkqTlhG+gqzIMrLrSAHdN30tWcHQTeNtNo2iAqv4uVUX4BnNa1HfFrWpnUbvpXACYgQAEPE4ETNi5U6ddXJ1Am8HUKt356zuSGK0pwB6569dmsvzvXFcJt1ZWsYYWYxsPZRPVjGyjBgJOxEDWtWhxTuPebh5FOXgY+xXYI2Niaav7uhPkQACWAEAEQ9vAn9nZC+fu2lYho8TOFGszQi+I7C6QqsZ+GDXk4WltDXnlG2R5U1hMqNDlWSjqNj/ndPG/Tw+VlzoTP0d+eM7C0UgRgAAEQ8/QIzoVF/XpN+oZhK9eEUXj+lWIhhSE50xSvsjjuBgbrH7+ZKdRy0vz1tCWKl21g3YCL6ku1NBpuw71hOMKRYDRFvWS1Ecf37f9MrBi0wjfVrUpkgHYgQAEPGoTel6d6/f3jWAruvT1CvLSI0EZywj/g5qZAXIrGKm4qrZwmXeBJkaWSSsWkbeH9VbKUY0Ykb0KrCyTtPPXtaZ7hjckiIdiBEAQMTjGWSoc9HUyhqxYBuJ40z63uBvkz+zWEz8YaOlZezUddMSI3zGilWMDhVfx4P/Lge3rUcTL+4ojF/hhQa/b/w5kpmWqNwP7nmz2sk0sl8zSogN774zZoAYAQBEPJ6dVXVGLs0UVvOH0WYVeA+8bDRri0+WKGt/OGkZkQN7fWEZYV+py2Q5d55PbulDowe28JjOhAh/3ijKwUdFSbFHwztm0sMj2qn2AzEjIiBGAAARj1Ypc+FFU2MwsZJNY7VGhtV98RYWF+MUfDaNEW//q6duzIiWYAjEsWLCiF8nv29sMos9em9Ub6qZrCy0BykiBmIEABDxNEhXmtL10BpMrIRGmCmJHsjU3qt7N6Zayc7EtZht1Mcwcld446apCmB1rhcQC1jVihnRi+WBYUQM6owAACKeq3s1ps0H8mhg6zqGx0LLzG4lZsRut15/BbCyu3y+vkjnRmm0MTvP1rqsNAM0St31LoCVHIUde/74J2g0ylPD96bxbZvE0AKWEQBAxMOKS7GshvM6NzC+aDoSM+L9MPTK1d0suZesCohTXBXUUYKqtWYprajur+KtuPKmPovTsRqxajHCFU3TDTmCn0YIxAgAAFhAq+uqFXkh8ly0rJtievm6NRLoql6NdZvssb41dmEBpLxe8sYdVGqhUJqRGGGf2y5aq2YWjdvPamkzZkS5nuptQXFYBWIEAAAsEKVx1bRi7Di3U6bHtEfOa296eTl0Qs8yIirMZpYyVf8Yb9xB3605YHpeow7GtSz09OnXsjY9fkF7Q4Gw6ekR9PgFHchOzAgvTE27aaBThECMAACABbTGEisxI10b1/TIWLEy3suDnd7g7Y2AUMd5eFP59OgpcQM9EUZuJyv7wfRULS6TRUsgMBedHdS7yve20fsYWpa1SAdiBAAArFw0tQSAxTCQ1hk1zK1XQJSp4mz2B73MtATNQE01Y4e0IqcwsoxY+UysO3A8Z63w1iKxZPw5VCclXjMGRWEZQTaNZSBGAADAAg5pEa8sGfJAqOcashPnwbKKrjujiSJgNSU+Rndd6gqjvjoGrJ+PFQMNSynmU4W9jeNokJ5EbTKVApIn3nQ2DRABMQIAABbQGmjMVhr96Z5B0n/13FbGSnlevS3acdNc0r0hvXhlV8nlIAd1PnVJJ4rRSanlLQLeomVR6NY4nX4fN9jQcsLDCqcprRXeV749nK90OdVNrbaUJHON9nTdNAgaEYI6IwAA4MDdqxktcsdZLalL43RNkXNR1wb08/qDpgWRXoqwnbTflITqIWH8+e3p1kEtJMvHou3a3Xr5WAlv0RIbIzrXl/Yj63ihJcuIWWuFWfapts8EyPyHzpaEXwGXCg3LiHVgGQEAACsXTS+KnvFzdGyQpniPDWhvXt9DaszmiGXExuBbgxMj7A5edsHoWVmctYyIp8edfoM/9s3rJNP/buihGzOizHDxfv9EZeqb102hJrWTFcG1qDNiHYgRAABwImbEhGWEt2TcO7Q1dWqYplgvEwApCcaWBvcu6GzTTgIMbxkxW/nUTIaL2fgVrXXJYohfDbN66FkgylWWEXZszWY8DWhVV/qflqg8HlP/1UuqdfLFbX11q8fqp/YiakQE3DQAAGABrYHm0u4N6ePFe3SX5QULM/GPP78D/euDZYr1mnEnuN00OoOrHctIsobLRc8yYiY2JSk+hvKLq90YVvdZHuj5bbXNTNUXIxX23TRPX9pJCla9uGtDxfTzOteXHuJ9jPZ5mf5wBpYRAACwQJQgy2PhI0OoR9Na1i/A3BXYihhxu2l0bvStDohJcTGUlhRn3bJhYjNmhIium+b0QM8fmyt7NlZ8xuv7NDHIpiHTsLLzd5/dWnK/mIW3HpnVPU427gt1IEYAAMAC6oGGDZRmBy2XjltCvrEuLjPu5SIPynpBs1YCWCdc1JFWPTlMU8DodqEl59DaTu3T9T1KuNLynRulu48ZgwXb6tUZ8XWJdjmuRQIawzIQIwAAYAG1z5+/u33pqq62B195vblFZSb2wXjdVuqMsMBVPjXVc13RfomB0BIMdU73pMlIrS7GVi81QTE/28dULuaFxYw4GVxrRBy3LbZtYA3EjAAAgBfww841vZvQ8t3H6ZtV+8XzurTFSLQFMWImZsRSRVeDWfUsI06GR2htp26NKssIs0BNG9PPXSGWn589r+QOcIUqZoS5bXzpFeHFH78fwBywjAAAgA7/va47NUjXrjLqITB0Rna1eOAHMPlpngkxorVtxX5YUAlGwkXPymLH/fH3Y+cIp8cYWEYY/VvVoZb1anjMz9xSLlWzv3jOj8PcNr6ED2A1K0Ya1zIfkxLuwDICAAA6XNq9keSKuHfaGlNBiHpjsxnLSH6JcbCn7BrhB1szAztrzpd9sshjulF2rpMxI2yf1U0C5borWnEurCS9CH5+9nn543tZ90YKN42oRoiT8MdI3WhQzR8PDqbC0grJ1QSqgGUEAABspryKBIaVGAqRGHnjuh6SJaJ3M+3sHHkxVr6dlUq/eUBzz3kEA3u7+qk0/fZ+ljvJ6tUZ4T8v7xbRXkD5sn39VHrj+h7C2h2ibWgev2ilRWLixZ2k5UZ0yqQeTWtKqcD+8p4YWUaYZYcF4IJqYBkBAACDwZfVydBCPexY6UivHkwZQ9pn0KZnRtCcjTm0cu8J4XLy2MxKsf9wzyApA8eoxom0ry6X0OViNWaEiY7S05kt/LIsEPZ4eanuutSbT4iLoUu6Ket5mIVfl2QZEXxn74zsLX1ufxYbq/CtRygsgRgBAAAVIzrVl9wGZzSvsk70alZLMqm3qJNi7Kbhbv3XTTxXUivdnvlVOC/vSuGfs/oYeoOnOk7DbNwG27qdQVmRtioVbItxi5H6XNdeSYwUlLozX9SN5eQaHjxOSQQmmLTqdri7HPsp59bX8SnhCMQIAACoYBaH2fedqXi9+LFzhHEY6uGNL/Gerioi5jKR2ut+z0Ibel8X/FR37WUF0t66rScdKyiVerOIettMuqIL3frJSo91qY+JUwYLKYDVotYY2j6DfEFTC8XSQBUQIwAAYDFbgkc9AF7du4kUnNinRW3DeXl3kFpQWGlDrxVg2qVROm3IznW/rtonl+UYB7UwqpUcTwNaV/VvYTRMT6QDucV0Rc9GtHlWnjStdUYNmnXvIHrs2w30yHntaOQHy8VihJyBWYesaJFnL+1El3RrRE7y7V39ac/RQlvVeCMdiBEAALDBa9d2ownfb6J3RvXyEAa3qKqBaqFITVUN+HruFPVb6nlZnxzG+zf1ps+W7KUz29SlnUcK6KpejWlD9kmP9Rl5FXixw6wik6/tpnh/7gNn0f4TRYr9Yk33mtVJoZ/+b5BiXnXJeadiOdixtFLfY2R/z6Bfb+nVrLb0ANaBGAEAABtc3qMxXdqtkaWy6+qYBVE2TfVr7fXobfG9Ub1peMdM6XlmWiI9NKKd9LxvyzpV+yAYr42GcOameurijlJ67JizWnq8n5oYRx0axNHmA1VWEbXLhqe5IO7GPsoAYNQaC10gRgAAwCZWhIjwAswFhqqFil5Qqv571vfDTMO2mwcaW3vqplZVSmWoS7Gz4nE/rj1A9w9v45sAVj9mywDngRgBAAA/oR7z+SQVdTsTvUJkeuOunYqoTlkUMlIT6fNb+1Jygmc2ECsexx5qnNIQzMrEgoc3HcjTtMrAchK8QIwAAICf0Ovaq7ZO6A2cVuJJ1DSq5Vn91MleKoPaVAe2msGo4Jrp9URF0Tsje9GUP3fQLSasOCC4gBgBAAA/oVcOXi0HynQqZ+kN30aWkQbpSVK1U5bVctGbi6RpAW0y66B3hfV6mXSFdudktK8LXiBGAADAb2gHsKotIyWni4qZFRwsY2b7oVPClGI1A7m03Kq9coWkFmnJ1TgBoQ3ECAAABAimRViBrPziMmpaWzmwyhVORYiMH5/e0ocqKl0Ua6Ue/WkCaRkxciuNHqidglsrJZ4WPjJEyvYxA6scC4ITiBEAAPAToqZ6rIMrEwPqJnOlOm4aVsxMDVuXXkM7SzsWJLBgWCNLTxML1U6fuaQz3fbpCrrtTM/0ZBBYIEYAAMBPiMZ8LUuGyDIy+94zae6mHLpzcCtH9yuQlpFGNZMdC4Y1ommdZPr1gcGOrhM4A8QIAAD4CSuxGeUVnvN2bJgmPZzGyWwaszC30lcr99ETF3bwiKNh7iYQWVh3LgIAAPA5V/duLHW+9QeB8NKc1bYeTbmhJ9VOqS6UJiqWBiIDfOsAAOAnrAz6NZPjaen4oVLDOV8TCMuIFhAjkQnECAAABHG5eX8IBaea1TmB2cwYEF5AjAAAgJ9ok2nDyuFDLXLboBbUom4KXdO7MQULsIxEJghgBQAAH/P92IH019YjdPMA62XKfWkX+fdFHaVHMAHLSGQCywgAAPiY7k1q0n3D2njUEjHD4Lb1pP91VIGe4YpcA2RIu6rPDSIDWEYAACCIefS89tQqowYNbZ9BkcCVPRtJRd2Y+whEDhAjAAAQxCTFx9DIfs0oUmDBtO3qpwZ6N4CfgZsGAAAAAAEFYgQAAAAAoSdGpkyZQs2bN6fExETq27cvLV++3NRy06dPl0xwl112mZ3NAgAAACAMsSxGvvrqKxo3bhxNnDiRVq9eTd26daMRI0bQ4cOHdZfbs2cPPfTQQ3TmmWd6s78AAAAAiHQxMnnyZBozZgyNHj2aOnbsSFOnTqXk5GT68MMPNZepqKigG2+8kZ5++mlq2RKtmwEAAABgU4yUlpbSqlWraNiwYdUriI6WXi9ZskRzuWeeeYYyMjLo1ltvNbWdkpISysvLUzwAAAAAEJ5YEiNHjx6VrByZmZmK6ex1Tk6OcJlFixbRBx98QO+9957p7UyaNInS09PdjyZNmljZTQAAAACEED7NpsnPz6eRI0dKQqRu3bqmlxs/fjzl5ua6H/v27fPlbgIAAAAgVIqeMUERExNDhw4dUkxnr+vXr+8x/86dO6XA1Ysvvtg9rbKysmrDsbG0detWatWqlcdyCQkJ0gMAAAAA4Y8ly0h8fDz16tWL5s2bpxAX7HX//v095m/fvj1t2LCB1q5d635ccsklNGTIEOk53C8AAAAAsFwOnqX13nTTTdS7d2/q06cPvf7661RQUCBl1zBGjRpFjRo1kuI+WB2Szp07K5avWbOm9F89HQAAAACRiWUxcu2119KRI0dowoQJUtBq9+7dac6cOe6g1qysLCnDBgAAAADADFEul8tFQQ5L7WVZNSyYNS0tLdC7AwAAAAAHx++Q6Nor6yXUGwEAAABCB3ncNrJ7hIQYYSnCDAS8AgAAAKEHG8eZhSSk3TQsY+fAgQOUmpoqNdpzUrExgcPqmMD9g2PlJDi3cKx8Ac4rHKtQO6+YxGBCpGHDhrrxpCFhGWEfoHHjxj5bPzv4ECM4Vji3Agt+hzhWOK/C8zeoZxGRQdoLAAAAAAIKxAgAAAAAAkpEixFWcn7ixIkoPY9jhXMLv8OQANcsHKtwPa9CIoAVAAAAAOFLRFtGAAAAABB4IEYAAAAAEFAgRgAAAAAQUCBGAAAAABBQIk6MXHLJJdS0aVNKTEykBg0a0MiRI6XqrnoUFxfT2LFjqU6dOlSjRg268sor6dChQxTO7Nmzh2699VZq0aIFJSUlUatWraRo69LSUt3lzj77bKlKLv+48847KZyxe6wi8bxiPP/88zRgwABKTk6mmjVrmlrm5ptv9jivzjvvPIoE7BwvlpfAOquzaxw7J4cNG0bbt2+ncOf48eN04403SoW72LFiv8tTp07pLhMp16wpU6ZQ8+bNpbGvb9++tHz5ct35v/76a2rfvr00f5cuXWj27Nk+3b+IEyNDhgyhGTNm0NatW+nbb7+lnTt30lVXXaW7zAMPPEA//fST9OX89ddfkni54oorKJzZsmWLVIb/nXfeoU2bNtFrr71GU6dOpccff9xw2TFjxtDBgwfdj5deeonCGbvHKhLPKwYTaVdffTXdddddlpZj4oM/r6ZNm0aRgJ3jxX5zb7zxhnQeLlu2jFJSUmjEiBGSAA5nmBBhv8HffvuNfv75Z1qwYAHdfvvtFOnXrK+++orGjRsn3SStXr2aunXrJp0Phw8fFs6/ePFiuv766yUxt2bNGrrsssukx8aNG323k64I54cffnBFRUW5SktLhe+fPHnSFRcX5/r666/d0/755x+WDu1asmSJK5J46aWXXC1atNCdZ/Dgwa777rvPFekYHSucVy7XRx995EpPTzd1PG+66SbXpZde6opkzB6vyspKV/369V0vv/yy4nxLSEhwTZs2zRWubN68Wbour1ixwj3tl19+ka7v2dnZEX3N6tOnj2vs2LHu1xUVFa6GDRu6Jk2aJJz/mmuucV144YWKaX379nXdcccdPtvHiLOMqE16X3zxhWQCjYuLE86zatUqKisrk8ycMsx0xVw9S5YsoUgiNzeXateubTgfO6Z169alzp070/jx46mwsJAiDaNjhfPKOvPnz6eMjAxq166dZCU4duyYV99RuLJ7927KyclRXLNYbxBmmg/naxb7bMw107t3b/c0dgxYbzNmHYrUa1Zpaal0veHPB3ZM2Gut84FN5+dnMEuKL8+fkGiU5zSPPvoo/e9//5NOuH79+knmPC3Yjzo+Pt7DV5uZmSm9Fyns2LGD3nzzTXrllVd057vhhhuoWbNmUofG9evXS8eaucRmzpxJkYKZY4Xziiy7aJgLi8XlMNcqc4Gdf/750sUxJibG6+8snJCvS+waFUnXLPbZmFjliY2NlW4K9D53uF+zjh49ShUVFcLzgbmYRbDj5e/zJywsI4899phHAJL6wR/0hx9+WPKD/frrr9KFbNSoUVLAVyRg9VgxsrOzpcGA+a2Zb1UP5p9lCpoFPDH/7aeffkrfffedNICEGr4+VuGEnWNlheuuu04KPmfnFfNdsxuIFStWSNaSUMTXxyuc8PWxCqdrVigTFpaRBx98UIq216Nly5bu58wcxx5t27alDh06UJMmTWjp0qXUv39/j+Xq168vmblOnjypsI6wrAf2XrgfKxZUyYJ+mSvr3Xfftbw9ZhqWrQUsyySU8OWxivTzylvYuthvmJ1XQ4cOpVDDl8dLPn/YucSyaWTY6+7du1O4Hiv2udUBmeXl5ZI73spvKpSvWSLY74TddKsz9fSuNWy6lfmdICzESL169aSHHVgWBKOkpET4fq9evaR4knnz5kmplwxmwsvKyhKKl3A6Vuwunw2u7Bh89NFHkp/RKmvXrpX+8xfFUMGXxyqSzysn2L9/vxQzEornla+PF3NlsUGDnVuy+MjLy5PiJqxmMIXSsWK/GybuWXwE+30x/vjjD+kaLwuMcL9miWBhBux4sPOBWRUZ7Jiw1/fcc4/msWTv33///e5pLEPJp9cmVwSxdOlS15tvvulas2aNa8+ePa558+a5BgwY4GrVqpWruLhYmmf//v2udu3auZYtW+Ze7s4773Q1bdrU9ccff7hWrlzp6t+/v/QIZ9hxaN26tWvo0KHS84MHD7of/Dz8sdqxY4frmWeekY7R7t27pUylli1bus466yxXOGPnWEXqecXYu3ev9Bt8+umnXTVq1JCes0d+fr57HnasZs6cKT1n0x966CEpe42dV7///rurZ8+erjZt2rh/t+GM1ePFePHFF101a9aUfoPr16+XMpFYdldRUZErnDnvvPNcPXr0kH5nixYtks6R66+/3hXp16zp06dL2VQff/yxlHV0++23S+dHTk6O9P7IkSNdjz32mHv+v//+2xUbG+t65ZVXpOzRiRMnSlmlGzZs8Nk+RpQYYT/KIUOGuGrXri19Mc2bN5cGBHaCyrATkmm0P//80z2N/YDvvvtuV61atVzJycmuyy+/XDHQhGsaITsOoofWscrKypJ+xPLxZQP0ww8/7MrNzXWFM3aOVaSeV3KaruhY8ceGvWbHlVFYWOg699xzXfXq1ZMuiM2aNXONGTPGfSENd6weLzm998knn3RlZmZKv0UmlLdu3eoKd44dOyaJDyba0tLSXKNHj1aItki+Zr355pvSzU98fLyU6stuzvn0Znae8cyYMcPVtm1baf5OnTq5Zs2a5dP9i2J/fGd3AQAAAACIgGwaAAAAAIQuECMAAAAACCgQIwAAAAAIKBAjAAAAAAgoECMAAAAACCgQIwAAAAAIKBAjAAAAAAgoECMAAAAACCgQIwAAAAAIKBAjAAAAAAgoECMAAAAACCgQIwAAAACgQPL/+zUf4Q0XpCAAAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.plot(lrei, lossi)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.14.2" } }, "nbformat": 4, "nbformat_minor": 5 }