{ "cells": [ { "cell_type": "markdown", "id": "64f38841-429e-4473-b525-c4a836eceea6", "metadata": {}, "source": [ "# `micrograd`: construction d'une bibliothèque de rétropropagation du gradient (partie 1)\n", "\n", "Ce notebook est dérivé du travail d'[Andrej Karpathy](https://karpathy.ai/) et reprend pas à pas les étapes exposées dans sa première séance de cours sur la construction d'un outil en Python pour le calcul du gradient et sa propagation arrière:\n", "\n", "[The spelled-out intro to neural networks and backpropagation: building micrograd](https://www.youtube.com/watch?v=VMj-3S1tku0&list=PLAqhIrjkxbuWI23v9cThsA9GvCAUhRvKZ)\n", "\n", "Voici les ressources originales associées à la vidéo youtube d'Andrej:\n", "\n", "- micrograd on github: https://github.com/karpathy/micrograd\n", "- notebook original: https://github.com/karpathy/nn-zero-to-hero/tree/master/lectures/micrograd\n", "- exercices: https://colab.research.google.com/drive/1FPTx1RXtBfc4MaTkf7viZZD4U2F9gtKN?usp=sharing" ] }, { "cell_type": "markdown", "id": "dfb3ee5b-a9b4-42db-b779-d01af8978193", "metadata": {}, "source": [ "## Importation de paquetages tiers" ] }, { "cell_type": "code", "execution_count": 1, "id": "28e8d8c5-2af4-46ea-85ac-9d283552c5d7", "metadata": {}, "outputs": [], "source": [ "# Imports de la librairie standard Python\n", "import math" ] }, { "cell_type": "code", "execution_count": 2, "id": "2826a6c9-789f-4b4f-8680-4dacc46d13a0", "metadata": {}, "outputs": [], "source": [ "# Imports spécifiques (doivent être présent dans l'environnement Python de ce notebook)\n", "import numpy as np\n", "import matplotlib.pyplot as plt\n", "%matplotlib inline\n", "from graphviz import Digraph" ] }, { "cell_type": "markdown", "id": "1ac17d7a-5ad7-4cb1-955f-c0a83e03b7df", "metadata": {}, "source": [ "## Introduction" ] }, { "cell_type": "code", "execution_count": 3, "id": "400227c3-43ed-4e5e-8413-3fc2ac3bf7b2", "metadata": {}, "outputs": [], "source": [ "# Définissons une fonction y <- f(x) arbitraire \n", "def f(x):\n", " return 3 * x**2 - 4*x + 5" ] }, { "cell_type": "code", "execution_count": 4, "id": "57dbc49b-89cd-4c9c-8743-781e9089caac", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "20.0" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "f(3.0)" ] }, { "cell_type": "code", "execution_count": 5, "id": "be6940ed-4b11-4ce6-9078-ca3ba916010b", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([-5. , -4.75, -4.5 , -4.25, -4. , -3.75, -3.5 , -3.25, -3. ,\n", " -2.75, -2.5 , -2.25, -2. , -1.75, -1.5 , -1.25, -1. , -0.75,\n", " -0.5 , -0.25, 0. , 0.25, 0.5 , 0.75, 1. , 1.25, 1.5 ,\n", " 1.75, 2. , 2.25, 2.5 , 2.75, 3. , 3.25, 3.5 , 3.75,\n", " 4. , 4.25, 4.5 , 4.75])" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Tableaux de points entre -5 et 5 pour les valeurs de x, avec un pas de 0.25\n", "xs = np.arange(-5, 5, 0.25)\n", "xs" ] }, { "cell_type": "code", "execution_count": 6, "id": "4936a849-2d31-4ec1-87eb-7605744c56c0", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([100. , 91.6875, 83.75 , 76.1875, 69. , 62.1875,\n", " 55.75 , 49.6875, 44. , 38.6875, 33.75 , 29.1875,\n", " 25. , 21.1875, 17.75 , 14.6875, 12. , 9.6875,\n", " 7.75 , 6.1875, 5. , 4.1875, 3.75 , 3.6875,\n", " 4. , 4.6875, 5.75 , 7.1875, 9. , 11.1875,\n", " 13.75 , 16.6875, 20. , 23.6875, 27.75 , 32.1875,\n", " 37. , 42.1875, 47.75 , 53.6875])" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Calcul des y pour ces points\n", "ys = f(xs)\n", "ys" ] }, { "cell_type": "code", "execution_count": 7, "id": "a91f07c9-5727-4024-9dcc-ee670a5ddd93", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[]" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAigAAAGdCAYAAAA44ojeAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjgsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvwVt1zgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAP7JJREFUeJzt3Qd0VVX6/vEnvScQIAkhCYQaem9iF0HFggVlxIYMNnAE28jMiDM/HbGN+scC6lhAQRQVFUUcRIVBQhek9xIIKRBSSEjPf50dkgFFpdybc8v3s9ZdObeQbE9i7pN99n5fn6qqqioBAAC4EF+7BwAAAPBzBBQAAOByCCgAAMDlEFAAAIDLIaAAAACXQ0ABAAAuh4ACAABcDgEFAAC4HH+5ocrKSqWnpysiIkI+Pj52DwcAAJwEqzZsQUGB4uPj5evr63kBxQoniYmJdg8DAACchrS0NCUkJHheQLFmTmr+AyMjI+0eDgAAOAn5+flmgqHmfdzjAkrNZR0rnBBQAABwLyezPINFsgAAwOUQUAAAgMshoAAAAJdDQAEAAC6HgAIAAFwOAQUAALgcAgoAAHA5BBQAAOByCCgAAMD9A8rChQt1xRVXmEY/ViW4Tz/99BeNgMaPH6/GjRsrJCRE/fv319atW497TU5OjoYNG2aqwNarV08jRozQ4cOHz/y/BgAAeGdAKSwsVOfOnfXKK6+c8PlnnnlGEydO1OTJk7V06VKFhYVp4MCBKi4urn2NFU7Wr1+vefPm6YsvvjCh54477jiz/xIAAOAxfKqsKY/T/cc+Ppo1a5YGDx5s7lufyppZeeCBB/Tggw+ax/Ly8hQbG6t33nlHQ4cO1caNG9WuXTstX75cPXr0MK+ZO3euLrvsMu3du9f8+5NpNhQVFWU+N714AABwD6fy/u3QNSg7d+5URkaGuaxTwxpI7969lZqaau5bH63LOjXhxGK93tfX18y4nEhJSYn5jzr25gybMvL111lrNXtNulM+PwAAODkODShWOLFYMybHsu7XPGd9jImJOe55f39/RUdH177m5yZMmGCCTs3NatXsDPM3Zmna0j16Z/Eup3x+AADgQbt4xo0bZ6aDam5paWlO+TpDeiTI39dHK3cf0uaMAqd8DQAAUMcBJS4uznzMzMw87nHrfs1z1sesrKzjni8vLzc7e2pe83NBQUHmWtWxN2eIiQhW/7bVsz/vL9vjlK8BAADqOKAkJyebkDF//vzax6z1Itbakr59+5r71sfc3FytXLmy9jXffvutKisrzVoVu/2hd5L5+MmqvSouq7B7OAAAeCX/U/0HVr2Sbdu2HbcwdvXq1WYNSVJSksaMGaMnnnhCrVq1MoHl0UcfNTtzanb6tG3bVpdccolGjhxptiKXlZVp9OjRZofPyezgcbZzWjZUk3oh2pd7RHPW7tc13RLsHhIAAF7nlGdQVqxYoa5du5qb5f777zfHVnE2y8MPP6x7773X1DXp2bOnCTTWNuLg4ODazzFt2jSlpKTooosuMtuLzz77bL3++utyBb6+PvpDr+pFuFzmAQDADeug2MXZdVCy8ovV96lvVVFZpXljz1Wr2AiHfw0AALxNvl11UDxFTKS1WLZ6K/T7y5yzYwgAAPw6Asqv+EOv6sWyH7NYFgCAOkdA+RXntGpkFsvmHSnTV+v21+13BQAAL0dA+RV+vj4a2vPoYtmlXOYBAKAuEVB+w5AeiSaoLNuVo21ZVJYFAKCuEFB+Q1xUsC5MYbEsAAB1jYDyO25ksSwAAHWOgPI7zm3dSPFRwcotKtPX60/cbRkAADgWAeV3WGtQbuhZveV4+lIaCAIAUBcIKCfh+p4J8vWRlu7M0fbsw87/rgAA4OUIKCehcVRI7WLZGcuYRQEAwNkIKKdYWfajlXtVUl7hzO8JAABej4Byks5r3UiNo4J1yCyWzfT6HxwAAJyJgHKS/P18dX2PmsqyXOYBAMCZCCin4PqeiWaxbOqOg9rBYlkAAJyGgHIKrOaB57c5ulh2Of15AABwFgLKKWKxLAAAzkdAOUUXtGmk2Mgg5RSW6j8slgUAwCkIKKexWPaGmsWy1EQBAMApCCinuVjWx0davP2gdh4odPx3BQAAL0dAOQ0J9UN1futG5njGcrYcAwDgaASUM10su2KvSssrHfk9AQDA6xFQTpPVmycmIkgHC0s1d32G1/8gAQDgSASUM1gsO/ToLMp7qbsd+T0BAMDrEVDOwI29kuTn66Nlu3K0KSPf63+YAABwFALKGYiLCtbA9rHmeCqzKAAAOAwB5Qzd3KeZ+fjpj/uUX1zmiO8JAABej4Byhvo0j1br2HAVlVbo45V7vf4HCgAARyCgnCEfHx/d3KepOX53yW5VVVU54vsCAIBXI6A4wNXdEhQe5K8d2YX6YdtBR3xKAAC8GgHFAaxwck23JuZ4auouR3xKAAC8GgHFQWou83yzMVP7co846tMCAOCVCCgO0io2wiyYrayS3l9Kfx4AAM4EAcWBbunbrLaBYEl5hSM/NQAAXoWA4kAXt4tVbGSQDhwu1dx19OcBAOB0EVAcKMDPVzf2ql6LQmVZAABOHwHFwf7QK1H+vj5aufuQ1qfnOfrTAwDgFQgoDhYTGaxLOsSZ43fpzwMAwGkhoDhxseynq/cpr4j+PAAAnCoCihP0bFZfKXERKi6r1MyVac74EgAAeDQCirP68/StXiw7bekeVVrFUQAAwEkjoDjJ4C5NFBHkr50HCrVo2wFnfRkAADwSAcVJwoL8dW33BHPMlmMAAE4NAcWJbjran+fbTZnae6jImV8KAACPQkBxopYx4erXsoHpz2OtRQEAACeHgOJkN/ep3nL8wfI0FZfRnwcAgJNBQHGy/m1jFB8VrJzCUs1Zu9/ZXw4AAI9AQHEyf6s/T+8kc8xiWQAATg4BpQ7c0DNJAX4+Wp2Wq7V76c8DAMDvIaDUgUYRQbqsY2NzPDV1V118SQAA3BoBpY7ccrSy7Gdr0nXwcEldfVkAANwSAaWOdEuqr84JUSotr9R0thwDAPCbCCh12J/n9rOTzfHUJbtNUAEAACdGQKlDl3ZorNjIIGUXlOjLtel1+aUBAHArBJQ6FOjvq1v6Vhdue3PRTlVV0eUYAIATIaDUsRt7JSnI31fr9uVr+a5Ddf3lAQBwCwSUOlY/LFDXdKvucvzWop11/eUBAHALBBQb3N6v+jLPfzZkKC2HLscAAPwcAcUGrWIjdE6rhqbL8ZTFFG4DAMDpAaWiokKPPvqokpOTFRISohYtWujxxx8/bkGodTx+/Hg1btzYvKZ///7aunWrvEnNlmOry/HhknK7hwMAgGcHlKefflqTJk3Syy+/rI0bN5r7zzzzjF566aXa11j3J06cqMmTJ2vp0qUKCwvTwIEDVVxcLG9xXqtGatEoTAUl5Zq5Is3u4QAA4NkBZfHixbrqqqs0aNAgNWvWTNddd50GDBigZcuW1c6evPjii/rb3/5mXtepUydNnTpV6enp+vTTT+UtfH19NLxf9SzKO4t3qcK63gMAAJwTUM466yzNnz9fW7ZsMffXrFmjRYsW6dJLLzX3d+7cqYyMDHNZp0ZUVJR69+6t1NTUE37OkpIS5efnH3fzBNd0a6KokADtPlikbzdl2T0cAAA8N6A88sgjGjp0qFJSUhQQEKCuXbtqzJgxGjZsmHneCieW2NjY4/6ddb/muZ+bMGGCCTE1t8TERHmC0EB//aFXkjlmyzEAAE4MKB9++KGmTZum6dOna9WqVZoyZYqee+458/F0jRs3Tnl5ebW3tLQ0j+py7Ofro9QdB7Uh3TNmhgAAcLmA8tBDD9XOonTs2FE333yzxo4da2ZBLHFxceZjZmbmcf/Oul/z3M8FBQUpMjLyuJuniK8Xoss6NjbHb/1A4TYAAJwSUIqKiuTre/yn9fPzU2Vldfdea/uxFUSsdSo1rDUl1m6evn37enXhts9Xp5tGggAAeDuHB5QrrrhC//znP/Xll19q165dmjVrlp5//nldffXV5nkfHx+zJuWJJ57Q559/rrVr1+qWW25RfHy8Bg8eLG/UNam+uibVU2lFpaYt3W33cAAAsJ2/oz+hVe/EKtR2zz33KCsrywSPO++80xRmq/Hwww+rsLBQd9xxh3Jzc3X22Wdr7ty5Cg4Olre6vV+y7t3zo95bslt3n99CQf5+dg8JAADb+FQdW+LVTViXhKzdPNaCWU9Zj1JeUalzn/lO6XnFem5IZ13XvbqhIAAAnuJU3r/pxeMi/P18dctZ1WtR3ly087jWAAAAeBsCigsZ2jNRIQF+2rg/X0t25Ng9HAAAbENAcSH1QgN1bfcm5pgtxwAAb0ZAcTE1/Xm+2Zip3QcL7R4OAAC2IKC4mBaNwnVBm0aylqBYTQQBAPBGBBQXdPvZ1bMoM1fsVUFxmd3DAQCgzhFQXNDZLRuqdWy4DpeU6/1le+weDgAAdY6A4oKsart/PLu5OX5r0S6Vlle3CQAAwFsQUFzUVV3jFRMRpIz8Yn2+Jt3u4QAAUKcIKC7KKnVfsxbl9YXbVVlJ4TYAgPcgoLiwG3snKTzIX1syD+v7LVl2DwcAgDpDQHFhkcEBGtY7yRxPXrDD7uEAAFBnCChuULgtwM9Hy3bmaNWeQ3YPBwCAOkFAcXFxUcEa3KW6/P3rzKIAALwEAcUN3Hle9ZbjrzdkaEf2YbuHAwCA0xFQ3EDLmAj1bxtryt+/8d+ddg8HAACnI6C4ibuOzqJ8vGqvsgqK7R4OAABORUBxEz2aRat70/qmquwUmggCADwcAcWN3Hlu9SzKu6m7TZ8eAAA8FQHFjVjrUJo3ClN+cblm0EQQAODBCChuxNfXp3YW5c1FO2kiCADwWAQUNzO4axM1igjS/rxizaaJIADAQxFQ3LGJYL/qJoKvLdyuKmvvMQAAHoaA4u5NBDdn2z0cAAAcjoDihqJCAkxIsUxesN3u4QAA4HAEFDc1vF8z00Rw6c4c/UgTQQCAhyGguKnGUSG6qqaJ4MIddg8HAACHIqC4sTuObjmeuz5DOw8U2j0cAAAchoDixlrHRuiilJijTQSZRQEAeA4Cipu787wW5uNHK/cqu6DE7uEAAOAQBBQ317NZfXVNqkcTQQCARyGguDkfH6v8ffUsytTUXcovLrN7SAAAnDECigcY0C5WLWPCTRNBq9MxAADujoDiIU0ER13QoraJYFFpud1DAgDgjBBQPMQVneLVtEGocgpLNX3pHruHAwDAGSGgeAh/P1/dc371LMprC3eouKzC7iEBAHDaCCge5OquCWpSL8RsN/5wRZrdwwEA4LQRUDxIoL+v7jqvurrs5O+3m63HAAC4IwKKhxnSI1ExEUFKzyvWJ6v22j0cAABOCwHFwwQH+NX26Hn1++0qr2AWBQDgfggoHujG3klqEBaoPTlF+nxNut3DAQDglBFQPFBooL9GnJNsjl/+bpsqKqvsHhIAAKeEgOKhbu7TVFEhAdqRXaiv1u23ezgAAJwSAoqHiggO0PB+zczxy99uUyWzKAAAN0JA8WDDz0pWeJC/NmUU6JuNmXYPBwCAk0ZA8WBRoQG6pW9Tc/zSt9tUVcVaFACAeyCgeLgRZycrJMBPa/fl6fst2XYPBwCAk0JA8XANwoM0rHeSOX5p/lZmUQAAboGA4gWswm1WGfxVe3KVuv2g3cMBAOB3EVC8QExksIb2TKxdiwIAgKsjoHiJO89roQA/H6XuOKgVu3LsHg4AAL+JgOIlmtQL0bXdEswxsygAAFdHQPEi95zfUn6+PlqwJVtr0nLtHg4AAL+KgOJFkhqE6qrO8bU9egAAcFUEFC9zzwUt5eMjzduQqQ3p+XYPBwCAEyKgeJmWMeG6rGNjczxx/la7hwMAwAkRULzQmItamVmUuesztHZvnt3DAQDgFwgoXqhVbIQGd2lijp+ft9nu4QAA8AsEFC9130WtzI6e7zZna+XuQ3YPBwAA5weUffv26aabblKDBg0UEhKijh07asWKFbXPW111x48fr8aNG5vn+/fvr61bWQ9Rl5o1DNN1R+uiMIsCAPD4gHLo0CH169dPAQEB+uqrr7Rhwwb961//Uv369Wtf88wzz2jixImaPHmyli5dqrCwMA0cOFDFxcWOHg5+w70XtTTVZX/YdlCLtx/gXAEAXIZPlTWd4UCPPPKIfvjhB/33v/894fPWl4uPj9cDDzygBx980DyWl5en2NhYvfPOOxo6dOjvfo38/HxFRUWZfxcZGenI4Xud8Z+t09TU3erRtL5m3tVXPtbqWQAAnOBU3r8dPoPy+eefq0ePHhoyZIhiYmLUtWtXvfHGG7XP79y5UxkZGeayTg1rsL1791ZqauoJP2dJSYn5jzr2BscYdUFLBfn7asXuQ6bCLAAArsDhAWXHjh2aNGmSWrVqpa+//lp33323/vSnP2nKlCnmeSucWKwZk2NZ92ue+7kJEyaYEFNzS0ys7syLMxcbGayb+zQ1x8/P22JmuAAA8LiAUllZqW7duunJJ580syd33HGHRo4cadabnK5x48aZ6aCaW1pamkPH7O3uOr+FQgP99NPePFNhFgAAjwso1s6cdu3aHfdY27ZttWfPHnMcFxdnPmZmHv9GaN2vee7ngoKCzLWqY29wnIbhQRrer1ntLEplJbMoAAAPCyjWDp7Nm48v/rVlyxY1bVp9GSE5OdkEkfnz59c+b60psXbz9O3b19HDwUm645wWigj216aMAn25dj/nDQDgWQFl7NixWrJkibnEs23bNk2fPl2vv/66Ro0aZZ63domMGTNGTzzxhFlQu3btWt1yyy1mZ8/gwYMdPRycpKjQAI08p7k5fuGbLSqvqOTcAQA8J6D07NlTs2bN0vvvv68OHTro8ccf14svvqhhw4bVvubhhx/Wvffea9anWK8/fPiw5s6dq+DgYEcPB6fAusxTLzRAO7IL9enqdM4dAMBz6qDUBeqgOM/kBdv11FeblBgdom8fOF8BfnRDAAB4QB0UuLdb+jY1i2bTco5o5oq9dg8HAOClCCg4Tmigv0Zd0MIcv/TtVhWXVXCGAAB1joCCX/hDryQ1jgrW/rxivb+sens4AAB1iYCCXwgO8NPoC1ua41e+264jpcyiAADqFgEFJzSke6JZKHvgcImmpu7iLAEA6hQBBScU6O+r+y5qXbuzp6C4jDMFAKgzBBT8qsFd4tW8UZgOFZXp7R+YRQEA1B0CCn6Vv5+vxvavnkV5Y+EOHSos5WwBAOoEAQW/aVDHxmrbOFIFJeV6+bttnC0AQJ0goOC3f0B8fTTu0hRzbC2WTcsp4owBAJyOgILfdW7rRjqnVUOVVVTpuf8c36kaAABnIKDgpPz5khT5+EifrU7X2r15nDUAgFMRUHBSOjSJ0tVdmpjjJ+dslBv2mAQAuBECCk7a/QNam/ooqTsO6vst2Zw5APBAVVVV2pZVYPcwCCg4eQn1QzX8rGbm+Kk5m1RRySwKAHiaL37ar4tfWKjHPltn6ziYQcEpuef8looKCdDmzAJ9vGovZw8APEhxWYWenrtJ1lX86LAgW8dCQMEpiQoN0OgLqhsJPv+fLTQSBAAPMmXxLu09dESxkUEaeW6yrWMhoOCU3dy3qZrUC1FGfrHe+mEnZxAAPEBOYWltQc4HB7RRaKC/reMhoOCUBQf46aGBbczxpO+36+DhEs4iALi5//fNFhUUl6td40hd2y3B7uEQUHB6ruwcr/bxkTpcUq6XvqUEPgC4s+3ZhzVt6R5z/LdBbU0Vcbsxg4LT+8Hx9dFfLmtrjt9bslu7DhRyJgHATU2Ys0nllVW6KCVGZ7VsKFdAQMFp69eyoc5r3cj8UD9LCXwAcEup2w/qm42Z8rN6rx39w9MVEFBwRh65tLoE/pc/7dePew5xNgHAjVRWVumfczaY4xt7JallTLhcBQEFZ6TtMYupJnxl7Z2neBsAuItZP+7Tun35Cg/y15j+reRKCCg4Y/df3FpB/r5atjNH327K4owCgBs4UlpR26H+ngtaqEG4vYXZfo6AgjMWXy9Et59dXdDnqa82qbyikrMKAC7uzUU7tD+v2NS1ur2fvUXZToSAAoe4+/wWqh8aoK1Zh/XRSkrgA4AryyooNnWsLA9f0sbUt3I1BBQ4RGRwgO69sPr65fPztqiotJwzCwAu6oV5W1VYWqHOCVG6olO8XBEBBQ5zU5+mSooOVVZBid5YSAl8AHBFmzMK9MHyo0XZLm/nEkXZToSAAocJ9Pc1U4WWSQu2KT33CGcXAFzMk3M2qrJKuqR9nHo2i5arIqDAoQZ1bKxezaJVXFZpth0DAFzHwi3ZWrAlWwF+PqaOlSsjoMChfHx89NiV7WTNGM5ek262HgMA7FdRWWVmTyw392mmZg3D5MoIKHC49vFRGtoryRz//fP15n8KAIC9Zq5I06aMAkWFBOhPF7V0+W8HAQVO8eCANooM9teG/fmacXQxFgDAHoUl5frXvC3m+N4LW6peaKDLfysIKHCK6LBAjb24tTl+7uvNyisq40wDgE1eW7hD2QUlZqflzX2busX3gYACp247bhUTrkNFZXrhm+rkDgCoW/tyj+j1hdVF2ayFsUH+rleU7UQIKHCaAD9fPXZFe3P87pLd2pJZwNkGgDr2zy83mJ2V1g7LSzvEuc35J6DAqc5u1VAD28eahbL/N3sD3Y4BoA4t2npAc9ZmyM/XR/+4qr3ZaekuCChwur8NameKuC3adkD/2ZDJGQeAOlBaXqnHPl9njm/u01RtG0e61XknoMDpEqNDdcc5zc3xE2aqsYKzDgBO9vYPO7U9u1ANw/+3acGdEFBQJ+65oIXiIoOVlnNE//7vDs46ADhRRl6xJs7fao7/fEmKqX3ibggoqBOhgf4ad1l1WeVXvtuu/Xn06QEAZ3lyzkbTrbhrUj1d2y3BLU80AQV15srO8erZrL6OlFXoKfr0AIBTLNlxUJ+vSZe1Hvbxqzq4bLfi30NAQd326bnCWkUufbY6XSt20acHAByprKJSj3223hzf2CtJHZpEue0JJqCgTln/swztmWiO/z6bPj0A4Ejvpu7W5swC1Q8N0EMD27j1ySWgwJY+PRHB/lq3L18frkjjOwAADpBVUKwXjvbbeWhgilv02/ktBBTUuQbhQRrbv3rL27NWn54j9OkBgDP19FebVVBSrk4JUbrh6Ey1OyOgwBZWsyqrT09OYan+3zfVW+EAAKdn5e4cfbxqrzn+x5XtTeVYd0dAgW19esZf0c4cT03dpc0Z9OkBgNNRUVml8UcXxt7QI1Fdk+p7xIkkoMA257RqZPr0lFdW6S+z1qqysorvBgCcounL9mh9er4ig/318CXuvTD2WAQU2OrvV7ZXWKCfVu4+pA9YMAsApySnsFTPfb3ZHD84sI1Z4+cpCCiwVeOoED0woDrxT5izUdkFJXxHAOAkPfv1JrPRwGoEaNU98SQEFNju1rOaqWOTKOUXl5tmggCA37cmLVczlleXanj8qvby9/Ost3TP+q+BW7JWmz95dUf5Hq0wu3BLtt1DAgCXVmkWxq5TVZV0Tdcm6tEsWp6GgAKX0DEhysykWB79bJ2KyyrsHhIAuKwPVqRpzd48hQf565GjjVg9DQEFLsNaixIXGazdB4v08rfb7B4OALhsxdgJczaa4zH9WykmIlieiIACl2H9JWDt6rG8tnC7tmRSGwUAfu4fn28wa/astXu3HZ159kQEFLgUqy5K/7axKquo0l+pjQIAx5m3IVNfrt1v1u5NuKajxy2MPZbn/pfBLfn4+OgfV7VXaKCflu86pJkraSYIAJaC4jI9+uk6c/zHc5JNd3hP5vSA8tRTT5k3nTFjxtQ+VlxcrFGjRqlBgwYKDw/Xtddeq8zMTGcPBW6iSb0Q3X9xdTPBJ+ds0oHD1EYBgGe/3qyM/GI1bRCqMRdV/470ZE4NKMuXL9drr72mTp06Hff42LFjNXv2bM2cOVMLFixQenq6rrnmGmcOBW7Guq7arnGkKUD0zy+rF4MBgDc3A3x3yW5zPOHqjgoJ9JOnc1pAOXz4sIYNG6Y33nhD9ev/r3FRXl6e3nzzTT3//PO68MIL1b17d7399ttavHixlixZ4qzhwM1Y11Wt66s+PtKsH/dp0dYDdg8JAGxRUl6hP3+81tQ8GdI9QWe1bOgV3wmnBRTrEs6gQYPUv3//4x5fuXKlysrKjns8JSVFSUlJSk1NPeHnKikpUX5+/nE3eL7OifV0S5+m5vhvn66lNgoArzTp++3alnVYDcMD9ddBbeUtnBJQZsyYoVWrVmnChAm/eC4jI0OBgYGqV6/ecY/Hxsaa507E+jxRUVG1t8TERGcMGy7ogYFtFBsZpF0Hi/Tqd9RGAeBdtmYW6JWjv/seu6K96oUGyls4PKCkpaXpvvvu07Rp0xQc7JjiMePGjTOXhmpu1teAd4gMDtDfr6iujTJpgfVXBLVRAHhPOftHPllryi5clBKjyzs1ljdxeECxLuFkZWWpW7du8vf3NzdrIezEiRPNsTVTUlpaqtzc3OP+nbWLJy4u7oSfMygoSJGRkcfd4D0u6RBn/ue0/if9yyyr90SV3UMCAKebtmyPVu4+pLBAPz0+uIPZEetNHB5QLrroIq1du1arV6+uvfXo0cMsmK05DggI0Pz582v/zebNm7Vnzx717dvX0cOBB9VGCQnw07KdOZq5Yq/dQwIAp9qfd0RPf7XJHD98SYri64V43Rn3d/QnjIiIUIcOHY57LCwszNQ8qXl8xIgRuv/++xUdHW1mQ+69914TTvr06ePo4cBDJNQP1diLW5m6KE98uUHntm6kuCjP7D8BwLtZs8SPfrpeh0vK1TWpnm46ulnA29hSSfaFF17Q5Zdfbgq0nXvuuebSzieffGLHUOBGbu+XrM4JUaYHxbhPfuJSDwCP9NW6DH2zMVMBfj56+tpOpqy9N/KpcsML+tY2Y2s3j7VglvUo3reifdDERSqtqNQz13XS9T3Y0QXAc+QVlan/CwuUXVCiP13YUvcPaCNPcirv3/TigVtpFRuh+wdUl3h+fPYGpecesXtIAOAwE77aaMJJi0ZhGnVhS68+swQUuJ2R5zQ312ULSsrNFjw3nAQEgF9I3X5QM5ZXl9F46tpOCvL3/HL2v4WAArdjXY99bkhnBfn7auGWbH1w9H9oAHBXxWUV+susteZ4WO8k9WwWLW9HQIFbatEoXA8NrL42+8SXG7X3UJHdQwKA0/b03E3aeaDQVM7+86UpnEkCCtzZ8H7J6tG0vtmK9+eP2dUDwD0t3nZAb/+wyxxbu3asCtpgBgVufqnH2skTHOCrH7Yd1LSle+weEgCckrwjZXpw5praSzvnt4nhDB7FJR64teaNwvXwwOrp0CfnbFRaDpd6ALiPf8xer/S8YjVtEKq/XOY9nYpPBgEFbu+2s5qpV7NoFZVW6OGPfjINtgDA1c1dt1+frNonqw7b89d3VliQw4u7uzUCCtyer6+Pnh3SyfTqSd1xUO8t3W33kADgN2UVFJvmp5a7zmuh7k3ZtfNzBBR4hKYNwjTusupLPRPmbNLug4V2DwkATsiq3fSXT9Yqp7BUbRtHakz/6uKTOB4BBR7jpt5N1bd5Ax0pq9BDXOoB4KKsjuzfbMxSoJ+vXrihswL9eSs+Ec4KPOpSj7WrJzTQT8t25mhKavW2PQBwFdZCfmthrOWBAa2VEvfb/Wi8GQEFHiUx+n8r4WsKHwGAK6iorNIDH65RYWmFejarrz+e09zuIbk0Ago8jlVL4OyWDVVcVqmHZq4xvxQAwG5vLdqpZbtyzCzvv4Z0MbWc8OsIKPA4Pj4+eurajgoP8teK3Yc0ecF2u4cEwMttzijQs19vNsePXt5OSQ1C7R6SyyOgwCMl1A/VY1e0M8fPz9uiVXsO2T0kAF6qtLxSYz9YrdKKSl2YEqOhPRPtHpJbIKDAY13XPUFXdo43l3j+9P6Pyi8us3tIALzQxPlbtWF/vuqHBpjZXWuWF7+PgAKPZf0SeOLqDkqMDtHeQ0dM3QGr/gAA1JWVuw/p1e+3meN/Xt1RMRHBnPyTRECBR7O6gk4c2lX+vj764qf9mrlyr91DAuAlikrL9cCHq2Wt07+6axNd1rGx3UNyKwQUeLyuSfV1/4DqSo2PfbZe27MP2z0kAF7giS83atfBIsVFBuvvV7a3ezhuh4ACr3DXuS3Ur2V1ldl7p/+okvIKu4cEwIN98VO6pi/dY46fG9JZUSEBdg/J7RBQ4DVVZp+/vouiwwLNYrWnv6re7gcAjrbrQKEe+XitOb7n/BY6u1VDTvJpIKDAa8RGBuu5IZ3M8Vs/7NS3mzLtHhIAD2PNzo5+f5UOl5SbarH3X0wjwNNFQIFXuTAlVred1cwcPzjzJ2XlF9s9JAAe5MkvN2rdvuotxRP/0FX+frzNni7OHLzOI5emmBbnVqvzsdYKe0rhA3CAr9bu15TU3ebYuqTcOCqE83oGCCjwOsEBfnrpD10VEuCnH7Yd1GsLd9g9JABubs/BIj380U/m+M7zmuuClBi7h+T2CCjwSi1jwvX3K6tL4f/rP5v1I6XwAZzhupOCknJ1b1pfDw5ow7l0AAIKvNb1PRI1qFNjlVul8GdQCh/A6Xnqq036aW+e2UpsrTsJYN2JQxBQ4NWl8J+8uqOa1AtRWs4R/W3WOkrhAzglX6/P0Ns/7DLH/xrS2fw+gWMQUODVqv/i6SI/Xx99viZdH1EKH8BJSssp0kMz15jjkeckq3+7WM6dAxFQ4PW6N43W2P6tzHl49LN12rg/3+vPCYDfVlpeqXtNl/RydUmsp4cvSeGUORgBBZB09/ktdU6rhiouq9Sd765UblEp5wXAr3pm7iatTstVZLC/Xr6RdSfOQEABJHOJx+p6nFA/RHtyinTfjNWqoD4KgBP4ZkOm/r1oZ22fnYT6oZwnJyCgAEfVDwvUazd3V5C/rxZsydaL32zh3AA4zr7cI3rg6LqT2/sla0D7OM6QkxBQgGO0j4/SU9d2NMcvfbvNrNAHAEtZRaXunb5KeUfK1DkhylSlhvMQUICfubprQm2/ngc+XKPt2Yc5RwD0+BcbtGpPriLMupNuCvTnLdSZOLvACfx1UFv1ahZtOpJai2atjwC81/vL9mhq6m75+EgvXN9FidGsO3E2AgpwAlYlyJeHdVVsZJC2ZR3Wgx+uoYgb4KWW78rR+M/WmeMHLm5NvZM6QkABfkVMRLAm3dRdAX4+mrs+Q5MWbOdcAV7GWhR717srVVZRZVpjjLqgpd1D8hoEFOA3dEuqr79f2d4cP/f1Zi3cks35ArzEkdIK3TF1hQ4Wlqpd40g9e10n0yIDdYOAAvyOG3sl6YYeibLKolhNBa3y1gA8W1VVlR76aI3Wp+erQVig3ri1h0ID/e0ellchoAC/w/qL6R9XtTfbCnOLysyiWesvKwCe69Xvt+uLn/bL39fHXOqlCWDdI6AAJyE4wM/8krL+ktqwP19/nbWWRbOAB1eKfe4/m82x9cdJr+Rou4fklQgowEmKrxeil27sasrif/LjPk1ZXN1iHYDn2JpZoDEfrFZVlXRTnyQN693U7iF5LQIKcArOatFQ445Wj3ziy41K3X6Q8wd4iLyiMo2cusLUPeqdHK3HrqheIA97EFCAUzTi7GRd2Tle5ZVVuvPdFdqWVcA5BNxceUWlRr+/SrsOFpn1Jq8O62bqIcE+nH3gNBbNPnNdJ3VLqqf84nLd9vZyZReUcB4BN/bUV5v0360HFBLgpzdu6aEG4UF2D8nrEVCA01w0a/0Sa9ogVHsPHdEfpyxXUSnl8AF39PHKvfr3op3m+F/Xd1a7+Ei7hwRmUIDTZ/2F9c7wXqofGqA1e/P0p/dXq8IqlgLAbfy455DGzVprjv90YUtd1rGx3UPCUcygAGcguWGYmUmxupp+szHTdDsF4B72HCzSyKkrVVpeqQHtYjWmf2u7h4RjEFCAM9SjWbSev76zOX5n8S69eXSqGIDrOnC4RLe8tdR8bNs4Us/f0EW+vpSxdyUEFMABLu8Ur0dqtx9v0Nx1GZxXwEUVlpRrxDvLzY6dhPohmjK8p8KDKGPvaggogIPceW5zDeudZAo83TfjR3NtG4BrKauo1N3TVpl1Y9FhgZp6ey/FRAbbPSycAAEFcGTPnivb64I2jVRSXqk/TllhrnEDcJ0GgH/+6CfTldzaTvzmrT3UvFG43cPCryCgAA7k7+erl2/spvbxkaZF+23vLFNuUSnnGHABT8/dbNpUWO0qrEJsXZPq2z0k/AYCCuBgYUH+euu2noqPCtaO7ELdMXWlSsrpfgzY6a1FOzV5wXZz/NQ1HXVBSgzfEBdHQAGcIDYyWG8P76WIIH8t25Wjh2b+pEpqpAC2mL0mXY9/WV0C4KGBbTSkRyLfCTdAQAGcpE1chCbd1F3+vj76fE26nj3avh1A3Vm87YAe+HCNWbx+a9+muuf8Fpx+bw0oEyZMUM+ePRUREaGYmBgNHjxYmzcf/4u5uLhYo0aNUoMGDRQeHq5rr71WmZmZjh4KYLuzWzXUhGs6muNJ3283NwB1Y316nu54d6VKKyp1Wcc4jb+ivVnMDi8NKAsWLDDhY8mSJZo3b57Kyso0YMAAFRYW1r5m7Nixmj17tmbOnGlen56ermuuucbRQwFcgjWd/PAlbczx03M3mWvhAJwrLafINPI8XFKu3slWMcUuZnEs3IdPlbXvyomys7PNTIoVRM4991zl5eWpUaNGmj59uq677jrzmk2bNqlt27ZKTU1Vnz59fvdz5ufnKyoqynyuyEiaOsE9PP+fzZr47TZz/OTVHXVj7yS7hwR4pIOHSzRkcqp2HChUSlyEPryrryKDA+weFnRq799OX4NiDcISHR1tPq5cudLMqvTv37/2NSkpKUpKSjIBBfBUYy9urTvObW6O//rpWn2yaq/dQwI8jtVV/PYpK0w4aVIvRFNu70U4cVNOre1bWVmpMWPGqF+/furQoYN5LCMjQ4GBgapXr95xr42NjTXPnUhJSYm5HZvAAHdjXfsed2mKSsoqNCV1tx6cucY0GbTK5ANwTDgZ/vZyrUnLVb3QABNOrB11cE9OnUGx1qKsW7dOM2bMOOOFt9aUUM0tMZEtYnDfkPLYFe01tGeirF3HY2as1n/W07cHcFQ4Wbozx2zvf/u2nmoZQ5VYd+a0gDJ69Gh98cUX+u6775SQkFD7eFxcnEpLS5Wbm3vc661dPNZzJzJu3DhzqajmlpaW5qxhA05ndUz959UdNbhLvMorqzR6+o9asCWbMw84KJxMGdGLKrEewOEBxVpza4WTWbNm6dtvv1VycvJxz3fv3l0BAQGaP39+7WPWNuQ9e/aob9++J/ycQUFBZjHNsTfAnVm7CZ4b0lmXdogzWyDvmLpCqdsP2j0swCPCSTdK2HsEX2dc1nnvvffMLh2rFoq1rsS6HTlyxDxvXaIZMWKE7r//fjO7Yi2aHT58uAknJ7ODB/Ckvj3/b2hXXZQSY5oLjpiyXCt359g9LMBtEE48m8O3Gf9aEZy3335bt912W22htgceeEDvv/++Wfw6cOBAvfrqq796iefn2GYMT1JcVqGRU1fov1sPmL8Ap43srU4Jxy8iB3A8wol7OpX3b6fXQXEGAgo8zZHSCt369jIt25mjqJAAzbijj9o25lImcCKEE/flUnVQAPy+kEA/0wG5S2I95R0p003/XqqtmQWcOuBnCCfeg4ACuIhwa4Hf7b3UPj5SBwtLdf1rqVqddvxuN8CbEU68CwEFcCHW5Z33RvRW58R6OlRUphvfWKIfth2we1iA7Qgn3oeAAriY+mGBmvbH3urXsoGKSivMFsq56/bbPSzANoQT70RAAVz0co+1JqWmTso901ZpxrI9dg8LqHPWmqzb3qLOiTcioAAuKsjfTy/f2K22LP4jn6zV5AXb7R4WUGfSc49oyOTFWraLImzeiIACuHjF2QnXdNRd57Uw95/6apMmzNloKjYDnmxTRr6ueXWxtmQeVmxkkD64sy8VYr0MAQVwcVbxw0cuTTGdkC2vLdyhRz5eq/KKSruHBjjF4m0HNGRSqjLyi9UqJlyf3NNP7eKpC+RtCCiAm7jzvBZ65tpO8vWRPliRZpoMWlVoAU/y2ep9pmhhQUm5eiVH66O7zlKTeiF2Dws2IKAAbuT6nol6dVh3Bfr5au76DN3+znIdLim3e1jAGbMuW762YLvum7FaZRVVGtSpsabe3ktRoQGcXS9FQAHczCUd4vTO7T0VFuinxdsPmlopOYWldg8LOG0VlVX6x+wNmvDVJnN/xNnJemloVwUH+HFWvRgBBXBDZ7VoqPfv6KPosED9tDdP101erJ0HCu0eFnDKrMuUo6at0juLd5n7fxvUVo9e3k6+1rVMeDUCCuCmrI7HH97ZV/FRwdqRXairXl6kBVuy7R4WcNIOFZaavlPW5UrrsuXLN3bVH89pzhmEQUAB3FjLmHB9Orqfujetr/zicg1/e5m5js82ZLi6tJwiXTt5sVbsPqTIYH9NHdFLl3eKt3tYcCEEFMDNxUQEa/rI3rqhR3VBN+s6/pgPVrPDBy5rTVqurpm02Mz8WTOAH919lvo0b2D3sOBiCCiAh1Sdferajvq/q9rL39dHn61ON+tS9uUesXtoQC1rZm/a0t0aMjlV2QUlSomLMDVOWsdGcJbwCwQUwIMKut3St5neHdHbLJ5dty/frEtZvivH7qEBOlJaoQdmrtFfZ60z/aUGtIvVh3f1VVxUMGcHJ0RAATxM3xYN9PnofmrbOFIHDpeabcjWX62AXawdZle/+oM+WbXPtG+wqiK/dnN3RQZT4wS/joACeKCE+qH6+O6+ptiVVfTK+qv1r7PWqrSc8vioW1+vz9CVLy3SpowCNQwP0rQ/9jZVka0ZP+C3EFAADxUa6K+X/9BVDw1sI+u9YNrSPRr27yXm2j/gbFavqAlfbdSd7640Zet7NquvOX86m8WwOGkEFMCDWX+ljrqgpd68tYcigvy1fNchXfnyIq1Oy7V7aPBgWQXFGvbvpXptwQ5zf+Q5yZo+so9iIllvgpNHQAG8wIUpsZo1qp+aNwzT/rxiXTtpsf7fN1vpiAyHsxZlXz5xkZbuzFF4kL9eHdZNfx3UTgF+vN3g1PATA3hRUTcrpFjrUqzeJy98s0VDXkvVLkrkw0FbiP/93x0a+voSZRWUqHVsuD4b3U+XdWzM+cVpIaAAXiQqJMCsS3nxhi6KCPbXj3tyddnE/+r9ZXuoPovTZjWrvPu9VXriy40m/F7VJV6fjuqnFo3COas4bT5VblgTOz8/X1FRUcrLy1NkZKTdwwHcklXE7YEPV2vJjuo6Kf3bxmjCNZ3UKCLI7qHBjXz5036N/2ydDhaWKsDPR+Mvb6eb+jRllw7O+P2bgAJ4scrKKr25aKee/XqzKZ7VICxQT13bSRe3i7V7aHBx1m4wK5h8tS7D3G8TG6HnhnRWx4Qou4cGF0ZAAXBKNu7P19gPVptaFZahPRNNy/uwIH/OJI5jTbpbrRT+Pnu9covKTGuFey5oqdEXtFSgP6sG8NsIKABOWXFZhZ6ft0Vv/HeHrAu/TRuE6vnru5hOyYAlK79Yf5m1Tt9szDT32zWO1LNDOql9PLMmODkEFACnLXX7QbM2JT2vWL4+0j3nt9ToC1sqOMCPs+rFsyYfr9qn/5u9XvnF5WatyZ8ubKW7zm/B9mGcEgIKgDOSd6RMf/98vWb9uM/cT4wO0d8GtTMN3ihR7l325x3RuE/W6vvN2eZ+p4QoPXtdZ7WJowMxTh0BBYBDzFm7X/83e4My8ovN/bNbNtRjV7RTq1jenLxh1mTG8jQ9+eVGU6reWl8ytn9rUxXWn6JrOE0EFAAOU1hSrknfb9frC3eYnT5WN9pb+zbTff1bmboq8Dwrd+foyTmbtHL3IXO/a1I9PXtdJ7WMIZjizBBQADjc7oOFphDXvA3VCyStLclWI8IhPRJNaIH725F9WM/M3ay566u3DgcH+OrBAW00vF8y32M4BAEFgNMs3JKtf8xer+3ZheZ+xyZR+vuV7dS9aTRn3Y1rmkycv1XTl+0xlWCtvHl9j0SNvbi1YmnwBwcioABwqrKKSk1N3a0X520x6xMsV3dtokcuTeENzY0UlZbr3//dqdcWbFdhaYV57KKUGP350hS1Zp0RnICAAqBOHDhcomfnbtaHK9NM7ZTQQD+NPKe5bjurmeqHBfJdcFHlFZWauXKvXpi3xTT2s3ROiNK4y9qqT/MGdg8PHiyfUvcA6tJPe3PNtuRVe3LNfSuo/KFXkv54TrIaR4XwzXChnTnzN2bpqbmbtC3rsHksKTpUD1/SRoM6NmYLOZyOgALAlje/OWsz9Mp327Rhf755zCrodU3XBN15XnM1p7Otbax1Jd9uytIbC3do2a7q5pD1QwN074WtNKxPkoL8KcKHukFAAWBrUFmwJdtsTV66s/rN0MdHurRDnO4+ryXN5OpQQXGZPlyxV1MW79KenCLzWJC/r24/O1l3ndeCbeKocwQUAC7BqqMx6ftt+mZjVu1j57RqqLvPb6G+zRtwScFJdh4oNKFk5oq02sWvkcH+5rLbbf2acdkNtiGgAHApmzLy9dqCHfp8Tbq53GDpkljP/BXfv20MlUkdNHO1aNsBvf3DLn23OcssWra0jAk3i5av6dZEoYF0p4a9CCgAXFJaTpGpSPvhijSVlFfWFny7onO8BndtYnaS0Ovn1BwprdAnP+7VOz/s0tajC18tF7RpZAqsWTNWnFO4CgIKAJcvDPb2Dzv1wfI0HSwsrX08uWGYruoSr8FdmqhZwzBbx+jq24Stxa5z12Xos9XpprmjJSzQT9d1T9CtZzVjUTJcEgEFgNsUfLMuS3z64z59vT5DxWXVsyo1/V+soHJ5p8ZqEB4kb1dSXqEfth0wocRqN3CoqDqU1HSbtvojXd8zUZHB9EeC6yKgAHA7h0vK9Z/1Gfp0dboWbc3W0aUqpgfMua0amktA/dvGKizI36saNX6/Odv0xvluU5Y5RzWsbcIXt4vVpR0a69zWjeiVA7dAQAHg1rIKijV7zX59tnqfftqbV/u4v6+POjSJUu/m0eqT3EDdm9X3uBmD3KJSU0zNCiVW36OatTqW2MggXdI+TgM7xKlXs2gWF8PtEFAAeAyr4qkVVKwdQLsPVtfyqGE1tWsXH6neyQ3UOzlavZKjVS800K0u22zcX2Aq8a5JyzMft2Ufrt2BY2naIFSXdIgzwaRzQj350jkaboyAAsBjdwFZxd+W7TxoPv48sFhS4iJMWOmZHK0WjcKVGB2qcBe4LGRtr7bC1pq0XK3Zm2tmhqzt12UVx6SRo9rERphQcmnHOHPMLhx4CgIKAK+QkVespUfDytIdB7U9u/CEr4sOCzRBxeo7kxQdYj4m1g81jzWOCnbIpRJru6+1Oyn7cHH1x5rb4RJtzyrUuvQ8FR0tmvbzsXVKiFKnhHpmm7X1sVEEi4LhmQgoALySFQiWHZ1hWZ2Wa8q7H7vb5USsdS1N6oeYoBDg6yt/Px8TWKzHrVuA39HHfI8+5ucjXx8f5RSVmq934GgQKThmAeuvsbYBW2toOifWM6HEumSTUD+EGRJ4jXy6GQPA0V+IxWXm0lBazhHzcc/Rm3W899ARlVb8bxHqmbL63MREBqlReJCZBTG38GATQqxAYjVMtHYlAd4q/xQCiv0XZgHAiaxdPu3jo8zt5yorq5RZUGzWsuQfKVN5ZZWpzVJeUaXyykqzPqSi5rHKKlMgzXqssqpK9UMD/xdCjt4igvyZDQEchIACwGtZO2IaR4XQPA9wQWe+MgwAAMDBCCgAAMDlEFAAAIDLIaAAAACXQ0ABAAAuh4ACAABcjq0B5ZVXXlGzZs0UHBys3r17a9myZXYOBwAAeHtA+eCDD3T//ffrscce06pVq9S5c2cNHDhQWVlZdg0JAAB4e0B5/vnnNXLkSA0fPlzt2rXT5MmTFRoaqrfeesuuIQEAAG8OKKWlpVq5cqX69+//v4H4+pr7qampv3h9SUmJqd9/7A0AAHguWwLKgQMHVFFRodjY2OMet+5nZGT84vUTJkwwzYVqbomJiXU4WgAAUNfcYhfPuHHjTOfDmltaWprdQwIAAJ7WLLBhw4by8/NTZmbmcY9b9+Pi4n7x+qCgIHMDAADewZaAEhgYqO7du2v+/PkaPHiweayystLcHz169O/++6qqKvORtSgAALiPmvftmvdxlwsoFmuL8a233qoePXqoV69eevHFF1VYWGh29fyegoIC85G1KAAAuB/rfdxaU+qSAeWGG25Qdna2xo8fbxbGdunSRXPnzv3FwtkTiY+PN+tQIiIi5OPjUyfjdYdUagU267xERkbaPRyPx/nmnHs6fsY5585gzZxY4cR6H/89PlUnM88Ct/hlYqVRaxExAYXz7Yn4Ged8ezp+xt1wFw8AAPAuBBQAAOByCCgewtqGbfU1Yjs259tT8TPO+fZ0/IwfjzUoAADA5TCDAgAAXA4BBQAAuBwCCgAAcDkEFAAA4HIIKB6spKTEVOi1qu2uXr3a7uF4rF27dmnEiBFKTk5WSEiIWrRoYXZUlZaW2j00j/HKK6+oWbNmCg4OVu/evbVs2TK7h+SxJkyYoJ49e5pK3TExMaZf2ubNm+0eltd46qmnzO/sMWPGyNsRUDzYww8/fFLlhHFmNm3aZJpdvvbaa1q/fr1eeOEFTZ48WX/5y184tQ7wwQcfmN5dVuhbtWqVOnfurIEDByorK4vz6wQLFizQqFGjtGTJEs2bN09lZWUaMGCA6ZUG51q+fLn5PdKpUydOtcUqdQ/PM2fOnKqUlJSq9evXW60Mqn788Ue7h+RVnnnmmark5GS7h+ERevXqVTVq1Kja+xUVFVXx8fFVEyZMsHVc3iIrK8v8DlmwYIHdQ/FoBQUFVa1ataqaN29e1XnnnVd13333VXk7ZlA8UGZmpkaOHKl3331XoaGhdg/HK1k9kaKjo+0ehtuzLpOtXLlS/fv3r33M19fX3E9NTbV1bN70s2zh59m5rFmrQYMGHfez7u1s62YM57B6P952222666671KNHD7M+AnVr27Zteumll/Tcc89x6s/QgQMHVFFR8Ysu59Z969IanMu6dGmthejXr586dOjA6XaSGTNmmMuX1iUe/A8zKG7ikUceMQunfutm/cK23hitVtbjxo2ze8hec86PtW/fPl1yySUaMmSImcUC3P2v+nXr1pk3UDhHWlqa7rvvPk2bNs0sAsf/UOreTWRnZ+vgwYO/+ZrmzZvr+uuv1+zZs82bZw3rL1A/Pz8NGzZMU6ZMqYPRetc5DwwMNMfp6ek6//zz1adPH73zzjvmUgTO/BKPdZnyo48+MrtJatx6663Kzc3VZ599xil2ktGjR5vzu3DhQrNDDc7x6aef6uqrrza/o4/9ne3j42N+h1i7MY99zpsQUDzMnj17lJ+fX3vfetO0djxYv+Ct7ZkJCQm2js9TWTMnF1xwgbp376733nvPa3+hOIP1c9urVy8zO1hz2SEpKcm8gVqzXHD8ZeJ7771Xs2bN0vfff69WrVpxip3ImvHevXv3cY8NHz5cKSkp+vOf/+zVl9ZYg+JhrF/cxwoPDzcfrdochBPnhRNr5qRp06Zm3Yk181IjLi7OSV/Ve1hbjK0ZE2tNlRVUXnzxRbPl1folDudc1pk+fbqZPbFqoWRkZJjHo6KiTJ0fOJZ1jn8eQsLCwtSgQQOvDicWAgpwhqxaEdbCWOv28xBo/TWKM3PDDTeY0Dd+/HjzZmkVH5w7d+4vFs7CMSZNmmQ+WqH7WG+//bZZgA/UFS7xAAAAl8MqPgAA4HIIKAAAwOUQUAAAgMshoAAAAJdDQAEAAC6HgAIAAFwOAQUAALgcAgoAAHA5BBQAAOByCCgAAMDlEFAAAIDLIaAAAAC5mv8PpiKQomVHzY0AAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Traçé de la parabole\n", "plt.plot(xs,ys)" ] }, { "cell_type": "markdown", "id": "d633101a-60b7-405b-a15d-7367b8e8b845", "metadata": {}, "source": [ "## Dérivations\n", "\n" ] }, { "cell_type": "markdown", "id": "2abc21bc-4d91-47cb-8bb4-3d698f0c499f", "metadata": {}, "source": [ "### Pente de la tangente\n", "\n", "$$t_{x}(h) = {f(x+h)-f(x) \\over h}$$\n", "\n", "[Taux d'accroissement](https://fr.wikipedia.org/wiki/Dérivée): _comment une fonction répond-elle quand on accroit $x$ d'un petit pas $h$_?" ] }, { "cell_type": "code", "execution_count": 8, "id": "58dbf7df-2c65-40ec-b9b4-0c9141308b00", "metadata": {}, "outputs": [], "source": [ "def pente(fct, x, h):\n", " print(f\"f(x) = {fct(x)}\")\n", " print(f\"f(x + h) = {fct(x + h)}\")\n", " print(f\"f(x + h) - f(x) = {fct(x + h) - fct(x)}\")\n", " print(f\"(f(x + h) - f(x)) / h= {(fct(x + h) - fct(x)) / h}\")\n", " return (fct(x + h) - fct(x)) / h" ] }, { "cell_type": "code", "execution_count": 9, "id": "ffc39abc-a6d0-45c5-8b18-2530c6800074", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "f(x) = 20.0\n", "f(x + h) = 20.00000014\n", "f(x + h) - f(x) = 1.400000009255109e-07\n", "(f(x + h) - f(x)) / h= 14.00000009255109\n" ] }, { "data": { "text/plain": [ "14.00000009255109" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "f = lambda x: 3 * x**2 - 4*x + 5\n", "pente(f, 3.0, 0.00000001)" ] }, { "cell_type": "code", "execution_count": 10, "id": "d02ce368-70fe-4ae6-a40f-4a4f24867e7f", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "f(x) = 44.0\n", "f(x + h) = 43.999999779999996\n", "f(x + h) - f(x) = -2.200000039920269e-07\n", "(f(x + h) - f(x)) / h= -22.00000039920269\n" ] }, { "data": { "text/plain": [ "-22.00000039920269" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "pente(f, -3.0, 0.00000001)" ] }, { "cell_type": "code", "execution_count": 11, "id": "5da198e9-6b85-431d-bb34-18c413246005", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "f(x) = 3.666666666666667\n", "f(x + h) = 3.666666666666667\n", "f(x + h) - f(x) = 0.0\n", "(f(x + h) - f(x)) / h= 0.0\n" ] }, { "data": { "text/plain": [ "0.0" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "pente(f, 2/3, 0.00000001)" ] }, { "cell_type": "markdown", "id": "6deaf726-2dbb-4c90-bd34-f840be5e45c8", "metadata": {}, "source": [ "### Expression arithmétique plus complexe" ] }, { "cell_type": "code", "execution_count": 12, "id": "0866addf-e765-49f4-8a8b-07c59577b4e9", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "4.0\n" ] } ], "source": [ "# Prenons une expression arithmétique plus complexe\n", "a = 2.0\n", "b = -3.0\n", "c = 10.0\n", "f = lambda x, y, z: x*y + z\n", "print(f(a,b,c))" ] }, { "cell_type": "code", "execution_count": 13, "id": "6c894dc0-48c1-4131-8a61-0085ca969fc8", "metadata": {}, "outputs": [], "source": [ "# Dérivée de f par rapport à a, b, c?\n", "h = 0.0001\n", "# Entrées\n", "a = 2.0\n", "b = -3.0\n", "c = 10.0" ] }, { "cell_type": "code", "execution_count": 14, "id": "a776697c-95a3-4ba3-8479-1e5beac082b1", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "4.0\n", "3.999699999999999\n", "pente -3.000000000010772\n" ] } ], "source": [ "# Dérivée de f par rapport à a\n", "f_1 = f(a, b, c)\n", "f_2 = f(a+h, b, c)\n", "print(f_1)\n", "print(f_2)\n", "print(\"pente\", (f_2-f_1)/h)\n" ] }, { "cell_type": "code", "execution_count": 15, "id": "8d152881-1640-4100-b841-19931881dfa5", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "4.0\n", "4.0002\n", "pente 2.0000000000042206\n" ] } ], "source": [ "# Dérivée de f par rapport à b\n", "f_1 = f(a, b, c)\n", "f_2 = f(a, b+h, c)\n", "print(f_1)\n", "print(f_2)\n", "print(\"pente\", (f_2-f_1)/h)\n" ] }, { "cell_type": "code", "execution_count": 16, "id": "2106c0a4-fc7c-4d39-9b54-7ff8f80062fd", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "4.0\n", "4.0001\n", "pente 0.9999999999976694\n" ] } ], "source": [ "# Dérivée de f par rapport à c\n", "f_1 = f(a, b, c)\n", "f_2 = f(a, b, c+h)\n", "print(f_1)\n", "print(f_2)\n", "print(\"pente\", (f_2-f_1)/h)\n" ] }, { "cell_type": "markdown", "id": "2fe8cc86-bfe1-44cc-b8ba-27f622c70768", "metadata": {}, "source": [ "## Création d'une classe Value" ] }, { "cell_type": "markdown", "id": "9b28572e-ef10-4680-8d3d-c91e5d1e6fa7", "metadata": {}, "source": [ "### Première version" ] }, { "cell_type": "code", "execution_count": 17, "id": "10925719-7561-4b2d-817e-0f5bc7df2c54", "metadata": {}, "outputs": [], "source": [ "class Value:\n", " \n", " def __init__(self, data):\n", " self.data = data\n", "\n", " def __repr__(self):\n", " return f\"Value(data={self.data})\"" ] }, { "cell_type": "code", "execution_count": 18, "id": "a571de2a-a18e-4c2b-b59a-1640481ab7cb", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Value(data=2.0)" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a = Value(2.0)\n", "a" ] }, { "cell_type": "code", "execution_count": 19, "id": "4b097b96-23eb-4deb-9beb-630a87749a8c", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Value(data=-3.0)" ] }, "execution_count": 19, "metadata": {}, "output_type": "execute_result" } ], "source": [ "b = Value(-3.0)\n", "b" ] }, { "cell_type": "markdown", "id": "f0b095eb-65e8-4107-acea-366c33710c5a", "metadata": {}, "source": [ "### Addition" ] }, { "cell_type": "code", "execution_count": 20, "id": "e06270bf-0509-4f10-8d68-373433b10f75", "metadata": {}, "outputs": [], "source": [ "class Value:\n", " \n", " def __init__(self, data):\n", " self.data = data\n", "\n", " def __repr__(self):\n", " return f\"Value(data={self.data})\"\n", "\n", " def __add__(self, other):\n", " return self.__class__(self.data + other.data)" ] }, { "cell_type": "code", "execution_count": 21, "id": "b60e3a61-df09-4e5f-8e9f-5fb1fafbeeab", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Value(data=-1.0)" ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a = Value(2.0)\n", "b = Value(-3.0)\n", "a + b" ] }, { "cell_type": "markdown", "id": "84477113-af0e-4982-bafc-722f7168edb9", "metadata": {}, "source": [ "### Multiplication" ] }, { "cell_type": "code", "execution_count": 21, "id": "e231ef66-90a8-4137-9ec3-59023b09de75", "metadata": {}, "outputs": [], "source": [ "class Value:\n", " \n", " def __init__(self, data):\n", " self.data = data\n", "\n", " def __repr__(self):\n", " return f\"Value(data={self.data})\"\n", "\n", " def __add__(self, other):\n", " return self.__class__(self.data + other.data)\n", "\n", " def __mul__(self, other):\n", " return self.__class__(self.data * other.data)" ] }, { "cell_type": "code", "execution_count": 22, "id": "ff8ab2a3-be0b-4e74-aa7d-d159dfa4756d", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Value(data=-6.0)" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a = Value(2.0)\n", "b = Value(-3.0)\n", "a * b" ] }, { "cell_type": "code", "execution_count": 23, "id": "3a7c8022-196b-4e53-b610-ed1bb227be6c", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Value(data=4.0)" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "c = Value(10.0)\n", "a*b + c" ] }, { "cell_type": "markdown", "id": "3dd1ffeb-04c2-44c1-b48b-58c8d7ca6627", "metadata": {}, "source": [ "### Construction d'un graphe correspondant à l'expression" ] }, { "cell_type": "code", "execution_count": 24, "id": "d353dd6c-4927-4a83-b8f9-e9485b21e68a", "metadata": {}, "outputs": [], "source": [ "class Value:\n", " \n", " def __init__(self, data, children=(), op=''):\n", " self.data = data\n", " self._prev = set(children)\n", " self._op = op\n", "\n", " def __repr__(self):\n", " return f\"Value(data={self.data})\"\n", "\n", " def __add__(self, other):\n", " return self.__class__(self.data + other.data, children=(self, other), op='+')\n", "\n", " def __mul__(self, other):\n", " return self.__class__(self.data * other.data, children=(self, other), op='*')" ] }, { "cell_type": "code", "execution_count": 25, "id": "cee3f062-7544-4d9d-b9af-bfc2f19c4dc7", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Value(data=4.0)" ] }, "execution_count": 25, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a = Value(2.0)\n", "b = Value(-3.0)\n", "c = Value(10.0)\n", "d = a * b + c\n", "d" ] }, { "cell_type": "code", "execution_count": 26, "id": "a460db74-0ba0-4c89-b318-75021350129e", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{Value(data=-6.0), Value(data=10.0)}" ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" } ], "source": [ "d._prev" ] }, { "cell_type": "markdown", "id": "6461edf5-a3e9-4d75-8411-3f4648d6d954", "metadata": {}, "source": [ "### Affichage du graphe avec graphviz" ] }, { "cell_type": "code", "execution_count": 30, "id": "2402e85d-70d8-4e44-ac1e-5b62dbe7b873", "metadata": {}, "outputs": [], "source": [ "a = Value(2.0)\n", "b = Value(-3.0)\n", "c = Value(10.0)\n", "d = a * b + c" ] }, { "cell_type": "code", "execution_count": 31, "id": "f6dd0f57-ccb4-4cbb-af6c-3524762b6dc7", "metadata": {}, "outputs": [], "source": [ "def trace(root):\n", " # builds a set of all nodes and edges in a graph\n", " nodes, edges = set(), set()\n", " def build(v):\n", " if v not in nodes:\n", " nodes.add(v)\n", " for child in v._prev:\n", " edges.add((child, v))\n", " build(child)\n", " build(root)\n", " return nodes, edges\n", "\n", "def draw_dot(root):\n", " dot = Digraph(format='svg', graph_attr={'rankdir': 'LR'}) # LR = left to right\n", " \n", " nodes, edges = trace(root)\n", " for n in nodes:\n", " uid = str(id(n))\n", " # for any value in the graph, create a rectangular ('record') node for it\n", " dot.node(name = uid, label = \"{ data %.4f }\" % n.data, shape='record')\n", " if n._op:\n", " # if this value is a result of some operation, create an op node for it\n", " dot.node(name = uid + n._op, label = n._op)\n", " # and connect this node to it\n", " dot.edge(uid + n._op, uid)\n", "\n", " for n1, n2 in edges:\n", " # connect n1 to the op node of n2\n", " dot.edge(str(id(n1)), str(id(n2)) + n2._op)\n", "\n", " return dot" ] }, { "cell_type": "code", "execution_count": 32, "id": "0641677f-74fb-4a63-a73a-8e7d90c988e4", "metadata": {}, "outputs": [ { "data": { "image/svg+xml": [ "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "4538923536\n", "\n", "data -6.0000\n", "\n", "\n", "\n", "4539358192+\n", "\n", "+\n", "\n", "\n", "\n", "4538923536->4539358192+\n", "\n", "\n", "\n", "\n", "\n", "4538923536*\n", "\n", "*\n", "\n", "\n", "\n", "4538923536*->4538923536\n", "\n", "\n", "\n", "\n", "\n", "4507759824\n", "\n", "data 2.0000\n", "\n", "\n", "\n", "4507759824->4538923536*\n", "\n", "\n", "\n", "\n", "\n", "4539258816\n", "\n", "data -3.0000\n", "\n", "\n", "\n", "4539258816->4538923536*\n", "\n", "\n", "\n", "\n", "\n", "4538923984\n", "\n", "data 10.0000\n", "\n", "\n", "\n", "4538923984->4539358192+\n", "\n", "\n", "\n", "\n", "\n", "4539358192\n", "\n", "data 4.0000\n", "\n", "\n", "\n", "4539358192+->4539358192\n", "\n", "\n", "\n", "\n", "\n" ], "text/plain": [ "" ] }, "execution_count": 32, "metadata": {}, "output_type": "execute_result" } ], "source": [ "draw_dot(d)" ] }, { "cell_type": "markdown", "id": "8b3598d6-20c0-4b94-be82-662162d32ae5", "metadata": {}, "source": [ "### Amélioration de l'affichage avec un label" ] }, { "cell_type": "code", "execution_count": 33, "id": "e445d7b2-108f-4c4e-be84-5676eae51f8e", "metadata": {}, "outputs": [], "source": [ "class Value:\n", " \n", " def __init__(self, data, children=(), op='', label=''):\n", " self.data = data\n", " self._prev = set(children)\n", " self._op = op\n", " self.label = label\n", "\n", " def __repr__(self):\n", " return f\"Value(data={self.data}, label={self.label})\"\n", "\n", " def __add__(self, other):\n", " return self.__class__(self.data + other.data, children=(self, other), op='+')\n", "\n", " def __mul__(self, other):\n", " return self.__class__(self.data * other.data, children=(self, other), op='*')" ] }, { "cell_type": "code", "execution_count": 34, "id": "f5697d7d-6b0b-496c-8b96-44651ddc8aca", "metadata": {}, "outputs": [], "source": [ "def draw_dot(root):\n", " dot = Digraph(format='svg', graph_attr={'rankdir': 'LR'}) # LR = left to right\n", " \n", " nodes, edges = trace(root)\n", " for n in nodes:\n", " uid = str(id(n))\n", " # for any value in the graph, create a rectangular ('record') node for it\n", " dot.node(name = uid, label = \"{ %s | data %.4f }\" % (n.label, n.data), shape='record')\n", " if n._op:\n", " # if this value is a result of some operation, create an op node for it\n", " dot.node(name = uid + n._op, label = n._op)\n", " # and connect this node to it\n", " dot.edge(uid + n._op, uid)\n", "\n", " for n1, n2 in edges:\n", " # connect n1 to the op node of n2\n", " dot.edge(str(id(n1)), str(id(n2)) + n2._op)\n", "\n", " return dot" ] }, { "cell_type": "code", "execution_count": 35, "id": "774f25ba-06dd-445e-97be-28ccf5433760", "metadata": {}, "outputs": [], "source": [ "a = Value(2.0, label='a')\n", "b = Value(-3.0, label='b')\n", "c = Value(10.0, label='c')\n", "e = a * b\n", "e.label = 'e'\n", "d = e + c\n", "d.label = 'd'" ] }, { "cell_type": "code", "execution_count": 36, "id": "493b3068-009e-4068-8cff-30a7c12a3cd0", "metadata": {}, "outputs": [ { "data": { "image/svg+xml": [ "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "4539029520\n", "\n", "d\n", "\n", "data 4.0000\n", "\n", "\n", "\n", "4539029520+\n", "\n", "+\n", "\n", "\n", "\n", "4539029520+->4539029520\n", "\n", "\n", "\n", "\n", "\n", "4539028000\n", "\n", "e\n", "\n", "data -6.0000\n", "\n", "\n", "\n", "4539028000->4539029520+\n", "\n", "\n", "\n", "\n", "\n", "4539028000*\n", "\n", "*\n", "\n", "\n", "\n", "4539028000*->4539028000\n", "\n", "\n", "\n", "\n", "\n", "4538805264\n", "\n", "c\n", "\n", "data 10.0000\n", "\n", "\n", "\n", "4538805264->4539029520+\n", "\n", "\n", "\n", "\n", "\n", "4539007056\n", "\n", "a\n", "\n", "data 2.0000\n", "\n", "\n", "\n", "4539007056->4539028000*\n", "\n", "\n", "\n", "\n", "\n", "4538804944\n", "\n", "b\n", "\n", "data -3.0000\n", "\n", "\n", "\n", "4538804944->4539028000*\n", "\n", "\n", "\n", "\n", "\n" ], "text/plain": [ "" ] }, "execution_count": 36, "metadata": {}, "output_type": "execute_result" } ], "source": [ "draw_dot(d)" ] }, { "cell_type": "markdown", "id": "4666d7e1-d263-4f9b-a1dc-f34a9cd70d15", "metadata": {}, "source": [ "### Fonction objectif: fonction de perte" ] }, { "cell_type": "code", "execution_count": 37, "id": "fd2cb8bc-fd4b-4730-ad4e-c4aa99b70f52", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Value(data=-8.0, label=L)" ] }, "execution_count": 37, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a = Value(2.0, label='a')\n", "b = Value(-3.0, label='b')\n", "c = Value(10.0, label='c')\n", "e = a * b\n", "e.label = 'e'\n", "d = e + c\n", "d.label = 'd'\n", "f = Value(-2.0, label='f')\n", "L = d * f\n", "L.label = 'L'\n", "L\n" ] }, { "cell_type": "code", "execution_count": 38, "id": "09d3a7fd-dfc8-4aa7-9604-061eedde47b0", "metadata": {}, "outputs": [ { "data": { "image/svg+xml": [ "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "4539211856\n", "\n", "e\n", "\n", "data -6.0000\n", "\n", "\n", "\n", "4539210576+\n", "\n", "+\n", "\n", "\n", "\n", "4539211856->4539210576+\n", "\n", "\n", "\n", "\n", "\n", "4539211856*\n", "\n", "*\n", "\n", "\n", "\n", "4539211856*->4539211856\n", "\n", "\n", "\n", "\n", "\n", "4539259056\n", "\n", "f\n", "\n", "data -2.0000\n", "\n", "\n", "\n", "4539258096*\n", "\n", "*\n", "\n", "\n", "\n", "4539259056->4539258096*\n", "\n", "\n", "\n", "\n", "\n", "4537604288\n", "\n", "c\n", "\n", "data 10.0000\n", "\n", "\n", "\n", "4537604288->4539210576+\n", "\n", "\n", "\n", "\n", "\n", "4537280720\n", "\n", "a\n", "\n", "data 2.0000\n", "\n", "\n", "\n", "4537280720->4539211856*\n", "\n", "\n", "\n", "\n", "\n", "4539258096\n", "\n", "L\n", "\n", "data -8.0000\n", "\n", "\n", "\n", "4539258096*->4539258096\n", "\n", "\n", "\n", "\n", "\n", "4539210576\n", "\n", "d\n", "\n", "data 4.0000\n", "\n", "\n", "\n", "4539210576->4539258096*\n", "\n", "\n", "\n", "\n", "\n", "4539210576+->4539210576\n", "\n", "\n", "\n", "\n", "\n", "4537604016\n", "\n", "b\n", "\n", "data -3.0000\n", "\n", "\n", "\n", "4537604016->4539211856*\n", "\n", "\n", "\n", "\n", "\n" ], "text/plain": [ "" ] }, "execution_count": 38, "metadata": {}, "output_type": "execute_result" } ], "source": [ "draw_dot(L)" ] }, { "cell_type": "markdown", "id": "ab1d1165-7a1e-4600-96f2-2808858a9d50", "metadata": {}, "source": [ "À ce stade, nous pouvons faire une passe avant (_forward pass_), dans une expression mathématique. Nous voulons maintenant pouvoir faire une propagation arrière du gradient, à partir de $L$, pour déterminer les gradients sur les chemins du graphe en remontant. On va donc calculer les dérivées de chaque noeud par rapport à $L$.\n" ] }, { "cell_type": "markdown", "id": "586c8117-de9e-412e-895e-44c306bf48a7", "metadata": {}, "source": [ "## Calcul des gradients à la main" ] }, { "cell_type": "markdown", "id": "601fdb09-94be-4621-94db-87089c8d35eb", "metadata": {}, "source": [ "### Stockage des gradients dans Value" ] }, { "cell_type": "code", "execution_count": 39, "id": "9ccaf273-11e7-408c-a461-dc08fe4f32e5", "metadata": {}, "outputs": [], "source": [ "class Value:\n", " \n", " def __init__(self, data, children=(), op='', label=''):\n", " self.data = data\n", " self._prev = set(children)\n", " self._op = op\n", " self.label = label\n", " self.grad = 0.0\n", "\n", " def __repr__(self):\n", " return f\"Value(data={self.data}, label={self.label}, grad={self.grad})\"\n", "\n", " def __add__(self, other):\n", " return self.__class__(self.data + other.data, children=(self, other), op='+')\n", "\n", " def __mul__(self, other):\n", " return self.__class__(self.data * other.data, children=(self, other), op='*')" ] }, { "cell_type": "markdown", "id": "f2fa8045-933d-4422-8baf-c7738ed45990", "metadata": {}, "source": [ "Affichage du gradient dans les noeuds du graphe:" ] }, { "cell_type": "code", "execution_count": 2, "id": "323af9b8-7c4d-4fb3-9905-f1e0df4c81fd", "metadata": {}, "outputs": [], "source": [ "def draw_dot(root):\n", " dot = Digraph(format='svg', graph_attr={'rankdir': 'LR'}) # LR = left to right\n", " \n", " nodes, edges = trace(root)\n", " for n in nodes:\n", " uid = str(id(n))\n", " # for any value in the graph, create a rectangular ('record') node for it\n", " dot.node(name = uid, label = \"{ %s | data %.4f | grad %.4f }\" % (n.label, n.data, n.grad), shape='record')\n", " if n._op:\n", " # if this value is a result of some operation, create an op node for it\n", " dot.node(name = uid + n._op, label = n._op)\n", " # and connect this node to it\n", " dot.edge(uid + n._op, uid)\n", "\n", " for n1, n2 in edges:\n", " # connect n1 to the op node of n2\n", " dot.edge(str(id(n1)), str(id(n2)) + n2._op)\n", "\n", " return dot" ] }, { "cell_type": "code", "execution_count": 3, "id": "330632e4-cef8-4077-b4d9-095cc482878b", "metadata": {}, "outputs": [ { "ename": "NameError", "evalue": "name 'Value' is not defined", "output_type": "error", "traceback": [ "\u001b[31m---------------------------------------------------------------------------\u001b[39m", "\u001b[31mNameError\u001b[39m Traceback (most recent call last)", "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[3]\u001b[39m\u001b[32m, line 1\u001b[39m\n\u001b[32m----> \u001b[39m\u001b[32m1\u001b[39m a = \u001b[43mValue\u001b[49m(\u001b[32m2.0\u001b[39m, label=\u001b[33m'\u001b[39m\u001b[33ma\u001b[39m\u001b[33m'\u001b[39m)\n\u001b[32m 2\u001b[39m b = Value(-\u001b[32m3.0\u001b[39m, label=\u001b[33m'\u001b[39m\u001b[33mb\u001b[39m\u001b[33m'\u001b[39m)\n\u001b[32m 3\u001b[39m c = Value(\u001b[32m10.0\u001b[39m, label=\u001b[33m'\u001b[39m\u001b[33mc\u001b[39m\u001b[33m'\u001b[39m)\n", "\u001b[31mNameError\u001b[39m: name 'Value' is not defined" ] } ], "source": [ "a = Value(2.0, label='a')\n", "b = Value(-3.0, label='b')\n", "c = Value(10.0, label='c')\n", "e = a * b\n", "e.label = 'e'\n", "d = e + c\n", "d.label = 'd'\n", "f = Value(-2.0, label='f')\n", "L = d * f\n", "L.label = 'L'\n", "L" ] }, { "cell_type": "code", "execution_count": 42, "id": "5ca06535-faa7-4023-91a5-b3493b65acac", "metadata": {}, "outputs": [ { "data": { "image/svg+xml": [ "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "4539010080\n", "\n", "a\n", "\n", "data 2.0000\n", "\n", "grad 0.0000\n", "\n", "\n", "\n", "4539029216*\n", "\n", "*\n", "\n", "\n", "\n", "4539010080->4539029216*\n", "\n", "\n", "\n", "\n", "\n", "4537603200\n", "\n", "L\n", "\n", "data -8.0000\n", "\n", "grad 0.0000\n", "\n", "\n", "\n", "4537603200*\n", "\n", "*\n", "\n", "\n", "\n", "4537603200*->4537603200\n", "\n", "\n", "\n", "\n", "\n", "4538803344\n", "\n", "c\n", "\n", "data 10.0000\n", "\n", "grad 0.0000\n", "\n", "\n", "\n", "4539031344+\n", "\n", "+\n", "\n", "\n", "\n", "4538803344->4539031344+\n", "\n", "\n", "\n", "\n", "\n", "4539029216\n", "\n", "e\n", "\n", "data -6.0000\n", "\n", "grad 0.0000\n", "\n", "\n", "\n", "4539029216->4539031344+\n", "\n", "\n", "\n", "\n", "\n", "4539029216*->4539029216\n", "\n", "\n", "\n", "\n", "\n", "4537281296\n", "\n", "f\n", "\n", "data -2.0000\n", "\n", "grad 0.0000\n", "\n", "\n", "\n", "4537281296->4537603200*\n", "\n", "\n", "\n", "\n", "\n", "4539031344\n", "\n", "d\n", "\n", "data 4.0000\n", "\n", "grad 0.0000\n", "\n", "\n", "\n", "4539031344->4537603200*\n", "\n", "\n", "\n", "\n", "\n", "4539031344+->4539031344\n", "\n", "\n", "\n", "\n", "\n", "4538803024\n", "\n", "b\n", "\n", "data -3.0000\n", "\n", "grad 0.0000\n", "\n", "\n", "\n", "4538803024->4539029216*\n", "\n", "\n", "\n", "\n", "\n" ], "text/plain": [ "" ] }, "execution_count": 42, "metadata": {}, "output_type": "execute_result" } ], "source": [ "draw_dot(L)" ] }, { "cell_type": "markdown", "id": "4ee03a7a-b6da-4594-a376-784760bed5fa", "metadata": {}, "source": [ "### Jouons avec les dérivées" ] }, { "cell_type": "code", "execution_count": 4, "id": "a57892ec-cf57-4366-943e-ea1a40d65ceb", "metadata": {}, "outputs": [ { "ename": "NameError", "evalue": "name 'Value' is not defined", "output_type": "error", "traceback": [ "\u001b[31m---------------------------------------------------------------------------\u001b[39m", "\u001b[31mNameError\u001b[39m Traceback (most recent call last)", "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[4]\u001b[39m\u001b[32m, line 35\u001b[39m\n\u001b[32m 31\u001b[39m L2 = L.data + h\n\u001b[32m 33\u001b[39m \u001b[38;5;28mprint\u001b[39m((L2-L1)/h)\n\u001b[32m---> \u001b[39m\u001b[32m35\u001b[39m \u001b[43mplayground\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m \n", "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[4]\u001b[39m\u001b[32m, line 5\u001b[39m, in \u001b[36mplayground\u001b[39m\u001b[34m()\u001b[39m\n\u001b[32m 1\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34mplayground\u001b[39m():\n\u001b[32m 3\u001b[39m h = \u001b[32m0.001\u001b[39m\n\u001b[32m----> \u001b[39m\u001b[32m5\u001b[39m a = \u001b[43mValue\u001b[49m(\u001b[32m2.0\u001b[39m, label=\u001b[33m'\u001b[39m\u001b[33ma\u001b[39m\u001b[33m'\u001b[39m)\n\u001b[32m 6\u001b[39m b = Value(-\u001b[32m3.0\u001b[39m, label=\u001b[33m'\u001b[39m\u001b[33mb\u001b[39m\u001b[33m'\u001b[39m)\n\u001b[32m 7\u001b[39m c = Value(\u001b[32m10.0\u001b[39m, label=\u001b[33m'\u001b[39m\u001b[33mc\u001b[39m\u001b[33m'\u001b[39m)\n", "\u001b[31mNameError\u001b[39m: name 'Value' is not defined" ] } ], "source": [ "def playground():\n", "\n", " h = 0.001\n", " \n", " a = Value(2.0, label='a')\n", " b = Value(-3.0, label='b')\n", " c = Value(10.0, label='c')\n", " e = a * b\n", " e.label = 'e'\n", " d = e + c\n", " d.label = 'd'\n", " f = Value(-2.0, label='f')\n", " L = d * f\n", " L.label = 'L'\n", " L1 = L.data\n", "\n", " a = Value(2.0, label='a')\n", " b = Value(-3.0, label='b')\n", " c = Value(10.0, label='c')\n", " e = a * b\n", " e.label = 'e'\n", " #e.data += h\n", " d = e + c\n", " d.label = 'd'\n", " #d.data += h\n", " f = Value(-2.0, label='f')\n", " #f.data += h\n", " L = d * f\n", " L.label = 'L'\n", " L2 = L.data\n", " L2 = L.data + h\n", "\n", " print((L2-L1)/h)\n", "\n", "playground() " ] }, { "cell_type": "code", "execution_count": 51, "id": "934734dd-1c20-426a-a242-a30fca07a6fc", "metadata": {}, "outputs": [], "source": [ "a = Value(2.0, label='a')\n", "b = Value(-3.0, label='b')\n", "c = Value(10.0, label='c')\n", "e = a * b\n", "e.label = 'e'\n", "d = e + c\n", "d.label = 'd'\n", "f = Value(-2.0, label='f')\n", "L = d * f\n", "L.label = 'L'\n", "L.grad = 1" ] }, { "cell_type": "code", "execution_count": 52, "id": "dff97940-1af4-4da3-8f8f-f8d55baae7f9", "metadata": {}, "outputs": [ { "data": { "image/svg+xml": [ "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "4539085920\n", "\n", "c\n", "\n", "data 10.0000\n", "\n", "grad 0.0000\n", "\n", "\n", "\n", "4539086480+\n", "\n", "+\n", "\n", "\n", "\n", "4539085920->4539086480+\n", "\n", "\n", "\n", "\n", "\n", "4539086480\n", "\n", "d\n", "\n", "data 4.0000\n", "\n", "grad 0.0000\n", "\n", "\n", "\n", "4539084688*\n", "\n", "*\n", "\n", "\n", "\n", "4539086480->4539084688*\n", "\n", "\n", "\n", "\n", "\n", "4539086480+->4539086480\n", "\n", "\n", "\n", "\n", "\n", "4539081888\n", "\n", "b\n", "\n", "data -3.0000\n", "\n", "grad 0.0000\n", "\n", "\n", "\n", "4539086032*\n", "\n", "*\n", "\n", "\n", "\n", "4539081888->4539086032*\n", "\n", "\n", "\n", "\n", "\n", "4539086032\n", "\n", "e\n", "\n", "data -6.0000\n", "\n", "grad 0.0000\n", "\n", "\n", "\n", "4539086032->4539086480+\n", "\n", "\n", "\n", "\n", "\n", "4539086032*->4539086032\n", "\n", "\n", "\n", "\n", "\n", "4539084576\n", "\n", "a\n", "\n", "data 2.0000\n", "\n", "grad 0.0000\n", "\n", "\n", "\n", "4539084576->4539086032*\n", "\n", "\n", "\n", "\n", "\n", "4539086704\n", "\n", "f\n", "\n", "data -2.0000\n", "\n", "grad 0.0000\n", "\n", "\n", "\n", "4539086704->4539084688*\n", "\n", "\n", "\n", "\n", "\n", "4539084688\n", "\n", "L\n", "\n", "data -8.0000\n", "\n", "grad 1.0000\n", "\n", "\n", "\n", "4539084688*->4539084688\n", "\n", "\n", "\n", "\n", "\n" ], "text/plain": [ "" ] }, "execution_count": 52, "metadata": {}, "output_type": "execute_result" } ], "source": [ "draw_dot(L)\n" ] }, { "cell_type": "code", "execution_count": 53, "id": "b586a4dd-a3c3-4c8d-b17e-bce6fed7981b", "metadata": {}, "outputs": [], "source": [ "d.grad = -2.0 # dL/dd = d(d*f)/dd = f\n", "f.grad = 4.0 # dL/df = d(d*f)/df = d" ] }, { "cell_type": "code", "execution_count": 54, "id": "17a64096-b601-47af-a31a-c1f4b4e64923", "metadata": {}, "outputs": [ { "data": { "image/svg+xml": [ "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "4539085920\n", "\n", "c\n", "\n", "data 10.0000\n", "\n", "grad 0.0000\n", "\n", "\n", "\n", "4539086480+\n", "\n", "+\n", "\n", "\n", "\n", "4539085920->4539086480+\n", "\n", "\n", "\n", "\n", "\n", "4539086480\n", "\n", "d\n", "\n", "data 4.0000\n", "\n", "grad -2.0000\n", "\n", "\n", "\n", "4539084688*\n", "\n", "*\n", "\n", "\n", "\n", "4539086480->4539084688*\n", "\n", "\n", "\n", "\n", "\n", "4539086480+->4539086480\n", "\n", "\n", "\n", "\n", "\n", "4539081888\n", "\n", "b\n", "\n", "data -3.0000\n", "\n", "grad 0.0000\n", "\n", "\n", "\n", "4539086032*\n", "\n", "*\n", "\n", "\n", "\n", "4539081888->4539086032*\n", "\n", "\n", "\n", "\n", "\n", "4539086032\n", "\n", "e\n", "\n", "data -6.0000\n", "\n", "grad 0.0000\n", "\n", "\n", "\n", "4539086032->4539086480+\n", "\n", "\n", "\n", "\n", "\n", "4539086032*->4539086032\n", "\n", "\n", "\n", "\n", "\n", "4539084576\n", "\n", "a\n", "\n", "data 2.0000\n", "\n", "grad 0.0000\n", "\n", "\n", "\n", "4539084576->4539086032*\n", "\n", "\n", "\n", "\n", "\n", "4539086704\n", "\n", "f\n", "\n", "data -2.0000\n", "\n", "grad 4.0000\n", "\n", "\n", "\n", "4539086704->4539084688*\n", "\n", "\n", "\n", "\n", "\n", "4539084688\n", "\n", "L\n", "\n", "data -8.0000\n", "\n", "grad 1.0000\n", "\n", "\n", "\n", "4539084688*->4539084688\n", "\n", "\n", "\n", "\n", "\n" ], "text/plain": [ "" ] }, "execution_count": 54, "metadata": {}, "output_type": "execute_result" } ], "source": [ "draw_dot(L)" ] }, { "cell_type": "markdown", "id": "eeb2d149-6b4c-406d-9a2a-360a17979f29", "metadata": {}, "source": [ "### Propagation des gradients locaux: dérivation des fonctions composées (chain rule)\n", "\n", "$$\\frac{dz}{dx} = \\frac{dz}{dy} \\cdot \\frac{dy}{dx}$$\n" ] }, { "cell_type": "code", "execution_count": 55, "id": "eb9721c8-d8b0-4cc1-8b14-22765b81e5fe", "metadata": {}, "outputs": [], "source": [ "# d = e + c\n", "# dd/dc = 1.0 \n", "# dd/de = 1.0\n", "# (dérivées locales)\n", "# dL/dc = dL/dd * dd/dc = d.grad * 1.0\n", "# dL/de = dL/dd * dd/de = d.grad * 1.0\n", "c.grad = d.grad\n", "e.grad = d.grad" ] }, { "cell_type": "code", "execution_count": 56, "id": "5bb7c744-a887-4f53-9914-10413fb67436", "metadata": {}, "outputs": [ { "data": { "image/svg+xml": [ "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "4539085920\n", "\n", "c\n", "\n", "data 10.0000\n", "\n", "grad -2.0000\n", "\n", "\n", "\n", "4539086480+\n", "\n", "+\n", "\n", "\n", "\n", "4539085920->4539086480+\n", "\n", "\n", "\n", "\n", "\n", "4539086480\n", "\n", "d\n", "\n", "data 4.0000\n", "\n", "grad -2.0000\n", "\n", "\n", "\n", "4539084688*\n", "\n", "*\n", "\n", "\n", "\n", "4539086480->4539084688*\n", "\n", "\n", "\n", "\n", "\n", "4539086480+->4539086480\n", "\n", "\n", "\n", "\n", "\n", "4539081888\n", "\n", "b\n", "\n", "data -3.0000\n", "\n", "grad 0.0000\n", "\n", "\n", "\n", "4539086032*\n", "\n", "*\n", "\n", "\n", "\n", "4539081888->4539086032*\n", "\n", "\n", "\n", "\n", "\n", "4539086032\n", "\n", "e\n", "\n", "data -6.0000\n", "\n", "grad -2.0000\n", "\n", "\n", "\n", "4539086032->4539086480+\n", "\n", "\n", "\n", "\n", "\n", "4539086032*->4539086032\n", "\n", "\n", "\n", "\n", "\n", "4539084576\n", "\n", "a\n", "\n", "data 2.0000\n", "\n", "grad 0.0000\n", "\n", "\n", "\n", "4539084576->4539086032*\n", "\n", "\n", "\n", "\n", "\n", "4539086704\n", "\n", "f\n", "\n", "data -2.0000\n", "\n", "grad 4.0000\n", "\n", "\n", "\n", "4539086704->4539084688*\n", "\n", "\n", "\n", "\n", "\n", "4539084688\n", "\n", "L\n", "\n", "data -8.0000\n", "\n", "grad 1.0000\n", "\n", "\n", "\n", "4539084688*->4539084688\n", "\n", "\n", "\n", "\n", "\n" ], "text/plain": [ "" ] }, "execution_count": 56, "metadata": {}, "output_type": "execute_result" } ], "source": [ "draw_dot(L)" ] }, { "cell_type": "code", "execution_count": 57, "id": "51bfd765-d52f-47cc-9fb0-8e81106626c9", "metadata": {}, "outputs": [], "source": [ "# dL/de = -2.0\n", "# dL/da = dL/de * de/da\n", "# dL/db = dL/de * de/db\n", "# de/da = b = -3.0\n", "# de/db = a = 2.0\n", "# dL/da = -2.0 * -3.0\n", "# dL/db = -2.0 * 2.0\n", "a.grad = d.grad * b.data\n", "b.grad = d.grad * a.data" ] }, { "cell_type": "code", "execution_count": 58, "id": "2515962c-8660-4b02-a90f-c453b0c1c996", "metadata": {}, "outputs": [ { "data": { "image/svg+xml": [ "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "4539085920\n", "\n", "c\n", "\n", "data 10.0000\n", "\n", "grad -2.0000\n", "\n", "\n", "\n", "4539086480+\n", "\n", "+\n", "\n", "\n", "\n", "4539085920->4539086480+\n", "\n", "\n", "\n", "\n", "\n", "4539086480\n", "\n", "d\n", "\n", "data 4.0000\n", "\n", "grad -2.0000\n", "\n", "\n", "\n", "4539084688*\n", "\n", "*\n", "\n", "\n", "\n", "4539086480->4539084688*\n", "\n", "\n", "\n", "\n", "\n", "4539086480+->4539086480\n", "\n", "\n", "\n", "\n", "\n", "4539081888\n", "\n", "b\n", "\n", "data -3.0000\n", "\n", "grad -4.0000\n", "\n", "\n", "\n", "4539086032*\n", "\n", "*\n", "\n", "\n", "\n", "4539081888->4539086032*\n", "\n", "\n", "\n", "\n", "\n", "4539086032\n", "\n", "e\n", "\n", "data -6.0000\n", "\n", "grad -2.0000\n", "\n", "\n", "\n", "4539086032->4539086480+\n", "\n", "\n", "\n", "\n", "\n", "4539086032*->4539086032\n", "\n", "\n", "\n", "\n", "\n", "4539084576\n", "\n", "a\n", "\n", "data 2.0000\n", "\n", "grad 6.0000\n", "\n", "\n", "\n", "4539084576->4539086032*\n", "\n", "\n", "\n", "\n", "\n", "4539086704\n", "\n", "f\n", "\n", "data -2.0000\n", "\n", "grad 4.0000\n", "\n", "\n", "\n", "4539086704->4539084688*\n", "\n", "\n", "\n", "\n", "\n", "4539084688\n", "\n", "L\n", "\n", "data -8.0000\n", "\n", "grad 1.0000\n", "\n", "\n", "\n", "4539084688*->4539084688\n", "\n", "\n", "\n", "\n", "\n" ], "text/plain": [ "" ] }, "execution_count": 58, "metadata": {}, "output_type": "execute_result" } ], "source": [ "draw_dot(L)" ] }, { "cell_type": "code", "execution_count": 60, "id": "8684d681-0e90-435a-a198-46593c7a1948", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Value(data=-6.586368000000001, label=L, grad=0.0)\n" ] } ], "source": [ "a.data += 0.01 * a.grad\n", "b.data += 0.01 * b.grad\n", "c.data += 0.01 * c.grad\n", "f.data += 0.01 * f.grad\n", "d = a * b; d.label = 'd'\n", "e = d + c; e.label = 'e'\n", "L = e * f; L.label = 'L'\n", "print(L)" ] }, { "cell_type": "markdown", "id": "0b34b0e4-e701-4c48-9e46-a9d19267d8ff", "metadata": {}, "source": [ "## Méthodes pour la propagation arrière automatique du gradient" ] }, { "cell_type": "markdown", "id": "4539ac0b-4ab6-4c37-92e9-367fa545de39", "metadata": {}, "source": [ "### Ajout d'une fonction tanh\n", "\n", "$$\\tanh x={\\frac {\\sinh x}{\\cosh x}}={\\frac {e^{x}-e^{-x}}{e^{x}+e^{-x}}}={\\frac {e^{2x}-1}{e^{2x}+1}}$$" ] }, { "cell_type": "code", "execution_count": 61, "id": "79137c51-87e4-4693-8ed0-cd1c4b489001", "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjgAAAGdCAYAAAAfTAk2AAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjgsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvwVt1zgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAQpVJREFUeJzt3Qd8VeXh//FvbnYIYSdhBMKSIRsK4lam0IqjClV/CFX8O7BVrFb8CcpQXFUcVKp1Vin+tNVqRQRRsMosQxyATMPKYISQhMx7/6/nSW5MIIxA7v68+zqvc+655x7OfXpz79dnnTCXy+USAABAEHH4+gIAAABqGwEHAAAEHQIOAAAIOgQcAAAQdAg4AAAg6BBwAABA0CHgAACAoEPAAQAAQSdCIcjpdGrPnj2qW7euwsLCfH05AADgFJi5iQ8fPqxmzZrJ4ThxHU1IBhwTblJSUnx9GQAA4DTs3LlTLVq0OOExIRlwTM2Nu4ASEhJ8fTl+obi4WAsWLNDgwYMVGRnp68sJepQ35R3M+HxT3p6Sk5NjKyjcv+MnEpIBx90sZcINAefnL6S4uDhbHgQcz6O8vYvypryDWSh+vsNOoXsJnYwBAEDQIeAAAICgQ8ABAABBh4ADAACCDgEHAAAEHQIOAAAIOgQcAAAQdAg4AAAg6BBwAABA0PFowPnyyy/1q1/9yt4Uy8w6+MEHH5z0NYsXL1avXr0UHR2tdu3a6fXXXz/mmFmzZik1NVUxMTHq16+fVq5c6aF3AAAAApFHA05eXp66d+9uA8mp2L59u4YPH65LLrlE69at01133aWbb75Zn376acUx77zzjiZMmKCHHnpIa9assecfMmSIMjMzPfhOAABAIPHovaguu+wyu5yq2bNnq3Xr1vrTn/5kH3fq1ElfffWVnnnmGRtijKefflrjxo3T2LFjK17z8ccf69VXX9X999/voXcCAAACiV/dbHPZsmUaOHBglX0m2JiaHKOoqEirV6/WxIkTK553OBz2Nea1x1NYWGiXyncjdd+gzCwoKwt3mcDzKG/vorwp79rgcrlU4nSpuNSpklKXip0ulZhtu3apqHx/ibNsX2nlxeWS0xznNGvZY5wuyVm+v2K7Yl227aq0z1X+2KXKj6WS0lJt3hWmrYs2K8zhsM/LPGf+Z9dlx9lX2id/3le2XXace7/7vR77/o9/TOWj3bt7tayv4V2Ta/XDV5PfKL8KOOnp6UpKSqqyzzw2geTIkSM6ePCgSktLqz1m48aNxz3vjBkzNGXKlGP2L1iwwN6BFT9buHAhxeFFlLd3Ud6hUd4lTim/RDpSWrbOLwlTQalUWL4UOaWi0jAV2nXZ47L9YSq24UMqcenn7UqPXTr5Xax9I1zauV3+ZMv2HQrb6azVc+bn5wdmwPEUU+Nj+u24mcCUkpKiwYMH29vLoywVmy+jQYMGKTIykiLxMMrbuyjvwC5vU8Nx8Eixsg4X2iXTvc4tsut9uYXKOVKiQwXFyjlSrCMmiXhJuCNMEWYJD1Okw6HIcLPtqNjvCCtfux87pAiHQ46wsteGh4XZQTjhDtm12W9eU7aU7TPHmFxlH8u9v+w5s3Y5Xdq7Z7datGguhyO87DmVH2P+V/G4LJyVr6ruq7TfrfJzP+9zv/bnvce8rnzdrUU9DeyUWKvl7W6BCbiAk5ycrIyMjCr7zGMTQmJjYxUeHm6X6o4xrz0eMyLLLEczf3j8mFMmvsRnkPIOZjX5fOcVlmjH/jzt2Jdv19v3me087c4+YkOMadqpqboxEaoXG2mXhJhI1YmOUFxUuF1io8JVJyrCruMqbcdEhisqwqHo8qVsO7zKY7OYkOIOLv4QKOfN26lhw7oG/W9aZA3en18FnP79+2vevHlV9pn/CjD7jaioKPXu3VuLFi3SFVdcYfc5nU77ePz48T65ZgDAqTtSVKrv9hzSNzuztTkjV9ttqMmztTIn06hOlJrUjVZiQowSzbp8aVw3WvVjoyrCjFniYyJsDQlCl0cDTm5urrZs2VJlGLgZ/t2wYUO1bNnSNh3t3r1bb775pn3+1ltv1QsvvKD77rtPv/3tb/X555/r//7v/+woKTfT1HTjjTeqT58+6tu3r2bOnGmHo7tHVQEA/INpWtqccVhrd2ZrnVnSsrUp47DtdFudhnWilNooTqmN66h1ozp23bJhnBITotU4PlqRph0H8IeA89///tfOaePm7gdjAoqZwG/v3r1KS0ureN4METdh5u6779azzz6rFi1a6K9//WvFEHFj5MiRysrK0uTJk22n5B49emj+/PnHdDwGAHg/0HyzK1sLv0/Xwu8demDN58ozvXePYmpdeqTUV6emCWrTpI5Sy8OMqXkBAiLgXHzxxdUONXOrbpZi85q1a9ee8LymOYomKQDwvaISp1Zs369PTaj5IUMZOe6mJlPbUqrYyHB1bVFPPVPq21DTPaW+mtaLqejACniKX/XBAQD4P9MheMmPWVrwfboWbczU4YKSiufqRIXrwvaNlXBkj64fer46N69vRxUB3kbAAQCcUvPT4h8zNWdFmr7cvM/W3Lg1jo/SoM5JGtw5Wee2aySHy6l583arU9O6hBv4DAEHAHBcJsj8a91uvfyfbfoxI7div+n8O+TsJA05O1k9WzaoMmKp2Ivz0ADHQ8ABABzjcEGx/r4yTa9+tUPpOQV2X3x0hEb9IkW/7tNCHZLq0o8Gfo2AAwCokJFToFe/3q45y9N0uLCkYtTT2PNa67p+LRnphIBBwAEAKG1/vp7/fLM+WLdbxaVlo1/bJcbrlgvbaESPZnY2XyCQEHAAIMQ7D7++dIee+HSjCsr7zvRNbaj/d1EbXdIh0S9uRQCcDgIOAIQoc7+n+977Rqt2HLSP+7dppHuHdlCvlg18fWnAGSPgAECIMbdKeO3r7Xry000qLHHauWseGN5J1/VtScdhBA0CDgCEkK1Zubr33W+0Ji3bPj6/XWM9dnVXtWgQ5+tLA2oVAQcAQqTW5pWvtulPC360tTZmyPeDwztp5C9SqLVBUCLgAECQ25KZq3vf+0Zry2ttLjyriWZc1VXN68f6+tIAjyHgAEAQW7n9gG56fZWd06ZudIQm/bKzrunTglobBD0CDgAEKXNDzP/3t//a4d9m6Pezv+mhpvWotUFoIOAAQBD65Nu9+t3ctXbSvks6NNGLN/RWTCST9SF0EHAAIMi8t3qXnd/G6ZKGd22qZ0b2UFSEw9eXBXgVAQcAgsgbS3fooQ+/t9vX9mmhGVd1q3KnbyBUEHAAIEjM+mKLnbzP+O15re0wcG61gFBFwAGAAOdyufT4/E2avWSrffz7Ae1118D2jJRCSCPgAECA3yxz8off6a3lafbx/w7rpHEXtvH1ZQE+R8ABgAAON3949xv9c+1uhYVJj17ZVb/p29LXlwX4BQIOAASoF5dsteEmwhGmp0f20OXdm/n6kgC/wbhBAAhAy7bu158WlHUofuTKLoQb4CgEHAAIMJmHC+wkfmaem6t7tdC1fVJ8fUmA3yHgAECA3RX8939fp6zDheqQVFfTr+jCaCmgGgQcAAggMz/7Ucu27VedqHDNur6XYqO4/QJQHQIOAASIxZsy9fznW+z2o1d1VbvEeF9fEuC3CDgAEAD2ZB/R3e+ss9s3nNNSI3o09/UlAX6NgAMAfq641Knxc9boYH6xujavp0m/7OzrSwL8HgEHAPzcY59s1Jq0bNWNidCs63opOoJ+N8DJEHAAwI/N/y5dr3y13W7/6ZruatkozteXBAQErwScWbNmKTU1VTExMerXr59Wrlx53GMvvvhiO+Tx6GX48OEVx4wZM+aY54cOHeqNtwIAXvPT/jzd++43dnvcBa01+OxkSh/wl1s1vPPOO5owYYJmz55tw83MmTM1ZMgQbdq0SYmJiccc/89//lNFRUUVj/fv36/u3bvrmmuuqXKcCTSvvfZaxePo6GgPvxMA8J6C4lLd/vYaHS4sUe9WDXTf0I4UP+BPNThPP/20xo0bp7Fjx6pz58426MTFxenVV1+t9viGDRsqOTm5Ylm4cKE9/uiAYwJN5eMaNGjg6bcCAF7z1/9s0/d7ctSwTpReuK6nIsPpUQD4TQ2OqYlZvXq1Jk6cWLHP4XBo4MCBWrZs2Smd45VXXtGoUaNUp06dKvsXL15sa4BMsLn00ks1ffp0NWrUqNpzFBYW2sUtJyfHrouLi+2CsrJwlwk8j/L2rkArbzNL8Z8Xb7Xb/3tZBzWOiwiYaw/E8g50oVTexTV4j2Eul8vlqQvZs2ePmjdvrqVLl6p///4V+++77z4tWbJEK1asOOHrTV8d06xljuvbt2/F/rlz59pandatW2vr1q164IEHFB8fb0NTePixowsefvhhTZky5Zj9c+bMsecBAH8yd6tDyzIdahXv0t1dShUW5usrAvxDfn6+rrvuOh06dEgJCQm+7YNzJkztTdeuXauEG8PU6LiZ57t166a2bdvaWp0BAwYccx5Tg2T6AVWuwUlJSdHgwYNPWkChwqRi0xw4aNAgRUZG+vpygh7lTXkfz8b0w1q+vKyG+4nf9FOvlvUVaPh8U96e4m6BORUeDTiNGze2NSoZGRlV9pvHpt/MieTl5dmamqlTp57032nTpo39t7Zs2VJtwDH9darrhGx+yPkxp0x8ic8g5V2ZqVB//NPNMvXqw7s2Vb+2TRTI+HxT3rWtJr/ZHu21FhUVpd69e2vRokUV+5xOp31cucmqOu+++67tN3PDDTec9N/ZtWuXHW3VtGnTWrluAPCFxZuy9NWWfYoKd+iPjJoCzojHu+WbpqGXX35Zb7zxhjZs2KDbbrvN1s6YUVXG6NGjq3RCrtw8dcUVVxzTcTg3N1f33nuvli9frh07dtiwNGLECLVr184OPweAQFRS6tQj8zbY7THnpTKhH3CGPN4HZ+TIkcrKytLkyZOVnp6uHj16aP78+UpKSrLPp6Wl2ZFVlZk5cr766istWLDgmPOZJq/169fbwJSdna1mzZrZvjTTpk1jLhwAAevvq3ZqS2auGsRF6o5L2vn6coCA55VOxuPHj7dLdUzH4KN16NDBtkVXJzY2Vp9++mmtXyMA+EpOQbGeWfij3b570FmqF0tHf+BMMXMUAPjYrC+26EBekdo2qaPf9G3p68sBggIBBwB8aOeBfL321Q67/cCwTsxYDNQSAg4A+NDj8zeqqNSp89o10qUdj70/H4DTQ8ABAB9Z/dNB/Xv9XjtT8f8O66wwpiwGag0BBwB8wAykmP7xD3b72t4p6tyMWdWB2kTAAQAfMDU3a9OyFRcVrnsGn8X/B0AtI+AAgJcVFJfqsU822u1bL2qrxIQY/j8AahkBBwC87K3lP2l39hElJ8Ro3AVtKH/AAwg4AODlWzK89nXZsPDfD2yv2Khwyh/wAAIOAHjRZxsybe2NuSXDlT2bU/aAhxBwAMCLXvt6u11f16+lYiKpvQE8hYADAF7yw54crdh+QOGOMN1wTivKHfAgAg4AeMnrS8tqby7rkqym9WIpd8CDCDgA4AXmZpofrNtjt8eel0qZAx5GwAEAL/j7yjQVlTjVtXk99WrZgDIHPIyAAwAeVlzq1N+W/VRRe8M9pwDPI+AAgIfN/y5d6TkFahwfreHdmlLegBcQcADAw15fWjax3/X9Wio6gqHhgDcQcADAg9bvytbqnw4qMjzMBhwA3kHAAQAPer38tgzDuzblppqAFxFwAMBDMg8X6KP17qHhrSlnwIsIOADgIXNWpKm41KWeLeure0p9yhnwIgIOAHiAmfPmreVpdpvaG8D7CDgA4AEff7tH+3ILlZQQbW/NAMC7CDgAUMtcLpdeK+9cfEO/VooM56sW8Db+6gCglq1Jy9b6XYcUFeHQdQwNB3yCgAMAHprY7/LuzdQoPpryBXyAgAMAtSj9UIE++Xav3R5zLncNB3yFgAMAteit5T+pxOlS39SG6tK8HmUL+AgBBwBqidPp0j/W7LLbo89tRbkCPkTAAYBasmrHAe09VKC60REa2CmJcgWCPeDMmjVLqampiomJUb9+/bRy5crjHvv6668rLCysymJed/QQzMmTJ6tp06aKjY3VwIEDtXnzZi+8EwA4vg+/Kbstw5AuyYqJ5K7hQFAHnHfeeUcTJkzQQw89pDVr1qh79+4aMmSIMjMzj/uahIQE7d27t2L56aefqjz/xBNP6LnnntPs2bO1YsUK1alTx56zoKDA028HAKpVXOrUvPLOxWb0FIAgDzhPP/20xo0bp7Fjx6pz5842lMTFxenVV1897mtMrU1ycnLFkpSUVKX2ZubMmXrwwQc1YsQIdevWTW+++ab27NmjDz74wNNvBwCq9dXmfTqYX6zG8VE6t20jSgnwsQhPnryoqEirV6/WxIkTK/Y5HA7bpLRs2bLjvi43N1etWrWS0+lUr1699Oijj+rss8+2z23fvl3p6en2HG716tWzTV/mnKNGjTrmfIWFhXZxy8nJsevi4mK7oKws3GUCz6O8g6+8P1hb1rn4srOT5HKWqthZqlDF55vy9pSa/A17NODs27dPpaWlVWpgDPN448aN1b6mQ4cOtnbH1MwcOnRITz31lM4991x9//33atGihQ037nMcfU73c0ebMWOGpkyZcsz+BQsW2Nok/GzhwoUUhxdR3sFR3kWl0vxvTZ+bMDXK265587Z75N8JNHy+Ke/alp+f7x8B53T079/fLm4m3HTq1El/+ctfNG3atNM6p6lBMv2AKtfgpKSkaPDgwba/D8pSsfkyGjRokCIjIykSD6O8g6u8532brsKV69W8foxuv/YC28weyvh8U96e4m6B8XnAady4scLDw5WRkVFlv3ls+tacCvNl1LNnT23ZssU+dr/OnMOMoqp8zh49elR7jujoaLtUd25+zCkTX+IzGBzl/fF3Zd9xl/dorqioqFo/f6Di801517aa/P16tJOx+UPv3bu3Fi1aVLHP9KsxjyvX0pyIaeL69ttvK8JM69atbcipfE6T6MxoqlM9JwDUlkNHirV4U5bdZvQU4D883kRlmoZuvPFG9enTR3379rUjoPLy8uyoKmP06NFq3ry57SdjTJ06Veecc47atWun7OxsPfnkk3aY+M0332yfN1W/d911l6ZPn6727dvbwDNp0iQ1a9ZMV1xxhaffDgBU8en36Soqdap9Yrw6JteldIBQCTgjR45UVlaWnZjPdAI2zUjz58+v6CSclpZmR1a5HTx40A4rN8c2aNDA1gAtXbrUDjF3u++++2xIuuWWW2wIOv/88+05j54QEAA87aPyyf1M7U2o970B/IlXOhmPHz/eLtVZvHhxlcfPPPOMXU7EfImYmh6zAICvZB4u0Ndb9tnty3swuR/gT7gXFQCcpnnr98rpkrqn1FerRnUoR8CPEHAA4AzvPUXnYsD/EHAA4DTsPJCvNWnZMt1uftXt5ykrAPgHAg4AnIaP1pfV3vRv00iJCQxwAPwNAQcATsOH62ieAvwZAQcAaujHjMPamH5YkeFhuqwLzVOAPyLgAMBp1t5cdFai6sVx7zbAHxFwAKAGXC7Xz6OnmPsG8FsEHACogXU7s5V2IF+xkeEa2CmRsgP8FAEHAGrAXXszqHOS4qK8Mhk8gNNAwAGAU1TqdOnf6/fa7RE0TwF+jYADAKdoxbb9yjpcqHqxkbqgfRPKDfBjBBwAqGHz1LCuyYqK4OsT8Gf8hQLAKSgpdeqT79Lt9q+6c+dwwN8RcADgFPz3p4M6dKRYDetEqV/rRpQZ4OcIOABwChZtyLDrizs0UbgjjDID/BwBBwBOwaINmXY9oGMS5QUEAAIOAJzEtqxcbduXZ+89deFZjSkvIAAQcADgJD7fWFZ7Y/re1I3h3lNAICDgAMBJfFbe/2YAt2YAAgYBBwBO4FB+sVbtOGi36X8DBA4CDgCcwOIfM+0tGtonxqtlozjKCggQBBwAOJXRU50YPQUEEgIOAJxg9uLFm8oCzkD63wABhYADACeYvTinoEQN4iLVs2UDygkIIAQcADjJ7MWXdEhk9mIgwBBwAOA46H8DBC4CDgBUg9mLgcBGwAGAE9TeMHsxEJgIOABQjUUbmb0YCGQEHAA4CrMXA4HPKwFn1qxZSk1NVUxMjPr166eVK1ce99iXX35ZF1xwgRo0aGCXgQMHHnP8mDFjFBYWVmUZOnSoF94JgFDA7MVA4PN4wHnnnXc0YcIEPfTQQ1qzZo26d++uIUOGKDOzrH37aIsXL9ZvfvMbffHFF1q2bJlSUlI0ePBg7d69u8pxJtDs3bu3Yvn73//u6bcCIEQwegoIfB4POE8//bTGjRunsWPHqnPnzpo9e7bi4uL06quvVnv822+/rdtvv109evRQx44d9de//lVOp1OLFi2qclx0dLSSk5MrFlPbAwBnqpjZi4GgEOHJkxcVFWn16tWaOHFixT6Hw2GbnUztzKnIz89XcXGxGjZseExNT2Jiog02l156qaZPn65GjRpVe47CwkK7uOXk5Ni1Oa9ZUFYW7jKB51He/lveK7YfqJi9uEvTeP4mPFzeOHOhVN7FNXiPHg04+/btU2lpqZKSqt6kzjzeuHHjKZ3jj3/8o5o1a2ZDUeXmqauuukqtW7fW1q1b9cADD+iyyy6zoSk8PPyYc8yYMUNTpkw5Zv+CBQtsbRJ+tnDhQorDiyhv/yvvD3aYim2H2sUV6tP5n3jluoIVn2/Ku7aZSg+/CDhn6rHHHtPcuXNtbY3poOw2atSoiu2uXbuqW7duatu2rT1uwIABx5zH1CCZfkCVa3DcfXsSEhK88E4CIxWbL6NBgwYpMjLS15cT9Chv/y3vmTO/Ml+j+p8BPXRZl2SvXWMw4fNNeXuKuwXG5wGncePGtkYlI6NsPgk389j0mzmRp556ygaczz77zAaYE2nTpo39t7Zs2VJtwDH9dcxyNPNFx485ZeJLfAb9q7zN7MXb9+crMjxMl3RK5vvBw+WN2hUK5R1Zg/fn0U7GUVFR6t27d5UOwu4Ow/379z/u65544glNmzZN8+fPV58+fU767+zatUv79+9X06ZNa+3aAYQeZi8GgofHR1GZpiEzt80bb7yhDRs26LbbblNeXp4dVWWMHj26Sifkxx9/XJMmTbKjrMzcOenp6XbJzc21z5v1vffeq+XLl2vHjh02LI0YMULt2rWzw88B4ExnL760YyKFCAQ4j/fBGTlypLKysjR58mQbVMzwb1Mz4+54nJaWZkdWub344ot29NWvf/3rKucx8+g8/PDDtslr/fr1NjBlZ2fbDsimL42p8amuGQoAajp78cBOVQdGAAg8XulkPH78eLtUx3QMrszUypxIbGysPv3001q9PgBg9mIguHAvKgBg9mIg6BBwAIQ8Zi8Ggg8BB0DIW5uWbWcvrh8XqZ4tue0LEAwIOABC3pc/ZtkyuKB9E4U7wkK+PIBgQMABEPK+3FwWcC5s3zjkywIIFgQcACHtQF6Rvt19yG5feFYTX18OgFpCwAEQ0v6zOUsul9Qxua6SEn6+5x2AwEbAARDSvvxxn11TewMEFwIOgJDlcrlsDY5xYXuap4BgQsABELI2ph9W5uFCxUQ61CeV4eFAMCHgAFCoDw8/p00jxUSG+/pyANQiAg6AkPXz8HCap4BgQ8ABEJLyi0q0anvZ3cPpYAwEHwIOgJC0YtsBFZU61bx+rNo2qePrywFQywg4AELSkvL+Nxee1VhhYdyeAQg2BBwAIYn+N0BwI+AACDm7DuZrW1aevbHmue24/xQQjAg4AEJ29uKeKfVVLzbS15cDwAMIOABCdv4bRk8BwYuAAyCklJQ69fVW7j8FBDsCDoCQsm5ntg4XlKh+XKS6Nq/n68sB4CEEHAAh2Tx1frvGtpMxgOBEwAEQUpZspnkKCAUEHAAh42B+kdbvyrbb3H8KCG4EHAAhY+nWA3K5pA5JdZVcL8bXlwPAgwg4AELGf7a4m6eY3A8IdgQcACHB1Nx8tXm/3Wb+GyD4EXAAhIS9R6SMw4WKiXToF6kNfX05ADyMgAMgJGzMLhsS3q91I8VEhvv6cgB4GAEHQEgFHJqngNBAwAEQ9I4UlWprTlnAuYgOxkBI8ErAmTVrllJTUxUTE6N+/fpp5cqVJzz+3XffVceOHe3xXbt21bx586o873K5NHnyZDVt2lSxsbEaOHCgNm/e7OF3ASBQrfrpoEpcYWpaL0Ztm8T7+nIABEPAeeeddzRhwgQ99NBDWrNmjbp3764hQ4YoMzOz2uOXLl2q3/zmN7rpppu0du1aXXHFFXb57rvvKo554okn9Nxzz2n27NlasWKF6tSpY89ZUFDg6bcDIAD9p3z24gvaNVJYGLdnAEKBxwPO008/rXHjxmns2LHq3LmzDSVxcXF69dVXqz3+2Wef1dChQ3XvvfeqU6dOmjZtmnr16qUXXnihovZm5syZevDBBzVixAh169ZNb775pvbs2aMPPvjA028HQAD6z5ay4eHnt2vk60sB4CURnjx5UVGRVq9erYkTJ1bsczgctklp2bJl1b7G7Dc1PpWZ2hl3eNm+fbvS09PtOdzq1atnm77Ma0eNGnXMOQsLC+3ilpOTY9fFxcV2QVlZuMsEnkd5e8/eQwXampWnMLn0i5YJfMa9gM+3d4VSeRfX4D16NODs27dPpaWlSkpKqrLfPN64cWO1rzHhpbrjzX738+59xzvmaDNmzNCUKVOO2b9gwQJbm4SfLVy4kOLwIsrb85ZlmCapcLWKl1Z+tdgL/yLc+Hx7VyiUd35+vn8EHH9hapAq1wqZGpyUlBQNHjxYCQkJPr02f0rF5o9j0KBBioyM9PXlBD3K23s+mfuNpAx1rO/k8+0lfL69K5TKO6e8BcbnAadx48YKDw9XRkZGlf3mcXJycrWvMftPdLx7bfaZUVSVj+nRo0e154yOjrbL0cwHIdg/DDVFmVDewaSk1KmlW8v633Sq7+Lz7WV8n1Deta0mv9ke7WQcFRWl3r17a9GiRRX7nE6nfdy/f/9qX2P2Vz7eMMnUfXzr1q1tyKl8jEl0ZjTV8c4JIDR9sytbOQUlqhcboZaMDgdCisebqEzT0I033qg+ffqob9++dgRUXl6eHVVljB49Ws2bN7f9ZIzf//73uuiii/SnP/1Jw4cP19y5c/Xf//5XL730kn3eDPG86667NH36dLVv394GnkmTJqlZs2Z2ODkAuC35sWx4+HltG8kRtpuCAUKIxwPOyJEjlZWVZSfmM52ATTPS/PnzKzoJp6Wl2ZFVbueee67mzJljh4E/8MADNsSYEVRdunSpOOa+++6zIemWW25Rdna2zj//fHtOMzEgALh9+WOWXZ/frrGUQcABQolXOhmPHz/eLtVZvPjYUQ3XXHONXY7H1OJMnTrVLgBQnez8Iq3flV0x/83aql37AAQ57kUFICh9tWWfnC7prKR4e4sGAKGFgAMgqJunLmzfxNeXAsAHCDgAgo65pcuX5R2MLzyLgAOEIgIOgKCzOTNX6TkFio5wqG/rhr6+HAA+QMABELTNU/3aNFJMZLivLweADxBwAASdJRX9bxr7+lIA+AgBB0BQKSgu1crtB+z2RfS/AUIWAQdAUFmx/YAKS5x2aHi7RO7PAIQqAg6AoB0ebiYFBRCaCDgAgjPg0DwFhDQCDoCgsSf7iB0i7ggrv/8UgJBFwAEQNP6zuaz2pntKfdWLi/T15QDwIQIOgKBRMXsxt2cAQh4BB0BQKHW67A02DfrfACDgAAgK3+zK1qEjxUqIiVD3FvV8fTkAfIyAAyCoRk+d376xIsL5agNCHd8CAIJu/hsAIOAACHiH8ou1bme23ab/DQCDgAMg4H29dZ+cLtlbMzSrH+vrywHgBwg4AAIezVMAjkbAARDQXC6XlpT3v7moA/1vAJQh4AAIaFsyc7X3UIGiIxzq17qhry8HgJ8g4AAIaO7am76tGyomMtzXlwPATxBwAAS0LzeXzV58EXcPB1AJAQdAwCooLtWKbfvtNsPDAVRGwAEQsFZuP6DCEqeSE2LUPjHe15cDwI8QcAAEfP+bC89qrLCwMF9fDgA/QsABELA+35hp15d0SPT1pQDwMwQcAAFpa1autu/LU1S4QxfQwRjAUQg4AALSog0Zdt2vTUPFR0f4+nIA+BkCDoCA9NmGsuapgZ2SfH0pAEIt4Bw4cEDXX3+9EhISVL9+fd10003Kzc094fF33nmnOnTooNjYWLVs2VK/+93vdOjQoSrHmc6ERy9z58715FsB4Eey84u0+qeDdvvSjvS/AXAsj9brmnCzd+9eLVy4UMXFxRo7dqxuueUWzZkzp9rj9+zZY5ennnpKnTt31k8//aRbb73V7nvvvfeqHPvaa69p6NChFY9NgAIQGhZvylKp06UOSXWV0jDO15cDIJQCzoYNGzR//nytWrVKffr0sfuef/55DRs2zAaYZs2aHfOaLl266B//+EfF47Zt2+qRRx7RDTfcoJKSEkVERFQJNMnJyZ66fAB+bFH56KkBnai9AeDlgLNs2TIbQtzhxhg4cKAcDodWrFihK6+88pTOY5qnTBNX5XBj3HHHHbr55pvVpk0bW8tjaoeONw9GYWGhXdxycnLs2tQqmQVlZeEuE3ge5X0GZVfq1OJNZQHn4vaNTukzS3l7F+VNeXtKTX6jPBZw0tPTlZhY9b+uTEhp2LChfe5U7Nu3T9OmTbPNWpVNnTpVl156qeLi4rRgwQLdfvvttm+P6a9TnRkzZmjKlCnH7DevNefAz0xzIryH8q65zYfCdLggXPERLu3+dqn2fkd5+ys+35R3bcvPz/dcwLn//vv1+OOPn7R56kyZWpbhw4fbvjgPP/xwlecmTZpUsd2zZ0/l5eXpySefPG7AmThxoiZMmFDl3CkpKRo8eLCtHUJZKjZfRoMGDVJkZCRF4mGU9+l79JNNkn7SoK7N9cvhXShvP8Tnm/L2FHcLjEcCzj333KMxY8ac8BjTbGT6x2RmllUju5l+NGak1Mn6zhw+fNh2IK5bt67ef//9k/7g9uvXz9b0mGao6OjoY543+6rbb87Ljzll4kt8BmvG5XLpi01lt2cY3Dm5xn+/lLd3Ud6Ud22ryd98jQNOkyZN7HIy/fv3V3Z2tlavXq3evXvbfZ9//rmcTqcNJCdKZ0OGDLGB5MMPP1RMTMxJ/61169apQYMG1YYYAMFj27487difz+zFAHzXB6dTp062FmbcuHGaPXu2rbIcP368Ro0aVTGCavfu3RowYIDefPNN9e3b14Yb02xk2tjeeust+9hdHWVCVXh4uD766CNlZGTonHPOseHHNKs8+uij+sMf/uCptwLATzB7MQC/mAfn7bfftqHGhBgzeurqq6/Wc889V/G8CT2bNm2q6DS0Zs0aO8LKaNeuXZVzbd++XampqbZ6atasWbr77rttdbU57umnn7ZBCkBozF48gMn9APgy4JgRU8eb1M8wgcWEFLeLL764yuPqmFqhyhP8AQi92YsHcHsGACfBvagABARmLwZQEwQcAAGB2YsB1AQBB0BAzV5M8xSAU0HAAeD3Vu04oMMFJWpYJ0o9UrixLoCTI+AA8HuLykdPXdIhUeGO6u85BwCVEXAA+DUzstI9/81A7h4O4BQRcAD4ta1ZzF4MoOYIOAD82ucby2pv+rVpqPhoj07dBSCIEHAA+DVmLwZwOgg4APwWsxcDOF0EHAB+i9mLAZwuAg4Av/VZ+eipAYyeAlBDBBwAfjt78ZIfs+w2sxcDqCkCDgC/xOzFAM4EAQeAX2L2YgBngoADwC9nL57/XbrdZvZiAKeDgAPA76xJO6jd2UdUJypcl3RM9PXlAAhABBwAfudf6/bY9ZCzkxUTGe7rywEQgAg4APxKSalT877da7d/1aOZry8HQIAi4ADwK0u37te+3CI1iIvU+e0a+/pyAAQoAg4Av/LhN2XNU8O7NVVkOF9RAE4P3x4A/EZBcak+LR89dXn35r6+HAABjIADwG8s3pSpw4UlalovRn1aNfD15QAIYAQcAH7XPPWr7s3kcIT5+nIABDACDgC/cLiguGL24su7M3oKwJkh4ADwCwt/yFBhiVNtmtTR2c0SfH05AAIcAQeAX03uZ2pvwsJongJwZgg4AHxuf26hvtqyz27TPAWgNhBwAPjcvO/SVep0qWvzemrTJN7XlwMgCBBwAPjcR5WapwCgNhBwAPjUnuwjWrnjgEy3m192b8r/GwD8P+AcOHBA119/vRISElS/fn3ddNNNys3NPeFrLr74YtvBsPJy6623VjkmLS1Nw4cPV1xcnBITE3XvvfeqpKTEk28FgId8VD73zS9SG6ppvVjKGUCtiJAHmXCzd+9eLVy4UMXFxRo7dqxuueUWzZkz54SvGzdunKZOnVrx2AQZt9LSUhtukpOTtXTpUnv+0aNHKzIyUo8++qgn3w4AD07uN4I7hwMIhICzYcMGzZ8/X6tWrVKfPn3svueff17Dhg3TU089pWbNjt/WbgKNCTDVWbBggX744Qd99tlnSkpKUo8ePTRt2jT98Y9/1MMPP6yoqChPvSUAtWxrVq6+35OjCEeYhnWheQpAAAScZcuW2WYpd7gxBg4cKIfDoRUrVujKK6887mvffvttvfXWWzbk/OpXv9KkSZMqanHMebt27WrDjduQIUN022236fvvv1fPnj2POV9hYaFd3HJycuza1CqZBWVl4S4TeB7lXeaDNTvt+rx2jRQfFeaxzx/l7V2UN+XtKTX5jvBYwElPT7f9Y6r8YxERatiwoX3ueK677jq1atXK1vCsX7/e1sxs2rRJ//znPyvOWzncGO7HxzvvjBkzNGXKlGprgyo3f0G2ORHeE8rl7XJJc9eFSwpTijND8+bN8/i/Gcrl7QuUN+Vd2/Lz8z0XcO6//349/vjjJ22eOl2mj46bqalp2rSpBgwYoK1bt6pt27andc6JEydqwoQJVWpwUlJSNHjwYNsBGmWp2HwZDRo0yPZngmdR3tJ3u3OUtXy5YiIdumfUpYqP9lyXQMrbuyhvyttT3C0wp6LG3yj33HOPxowZc8Jj2rRpY5uXMjPLbpznZkY6mZFVx+tfU51+/frZ9ZYtW2zAMa9duXJllWMyMjLs+njnjY6OtsvRzA85P+aUiS+F8mdw3vdlf7cDOiWpQbx3Rk+Fcnn7AuVNede2mvz91jjgNGnSxC4n079/f2VnZ2v16tXq3bu33ff555/L6XRWhJZTsW7dOrs2NTnu8z7yyCM2PLmbwEzNg6mJ6dy5c03fDgAfcDpd+vf6vXabyf0ABNQ8OJ06ddLQoUPtkG9T4/L1119r/PjxGjVqVMUIqt27d6tjx44VNTKmGcqMiDKhaMeOHfrwww/tEPALL7xQ3bp1s8eYZiUTZP7nf/5H33zzjT799FM9+OCDuuOOO6qtpQHgf1btOKC9hwpUNyZCF3c4+X8wAYBfTfRnRkOZAGP60Jjh4eeff75eeumlKu20pgOxu9OQGeJthn+bEGNeZ5rDrr76an300UcVrwkPD9e///1vuza1OTfccIMNQZXnzQHg3/5VPvfNZV2SFR1hOhoDQABN9GdGTJ1oUr/U1FS5zFCKcqbj75IlS056XjPKyhsjLgDUvoLiUs371t081ZwiBuAR3IsKgFf9a91uZecXq3n9WPVv24jSB+ARBBwAXmNqbF/7eofdvvHcVgp3hFH6ADyCgAPAa5ZvO6CN6YcVGxmukX1aUvIAPIaAA8BrXl+63a6v6tVc9eKYjwaA5xBwAHjFzgP5WvhD2eR+Y85NpdQBeBQBB4BX/G35T3K6pAvaN1b7pLqUOgCPIuAA8Lj8ohLNXZlmt6m9AeANBBwAHvfPNbuVU1CiVo3idEmHslusAIAnEXAAeHxo+OtLy4eG90+Vg6HhALyAgAPAo77ask9bMnNVJypcv+7TgtIG4BUEHAAe9Xr5xH7X9ElRQgxDwwF4BwEHgMfs2Jenzzdl2u3R/VtR0gC8hoADwGPeWLZD5n66l3RoojZN4ilpAF5DwAHgEYcLivXuf3fZ7THntaaUAXgVAQeAR/xj9S7lFpaoTZM6uqBdY0oZgFcRcADUOqfTpTeW/WS3x57L0HAA3kfAAVDrlvyYpe378lQ3JkJX9WJoOADvI+AAqHWvlU/sN7JPiupER1DCALyOgAOgVplJ/b78MUthYWZoOHcNB+AbBBwAteqN8tqbAR2T1LJRHKULwCcIOABqzcG8Iv1jTdnQ8N+eR+0NAN8h4ACoNc8u2qz8olJ1bpqg/m0bUbIAfIaAA6BWbMvK1VvLy4aGPzCsk8JMJxwA8BECDoBaMeOTjSpxunRpx0Sd356J/QD4FgEHwBlbtnW/Fv6QoXBHmB4Y1pESBeBzBBwAZzxr8fSPf7Db1/VtqXaJdSlRAD5HwAFwRv65dre+35OjutERumtge0oTgF8g4AA4bflFJXry0412e/yl7dQoPprSBOAXCDgATttLX25TRk6hUhrG6sZzmfcGgP8g4AA4LRk5BfrLkm12+49DOyomMpySBOA3CDgATstTn27SkeJS9WpZX8O7NqUUAYROwDlw4ICuv/56JSQkqH79+rrpppuUm5t73ON37NhhJwerbnn33Xcrjqvu+blz53ryrQCo5Lvdh/Re+S0ZHvxlZyb1A+B3Ijx5chNu9u7dq4ULF6q4uFhjx47VLbfcojlz5lR7fEpKij2+spdeeklPPvmkLrvssir7X3vtNQ0dOrTisQlQADzP5XLpkY83yOWSLu/eTL1aNqDYAYROwNmwYYPmz5+vVatWqU+fPnbf888/r2HDhumpp55Ss2bNjnlNeHi4kpOTq+x7//33de211yo+Pr7KfhNojj4WgOct2pCpZdv2KyrCofuGdqDIAYRWwFm2bJkNIe5wYwwcOFAOh0MrVqzQlVdeedJzrF69WuvWrdOsWbOOee6OO+7QzTffrDZt2ujWW2+1tUPHu/dNYWGhXdxycnLs2tQqmQVlZeEuE3heoJZ3calTj5RP6je2fyslxUcGxHsI1PIOVJQ35e0pNfkb9ljASU9PV2JiYtV/LCJCDRs2tM+dildeeUWdOnXSueeeW2X/1KlTdemllyouLk4LFizQ7bffbvv2/O53v6v2PDNmzNCUKVOO2W9ea86Bn5nmRHhPoJX3l3vDtH1/uOIjXGpTsFnz5m1WIAm08g50lDflXdvy8/M9F3Duv/9+Pf744ydtnjpTR44csX11Jk2adMxzlff17NlTeXl5tp/O8QLOxIkTNWHChCo1OKa/z+DBg20HaJSlYvNlNGjQIEVGRlIkHhaI5X3oSLEenvmVuXrdN6yzrvpFigJFIJZ3IKO8KW9PcbfAeCTg3HPPPRozZswJjzHNRqZ/TGZmZpX9JSUldmTVqfSdee+992xSGz169EmP7devn6ZNm2aboaKjj51J1eyrbr/5ouPLjjLxpUD5DJqOxQ/+a70O5herfWK8ruuXqojwwJtlIlDKO1hQ3pR3bavJ32+NA06TJk3scjL9+/dXdna27UfTu3dvu+/zzz+X0+m0geRUmqcuv/zyU/q3TD+dBg0aVBtiAJy5177eofnfpysyPExPXtM9IMMNgNDisT44pu+MGcY9btw4zZ4921ZZjh8/XqNGjaoYQbV7924NGDBAb775pvr27Vvx2i1btujLL7/UvHnzjjnvRx99pIyMDJ1zzjmKiYmx1c6PPvqo/vCHP3jqrQAhbU3aQT06r6zZ+X+HdVKPFKZkABDi8+C8/fbbNtSYEGNGT1199dV67rnnKp43oWfTpk3HdBp69dVX1aJFC9tHprrqKTOq6u6777bV5u3atdPTTz9tgxSA2nUwr0jj316jEqfLzlbM/aYABAqPBhwzYup4k/oZqampNqQczdTImKU6plao8gR/ADzD6XTp7v9bpz2HCtS6cR09dnVXZiwGEDBoSAdQrReXbNXiTVmKjnBo1nW9VDeGzrkAAgcBB8Axlm3drz8t2GS3p444W52bMZ0CgMBCwAFQRebhAt3597VyuqSre7XQtX0CZ74bAHAj4ACoUOp06Xd/X6t9uYU6Kyle0644m343AAISAQdAhWcW/qjl2w6oTlS4/nx9b8VFeXQcAgB4DAEHgPXFpky98MUWu/3oVV3VLjGekgEQsAg4ALQ7+4gmvLPOlsQN57TUiB7NKRUAAY36ZyDE7TyQr+v/usLeZ6pr83qa9MvOvr4kADhjBBwghG3JPGzDTUZOoVo2jNOLN/RSdES4ry8LAM4YAQcIUd/tPqTRr67Ugbwie4fwt27up6SEGF9fFgDUCgIOEIJW7Tig3762SocLS9StRT29PravGtaJ8vVlAUCtIeAAIebLH7N0y9/+q4Jip/q2bqhXbuzDbRgABB0CDhBC5n+XbifyKyp16qKzmmj2Db0VG0WfGwDBh4ADhIh/rtmle99bb2crHtY1WTNH9lRUBDNFAAhOBBwgBPxt2Q5N+tf3dvvXvVvosau6KiKccAMgeBFwgCBWVOLUzM9+1J8Xb7WPx5ybqsm/7CyHI8zXlwYAHkXAAYJ4GPgf3v1GG9MP28d3XtpOEwadxc0zAYQEAg4QZApLSvX8oi16cclW29/GDP+eOuJs/bJbM19fGgB4DQEHCCLrd2XbWpsfM3Lt4+Hdmmrq5WerUXy0ry8NALyKgAMEgYLiUj27aLNe+nKbrbVpHB+laSO66LKuTX19aQDgEwQcIMCtTTtoh39vySyrtbm8ezM9fPnZzEwMIKQRcIAAtSf7iK2xeXPZDjldUuP4aE2/oouGdkn29aUBgM8RcIAAs2Fvjl7+cps+/GaPSkyykXRlz+Z2+HcD7icFABYBBwgALpdLy7bt11+WbNOSH7Mq9vdv00i3X9JWF7Rv4tPrAwB/Q8AB/FhJqVOffJdum6K+3X3I7jNz9JnOw//vwjbq1qK+ry8RAPwSAQfwQ3sPHdG8b9P1+tLt2nngiN0XE+nQtX1SdPP5bdSyUZyvLxEA/BoBB/CTJigzCmrBDxn69Pt0rd9VVltjNIiL1I3npmp0/1RGRgHAKSLgAD5i+gev3ZmtRZv2aeH3Gdq2L6/iubAwqXfLBhrRo5l+3TtFsVHh/P8EADVAwAG8xOl0afv+PK1Ly9bK7fv1yTfhylm+suL5qHCHzmvXSIPPTtbATklqUpfZhwHgdBFwAA/Zl1tow8w3u7K1bme2vtmZrZyCkkpHhCk+OkKXdkzU4LOTdNFZTVQ3JpL/PwCgFhBwgDN0KL/Y1szs2Jen7fvytCUr14aZXQfLOgdXFh3hUNfm9dS1eYKiD2zT+JEDFR9LTQ0ABEzAeeSRR/Txxx9r3bp1ioqKUnZ29il1tHzooYf08ssv2+PPO+88vfjii2rfvn3FMQcOHNCdd96pjz76SA6HQ1dffbWeffZZxcfHe+qtIMSZ+zxlHS5U5uEC7ckuKAsylQLNwfzial9n+tG0axKv7in11aN86ZBcV5HhDhUXF2vevK028AAAAijgFBUV6ZprrlH//v31yiuvnNJrnnjiCT333HN644031Lp1a02aNElDhgzRDz/8oJiYGHvM9ddfr71792rhwoX2R2Ls2LG65ZZbNGfOHE+9FQRhYMk5UqxDRy0H8oqUaYJMTkHZuny7arNS9RLrRiu1cR21blRHrZvUKaulaVFPCTQ5AUBwBZwpU6bY9euvv35Kx5vam5kzZ+rBBx/UiBEj7L4333xTSUlJ+uCDDzRq1Cht2LBB8+fP16pVq9SnTx97zPPPP69hw4bpqaeeUrNmzTz1duBh5g7YxaVOe+sBM7ldcalLJU6nCoudKiotWxeWlKqoxKzdS6ldHykqVX5RqY4UlSjvqO2y50psSDEhxgQb85qaiopw2BCTnBCjVibENI4rCzSN6yi1UR3Viaa1FwD8id98K2/fvl3p6ekaOHBgxb569eqpX79+WrZsmQ04Zl2/fv2KcGOY401T1YoVK3TllVdWe+7CwkK7uOXk5Ni1qQEyS21Zk5atj79Nr7Kv7E5BR3G5qn2+0u7y51xV9ruqPc5lH1d+zryuymtcPx9T+fmy58oelzqdykh36N8H19q2FRM4zfNO92vLjzNDm81oILPfbpevTUBxVdq2i3lsAkv58XZdvt88X1LqUnF5sDn6vXuaaT5KiImwNSz1YiOVEBuhBnFRNsQ0qRulxHizLlvMPnNsmHlRtVw1/hy5j6/Nzx8ob3/B55vy9pSafGf6TcAx4cYwNTaVmcfu58w6MTGxyvMRERFq2LBhxTHVmTFjRkWNUmULFixQXFztzQi7NCNM72wL5PlKHNKBn+9z5GvhYS6ZLiqRYbJru4RJkRXbZc9HO8wQ65/XUQ6XosPd22VLbIQUF+FSbLhZyz7vCDNNTwVV/1ETtEz+zZFMrzGzbPbgezRNrfAeytu7KG/Ku7bl5+d7JuDcf//9evzxx094jGlG6tixo/zJxIkTNWHChCo1OCkpKRo8eLASEhJq7d9pseuQGm7MPGZ/mI79L//KlQGVnz26kqDitVVXVWoTzJb7YdjRz7n3h5Wdq2xddkzlbWdpqTZt2qjOnTopIiLcHmvueWSPMceWnyPcvq7suXDHz9sOs3b8vG2ei7CPy9fVPQ4PU6QjzHa6NdsRDoci7brs9cevMQmO/woxX/6DBg1SZCRDwynv4MLnm/L2FHcLTK0HnHvuuUdjxow54TFt2rTR6UhOTrbrjIwMNW3atGK/edyjR4+KYzIzqwaIkpISO7LK/frqREdH2+Vo5oelNn9cerdubJdAZEf1HNqgYf1T+cH1otr+DILy9id8vinv2laT78saBZwmTZrYxRPMqCkTUhYtWlQRaExSM31rbrvtNvvYjMgyw8dXr16t3r17232ff/65nE6n7asDAABgeGwSjrS0NDsHjlmXlpbabbPk5uZWHGOast5//327bZoj7rrrLk2fPl0ffvihvv32W40ePdqOjLriiivsMZ06ddLQoUM1btw4rVy5Ul9//bXGjx9vOyAzggoAAHi8k/HkyZPtfDZuPXv2tOsvvvhCF198sd3etGmTDh36+a7J9913n/Ly8uy8Nqam5vzzz7fDwt1z4Bhvv/22DTUDBgyomOjPzJ0DAADg8YBj5r852Rw4ZlhxZaYWZ+rUqXY5HjNiikn9AADAiTBPPAAACDoEHAAAEHQIOAAAIOgQcAAAQNAh4AAAgKBDwAEAAEGHgAMAAIIOAQcAAAQdAg4AAAg6HpvJ2J+5Z1CuyW3Xg525m3h+fr4tE+5uTXkHGz7flHcwC6XPd0757/bRd0KoTkgGnMOHD9t1SkqKry8FAACcxu94vXr1TnhMmOtUYlCQcTqd2rNnj+rWrWvvf4WyVGwC386dO5WQkECReBjl7V2UN+UdzELp8+1yuWy4adasmb3h9omEZA2OKZQWLVr4+jL8kvnjCPY/EH9CeVPewYzPN+XtCSeruXGjkzEAAAg6BBwAABB0CDiwoqOj9dBDD9k1PI/y9i7Km/IOZny+qxeSnYwBAEBwowYHAAAEHQIOAAAIOgQcAAAQdAg4AAAg6BBwcFyFhYXq0aOHne153bp1lJQH7NixQzfddJNat26t2NhYtW3b1o5mKyoqorxryaxZs5SamqqYmBj169dPK1eupGw9YMaMGfrFL35hZ4hPTEzUFVdcoU2bNlHWXvLYY4/Z7+q77rqLMi9HwMFx3XfffXY6bHjOxo0b7a1D/vKXv+j777/XM888o9mzZ+uBBx6g2GvBO++8owkTJtjQuGbNGnXv3l1DhgxRZmYm5VvLlixZojvuuEPLly/XwoUL7Q0gBw8erLy8PMraw1atWmW/Q7p160ZZV8IwcVTrk08+sT8M//jHP3T22Wdr7dq1tjYHnvfkk0/qxRdf1LZt2yjuM2RqbEytwgsvvGAfmzBp7tlz55136v7776d8PSgrK8vW5Jjgc+GFF1LWHpKbm6tevXrpz3/+s6ZPn26/p2fOnEl5U4OD6mRkZGjcuHH629/+pri4OArJyw4dOqSGDRtS7mfINPOtXr1aAwcOrHIfOvN42bJllK8XPscGn2XPMrVmw4cPr/I5RwjfbBPHZ+Z9HDNmjG699Vb16dPH9hGB92zZskXPP/+8nnrqKYr9DO3bt0+lpaVKSkqqst88Nk2D8BxTU2b6gpx33nnq0qULRe0hc+fOtU2vpokKx6IPTogw1fGmA9qJFvOlb35cza3oJ06c6OtLDonyrmz37t0aOnSorrnmGluDBgRyrcJ3331nf4DhGTt37tTvf/97vf3227YDPY5FH5wQag/fv3//CY9p06aNrr32Wn300Uf2B9jN/FdweHi4rr/+er3xxhteuNrQKe+oqCi7vWfPHl188cU655xz9Prrr9umFJx5E5VpYn3vvffsiB63G2+8UdnZ2frXv/5FEXvA+PHjbdl++eWXdnQgPOODDz7QlVdeab+bK39Xm+9u8/1RWFhY5blQRMBBFWlpacrJyal4bH54zagT8yNhOmy2aNGCEqtlpubmkksuUe/evfXWW2+F/JdSbTKf2b59+9qaSXfTScuWLe2PMJ2Ma79523Tefv/997V48WK1b9++lv8FVGZq2n/66acq+8aOHauOHTvqj3/8I02D9MHB0cyXf2Xx8fF2beZnIdx4JtyYmptWrVrZfjem5sctOTmZD+gZMiMBTY2N6U9mgo4ZXWKGLZsfAtR+s9ScOXNs7Y2ZCyc9Pd3ur1evnp3jCbXLlPHR/Zvq1KmjRo0aEW7K0ckY8CEzX4jpWGyWowOk+S9inJmRI0fa0Dh58mT7g2uG0M6fP/+Yjsc4c2ZqA8ME9spee+01O3AB8DaaqAAAQNChJyMAAAg6BBwAABB0CDgAACDoEHAAAEDQIeAAAICgQ8ABAABBh4ADAACCDgEHAAAEHQIOAAAIOgQcAAAQdAg4AAAg6BBwAACAgs3/B9sXg7QOR0wMAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.plot(np.arange(-5,5,0.2), np.tanh(np.arange(-5,5,0.2))); plt.grid() # Activation function (tanh or sigmoid)" ] }, { "cell_type": "code", "execution_count": 62, "id": "b3372da9-326a-4aad-b124-4d7d0e0b7e5e", "metadata": {}, "outputs": [], "source": [ "class Value:\n", " \n", " def __init__(self, data, children=(), op='', label=''):\n", " self.data = data\n", " self._prev = set(children)\n", " self._op = op\n", " self.label = label\n", " self.grad = 0.0\n", "\n", " def __repr__(self):\n", " return f\"Value(data={self.data}, label={self.label}, grad={self.grad})\"\n", "\n", " def __add__(self, other):\n", " return self.__class__(self.data + other.data, children=(self, other), op='+')\n", "\n", " def __mul__(self, other):\n", " return self.__class__(self.data * other.data, children=(self, other), op='*')\n", "\n", " def tanh(self):\n", " x = self.data\n", " t = (math.exp(2*x) - 1) / (math.exp(2*x) + 1)\n", " out = Value(t, children=(self, ), op='tanh')\n", " return out" ] }, { "cell_type": "markdown", "id": "7e356fa6-02d5-4c1d-a93e-8ac10b2d3249", "metadata": {}, "source": [ "### Implémentation d'un parcours arrière pour calculer les gradients (backward)\n" ] }, { "cell_type": "code", "execution_count": 63, "id": "13ce876e-67dc-4926-ab70-2f26102dbf9b", "metadata": {}, "outputs": [], "source": [ "class Value:\n", " \n", " def __init__(self, data, children=(), op='', label=''):\n", " self.data = data\n", " self._prev = set(children)\n", " self._op = op\n", " self.label = label\n", " self.grad = 0.0\n", " self._backward = lambda: None\n", "\n", " def __repr__(self):\n", " return f\"Value(data={self.data}, label={self.label}, grad={self.grad})\"\n", "\n", " def __add__(self, other):\n", " out = self.__class__(self.data + other.data, children=(self, other), op='+')\n", " def _backward():\n", " self.grad = out.grad \n", " other.grad = out.grad\n", " out._backward = _backward\n", " return out\n", "\n", " def __mul__(self, other):\n", " out = self.__class__(self.data * other.data, children=(self, other), op='*')\n", " def _backward():\n", " self.grad = other.data * out.grad \n", " other.grad = self.data * out.grad\n", " out._backward = _backward\n", " return out\n", " \n", " def tanh(self):\n", " x = self.data\n", " t = (math.exp(2*x) - 1) / (math.exp(2*x) + 1)\n", " out = Value(t, children=(self,), op='tanh')\n", " def _backward():\n", " self.grad = (1 - t**2) * out.grad\n", " out._backward = _backward\n", " return out" ] }, { "cell_type": "markdown", "id": "9781d309-dc46-433e-ba57-e3bd4a32411e", "metadata": {}, "source": [ "Utilisation à la main:" ] }, { "cell_type": "code", "execution_count": 64, "id": "dfd35787-d6dd-47d2-b9ea-8349f84a2a9f", "metadata": {}, "outputs": [], "source": [ "a = Value(2.0, label='a')\n", "b = Value(-3.0, label='b')\n", "c = Value(10.0, label='c')\n", "e = a * b\n", "e.label = 'e'\n", "d = e + c\n", "d.label = 'd'\n", "f = Value(-2.0, label='f')\n", "L = d * f\n", "L.label = 'L'\n", "L.grad = 1.0" ] }, { "cell_type": "code", "execution_count": 67, "id": "ba6b2357-2fdf-47ed-868e-2fced6c93012", "metadata": {}, "outputs": [ { "data": { "image/svg+xml": [ "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "4539036208\n", "\n", "e\n", "\n", "data -6.0000\n", "\n", "grad 0.0000\n", "\n", "\n", "\n", "4539036816+\n", "\n", "+\n", "\n", "\n", "\n", "4539036208->4539036816+\n", "\n", "\n", "\n", "\n", "\n", "4539036208*\n", "\n", "*\n", "\n", "\n", "\n", "4539036208*->4539036208\n", "\n", "\n", "\n", "\n", "\n", "4539036816\n", "\n", "d\n", "\n", "data 4.0000\n", "\n", "grad 0.0000\n", "\n", "\n", "\n", "4537602928*\n", "\n", "*\n", "\n", "\n", "\n", "4539036816->4537602928*\n", "\n", "\n", "\n", "\n", "\n", "4539036816+->4539036816\n", "\n", "\n", "\n", "\n", "\n", "4539806928\n", "\n", "c\n", "\n", "data 10.0000\n", "\n", "grad 0.0000\n", "\n", "\n", "\n", "4539806928->4539036816+\n", "\n", "\n", "\n", "\n", "\n", "4537602928\n", "\n", "L\n", "\n", "data -8.0000\n", "\n", "grad 1.0000\n", "\n", "\n", "\n", "4537602928*->4537602928\n", "\n", "\n", "\n", "\n", "\n", "4539806608\n", "\n", "b\n", "\n", "data -3.0000\n", "\n", "grad 0.0000\n", "\n", "\n", "\n", "4539806608->4539036208*\n", "\n", "\n", "\n", "\n", "\n", "4539931616\n", "\n", "a\n", "\n", "data 2.0000\n", "\n", "grad 0.0000\n", "\n", "\n", "\n", "4539931616->4539036208*\n", "\n", "\n", "\n", "\n", "\n", "4537281008\n", "\n", "f\n", "\n", "data -2.0000\n", "\n", "grad 0.0000\n", "\n", "\n", "\n", "4537281008->4537602928*\n", "\n", "\n", "\n", "\n", "\n" ], "text/plain": [ "" ] }, "execution_count": 67, "metadata": {}, "output_type": "execute_result" } ], "source": [ "draw_dot(L)" ] }, { "cell_type": "code", "execution_count": 68, "id": "ccc6c67f-5206-4e51-8bad-f2da167ec2cf", "metadata": {}, "outputs": [], "source": [ "L._backward()" ] }, { "cell_type": "code", "execution_count": 69, "id": "a53a7891-e755-436e-b548-c1aceb3056b7", "metadata": {}, "outputs": [ { "data": { "image/svg+xml": [ "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "4539036208\n", "\n", "e\n", "\n", "data -6.0000\n", "\n", "grad 0.0000\n", "\n", "\n", "\n", "4539036816+\n", "\n", "+\n", "\n", "\n", "\n", "4539036208->4539036816+\n", "\n", "\n", "\n", "\n", "\n", "4539036208*\n", "\n", "*\n", "\n", "\n", "\n", "4539036208*->4539036208\n", "\n", "\n", "\n", "\n", "\n", "4539036816\n", "\n", "d\n", "\n", "data 4.0000\n", "\n", "grad -2.0000\n", "\n", "\n", "\n", "4537602928*\n", "\n", "*\n", "\n", "\n", "\n", "4539036816->4537602928*\n", "\n", "\n", "\n", "\n", "\n", "4539036816+->4539036816\n", "\n", "\n", "\n", "\n", "\n", "4539806928\n", "\n", "c\n", "\n", "data 10.0000\n", "\n", "grad 0.0000\n", "\n", "\n", "\n", "4539806928->4539036816+\n", "\n", "\n", "\n", "\n", "\n", "4537602928\n", "\n", "L\n", "\n", "data -8.0000\n", "\n", "grad 1.0000\n", "\n", "\n", "\n", "4537602928*->4537602928\n", "\n", "\n", "\n", "\n", "\n", "4539806608\n", "\n", "b\n", "\n", "data -3.0000\n", "\n", "grad 0.0000\n", "\n", "\n", "\n", "4539806608->4539036208*\n", "\n", "\n", "\n", "\n", "\n", "4539931616\n", "\n", "a\n", "\n", "data 2.0000\n", "\n", "grad 0.0000\n", "\n", "\n", "\n", "4539931616->4539036208*\n", "\n", "\n", "\n", "\n", "\n", "4537281008\n", "\n", "f\n", "\n", "data -2.0000\n", "\n", "grad 4.0000\n", "\n", "\n", "\n", "4537281008->4537602928*\n", "\n", "\n", "\n", "\n", "\n" ], "text/plain": [ "" ] }, "execution_count": 69, "metadata": {}, "output_type": "execute_result" } ], "source": [ "draw_dot(L)" ] }, { "cell_type": "code", "execution_count": 70, "id": "0cae3c94-9e06-474b-a536-cbf06dc47117", "metadata": {}, "outputs": [], "source": [ "d._backward()" ] }, { "cell_type": "code", "execution_count": 71, "id": "415a16da-1a0e-42ef-a534-bb2a70d746c6", "metadata": {}, "outputs": [ { "data": { "image/svg+xml": [ "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "4539036208\n", "\n", "e\n", "\n", "data -6.0000\n", "\n", "grad -2.0000\n", "\n", "\n", "\n", "4539036816+\n", "\n", "+\n", "\n", "\n", "\n", "4539036208->4539036816+\n", "\n", "\n", "\n", "\n", "\n", "4539036208*\n", "\n", "*\n", "\n", "\n", "\n", "4539036208*->4539036208\n", "\n", "\n", "\n", "\n", "\n", "4539036816\n", "\n", "d\n", "\n", "data 4.0000\n", "\n", "grad -2.0000\n", "\n", "\n", "\n", "4537602928*\n", "\n", "*\n", "\n", "\n", "\n", "4539036816->4537602928*\n", "\n", "\n", "\n", "\n", "\n", "4539036816+->4539036816\n", "\n", "\n", "\n", "\n", "\n", "4539806928\n", "\n", "c\n", "\n", "data 10.0000\n", "\n", "grad -2.0000\n", "\n", "\n", "\n", "4539806928->4539036816+\n", "\n", "\n", "\n", "\n", "\n", "4537602928\n", "\n", "L\n", "\n", "data -8.0000\n", "\n", "grad 1.0000\n", "\n", "\n", "\n", "4537602928*->4537602928\n", "\n", "\n", "\n", "\n", "\n", "4539806608\n", "\n", "b\n", "\n", "data -3.0000\n", "\n", "grad 0.0000\n", "\n", "\n", "\n", "4539806608->4539036208*\n", "\n", "\n", "\n", "\n", "\n", "4539931616\n", "\n", "a\n", "\n", "data 2.0000\n", "\n", "grad 0.0000\n", "\n", "\n", "\n", "4539931616->4539036208*\n", "\n", "\n", "\n", "\n", "\n", "4537281008\n", "\n", "f\n", "\n", "data -2.0000\n", "\n", "grad 4.0000\n", "\n", "\n", "\n", "4537281008->4537602928*\n", "\n", "\n", "\n", "\n", "\n" ], "text/plain": [ "" ] }, "execution_count": 71, "metadata": {}, "output_type": "execute_result" } ], "source": [ "draw_dot(L)" ] }, { "cell_type": "code", "execution_count": 72, "id": "4cac8b5b-ea26-4274-9096-77da6f905d58", "metadata": {}, "outputs": [], "source": [ "e._backward()" ] }, { "cell_type": "code", "execution_count": 73, "id": "92885462-a2d6-4849-90b8-216c29d4b91f", "metadata": {}, "outputs": [ { "data": { "image/svg+xml": [ "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "4539036208\n", "\n", "e\n", "\n", "data -6.0000\n", "\n", "grad -2.0000\n", "\n", "\n", "\n", "4539036816+\n", "\n", "+\n", "\n", "\n", "\n", "4539036208->4539036816+\n", "\n", "\n", "\n", "\n", "\n", "4539036208*\n", "\n", "*\n", "\n", "\n", "\n", "4539036208*->4539036208\n", "\n", "\n", "\n", "\n", "\n", "4539036816\n", "\n", "d\n", "\n", "data 4.0000\n", "\n", "grad -2.0000\n", "\n", "\n", "\n", "4537602928*\n", "\n", "*\n", "\n", "\n", "\n", "4539036816->4537602928*\n", "\n", "\n", "\n", "\n", "\n", "4539036816+->4539036816\n", "\n", "\n", "\n", "\n", "\n", "4539806928\n", "\n", "c\n", "\n", "data 10.0000\n", "\n", "grad -2.0000\n", "\n", "\n", "\n", "4539806928->4539036816+\n", "\n", "\n", "\n", "\n", "\n", "4537602928\n", "\n", "L\n", "\n", "data -8.0000\n", "\n", "grad 1.0000\n", "\n", "\n", "\n", "4537602928*->4537602928\n", "\n", "\n", "\n", "\n", "\n", "4539806608\n", "\n", "b\n", "\n", "data -3.0000\n", "\n", "grad -4.0000\n", "\n", "\n", "\n", "4539806608->4539036208*\n", "\n", "\n", "\n", "\n", "\n", "4539931616\n", "\n", "a\n", "\n", "data 2.0000\n", "\n", "grad 6.0000\n", "\n", "\n", "\n", "4539931616->4539036208*\n", "\n", "\n", "\n", "\n", "\n", "4537281008\n", "\n", "f\n", "\n", "data -2.0000\n", "\n", "grad 4.0000\n", "\n", "\n", "\n", "4537281008->4537602928*\n", "\n", "\n", "\n", "\n", "\n" ], "text/plain": [ "" ] }, "execution_count": 73, "metadata": {}, "output_type": "execute_result" } ], "source": [ "draw_dot(L)" ] }, { "cell_type": "markdown", "id": "2f6c2755-ccba-47cd-8e54-faae31d09b60", "metadata": {}, "source": [ "On peut vérifier que les valeurs sont identiques à ce que nous avions calculé tout à l'heure à main." ] }, { "cell_type": "markdown", "id": "a6f781e9-33a4-4c73-809a-d055d5fad437", "metadata": {}, "source": [ "### Automatisation du calcul\n", "\n", "On va ajouter à la classe `Value` une méthode `backward` qui permettra d'appeler dans en remontant dans le graphe les méthodes `_backward` définies lors de la passe \"forward\" de construction du graphe, capturant le calcul à effectuer ainsi que l'objet `Value` parent qui fournira son gradient.\n", "\n", "On effectue un [tri topologique](https://fr.wikipedia.org/wiki/Tri_topologique) sur les noeuds du graphe (tri possible car les graphes que nous construisons sonr des DAG dans nos exemples), puis on applique en \"remontant\" `_backward`: le tri topologique assure que les gradients nécessaires au calcul ont bien été calculés avant." ] }, { "cell_type": "code", "execution_count": 76, "id": "8262a61d-2d7d-493a-a301-37ee95607f62", "metadata": {}, "outputs": [], "source": [ "# Passer d'un mode manuel à une automatisation de la back propagation\n", "# Tri topologique du DAG de l'expression\n", "def backward(self):\n", " topo = []\n", " visited = set()\n", " def build_topo(v):\n", " if v not in visited:\n", " visited.add(v)\n", " for child in v._prev:\n", " build_topo(child)\n", " topo.append(v)\n", " build_topo(self)\n", " self.grad = 1.0\n", " for node in reversed(topo):\n", " node._backward()\n", "Value.backward = backward" ] }, { "cell_type": "code", "execution_count": 77, "id": "ca8b8b8b-122f-4ef1-8fd3-98803fd83059", "metadata": {}, "outputs": [], "source": [ "a = Value(2.0, label='a')\n", "b = Value(-3.0, label='b')\n", "c = Value(10.0, label='c')\n", "e = a * b\n", "e.label = 'e'\n", "d = e + c\n", "d.label = 'd'\n", "f = Value(-2.0, label='f')\n", "L = d * f\n", "L.label = 'L'" ] }, { "cell_type": "code", "execution_count": 78, "id": "ff35545b-da5b-4c94-9ace-600ad0afe138", "metadata": {}, "outputs": [], "source": [ "L.backward()" ] }, { "cell_type": "code", "execution_count": 79, "id": "8b33d798-ab6e-4f35-a837-de73823a3bb3", "metadata": {}, "outputs": [ { "data": { "image/svg+xml": [ "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "4539210832\n", "\n", "c\n", "\n", "data 10.0000\n", "\n", "grad -2.0000\n", "\n", "\n", "\n", "4539261936+\n", "\n", "+\n", "\n", "\n", "\n", "4539210832->4539261936+\n", "\n", "\n", "\n", "\n", "\n", "4537599664\n", "\n", "a\n", "\n", "data 2.0000\n", "\n", "grad 6.0000\n", "\n", "\n", "\n", "4539258816*\n", "\n", "*\n", "\n", "\n", "\n", "4537599664->4539258816*\n", "\n", "\n", "\n", "\n", "\n", "4539214160\n", "\n", "b\n", "\n", "data -3.0000\n", "\n", "grad -4.0000\n", "\n", "\n", "\n", "4539214160->4539258816*\n", "\n", "\n", "\n", "\n", "\n", "4538922864\n", "\n", "L\n", "\n", "data -8.0000\n", "\n", "grad 1.0000\n", "\n", "\n", "\n", "4538922864*\n", "\n", "*\n", "\n", "\n", "\n", "4538922864*->4538922864\n", "\n", "\n", "\n", "\n", "\n", "4539258816\n", "\n", "e\n", "\n", "data -6.0000\n", "\n", "grad -2.0000\n", "\n", "\n", "\n", "4539258816->4539261936+\n", "\n", "\n", "\n", "\n", "\n", "4539258816*->4539258816\n", "\n", "\n", "\n", "\n", "\n", "4539637712\n", "\n", "f\n", "\n", "data -2.0000\n", "\n", "grad 4.0000\n", "\n", "\n", "\n", "4539637712->4538922864*\n", "\n", "\n", "\n", "\n", "\n", "4539261936\n", "\n", "d\n", "\n", "data 4.0000\n", "\n", "grad -2.0000\n", "\n", "\n", "\n", "4539261936->4538922864*\n", "\n", "\n", "\n", "\n", "\n", "4539261936+->4539261936\n", "\n", "\n", "\n", "\n", "\n" ], "text/plain": [ "" ] }, "execution_count": 79, "metadata": {}, "output_type": "execute_result" } ], "source": [ "draw_dot(L)" ] }, { "cell_type": "markdown", "id": "68eaac37-487e-4053-9259-18313319ce27", "metadata": {}, "source": [ "**fin de la première partie**" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.14.2" } }, "nbformat": 4, "nbformat_minor": 5 }