2962 lines
195 KiB
Plaintext
2962 lines
195 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"# Lecture 16: Decision trees"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "skip"
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"\n",
|
|
"[Run in colab](https://colab.research.google.com/drive/1P9IoqXN9dbjJ3TN50wa8wwDdvn9P6hX7)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"metadata": {
|
|
"editable": true,
|
|
"execution": {
|
|
"iopub.execute_input": "2025-02-27T23:21:19.492309Z",
|
|
"iopub.status.busy": "2025-02-27T23:21:19.492086Z",
|
|
"iopub.status.idle": "2025-02-27T23:21:19.498527Z",
|
|
"shell.execute_reply": "2025-02-27T23:21:19.497958Z"
|
|
},
|
|
"slideshow": {
|
|
"slide_type": "skip"
|
|
},
|
|
"tags": []
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Last executed: 2025-02-27 23:21:19\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"import datetime\n",
|
|
"now = datetime.datetime.now()\n",
|
|
"print(\"Last executed: \" + now.strftime(\"%Y-%m-%d %H:%M:%S\"))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "subslide"
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"Considered conceptually as a *flow diagram* or tree of decisions based on inspecting properties of data-set.\n",
|
|
"\n",
|
|
"- Can perform both classification and regression.\n",
|
|
"- A fundamental component of random forests (a powerful machine learning algorithm covered in the next lecture). \n",
|
|
"- We will learn how to visualise and make predictions using Decision Trees.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"## Conceptual example"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": ""
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"<img src=\"https://raw.githubusercontent.com/astro-informatics/course_mlbd_images/master/Lecture16_Images/DecisionTree.jpg\" alt=\"data-layout\" width=\"500\" style=\"display:block; margin:auto\"/>\n",
|
|
"\n",
|
|
"[[Image source](https://inside-machinelearning.com/en/decision-tree-and-hyperparameters/)]"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"## Walk-through of decision tree\n",
|
|
"\n",
|
|
"Let's consider an illustration using the Iris Data set (introduced in Lecture 3)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "subslide"
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"### Images of different Iris species\n",
|
|
"\n",
|
|
"#### Iris Setosa\n",
|
|
"\n",
|
|
"<img src=\"https://raw.githubusercontent.com/astro-informatics/course_mlbd_images/master/Lecture03_Images/iris_setosa.jpg\" width=\"300\" style=\"display:block; margin:auto\"/>\n",
|
|
"\n",
|
|
"#### Iris Versicolor\n",
|
|
"\n",
|
|
"<img src=\"https://raw.githubusercontent.com/astro-informatics/course_mlbd_images/master/Lecture03_Images/iris_versicolor.jpg\" width=\"300\" style=\"display:block; margin:auto\"/>\n",
|
|
"\n",
|
|
"#### Iris Virginica\n",
|
|
"\n",
|
|
"<img src=\"https://raw.githubusercontent.com/astro-informatics/course_mlbd_images/master/Lecture03_Images/iris_virginica.jpg\" width=\"300\" style=\"display:block; margin:auto\"/>\n",
|
|
"\n",
|
|
"[[Image source](https://github.com/jakevdp/sklearn_tutorial)]\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "subslide"
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"Load feature matrix, where each row correpsonds to an observed (*sampled*) flower, with a number of *features*, with corresponding target vector.\n",
|
|
"\n",
|
|
"Consider two features only for now (petal length and width)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"metadata": {
|
|
"editable": true,
|
|
"execution": {
|
|
"iopub.execute_input": "2025-02-27T23:21:19.532095Z",
|
|
"iopub.status.busy": "2025-02-27T23:21:19.531889Z",
|
|
"iopub.status.idle": "2025-02-27T23:21:20.377758Z",
|
|
"shell.execute_reply": "2025-02-27T23:21:20.377159Z"
|
|
},
|
|
"slideshow": {
|
|
"slide_type": ""
|
|
},
|
|
"tags": []
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/html": [
|
|
"<style>#sk-container-id-1 {\n",
|
|
" /* Definition of color scheme common for light and dark mode */\n",
|
|
" --sklearn-color-text: #000;\n",
|
|
" --sklearn-color-text-muted: #666;\n",
|
|
" --sklearn-color-line: gray;\n",
|
|
" /* Definition of color scheme for unfitted estimators */\n",
|
|
" --sklearn-color-unfitted-level-0: #fff5e6;\n",
|
|
" --sklearn-color-unfitted-level-1: #f6e4d2;\n",
|
|
" --sklearn-color-unfitted-level-2: #ffe0b3;\n",
|
|
" --sklearn-color-unfitted-level-3: chocolate;\n",
|
|
" /* Definition of color scheme for fitted estimators */\n",
|
|
" --sklearn-color-fitted-level-0: #f0f8ff;\n",
|
|
" --sklearn-color-fitted-level-1: #d4ebff;\n",
|
|
" --sklearn-color-fitted-level-2: #b3dbfd;\n",
|
|
" --sklearn-color-fitted-level-3: cornflowerblue;\n",
|
|
"\n",
|
|
" /* Specific color for light theme */\n",
|
|
" --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
|
|
" --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n",
|
|
" --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
|
|
" --sklearn-color-icon: #696969;\n",
|
|
"\n",
|
|
" @media (prefers-color-scheme: dark) {\n",
|
|
" /* Redefinition of color scheme for dark theme */\n",
|
|
" --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
|
|
" --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n",
|
|
" --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
|
|
" --sklearn-color-icon: #878787;\n",
|
|
" }\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-1 {\n",
|
|
" color: var(--sklearn-color-text);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-1 pre {\n",
|
|
" padding: 0;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-1 input.sk-hidden--visually {\n",
|
|
" border: 0;\n",
|
|
" clip: rect(1px 1px 1px 1px);\n",
|
|
" clip: rect(1px, 1px, 1px, 1px);\n",
|
|
" height: 1px;\n",
|
|
" margin: -1px;\n",
|
|
" overflow: hidden;\n",
|
|
" padding: 0;\n",
|
|
" position: absolute;\n",
|
|
" width: 1px;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-1 div.sk-dashed-wrapped {\n",
|
|
" border: 1px dashed var(--sklearn-color-line);\n",
|
|
" margin: 0 0.4em 0.5em 0.4em;\n",
|
|
" box-sizing: border-box;\n",
|
|
" padding-bottom: 0.4em;\n",
|
|
" background-color: var(--sklearn-color-background);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-1 div.sk-container {\n",
|
|
" /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n",
|
|
" but bootstrap.min.css set `[hidden] { display: none !important; }`\n",
|
|
" so we also need the `!important` here to be able to override the\n",
|
|
" default hidden behavior on the sphinx rendered scikit-learn.org.\n",
|
|
" See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n",
|
|
" display: inline-block !important;\n",
|
|
" position: relative;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-1 div.sk-text-repr-fallback {\n",
|
|
" display: none;\n",
|
|
"}\n",
|
|
"\n",
|
|
"div.sk-parallel-item,\n",
|
|
"div.sk-serial,\n",
|
|
"div.sk-item {\n",
|
|
" /* draw centered vertical line to link estimators */\n",
|
|
" background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n",
|
|
" background-size: 2px 100%;\n",
|
|
" background-repeat: no-repeat;\n",
|
|
" background-position: center center;\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Parallel-specific style estimator block */\n",
|
|
"\n",
|
|
"#sk-container-id-1 div.sk-parallel-item::after {\n",
|
|
" content: \"\";\n",
|
|
" width: 100%;\n",
|
|
" border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n",
|
|
" flex-grow: 1;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-1 div.sk-parallel {\n",
|
|
" display: flex;\n",
|
|
" align-items: stretch;\n",
|
|
" justify-content: center;\n",
|
|
" background-color: var(--sklearn-color-background);\n",
|
|
" position: relative;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-1 div.sk-parallel-item {\n",
|
|
" display: flex;\n",
|
|
" flex-direction: column;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-1 div.sk-parallel-item:first-child::after {\n",
|
|
" align-self: flex-end;\n",
|
|
" width: 50%;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-1 div.sk-parallel-item:last-child::after {\n",
|
|
" align-self: flex-start;\n",
|
|
" width: 50%;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-1 div.sk-parallel-item:only-child::after {\n",
|
|
" width: 0;\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Serial-specific style estimator block */\n",
|
|
"\n",
|
|
"#sk-container-id-1 div.sk-serial {\n",
|
|
" display: flex;\n",
|
|
" flex-direction: column;\n",
|
|
" align-items: center;\n",
|
|
" background-color: var(--sklearn-color-background);\n",
|
|
" padding-right: 1em;\n",
|
|
" padding-left: 1em;\n",
|
|
"}\n",
|
|
"\n",
|
|
"\n",
|
|
"/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n",
|
|
"clickable and can be expanded/collapsed.\n",
|
|
"- Pipeline and ColumnTransformer use this feature and define the default style\n",
|
|
"- Estimators will overwrite some part of the style using the `sk-estimator` class\n",
|
|
"*/\n",
|
|
"\n",
|
|
"/* Pipeline and ColumnTransformer style (default) */\n",
|
|
"\n",
|
|
"#sk-container-id-1 div.sk-toggleable {\n",
|
|
" /* Default theme specific background. It is overwritten whether we have a\n",
|
|
" specific estimator or a Pipeline/ColumnTransformer */\n",
|
|
" background-color: var(--sklearn-color-background);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Toggleable label */\n",
|
|
"#sk-container-id-1 label.sk-toggleable__label {\n",
|
|
" cursor: pointer;\n",
|
|
" display: flex;\n",
|
|
" width: 100%;\n",
|
|
" margin-bottom: 0;\n",
|
|
" padding: 0.5em;\n",
|
|
" box-sizing: border-box;\n",
|
|
" text-align: center;\n",
|
|
" align-items: start;\n",
|
|
" justify-content: space-between;\n",
|
|
" gap: 0.5em;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-1 label.sk-toggleable__label .caption {\n",
|
|
" font-size: 0.6rem;\n",
|
|
" font-weight: lighter;\n",
|
|
" color: var(--sklearn-color-text-muted);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-1 label.sk-toggleable__label-arrow:before {\n",
|
|
" /* Arrow on the left of the label */\n",
|
|
" content: \"▸\";\n",
|
|
" float: left;\n",
|
|
" margin-right: 0.25em;\n",
|
|
" color: var(--sklearn-color-icon);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {\n",
|
|
" color: var(--sklearn-color-text);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Toggleable content - dropdown */\n",
|
|
"\n",
|
|
"#sk-container-id-1 div.sk-toggleable__content {\n",
|
|
" max-height: 0;\n",
|
|
" max-width: 0;\n",
|
|
" overflow: hidden;\n",
|
|
" text-align: left;\n",
|
|
" /* unfitted */\n",
|
|
" background-color: var(--sklearn-color-unfitted-level-0);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-1 div.sk-toggleable__content.fitted {\n",
|
|
" /* fitted */\n",
|
|
" background-color: var(--sklearn-color-fitted-level-0);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-1 div.sk-toggleable__content pre {\n",
|
|
" margin: 0.2em;\n",
|
|
" border-radius: 0.25em;\n",
|
|
" color: var(--sklearn-color-text);\n",
|
|
" /* unfitted */\n",
|
|
" background-color: var(--sklearn-color-unfitted-level-0);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-1 div.sk-toggleable__content.fitted pre {\n",
|
|
" /* unfitted */\n",
|
|
" background-color: var(--sklearn-color-fitted-level-0);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n",
|
|
" /* Expand drop-down */\n",
|
|
" max-height: 200px;\n",
|
|
" max-width: 100%;\n",
|
|
" overflow: auto;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n",
|
|
" content: \"▾\";\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Pipeline/ColumnTransformer-specific style */\n",
|
|
"\n",
|
|
"#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
|
|
" color: var(--sklearn-color-text);\n",
|
|
" background-color: var(--sklearn-color-unfitted-level-2);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-1 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
|
|
" background-color: var(--sklearn-color-fitted-level-2);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Estimator-specific style */\n",
|
|
"\n",
|
|
"/* Colorize estimator box */\n",
|
|
"#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
|
|
" /* unfitted */\n",
|
|
" background-color: var(--sklearn-color-unfitted-level-2);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-1 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
|
|
" /* fitted */\n",
|
|
" background-color: var(--sklearn-color-fitted-level-2);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-1 div.sk-label label.sk-toggleable__label,\n",
|
|
"#sk-container-id-1 div.sk-label label {\n",
|
|
" /* The background is the default theme color */\n",
|
|
" color: var(--sklearn-color-text-on-default-background);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* On hover, darken the color of the background */\n",
|
|
"#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {\n",
|
|
" color: var(--sklearn-color-text);\n",
|
|
" background-color: var(--sklearn-color-unfitted-level-2);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Label box, darken color on hover, fitted */\n",
|
|
"#sk-container-id-1 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n",
|
|
" color: var(--sklearn-color-text);\n",
|
|
" background-color: var(--sklearn-color-fitted-level-2);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Estimator label */\n",
|
|
"\n",
|
|
"#sk-container-id-1 div.sk-label label {\n",
|
|
" font-family: monospace;\n",
|
|
" font-weight: bold;\n",
|
|
" display: inline-block;\n",
|
|
" line-height: 1.2em;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-1 div.sk-label-container {\n",
|
|
" text-align: center;\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Estimator-specific */\n",
|
|
"#sk-container-id-1 div.sk-estimator {\n",
|
|
" font-family: monospace;\n",
|
|
" border: 1px dotted var(--sklearn-color-border-box);\n",
|
|
" border-radius: 0.25em;\n",
|
|
" box-sizing: border-box;\n",
|
|
" margin-bottom: 0.5em;\n",
|
|
" /* unfitted */\n",
|
|
" background-color: var(--sklearn-color-unfitted-level-0);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-1 div.sk-estimator.fitted {\n",
|
|
" /* fitted */\n",
|
|
" background-color: var(--sklearn-color-fitted-level-0);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* on hover */\n",
|
|
"#sk-container-id-1 div.sk-estimator:hover {\n",
|
|
" /* unfitted */\n",
|
|
" background-color: var(--sklearn-color-unfitted-level-2);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-1 div.sk-estimator.fitted:hover {\n",
|
|
" /* fitted */\n",
|
|
" background-color: var(--sklearn-color-fitted-level-2);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Specification for estimator info (e.g. \"i\" and \"?\") */\n",
|
|
"\n",
|
|
"/* Common style for \"i\" and \"?\" */\n",
|
|
"\n",
|
|
".sk-estimator-doc-link,\n",
|
|
"a:link.sk-estimator-doc-link,\n",
|
|
"a:visited.sk-estimator-doc-link {\n",
|
|
" float: right;\n",
|
|
" font-size: smaller;\n",
|
|
" line-height: 1em;\n",
|
|
" font-family: monospace;\n",
|
|
" background-color: var(--sklearn-color-background);\n",
|
|
" border-radius: 1em;\n",
|
|
" height: 1em;\n",
|
|
" width: 1em;\n",
|
|
" text-decoration: none !important;\n",
|
|
" margin-left: 0.5em;\n",
|
|
" text-align: center;\n",
|
|
" /* unfitted */\n",
|
|
" border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
|
|
" color: var(--sklearn-color-unfitted-level-1);\n",
|
|
"}\n",
|
|
"\n",
|
|
".sk-estimator-doc-link.fitted,\n",
|
|
"a:link.sk-estimator-doc-link.fitted,\n",
|
|
"a:visited.sk-estimator-doc-link.fitted {\n",
|
|
" /* fitted */\n",
|
|
" border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
|
|
" color: var(--sklearn-color-fitted-level-1);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* On hover */\n",
|
|
"div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",
|
|
".sk-estimator-doc-link:hover,\n",
|
|
"div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",
|
|
".sk-estimator-doc-link:hover {\n",
|
|
" /* unfitted */\n",
|
|
" background-color: var(--sklearn-color-unfitted-level-3);\n",
|
|
" color: var(--sklearn-color-background);\n",
|
|
" text-decoration: none;\n",
|
|
"}\n",
|
|
"\n",
|
|
"div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",
|
|
".sk-estimator-doc-link.fitted:hover,\n",
|
|
"div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",
|
|
".sk-estimator-doc-link.fitted:hover {\n",
|
|
" /* fitted */\n",
|
|
" background-color: var(--sklearn-color-fitted-level-3);\n",
|
|
" color: var(--sklearn-color-background);\n",
|
|
" text-decoration: none;\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Span, style for the box shown on hovering the info icon */\n",
|
|
".sk-estimator-doc-link span {\n",
|
|
" display: none;\n",
|
|
" z-index: 9999;\n",
|
|
" position: relative;\n",
|
|
" font-weight: normal;\n",
|
|
" right: .2ex;\n",
|
|
" padding: .5ex;\n",
|
|
" margin: .5ex;\n",
|
|
" width: min-content;\n",
|
|
" min-width: 20ex;\n",
|
|
" max-width: 50ex;\n",
|
|
" color: var(--sklearn-color-text);\n",
|
|
" box-shadow: 2pt 2pt 4pt #999;\n",
|
|
" /* unfitted */\n",
|
|
" background: var(--sklearn-color-unfitted-level-0);\n",
|
|
" border: .5pt solid var(--sklearn-color-unfitted-level-3);\n",
|
|
"}\n",
|
|
"\n",
|
|
".sk-estimator-doc-link.fitted span {\n",
|
|
" /* fitted */\n",
|
|
" background: var(--sklearn-color-fitted-level-0);\n",
|
|
" border: var(--sklearn-color-fitted-level-3);\n",
|
|
"}\n",
|
|
"\n",
|
|
".sk-estimator-doc-link:hover span {\n",
|
|
" display: block;\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* \"?\"-specific style due to the `<a>` HTML tag */\n",
|
|
"\n",
|
|
"#sk-container-id-1 a.estimator_doc_link {\n",
|
|
" float: right;\n",
|
|
" font-size: 1rem;\n",
|
|
" line-height: 1em;\n",
|
|
" font-family: monospace;\n",
|
|
" background-color: var(--sklearn-color-background);\n",
|
|
" border-radius: 1rem;\n",
|
|
" height: 1rem;\n",
|
|
" width: 1rem;\n",
|
|
" text-decoration: none;\n",
|
|
" /* unfitted */\n",
|
|
" color: var(--sklearn-color-unfitted-level-1);\n",
|
|
" border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-1 a.estimator_doc_link.fitted {\n",
|
|
" /* fitted */\n",
|
|
" border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
|
|
" color: var(--sklearn-color-fitted-level-1);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* On hover */\n",
|
|
"#sk-container-id-1 a.estimator_doc_link:hover {\n",
|
|
" /* unfitted */\n",
|
|
" background-color: var(--sklearn-color-unfitted-level-3);\n",
|
|
" color: var(--sklearn-color-background);\n",
|
|
" text-decoration: none;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-1 a.estimator_doc_link.fitted:hover {\n",
|
|
" /* fitted */\n",
|
|
" background-color: var(--sklearn-color-fitted-level-3);\n",
|
|
"}\n",
|
|
"</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>DecisionTreeClassifier(max_depth=2, random_state=42)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" checked><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow\"><div><div>DecisionTreeClassifier</div></div><div><a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.6/modules/generated/sklearn.tree.DecisionTreeClassifier.html\">?<span>Documentation for DecisionTreeClassifier</span></a><span class=\"sk-estimator-doc-link fitted\">i<span>Fitted</span></span></div></label><div class=\"sk-toggleable__content fitted\"><pre>DecisionTreeClassifier(max_depth=2, random_state=42)</pre></div> </div></div></div></div>"
|
|
],
|
|
"text/plain": [
|
|
"DecisionTreeClassifier(max_depth=2, random_state=42)"
|
|
]
|
|
},
|
|
"execution_count": 2,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"from sklearn.datasets import load_iris\n",
|
|
"from sklearn.tree import DecisionTreeClassifier\n",
|
|
"\n",
|
|
"iris = load_iris(as_frame=True)\n",
|
|
"X_iris = iris.data[[\"petal length (cm)\", \"petal width (cm)\"]].values\n",
|
|
"y_iris = iris.target\n",
|
|
"\n",
|
|
"tree_clf = DecisionTreeClassifier(max_depth=2, random_state=42)\n",
|
|
"tree_clf.fit(X_iris, y_iris)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 3,
|
|
"metadata": {
|
|
"editable": true,
|
|
"execution": {
|
|
"iopub.execute_input": "2025-02-27T23:21:20.379798Z",
|
|
"iopub.status.busy": "2025-02-27T23:21:20.379575Z",
|
|
"iopub.status.idle": "2025-02-27T23:21:20.569651Z",
|
|
"shell.execute_reply": "2025-02-27T23:21:20.568896Z"
|
|
},
|
|
"slideshow": {
|
|
"slide_type": "skip"
|
|
},
|
|
"tags": []
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# We want to visualise the actual flow diagram of the tree, for this we can use graphviz\n",
|
|
"from sklearn.tree import export_graphviz\n",
|
|
"\n",
|
|
"export_graphviz(tree_clf, \n",
|
|
" out_file = './iris_tree.dot', \n",
|
|
" feature_names = iris.feature_names[ 2:], \n",
|
|
" class_names = iris.target_names, \n",
|
|
" rounded = True, \n",
|
|
" filled = True)\n",
|
|
"\n",
|
|
"#creates a dot file :( so need to convert to something more sensible\n",
|
|
"! dot -Tpng ./iris_tree.dot -o ./iris_tree.png"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "skip"
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"To run dot locally you will need to install [graphviz](https://graphviz.org/).\n",
|
|
"\n",
|
|
"You can install on Mac using Homebrew:\n",
|
|
"```bash\n",
|
|
"brew install graphviz\n",
|
|
"```\n",
|
|
"\n",
|
|
"You can install on Ubuntu using apt:\n",
|
|
"```bash\n",
|
|
"sudo apt install graphviz\n",
|
|
"```\n",
|
|
"\n",
|
|
"Installation instructions for other systems are available [here](https://graphviz.org/download/)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "subslide"
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"### Decision tree for Iris classification (depth 2)\n",
|
|
"\n",
|
|
"<img src=\"https://raw.githubusercontent.com/astro-informatics/course_mlbd_images/master/Lecture16_Images/iris_tree.png\" alt=\"data-layout\" width=\"500\" style=\"display:block; margin:auto\"/>"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": ""
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"Tree consists of a number of nodes.\n",
|
|
"- Top node is the _root node_.\n",
|
|
"- Intermediate _split nodes_.\n",
|
|
"- Lower nodes are _leaf nodes_.\n",
|
|
"\n",
|
|
"Decisions based on *features* and *thresholds*.\n",
|
|
"\n",
|
|
"Navigate tree to make predictions."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "subslide"
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"### Interpreting node outputs\n",
|
|
"\n",
|
|
"<img src=\"https://raw.githubusercontent.com/astro-informatics/course_mlbd_images/master/Lecture16_Images/iris_tree_1node.png\" alt=\"data-layout\" width=\"300\" style=\"display:block; margin:auto\"/>\n",
|
|
"\n",
|
|
"Arguments in the nodes are: \n",
|
|
"- Top argument shows the _threshold_ upon which the classification division was made.\n",
|
|
"- ```gini``` (see next slides) is a quantitative measure of impurity.\n",
|
|
"- ```samples``` denotes the number of training instances that satisfy the criteria.\n",
|
|
"- ```values``` denotes the number of training instances per class that satisfy the criteria.\n",
|
|
"- ```class``` prediction for the node."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "subslide"
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"### Decision boundaries"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": ""
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"<img src=\"https://raw.githubusercontent.com/astro-informatics/course_mlbd_images/master/Lecture16_Images/decision_tree_decision_boundaries_plot.png\" alt=\"data-layout\" width=\"600\" style=\"display:block; margin:auto\"/>\n",
|
|
"\n",
|
|
"We set ```max_depth=2```, so algorithm stopped after two divisions. "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "subslide"
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"### Estimating class probabilities"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": ""
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"Also want to know the _probability_ that an instance $i$ belongs to class $k$. \n",
|
|
"\n",
|
|
"Class probability founds by finding the leaf node for instance $i$, then returns ratio of training instances of class $k$ in this node.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "subslide"
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"#### Example \n",
|
|
"For case where flower has petals=5cm long and 1.5cm wide\n",
|
|
"corresponding leaf node is at depth-2 left node. \n",
|
|
"\n",
|
|
"<img src=\"https://raw.githubusercontent.com/astro-informatics/course_mlbd_images/master/Lecture16_Images/decision_tree_decision_boundaries_plot.png\" alt=\"data-layout\" width=\"600\" style=\"display:block; margin:auto\"/>"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "subslide"
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"<img src=\"https://raw.githubusercontent.com/astro-informatics/course_mlbd_images/master/Lecture16_Images/iris_tree_1node2.png\" alt=\"data-layout\" width=\"300\" style=\"display:block; margin:auto\"/>\n",
|
|
"\n",
|
|
"\n",
|
|
"So probabilities are: 0% (Setosa), 49/54=90.7% (Versicolor), 5/54=9.3% (Virginica)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"metadata": {
|
|
"editable": true,
|
|
"execution": {
|
|
"iopub.execute_input": "2025-02-27T23:21:20.572369Z",
|
|
"iopub.status.busy": "2025-02-27T23:21:20.572178Z",
|
|
"iopub.status.idle": "2025-02-27T23:21:20.577551Z",
|
|
"shell.execute_reply": "2025-02-27T23:21:20.576948Z"
|
|
},
|
|
"slideshow": {
|
|
"slide_type": ""
|
|
},
|
|
"tags": []
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"array([[0. , 0.907, 0.093]])"
|
|
]
|
|
},
|
|
"execution_count": 4,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"tree_clf.predict_proba([[5, 1.5]]).round(3)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"## Quality measures"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "subslide"
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"### Gini impurity\n",
|
|
"\n",
|
|
"Gini impurity is defined by\n",
|
|
"$$\n",
|
|
"G_i=1-\\sum_{k=1}^{n}p^2_{i,k} ,\n",
|
|
"$$\n",
|
|
"where $p_{i,k}$ is the ratio of class $k$ instances among training instances in the $i^{\\rm th}$ node. \n",
|
|
"\n",
|
|
"$G_i=0$ means the sample is 100% _pure_ i.e. all instances are in a single class. "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "subslide"
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"#### Gini example calculation\n",
|
|
"\n",
|
|
"<img src=\"https://raw.githubusercontent.com/astro-informatics/course_mlbd_images/master/Lecture16_Images/iris_tree_1node2.png\" alt=\"data-layout\" width=\"300\" style=\"display:block; margin:auto\"/>\n",
|
|
"\n",
|
|
"$$\n",
|
|
"G_i = 1 - (0/54)^2 - (49/54)^2 - (5/54)^2= 0.168\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "subslide"
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"### Entropy \n",
|
|
"\n",
|
|
"Alternative to Gini is to use entropy as the purity measure \n",
|
|
"$$\n",
|
|
"H_i=-\\sum_{k=1}^n p_{i,k}\\log_2(p_{i,k}),\n",
|
|
"$$\n",
|
|
"for $p_{i,k}\\not=0$. \n",
|
|
"\n",
|
|
"Measures the information content of variable (number of bits required to encode)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "subslide"
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"#### Entropy example calculation\n",
|
|
"\n",
|
|
"<img src=\"https://raw.githubusercontent.com/astro-informatics/course_mlbd_images/master/Lecture16_Images/iris_tree_1node2.png\" alt=\"data-layout\" width=\"300\" style=\"display:block; margin:auto\"/>\n",
|
|
"\n",
|
|
"$$\n",
|
|
"H_i = - (49/54) \\log_2(49/54) - (5/54) \\log_2(5/54)= 0.445\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "subslide"
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"### Does it make a difference what quality measure is used? \n",
|
|
"\n",
|
|
"Not usually, although Gini tends to isolate the most frequent classes, and entropy leads to more \"balanced\" trees. Entropy is slightly more expensive to compute."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "subslide"
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"### Decision tree using entropy criterion"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"metadata": {
|
|
"editable": true,
|
|
"execution": {
|
|
"iopub.execute_input": "2025-02-27T23:21:20.579634Z",
|
|
"iopub.status.busy": "2025-02-27T23:21:20.579461Z",
|
|
"iopub.status.idle": "2025-02-27T23:21:20.585935Z",
|
|
"shell.execute_reply": "2025-02-27T23:21:20.585283Z"
|
|
},
|
|
"slideshow": {
|
|
"slide_type": ""
|
|
},
|
|
"tags": []
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/html": [
|
|
"<style>#sk-container-id-2 {\n",
|
|
" /* Definition of color scheme common for light and dark mode */\n",
|
|
" --sklearn-color-text: #000;\n",
|
|
" --sklearn-color-text-muted: #666;\n",
|
|
" --sklearn-color-line: gray;\n",
|
|
" /* Definition of color scheme for unfitted estimators */\n",
|
|
" --sklearn-color-unfitted-level-0: #fff5e6;\n",
|
|
" --sklearn-color-unfitted-level-1: #f6e4d2;\n",
|
|
" --sklearn-color-unfitted-level-2: #ffe0b3;\n",
|
|
" --sklearn-color-unfitted-level-3: chocolate;\n",
|
|
" /* Definition of color scheme for fitted estimators */\n",
|
|
" --sklearn-color-fitted-level-0: #f0f8ff;\n",
|
|
" --sklearn-color-fitted-level-1: #d4ebff;\n",
|
|
" --sklearn-color-fitted-level-2: #b3dbfd;\n",
|
|
" --sklearn-color-fitted-level-3: cornflowerblue;\n",
|
|
"\n",
|
|
" /* Specific color for light theme */\n",
|
|
" --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
|
|
" --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n",
|
|
" --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
|
|
" --sklearn-color-icon: #696969;\n",
|
|
"\n",
|
|
" @media (prefers-color-scheme: dark) {\n",
|
|
" /* Redefinition of color scheme for dark theme */\n",
|
|
" --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
|
|
" --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n",
|
|
" --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
|
|
" --sklearn-color-icon: #878787;\n",
|
|
" }\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-2 {\n",
|
|
" color: var(--sklearn-color-text);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-2 pre {\n",
|
|
" padding: 0;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-2 input.sk-hidden--visually {\n",
|
|
" border: 0;\n",
|
|
" clip: rect(1px 1px 1px 1px);\n",
|
|
" clip: rect(1px, 1px, 1px, 1px);\n",
|
|
" height: 1px;\n",
|
|
" margin: -1px;\n",
|
|
" overflow: hidden;\n",
|
|
" padding: 0;\n",
|
|
" position: absolute;\n",
|
|
" width: 1px;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-2 div.sk-dashed-wrapped {\n",
|
|
" border: 1px dashed var(--sklearn-color-line);\n",
|
|
" margin: 0 0.4em 0.5em 0.4em;\n",
|
|
" box-sizing: border-box;\n",
|
|
" padding-bottom: 0.4em;\n",
|
|
" background-color: var(--sklearn-color-background);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-2 div.sk-container {\n",
|
|
" /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n",
|
|
" but bootstrap.min.css set `[hidden] { display: none !important; }`\n",
|
|
" so we also need the `!important` here to be able to override the\n",
|
|
" default hidden behavior on the sphinx rendered scikit-learn.org.\n",
|
|
" See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n",
|
|
" display: inline-block !important;\n",
|
|
" position: relative;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-2 div.sk-text-repr-fallback {\n",
|
|
" display: none;\n",
|
|
"}\n",
|
|
"\n",
|
|
"div.sk-parallel-item,\n",
|
|
"div.sk-serial,\n",
|
|
"div.sk-item {\n",
|
|
" /* draw centered vertical line to link estimators */\n",
|
|
" background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n",
|
|
" background-size: 2px 100%;\n",
|
|
" background-repeat: no-repeat;\n",
|
|
" background-position: center center;\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Parallel-specific style estimator block */\n",
|
|
"\n",
|
|
"#sk-container-id-2 div.sk-parallel-item::after {\n",
|
|
" content: \"\";\n",
|
|
" width: 100%;\n",
|
|
" border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n",
|
|
" flex-grow: 1;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-2 div.sk-parallel {\n",
|
|
" display: flex;\n",
|
|
" align-items: stretch;\n",
|
|
" justify-content: center;\n",
|
|
" background-color: var(--sklearn-color-background);\n",
|
|
" position: relative;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-2 div.sk-parallel-item {\n",
|
|
" display: flex;\n",
|
|
" flex-direction: column;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-2 div.sk-parallel-item:first-child::after {\n",
|
|
" align-self: flex-end;\n",
|
|
" width: 50%;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-2 div.sk-parallel-item:last-child::after {\n",
|
|
" align-self: flex-start;\n",
|
|
" width: 50%;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-2 div.sk-parallel-item:only-child::after {\n",
|
|
" width: 0;\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Serial-specific style estimator block */\n",
|
|
"\n",
|
|
"#sk-container-id-2 div.sk-serial {\n",
|
|
" display: flex;\n",
|
|
" flex-direction: column;\n",
|
|
" align-items: center;\n",
|
|
" background-color: var(--sklearn-color-background);\n",
|
|
" padding-right: 1em;\n",
|
|
" padding-left: 1em;\n",
|
|
"}\n",
|
|
"\n",
|
|
"\n",
|
|
"/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n",
|
|
"clickable and can be expanded/collapsed.\n",
|
|
"- Pipeline and ColumnTransformer use this feature and define the default style\n",
|
|
"- Estimators will overwrite some part of the style using the `sk-estimator` class\n",
|
|
"*/\n",
|
|
"\n",
|
|
"/* Pipeline and ColumnTransformer style (default) */\n",
|
|
"\n",
|
|
"#sk-container-id-2 div.sk-toggleable {\n",
|
|
" /* Default theme specific background. It is overwritten whether we have a\n",
|
|
" specific estimator or a Pipeline/ColumnTransformer */\n",
|
|
" background-color: var(--sklearn-color-background);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Toggleable label */\n",
|
|
"#sk-container-id-2 label.sk-toggleable__label {\n",
|
|
" cursor: pointer;\n",
|
|
" display: flex;\n",
|
|
" width: 100%;\n",
|
|
" margin-bottom: 0;\n",
|
|
" padding: 0.5em;\n",
|
|
" box-sizing: border-box;\n",
|
|
" text-align: center;\n",
|
|
" align-items: start;\n",
|
|
" justify-content: space-between;\n",
|
|
" gap: 0.5em;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-2 label.sk-toggleable__label .caption {\n",
|
|
" font-size: 0.6rem;\n",
|
|
" font-weight: lighter;\n",
|
|
" color: var(--sklearn-color-text-muted);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-2 label.sk-toggleable__label-arrow:before {\n",
|
|
" /* Arrow on the left of the label */\n",
|
|
" content: \"▸\";\n",
|
|
" float: left;\n",
|
|
" margin-right: 0.25em;\n",
|
|
" color: var(--sklearn-color-icon);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-2 label.sk-toggleable__label-arrow:hover:before {\n",
|
|
" color: var(--sklearn-color-text);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Toggleable content - dropdown */\n",
|
|
"\n",
|
|
"#sk-container-id-2 div.sk-toggleable__content {\n",
|
|
" max-height: 0;\n",
|
|
" max-width: 0;\n",
|
|
" overflow: hidden;\n",
|
|
" text-align: left;\n",
|
|
" /* unfitted */\n",
|
|
" background-color: var(--sklearn-color-unfitted-level-0);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-2 div.sk-toggleable__content.fitted {\n",
|
|
" /* fitted */\n",
|
|
" background-color: var(--sklearn-color-fitted-level-0);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-2 div.sk-toggleable__content pre {\n",
|
|
" margin: 0.2em;\n",
|
|
" border-radius: 0.25em;\n",
|
|
" color: var(--sklearn-color-text);\n",
|
|
" /* unfitted */\n",
|
|
" background-color: var(--sklearn-color-unfitted-level-0);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-2 div.sk-toggleable__content.fitted pre {\n",
|
|
" /* unfitted */\n",
|
|
" background-color: var(--sklearn-color-fitted-level-0);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-2 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n",
|
|
" /* Expand drop-down */\n",
|
|
" max-height: 200px;\n",
|
|
" max-width: 100%;\n",
|
|
" overflow: auto;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-2 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n",
|
|
" content: \"▾\";\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Pipeline/ColumnTransformer-specific style */\n",
|
|
"\n",
|
|
"#sk-container-id-2 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
|
|
" color: var(--sklearn-color-text);\n",
|
|
" background-color: var(--sklearn-color-unfitted-level-2);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-2 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
|
|
" background-color: var(--sklearn-color-fitted-level-2);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Estimator-specific style */\n",
|
|
"\n",
|
|
"/* Colorize estimator box */\n",
|
|
"#sk-container-id-2 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
|
|
" /* unfitted */\n",
|
|
" background-color: var(--sklearn-color-unfitted-level-2);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-2 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
|
|
" /* fitted */\n",
|
|
" background-color: var(--sklearn-color-fitted-level-2);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-2 div.sk-label label.sk-toggleable__label,\n",
|
|
"#sk-container-id-2 div.sk-label label {\n",
|
|
" /* The background is the default theme color */\n",
|
|
" color: var(--sklearn-color-text-on-default-background);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* On hover, darken the color of the background */\n",
|
|
"#sk-container-id-2 div.sk-label:hover label.sk-toggleable__label {\n",
|
|
" color: var(--sklearn-color-text);\n",
|
|
" background-color: var(--sklearn-color-unfitted-level-2);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Label box, darken color on hover, fitted */\n",
|
|
"#sk-container-id-2 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n",
|
|
" color: var(--sklearn-color-text);\n",
|
|
" background-color: var(--sklearn-color-fitted-level-2);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Estimator label */\n",
|
|
"\n",
|
|
"#sk-container-id-2 div.sk-label label {\n",
|
|
" font-family: monospace;\n",
|
|
" font-weight: bold;\n",
|
|
" display: inline-block;\n",
|
|
" line-height: 1.2em;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-2 div.sk-label-container {\n",
|
|
" text-align: center;\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Estimator-specific */\n",
|
|
"#sk-container-id-2 div.sk-estimator {\n",
|
|
" font-family: monospace;\n",
|
|
" border: 1px dotted var(--sklearn-color-border-box);\n",
|
|
" border-radius: 0.25em;\n",
|
|
" box-sizing: border-box;\n",
|
|
" margin-bottom: 0.5em;\n",
|
|
" /* unfitted */\n",
|
|
" background-color: var(--sklearn-color-unfitted-level-0);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-2 div.sk-estimator.fitted {\n",
|
|
" /* fitted */\n",
|
|
" background-color: var(--sklearn-color-fitted-level-0);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* on hover */\n",
|
|
"#sk-container-id-2 div.sk-estimator:hover {\n",
|
|
" /* unfitted */\n",
|
|
" background-color: var(--sklearn-color-unfitted-level-2);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-2 div.sk-estimator.fitted:hover {\n",
|
|
" /* fitted */\n",
|
|
" background-color: var(--sklearn-color-fitted-level-2);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Specification for estimator info (e.g. \"i\" and \"?\") */\n",
|
|
"\n",
|
|
"/* Common style for \"i\" and \"?\" */\n",
|
|
"\n",
|
|
".sk-estimator-doc-link,\n",
|
|
"a:link.sk-estimator-doc-link,\n",
|
|
"a:visited.sk-estimator-doc-link {\n",
|
|
" float: right;\n",
|
|
" font-size: smaller;\n",
|
|
" line-height: 1em;\n",
|
|
" font-family: monospace;\n",
|
|
" background-color: var(--sklearn-color-background);\n",
|
|
" border-radius: 1em;\n",
|
|
" height: 1em;\n",
|
|
" width: 1em;\n",
|
|
" text-decoration: none !important;\n",
|
|
" margin-left: 0.5em;\n",
|
|
" text-align: center;\n",
|
|
" /* unfitted */\n",
|
|
" border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
|
|
" color: var(--sklearn-color-unfitted-level-1);\n",
|
|
"}\n",
|
|
"\n",
|
|
".sk-estimator-doc-link.fitted,\n",
|
|
"a:link.sk-estimator-doc-link.fitted,\n",
|
|
"a:visited.sk-estimator-doc-link.fitted {\n",
|
|
" /* fitted */\n",
|
|
" border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
|
|
" color: var(--sklearn-color-fitted-level-1);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* On hover */\n",
|
|
"div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",
|
|
".sk-estimator-doc-link:hover,\n",
|
|
"div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",
|
|
".sk-estimator-doc-link:hover {\n",
|
|
" /* unfitted */\n",
|
|
" background-color: var(--sklearn-color-unfitted-level-3);\n",
|
|
" color: var(--sklearn-color-background);\n",
|
|
" text-decoration: none;\n",
|
|
"}\n",
|
|
"\n",
|
|
"div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",
|
|
".sk-estimator-doc-link.fitted:hover,\n",
|
|
"div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",
|
|
".sk-estimator-doc-link.fitted:hover {\n",
|
|
" /* fitted */\n",
|
|
" background-color: var(--sklearn-color-fitted-level-3);\n",
|
|
" color: var(--sklearn-color-background);\n",
|
|
" text-decoration: none;\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Span, style for the box shown on hovering the info icon */\n",
|
|
".sk-estimator-doc-link span {\n",
|
|
" display: none;\n",
|
|
" z-index: 9999;\n",
|
|
" position: relative;\n",
|
|
" font-weight: normal;\n",
|
|
" right: .2ex;\n",
|
|
" padding: .5ex;\n",
|
|
" margin: .5ex;\n",
|
|
" width: min-content;\n",
|
|
" min-width: 20ex;\n",
|
|
" max-width: 50ex;\n",
|
|
" color: var(--sklearn-color-text);\n",
|
|
" box-shadow: 2pt 2pt 4pt #999;\n",
|
|
" /* unfitted */\n",
|
|
" background: var(--sklearn-color-unfitted-level-0);\n",
|
|
" border: .5pt solid var(--sklearn-color-unfitted-level-3);\n",
|
|
"}\n",
|
|
"\n",
|
|
".sk-estimator-doc-link.fitted span {\n",
|
|
" /* fitted */\n",
|
|
" background: var(--sklearn-color-fitted-level-0);\n",
|
|
" border: var(--sklearn-color-fitted-level-3);\n",
|
|
"}\n",
|
|
"\n",
|
|
".sk-estimator-doc-link:hover span {\n",
|
|
" display: block;\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* \"?\"-specific style due to the `<a>` HTML tag */\n",
|
|
"\n",
|
|
"#sk-container-id-2 a.estimator_doc_link {\n",
|
|
" float: right;\n",
|
|
" font-size: 1rem;\n",
|
|
" line-height: 1em;\n",
|
|
" font-family: monospace;\n",
|
|
" background-color: var(--sklearn-color-background);\n",
|
|
" border-radius: 1rem;\n",
|
|
" height: 1rem;\n",
|
|
" width: 1rem;\n",
|
|
" text-decoration: none;\n",
|
|
" /* unfitted */\n",
|
|
" color: var(--sklearn-color-unfitted-level-1);\n",
|
|
" border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-2 a.estimator_doc_link.fitted {\n",
|
|
" /* fitted */\n",
|
|
" border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
|
|
" color: var(--sklearn-color-fitted-level-1);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* On hover */\n",
|
|
"#sk-container-id-2 a.estimator_doc_link:hover {\n",
|
|
" /* unfitted */\n",
|
|
" background-color: var(--sklearn-color-unfitted-level-3);\n",
|
|
" color: var(--sklearn-color-background);\n",
|
|
" text-decoration: none;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-2 a.estimator_doc_link.fitted:hover {\n",
|
|
" /* fitted */\n",
|
|
" background-color: var(--sklearn-color-fitted-level-3);\n",
|
|
"}\n",
|
|
"</style><div id=\"sk-container-id-2\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>DecisionTreeClassifier(criterion='entropy', max_depth=2)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-2\" type=\"checkbox\" checked><label for=\"sk-estimator-id-2\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow\"><div><div>DecisionTreeClassifier</div></div><div><a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.6/modules/generated/sklearn.tree.DecisionTreeClassifier.html\">?<span>Documentation for DecisionTreeClassifier</span></a><span class=\"sk-estimator-doc-link fitted\">i<span>Fitted</span></span></div></label><div class=\"sk-toggleable__content fitted\"><pre>DecisionTreeClassifier(criterion='entropy', max_depth=2)</pre></div> </div></div></div></div>"
|
|
],
|
|
"text/plain": [
|
|
"DecisionTreeClassifier(criterion='entropy', max_depth=2)"
|
|
]
|
|
},
|
|
"execution_count": 5,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"# Redo the first example but use entropy instead\n",
|
|
"tree_clf = DecisionTreeClassifier(max_depth = 2,criterion='entropy') #making a decision tree of depth 2 from the data \n",
|
|
"tree_clf.fit(X_iris, y_iris)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"metadata": {
|
|
"editable": true,
|
|
"execution": {
|
|
"iopub.execute_input": "2025-02-27T23:21:20.587706Z",
|
|
"iopub.status.busy": "2025-02-27T23:21:20.587538Z",
|
|
"iopub.status.idle": "2025-02-27T23:21:20.733060Z",
|
|
"shell.execute_reply": "2025-02-27T23:21:20.732336Z"
|
|
},
|
|
"slideshow": {
|
|
"slide_type": "skip"
|
|
},
|
|
"tags": []
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"export_graphviz(tree_clf, \n",
|
|
" out_file = './iris_tree_entropy.dot', \n",
|
|
" feature_names = iris.feature_names[ 2:], \n",
|
|
" class_names = iris.target_names, \n",
|
|
" rounded = True, \n",
|
|
" filled = True)\n",
|
|
"! dot -Tpng ./iris_tree_entropy.dot -o ./iris_tree_entropy.png"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "subslide"
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"#### Entropy tree\n",
|
|
"\n",
|
|
"<img src=\"https://raw.githubusercontent.com/astro-informatics/course_mlbd_images/master/Lecture16_Images/iris_tree_entropy.png\" alt=\"data-layout\" width=\"500\" style=\"display:block; margin:auto\"/>"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "subslide"
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"#### Gini tree (for reference)\n",
|
|
"\n",
|
|
"<img src=\"https://raw.githubusercontent.com/astro-informatics/course_mlbd_images/master/Lecture16_Images/iris_tree.png\" alt=\"data-layout\" width=\"500\" style=\"display:block; margin:auto\"/>"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"## CART training algorithm\n",
|
|
"\n",
|
|
"Classification And Regression Tree (CART) algorihtm can be used to train decision trees -- also called *growing trees* algorithm (used by SciKit Learn)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "subslide"
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"- Splits the sample into two subsets using a single feature $k$ at threshold $t_k$\n",
|
|
"- Chooses feature $k$ and threshold $t_k$ by finding pair that produces purest subset, weighted by their size.\n",
|
|
"\n",
|
|
"Cost function minimized for each split:\n",
|
|
"\n",
|
|
"$$\n",
|
|
"J(k,t_k)=\\frac{m_\\text{left}}{m}G_\\text{left}+\\frac{m_\\text{right}}{m}G_\\text{right} ,\n",
|
|
"$$\n",
|
|
"\n",
|
|
"where $G_\\text{left/right}$ measures the impurity of the left/right subset and $m_\\text{left/right}$ is the number of instances in the left/right subset."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "subslide"
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"Note that the CART algorithm:\n",
|
|
"- Only splits data in two at each stage, i.e. is binary.\n",
|
|
"- Is a greedy algorithm. It searches for the optimal split at each level, then repeats for subsequent levels. There is no guarantee that the overall optimal tree is found. Nevertheless, usually produces a tree that is reasonably good."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"## Regularisation and hyperparameters\n",
|
|
"\n",
|
|
"Decision Trees are **non-parametric** classification algorithms since the number of parameters is not determined prior to training.\n",
|
|
"\n",
|
|
"Tends to overfit if not careful. "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "subslide"
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"Need to regularise the problem. This can be done by the restricting the complexity of the tree, for example though the following SciKit Learn parameters.\n",
|
|
"\n",
|
|
"- `max_depth`: maximum depth of the tree.\n",
|
|
"- `max_features` maximum number of features used in splitting at each node.\n",
|
|
"- `max_leaf_nodes` maximum number of leaf nodes.\n",
|
|
"- `min_samples_split`: mimimum number of samples a node must have before it can be split.\n",
|
|
"- `min_samples_leaf`: minimum number a leaf can have to be created.\n",
|
|
"- `min_weight_fraction`: Same as `min_samples_leaf` but expressed as a fraction of total samples.\n",
|
|
"\n",
|
|
"Generally, increasing ```min_*``` hyperparameters or reducing ```max_*``` hyperparameters will regularise the model."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "subslide"
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"Other algorithms _prune_, i.e. make a (relatively) unrestricted tree then remove statistically insignificant nodes."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "subslide"
|
|
},
|
|
"tags": [
|
|
"exercise_pointer"
|
|
]
|
|
},
|
|
"source": [
|
|
"**Exercises:** *You can now complete Exercise 1 in the exercises associated with this lecture.*"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"## Decision trees for regression \n",
|
|
"\n",
|
|
"Decision trees can also be used for regression tasks."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "subslide"
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"### Train regression model"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 7,
|
|
"metadata": {
|
|
"editable": true,
|
|
"execution": {
|
|
"iopub.execute_input": "2025-02-27T23:21:20.735593Z",
|
|
"iopub.status.busy": "2025-02-27T23:21:20.735405Z",
|
|
"iopub.status.idle": "2025-02-27T23:21:20.743106Z",
|
|
"shell.execute_reply": "2025-02-27T23:21:20.742456Z"
|
|
},
|
|
"slideshow": {
|
|
"slide_type": ""
|
|
},
|
|
"tags": []
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/html": [
|
|
"<style>#sk-container-id-3 {\n",
|
|
" /* Definition of color scheme common for light and dark mode */\n",
|
|
" --sklearn-color-text: #000;\n",
|
|
" --sklearn-color-text-muted: #666;\n",
|
|
" --sklearn-color-line: gray;\n",
|
|
" /* Definition of color scheme for unfitted estimators */\n",
|
|
" --sklearn-color-unfitted-level-0: #fff5e6;\n",
|
|
" --sklearn-color-unfitted-level-1: #f6e4d2;\n",
|
|
" --sklearn-color-unfitted-level-2: #ffe0b3;\n",
|
|
" --sklearn-color-unfitted-level-3: chocolate;\n",
|
|
" /* Definition of color scheme for fitted estimators */\n",
|
|
" --sklearn-color-fitted-level-0: #f0f8ff;\n",
|
|
" --sklearn-color-fitted-level-1: #d4ebff;\n",
|
|
" --sklearn-color-fitted-level-2: #b3dbfd;\n",
|
|
" --sklearn-color-fitted-level-3: cornflowerblue;\n",
|
|
"\n",
|
|
" /* Specific color for light theme */\n",
|
|
" --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
|
|
" --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n",
|
|
" --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
|
|
" --sklearn-color-icon: #696969;\n",
|
|
"\n",
|
|
" @media (prefers-color-scheme: dark) {\n",
|
|
" /* Redefinition of color scheme for dark theme */\n",
|
|
" --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
|
|
" --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n",
|
|
" --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
|
|
" --sklearn-color-icon: #878787;\n",
|
|
" }\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-3 {\n",
|
|
" color: var(--sklearn-color-text);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-3 pre {\n",
|
|
" padding: 0;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-3 input.sk-hidden--visually {\n",
|
|
" border: 0;\n",
|
|
" clip: rect(1px 1px 1px 1px);\n",
|
|
" clip: rect(1px, 1px, 1px, 1px);\n",
|
|
" height: 1px;\n",
|
|
" margin: -1px;\n",
|
|
" overflow: hidden;\n",
|
|
" padding: 0;\n",
|
|
" position: absolute;\n",
|
|
" width: 1px;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-3 div.sk-dashed-wrapped {\n",
|
|
" border: 1px dashed var(--sklearn-color-line);\n",
|
|
" margin: 0 0.4em 0.5em 0.4em;\n",
|
|
" box-sizing: border-box;\n",
|
|
" padding-bottom: 0.4em;\n",
|
|
" background-color: var(--sklearn-color-background);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-3 div.sk-container {\n",
|
|
" /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n",
|
|
" but bootstrap.min.css set `[hidden] { display: none !important; }`\n",
|
|
" so we also need the `!important` here to be able to override the\n",
|
|
" default hidden behavior on the sphinx rendered scikit-learn.org.\n",
|
|
" See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n",
|
|
" display: inline-block !important;\n",
|
|
" position: relative;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-3 div.sk-text-repr-fallback {\n",
|
|
" display: none;\n",
|
|
"}\n",
|
|
"\n",
|
|
"div.sk-parallel-item,\n",
|
|
"div.sk-serial,\n",
|
|
"div.sk-item {\n",
|
|
" /* draw centered vertical line to link estimators */\n",
|
|
" background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n",
|
|
" background-size: 2px 100%;\n",
|
|
" background-repeat: no-repeat;\n",
|
|
" background-position: center center;\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Parallel-specific style estimator block */\n",
|
|
"\n",
|
|
"#sk-container-id-3 div.sk-parallel-item::after {\n",
|
|
" content: \"\";\n",
|
|
" width: 100%;\n",
|
|
" border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n",
|
|
" flex-grow: 1;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-3 div.sk-parallel {\n",
|
|
" display: flex;\n",
|
|
" align-items: stretch;\n",
|
|
" justify-content: center;\n",
|
|
" background-color: var(--sklearn-color-background);\n",
|
|
" position: relative;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-3 div.sk-parallel-item {\n",
|
|
" display: flex;\n",
|
|
" flex-direction: column;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-3 div.sk-parallel-item:first-child::after {\n",
|
|
" align-self: flex-end;\n",
|
|
" width: 50%;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-3 div.sk-parallel-item:last-child::after {\n",
|
|
" align-self: flex-start;\n",
|
|
" width: 50%;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-3 div.sk-parallel-item:only-child::after {\n",
|
|
" width: 0;\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Serial-specific style estimator block */\n",
|
|
"\n",
|
|
"#sk-container-id-3 div.sk-serial {\n",
|
|
" display: flex;\n",
|
|
" flex-direction: column;\n",
|
|
" align-items: center;\n",
|
|
" background-color: var(--sklearn-color-background);\n",
|
|
" padding-right: 1em;\n",
|
|
" padding-left: 1em;\n",
|
|
"}\n",
|
|
"\n",
|
|
"\n",
|
|
"/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n",
|
|
"clickable and can be expanded/collapsed.\n",
|
|
"- Pipeline and ColumnTransformer use this feature and define the default style\n",
|
|
"- Estimators will overwrite some part of the style using the `sk-estimator` class\n",
|
|
"*/\n",
|
|
"\n",
|
|
"/* Pipeline and ColumnTransformer style (default) */\n",
|
|
"\n",
|
|
"#sk-container-id-3 div.sk-toggleable {\n",
|
|
" /* Default theme specific background. It is overwritten whether we have a\n",
|
|
" specific estimator or a Pipeline/ColumnTransformer */\n",
|
|
" background-color: var(--sklearn-color-background);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Toggleable label */\n",
|
|
"#sk-container-id-3 label.sk-toggleable__label {\n",
|
|
" cursor: pointer;\n",
|
|
" display: flex;\n",
|
|
" width: 100%;\n",
|
|
" margin-bottom: 0;\n",
|
|
" padding: 0.5em;\n",
|
|
" box-sizing: border-box;\n",
|
|
" text-align: center;\n",
|
|
" align-items: start;\n",
|
|
" justify-content: space-between;\n",
|
|
" gap: 0.5em;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-3 label.sk-toggleable__label .caption {\n",
|
|
" font-size: 0.6rem;\n",
|
|
" font-weight: lighter;\n",
|
|
" color: var(--sklearn-color-text-muted);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-3 label.sk-toggleable__label-arrow:before {\n",
|
|
" /* Arrow on the left of the label */\n",
|
|
" content: \"▸\";\n",
|
|
" float: left;\n",
|
|
" margin-right: 0.25em;\n",
|
|
" color: var(--sklearn-color-icon);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-3 label.sk-toggleable__label-arrow:hover:before {\n",
|
|
" color: var(--sklearn-color-text);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Toggleable content - dropdown */\n",
|
|
"\n",
|
|
"#sk-container-id-3 div.sk-toggleable__content {\n",
|
|
" max-height: 0;\n",
|
|
" max-width: 0;\n",
|
|
" overflow: hidden;\n",
|
|
" text-align: left;\n",
|
|
" /* unfitted */\n",
|
|
" background-color: var(--sklearn-color-unfitted-level-0);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-3 div.sk-toggleable__content.fitted {\n",
|
|
" /* fitted */\n",
|
|
" background-color: var(--sklearn-color-fitted-level-0);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-3 div.sk-toggleable__content pre {\n",
|
|
" margin: 0.2em;\n",
|
|
" border-radius: 0.25em;\n",
|
|
" color: var(--sklearn-color-text);\n",
|
|
" /* unfitted */\n",
|
|
" background-color: var(--sklearn-color-unfitted-level-0);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-3 div.sk-toggleable__content.fitted pre {\n",
|
|
" /* unfitted */\n",
|
|
" background-color: var(--sklearn-color-fitted-level-0);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-3 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n",
|
|
" /* Expand drop-down */\n",
|
|
" max-height: 200px;\n",
|
|
" max-width: 100%;\n",
|
|
" overflow: auto;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-3 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n",
|
|
" content: \"▾\";\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Pipeline/ColumnTransformer-specific style */\n",
|
|
"\n",
|
|
"#sk-container-id-3 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
|
|
" color: var(--sklearn-color-text);\n",
|
|
" background-color: var(--sklearn-color-unfitted-level-2);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-3 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
|
|
" background-color: var(--sklearn-color-fitted-level-2);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Estimator-specific style */\n",
|
|
"\n",
|
|
"/* Colorize estimator box */\n",
|
|
"#sk-container-id-3 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
|
|
" /* unfitted */\n",
|
|
" background-color: var(--sklearn-color-unfitted-level-2);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-3 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
|
|
" /* fitted */\n",
|
|
" background-color: var(--sklearn-color-fitted-level-2);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-3 div.sk-label label.sk-toggleable__label,\n",
|
|
"#sk-container-id-3 div.sk-label label {\n",
|
|
" /* The background is the default theme color */\n",
|
|
" color: var(--sklearn-color-text-on-default-background);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* On hover, darken the color of the background */\n",
|
|
"#sk-container-id-3 div.sk-label:hover label.sk-toggleable__label {\n",
|
|
" color: var(--sklearn-color-text);\n",
|
|
" background-color: var(--sklearn-color-unfitted-level-2);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Label box, darken color on hover, fitted */\n",
|
|
"#sk-container-id-3 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n",
|
|
" color: var(--sklearn-color-text);\n",
|
|
" background-color: var(--sklearn-color-fitted-level-2);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Estimator label */\n",
|
|
"\n",
|
|
"#sk-container-id-3 div.sk-label label {\n",
|
|
" font-family: monospace;\n",
|
|
" font-weight: bold;\n",
|
|
" display: inline-block;\n",
|
|
" line-height: 1.2em;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-3 div.sk-label-container {\n",
|
|
" text-align: center;\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Estimator-specific */\n",
|
|
"#sk-container-id-3 div.sk-estimator {\n",
|
|
" font-family: monospace;\n",
|
|
" border: 1px dotted var(--sklearn-color-border-box);\n",
|
|
" border-radius: 0.25em;\n",
|
|
" box-sizing: border-box;\n",
|
|
" margin-bottom: 0.5em;\n",
|
|
" /* unfitted */\n",
|
|
" background-color: var(--sklearn-color-unfitted-level-0);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-3 div.sk-estimator.fitted {\n",
|
|
" /* fitted */\n",
|
|
" background-color: var(--sklearn-color-fitted-level-0);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* on hover */\n",
|
|
"#sk-container-id-3 div.sk-estimator:hover {\n",
|
|
" /* unfitted */\n",
|
|
" background-color: var(--sklearn-color-unfitted-level-2);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-3 div.sk-estimator.fitted:hover {\n",
|
|
" /* fitted */\n",
|
|
" background-color: var(--sklearn-color-fitted-level-2);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Specification for estimator info (e.g. \"i\" and \"?\") */\n",
|
|
"\n",
|
|
"/* Common style for \"i\" and \"?\" */\n",
|
|
"\n",
|
|
".sk-estimator-doc-link,\n",
|
|
"a:link.sk-estimator-doc-link,\n",
|
|
"a:visited.sk-estimator-doc-link {\n",
|
|
" float: right;\n",
|
|
" font-size: smaller;\n",
|
|
" line-height: 1em;\n",
|
|
" font-family: monospace;\n",
|
|
" background-color: var(--sklearn-color-background);\n",
|
|
" border-radius: 1em;\n",
|
|
" height: 1em;\n",
|
|
" width: 1em;\n",
|
|
" text-decoration: none !important;\n",
|
|
" margin-left: 0.5em;\n",
|
|
" text-align: center;\n",
|
|
" /* unfitted */\n",
|
|
" border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
|
|
" color: var(--sklearn-color-unfitted-level-1);\n",
|
|
"}\n",
|
|
"\n",
|
|
".sk-estimator-doc-link.fitted,\n",
|
|
"a:link.sk-estimator-doc-link.fitted,\n",
|
|
"a:visited.sk-estimator-doc-link.fitted {\n",
|
|
" /* fitted */\n",
|
|
" border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
|
|
" color: var(--sklearn-color-fitted-level-1);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* On hover */\n",
|
|
"div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",
|
|
".sk-estimator-doc-link:hover,\n",
|
|
"div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",
|
|
".sk-estimator-doc-link:hover {\n",
|
|
" /* unfitted */\n",
|
|
" background-color: var(--sklearn-color-unfitted-level-3);\n",
|
|
" color: var(--sklearn-color-background);\n",
|
|
" text-decoration: none;\n",
|
|
"}\n",
|
|
"\n",
|
|
"div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",
|
|
".sk-estimator-doc-link.fitted:hover,\n",
|
|
"div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",
|
|
".sk-estimator-doc-link.fitted:hover {\n",
|
|
" /* fitted */\n",
|
|
" background-color: var(--sklearn-color-fitted-level-3);\n",
|
|
" color: var(--sklearn-color-background);\n",
|
|
" text-decoration: none;\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Span, style for the box shown on hovering the info icon */\n",
|
|
".sk-estimator-doc-link span {\n",
|
|
" display: none;\n",
|
|
" z-index: 9999;\n",
|
|
" position: relative;\n",
|
|
" font-weight: normal;\n",
|
|
" right: .2ex;\n",
|
|
" padding: .5ex;\n",
|
|
" margin: .5ex;\n",
|
|
" width: min-content;\n",
|
|
" min-width: 20ex;\n",
|
|
" max-width: 50ex;\n",
|
|
" color: var(--sklearn-color-text);\n",
|
|
" box-shadow: 2pt 2pt 4pt #999;\n",
|
|
" /* unfitted */\n",
|
|
" background: var(--sklearn-color-unfitted-level-0);\n",
|
|
" border: .5pt solid var(--sklearn-color-unfitted-level-3);\n",
|
|
"}\n",
|
|
"\n",
|
|
".sk-estimator-doc-link.fitted span {\n",
|
|
" /* fitted */\n",
|
|
" background: var(--sklearn-color-fitted-level-0);\n",
|
|
" border: var(--sklearn-color-fitted-level-3);\n",
|
|
"}\n",
|
|
"\n",
|
|
".sk-estimator-doc-link:hover span {\n",
|
|
" display: block;\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* \"?\"-specific style due to the `<a>` HTML tag */\n",
|
|
"\n",
|
|
"#sk-container-id-3 a.estimator_doc_link {\n",
|
|
" float: right;\n",
|
|
" font-size: 1rem;\n",
|
|
" line-height: 1em;\n",
|
|
" font-family: monospace;\n",
|
|
" background-color: var(--sklearn-color-background);\n",
|
|
" border-radius: 1rem;\n",
|
|
" height: 1rem;\n",
|
|
" width: 1rem;\n",
|
|
" text-decoration: none;\n",
|
|
" /* unfitted */\n",
|
|
" color: var(--sklearn-color-unfitted-level-1);\n",
|
|
" border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-3 a.estimator_doc_link.fitted {\n",
|
|
" /* fitted */\n",
|
|
" border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
|
|
" color: var(--sklearn-color-fitted-level-1);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* On hover */\n",
|
|
"#sk-container-id-3 a.estimator_doc_link:hover {\n",
|
|
" /* unfitted */\n",
|
|
" background-color: var(--sklearn-color-unfitted-level-3);\n",
|
|
" color: var(--sklearn-color-background);\n",
|
|
" text-decoration: none;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-3 a.estimator_doc_link.fitted:hover {\n",
|
|
" /* fitted */\n",
|
|
" background-color: var(--sklearn-color-fitted-level-3);\n",
|
|
"}\n",
|
|
"</style><div id=\"sk-container-id-3\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>DecisionTreeRegressor(max_depth=2, random_state=42)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-3\" type=\"checkbox\" checked><label for=\"sk-estimator-id-3\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow\"><div><div>DecisionTreeRegressor</div></div><div><a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.6/modules/generated/sklearn.tree.DecisionTreeRegressor.html\">?<span>Documentation for DecisionTreeRegressor</span></a><span class=\"sk-estimator-doc-link fitted\">i<span>Fitted</span></span></div></label><div class=\"sk-toggleable__content fitted\"><pre>DecisionTreeRegressor(max_depth=2, random_state=42)</pre></div> </div></div></div></div>"
|
|
],
|
|
"text/plain": [
|
|
"DecisionTreeRegressor(max_depth=2, random_state=42)"
|
|
]
|
|
},
|
|
"execution_count": 7,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"import numpy as np\n",
|
|
"from sklearn.tree import DecisionTreeRegressor\n",
|
|
"\n",
|
|
"np.random.seed(42)\n",
|
|
"X_quad = np.random.rand(200, 1) - 0.5 # a single random input feature\n",
|
|
"y_quad = X_quad ** 2 + 0.025 * np.random.randn(200, 1)\n",
|
|
"\n",
|
|
"tree_reg = DecisionTreeRegressor(max_depth=2, random_state=42)\n",
|
|
"tree_reg.fit(X_quad, y_quad)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "subslide"
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"### Visualise tree"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 8,
|
|
"metadata": {
|
|
"editable": true,
|
|
"execution": {
|
|
"iopub.execute_input": "2025-02-27T23:21:20.744939Z",
|
|
"iopub.status.busy": "2025-02-27T23:21:20.744742Z",
|
|
"iopub.status.idle": "2025-02-27T23:21:20.896284Z",
|
|
"shell.execute_reply": "2025-02-27T23:21:20.895553Z"
|
|
},
|
|
"slideshow": {
|
|
"slide_type": "skip"
|
|
},
|
|
"tags": []
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"export_graphviz(\n",
|
|
" tree_reg,\n",
|
|
" out_file=\"./regression_tree.dot\",\n",
|
|
" feature_names=[\"x1\"],\n",
|
|
" rounded=True,\n",
|
|
" filled=True\n",
|
|
")\n",
|
|
"\n",
|
|
"#creates a dot file :( so need to convert to something more sensible\n",
|
|
"! dot -Tpng ./regression_tree.dot -o ./regression_tree.png"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": ""
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"<img src=\"https://raw.githubusercontent.com/astro-informatics/course_mlbd_images/master/Lecture16_Images/regression_tree.png\" alt=\"data-layout\" width=\"700\" style=\"display:block; margin:auto\"/>"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "subslide"
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"### Train a deeper model"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 9,
|
|
"metadata": {
|
|
"editable": true,
|
|
"execution": {
|
|
"iopub.execute_input": "2025-02-27T23:21:20.898435Z",
|
|
"iopub.status.busy": "2025-02-27T23:21:20.898251Z",
|
|
"iopub.status.idle": "2025-02-27T23:21:20.904714Z",
|
|
"shell.execute_reply": "2025-02-27T23:21:20.904136Z"
|
|
},
|
|
"slideshow": {
|
|
"slide_type": ""
|
|
},
|
|
"tags": []
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/html": [
|
|
"<style>#sk-container-id-4 {\n",
|
|
" /* Definition of color scheme common for light and dark mode */\n",
|
|
" --sklearn-color-text: #000;\n",
|
|
" --sklearn-color-text-muted: #666;\n",
|
|
" --sklearn-color-line: gray;\n",
|
|
" /* Definition of color scheme for unfitted estimators */\n",
|
|
" --sklearn-color-unfitted-level-0: #fff5e6;\n",
|
|
" --sklearn-color-unfitted-level-1: #f6e4d2;\n",
|
|
" --sklearn-color-unfitted-level-2: #ffe0b3;\n",
|
|
" --sklearn-color-unfitted-level-3: chocolate;\n",
|
|
" /* Definition of color scheme for fitted estimators */\n",
|
|
" --sklearn-color-fitted-level-0: #f0f8ff;\n",
|
|
" --sklearn-color-fitted-level-1: #d4ebff;\n",
|
|
" --sklearn-color-fitted-level-2: #b3dbfd;\n",
|
|
" --sklearn-color-fitted-level-3: cornflowerblue;\n",
|
|
"\n",
|
|
" /* Specific color for light theme */\n",
|
|
" --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
|
|
" --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n",
|
|
" --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
|
|
" --sklearn-color-icon: #696969;\n",
|
|
"\n",
|
|
" @media (prefers-color-scheme: dark) {\n",
|
|
" /* Redefinition of color scheme for dark theme */\n",
|
|
" --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
|
|
" --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n",
|
|
" --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
|
|
" --sklearn-color-icon: #878787;\n",
|
|
" }\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-4 {\n",
|
|
" color: var(--sklearn-color-text);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-4 pre {\n",
|
|
" padding: 0;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-4 input.sk-hidden--visually {\n",
|
|
" border: 0;\n",
|
|
" clip: rect(1px 1px 1px 1px);\n",
|
|
" clip: rect(1px, 1px, 1px, 1px);\n",
|
|
" height: 1px;\n",
|
|
" margin: -1px;\n",
|
|
" overflow: hidden;\n",
|
|
" padding: 0;\n",
|
|
" position: absolute;\n",
|
|
" width: 1px;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-4 div.sk-dashed-wrapped {\n",
|
|
" border: 1px dashed var(--sklearn-color-line);\n",
|
|
" margin: 0 0.4em 0.5em 0.4em;\n",
|
|
" box-sizing: border-box;\n",
|
|
" padding-bottom: 0.4em;\n",
|
|
" background-color: var(--sklearn-color-background);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-4 div.sk-container {\n",
|
|
" /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n",
|
|
" but bootstrap.min.css set `[hidden] { display: none !important; }`\n",
|
|
" so we also need the `!important` here to be able to override the\n",
|
|
" default hidden behavior on the sphinx rendered scikit-learn.org.\n",
|
|
" See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n",
|
|
" display: inline-block !important;\n",
|
|
" position: relative;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-4 div.sk-text-repr-fallback {\n",
|
|
" display: none;\n",
|
|
"}\n",
|
|
"\n",
|
|
"div.sk-parallel-item,\n",
|
|
"div.sk-serial,\n",
|
|
"div.sk-item {\n",
|
|
" /* draw centered vertical line to link estimators */\n",
|
|
" background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n",
|
|
" background-size: 2px 100%;\n",
|
|
" background-repeat: no-repeat;\n",
|
|
" background-position: center center;\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Parallel-specific style estimator block */\n",
|
|
"\n",
|
|
"#sk-container-id-4 div.sk-parallel-item::after {\n",
|
|
" content: \"\";\n",
|
|
" width: 100%;\n",
|
|
" border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n",
|
|
" flex-grow: 1;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-4 div.sk-parallel {\n",
|
|
" display: flex;\n",
|
|
" align-items: stretch;\n",
|
|
" justify-content: center;\n",
|
|
" background-color: var(--sklearn-color-background);\n",
|
|
" position: relative;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-4 div.sk-parallel-item {\n",
|
|
" display: flex;\n",
|
|
" flex-direction: column;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-4 div.sk-parallel-item:first-child::after {\n",
|
|
" align-self: flex-end;\n",
|
|
" width: 50%;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-4 div.sk-parallel-item:last-child::after {\n",
|
|
" align-self: flex-start;\n",
|
|
" width: 50%;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-4 div.sk-parallel-item:only-child::after {\n",
|
|
" width: 0;\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Serial-specific style estimator block */\n",
|
|
"\n",
|
|
"#sk-container-id-4 div.sk-serial {\n",
|
|
" display: flex;\n",
|
|
" flex-direction: column;\n",
|
|
" align-items: center;\n",
|
|
" background-color: var(--sklearn-color-background);\n",
|
|
" padding-right: 1em;\n",
|
|
" padding-left: 1em;\n",
|
|
"}\n",
|
|
"\n",
|
|
"\n",
|
|
"/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n",
|
|
"clickable and can be expanded/collapsed.\n",
|
|
"- Pipeline and ColumnTransformer use this feature and define the default style\n",
|
|
"- Estimators will overwrite some part of the style using the `sk-estimator` class\n",
|
|
"*/\n",
|
|
"\n",
|
|
"/* Pipeline and ColumnTransformer style (default) */\n",
|
|
"\n",
|
|
"#sk-container-id-4 div.sk-toggleable {\n",
|
|
" /* Default theme specific background. It is overwritten whether we have a\n",
|
|
" specific estimator or a Pipeline/ColumnTransformer */\n",
|
|
" background-color: var(--sklearn-color-background);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Toggleable label */\n",
|
|
"#sk-container-id-4 label.sk-toggleable__label {\n",
|
|
" cursor: pointer;\n",
|
|
" display: flex;\n",
|
|
" width: 100%;\n",
|
|
" margin-bottom: 0;\n",
|
|
" padding: 0.5em;\n",
|
|
" box-sizing: border-box;\n",
|
|
" text-align: center;\n",
|
|
" align-items: start;\n",
|
|
" justify-content: space-between;\n",
|
|
" gap: 0.5em;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-4 label.sk-toggleable__label .caption {\n",
|
|
" font-size: 0.6rem;\n",
|
|
" font-weight: lighter;\n",
|
|
" color: var(--sklearn-color-text-muted);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-4 label.sk-toggleable__label-arrow:before {\n",
|
|
" /* Arrow on the left of the label */\n",
|
|
" content: \"▸\";\n",
|
|
" float: left;\n",
|
|
" margin-right: 0.25em;\n",
|
|
" color: var(--sklearn-color-icon);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-4 label.sk-toggleable__label-arrow:hover:before {\n",
|
|
" color: var(--sklearn-color-text);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Toggleable content - dropdown */\n",
|
|
"\n",
|
|
"#sk-container-id-4 div.sk-toggleable__content {\n",
|
|
" max-height: 0;\n",
|
|
" max-width: 0;\n",
|
|
" overflow: hidden;\n",
|
|
" text-align: left;\n",
|
|
" /* unfitted */\n",
|
|
" background-color: var(--sklearn-color-unfitted-level-0);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-4 div.sk-toggleable__content.fitted {\n",
|
|
" /* fitted */\n",
|
|
" background-color: var(--sklearn-color-fitted-level-0);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-4 div.sk-toggleable__content pre {\n",
|
|
" margin: 0.2em;\n",
|
|
" border-radius: 0.25em;\n",
|
|
" color: var(--sklearn-color-text);\n",
|
|
" /* unfitted */\n",
|
|
" background-color: var(--sklearn-color-unfitted-level-0);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-4 div.sk-toggleable__content.fitted pre {\n",
|
|
" /* unfitted */\n",
|
|
" background-color: var(--sklearn-color-fitted-level-0);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-4 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n",
|
|
" /* Expand drop-down */\n",
|
|
" max-height: 200px;\n",
|
|
" max-width: 100%;\n",
|
|
" overflow: auto;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-4 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n",
|
|
" content: \"▾\";\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Pipeline/ColumnTransformer-specific style */\n",
|
|
"\n",
|
|
"#sk-container-id-4 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
|
|
" color: var(--sklearn-color-text);\n",
|
|
" background-color: var(--sklearn-color-unfitted-level-2);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-4 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
|
|
" background-color: var(--sklearn-color-fitted-level-2);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Estimator-specific style */\n",
|
|
"\n",
|
|
"/* Colorize estimator box */\n",
|
|
"#sk-container-id-4 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
|
|
" /* unfitted */\n",
|
|
" background-color: var(--sklearn-color-unfitted-level-2);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-4 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
|
|
" /* fitted */\n",
|
|
" background-color: var(--sklearn-color-fitted-level-2);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-4 div.sk-label label.sk-toggleable__label,\n",
|
|
"#sk-container-id-4 div.sk-label label {\n",
|
|
" /* The background is the default theme color */\n",
|
|
" color: var(--sklearn-color-text-on-default-background);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* On hover, darken the color of the background */\n",
|
|
"#sk-container-id-4 div.sk-label:hover label.sk-toggleable__label {\n",
|
|
" color: var(--sklearn-color-text);\n",
|
|
" background-color: var(--sklearn-color-unfitted-level-2);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Label box, darken color on hover, fitted */\n",
|
|
"#sk-container-id-4 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n",
|
|
" color: var(--sklearn-color-text);\n",
|
|
" background-color: var(--sklearn-color-fitted-level-2);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Estimator label */\n",
|
|
"\n",
|
|
"#sk-container-id-4 div.sk-label label {\n",
|
|
" font-family: monospace;\n",
|
|
" font-weight: bold;\n",
|
|
" display: inline-block;\n",
|
|
" line-height: 1.2em;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-4 div.sk-label-container {\n",
|
|
" text-align: center;\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Estimator-specific */\n",
|
|
"#sk-container-id-4 div.sk-estimator {\n",
|
|
" font-family: monospace;\n",
|
|
" border: 1px dotted var(--sklearn-color-border-box);\n",
|
|
" border-radius: 0.25em;\n",
|
|
" box-sizing: border-box;\n",
|
|
" margin-bottom: 0.5em;\n",
|
|
" /* unfitted */\n",
|
|
" background-color: var(--sklearn-color-unfitted-level-0);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-4 div.sk-estimator.fitted {\n",
|
|
" /* fitted */\n",
|
|
" background-color: var(--sklearn-color-fitted-level-0);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* on hover */\n",
|
|
"#sk-container-id-4 div.sk-estimator:hover {\n",
|
|
" /* unfitted */\n",
|
|
" background-color: var(--sklearn-color-unfitted-level-2);\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-4 div.sk-estimator.fitted:hover {\n",
|
|
" /* fitted */\n",
|
|
" background-color: var(--sklearn-color-fitted-level-2);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Specification for estimator info (e.g. \"i\" and \"?\") */\n",
|
|
"\n",
|
|
"/* Common style for \"i\" and \"?\" */\n",
|
|
"\n",
|
|
".sk-estimator-doc-link,\n",
|
|
"a:link.sk-estimator-doc-link,\n",
|
|
"a:visited.sk-estimator-doc-link {\n",
|
|
" float: right;\n",
|
|
" font-size: smaller;\n",
|
|
" line-height: 1em;\n",
|
|
" font-family: monospace;\n",
|
|
" background-color: var(--sklearn-color-background);\n",
|
|
" border-radius: 1em;\n",
|
|
" height: 1em;\n",
|
|
" width: 1em;\n",
|
|
" text-decoration: none !important;\n",
|
|
" margin-left: 0.5em;\n",
|
|
" text-align: center;\n",
|
|
" /* unfitted */\n",
|
|
" border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
|
|
" color: var(--sklearn-color-unfitted-level-1);\n",
|
|
"}\n",
|
|
"\n",
|
|
".sk-estimator-doc-link.fitted,\n",
|
|
"a:link.sk-estimator-doc-link.fitted,\n",
|
|
"a:visited.sk-estimator-doc-link.fitted {\n",
|
|
" /* fitted */\n",
|
|
" border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
|
|
" color: var(--sklearn-color-fitted-level-1);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* On hover */\n",
|
|
"div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",
|
|
".sk-estimator-doc-link:hover,\n",
|
|
"div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",
|
|
".sk-estimator-doc-link:hover {\n",
|
|
" /* unfitted */\n",
|
|
" background-color: var(--sklearn-color-unfitted-level-3);\n",
|
|
" color: var(--sklearn-color-background);\n",
|
|
" text-decoration: none;\n",
|
|
"}\n",
|
|
"\n",
|
|
"div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",
|
|
".sk-estimator-doc-link.fitted:hover,\n",
|
|
"div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",
|
|
".sk-estimator-doc-link.fitted:hover {\n",
|
|
" /* fitted */\n",
|
|
" background-color: var(--sklearn-color-fitted-level-3);\n",
|
|
" color: var(--sklearn-color-background);\n",
|
|
" text-decoration: none;\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* Span, style for the box shown on hovering the info icon */\n",
|
|
".sk-estimator-doc-link span {\n",
|
|
" display: none;\n",
|
|
" z-index: 9999;\n",
|
|
" position: relative;\n",
|
|
" font-weight: normal;\n",
|
|
" right: .2ex;\n",
|
|
" padding: .5ex;\n",
|
|
" margin: .5ex;\n",
|
|
" width: min-content;\n",
|
|
" min-width: 20ex;\n",
|
|
" max-width: 50ex;\n",
|
|
" color: var(--sklearn-color-text);\n",
|
|
" box-shadow: 2pt 2pt 4pt #999;\n",
|
|
" /* unfitted */\n",
|
|
" background: var(--sklearn-color-unfitted-level-0);\n",
|
|
" border: .5pt solid var(--sklearn-color-unfitted-level-3);\n",
|
|
"}\n",
|
|
"\n",
|
|
".sk-estimator-doc-link.fitted span {\n",
|
|
" /* fitted */\n",
|
|
" background: var(--sklearn-color-fitted-level-0);\n",
|
|
" border: var(--sklearn-color-fitted-level-3);\n",
|
|
"}\n",
|
|
"\n",
|
|
".sk-estimator-doc-link:hover span {\n",
|
|
" display: block;\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* \"?\"-specific style due to the `<a>` HTML tag */\n",
|
|
"\n",
|
|
"#sk-container-id-4 a.estimator_doc_link {\n",
|
|
" float: right;\n",
|
|
" font-size: 1rem;\n",
|
|
" line-height: 1em;\n",
|
|
" font-family: monospace;\n",
|
|
" background-color: var(--sklearn-color-background);\n",
|
|
" border-radius: 1rem;\n",
|
|
" height: 1rem;\n",
|
|
" width: 1rem;\n",
|
|
" text-decoration: none;\n",
|
|
" /* unfitted */\n",
|
|
" color: var(--sklearn-color-unfitted-level-1);\n",
|
|
" border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-4 a.estimator_doc_link.fitted {\n",
|
|
" /* fitted */\n",
|
|
" border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
|
|
" color: var(--sklearn-color-fitted-level-1);\n",
|
|
"}\n",
|
|
"\n",
|
|
"/* On hover */\n",
|
|
"#sk-container-id-4 a.estimator_doc_link:hover {\n",
|
|
" /* unfitted */\n",
|
|
" background-color: var(--sklearn-color-unfitted-level-3);\n",
|
|
" color: var(--sklearn-color-background);\n",
|
|
" text-decoration: none;\n",
|
|
"}\n",
|
|
"\n",
|
|
"#sk-container-id-4 a.estimator_doc_link.fitted:hover {\n",
|
|
" /* fitted */\n",
|
|
" background-color: var(--sklearn-color-fitted-level-3);\n",
|
|
"}\n",
|
|
"</style><div id=\"sk-container-id-4\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>DecisionTreeRegressor(max_depth=3, random_state=42)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-4\" type=\"checkbox\" checked><label for=\"sk-estimator-id-4\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow\"><div><div>DecisionTreeRegressor</div></div><div><a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.6/modules/generated/sklearn.tree.DecisionTreeRegressor.html\">?<span>Documentation for DecisionTreeRegressor</span></a><span class=\"sk-estimator-doc-link fitted\">i<span>Fitted</span></span></div></label><div class=\"sk-toggleable__content fitted\"><pre>DecisionTreeRegressor(max_depth=3, random_state=42)</pre></div> </div></div></div></div>"
|
|
],
|
|
"text/plain": [
|
|
"DecisionTreeRegressor(max_depth=3, random_state=42)"
|
|
]
|
|
},
|
|
"execution_count": 9,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"tree_reg2 = DecisionTreeRegressor(max_depth=3, random_state=42)\n",
|
|
"tree_reg2.fit(X_quad, y_quad)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "subslide"
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"### Plot models"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 10,
|
|
"metadata": {
|
|
"editable": true,
|
|
"execution": {
|
|
"iopub.execute_input": "2025-02-27T23:21:20.906565Z",
|
|
"iopub.status.busy": "2025-02-27T23:21:20.906396Z",
|
|
"iopub.status.idle": "2025-02-27T23:21:21.488766Z",
|
|
"shell.execute_reply": "2025-02-27T23:21:21.488082Z"
|
|
},
|
|
"slideshow": {
|
|
"slide_type": ""
|
|
},
|
|
"tags": []
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"Text(0.5, 1.0, 'max_depth=3')"
|
|
]
|
|
},
|
|
"execution_count": 10,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
},
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA1kAAAGKCAYAAAD+NIubAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAArfJJREFUeJzs3Xd8FGX+B/DP7ibZQEIKJCGQpRN6k1AVBJEzqFjOgvLTEzQUFc6So3oIeBakiFiwEsECgnq2OxQUhFMERUPvoQSygTRCdkOAtH1+fyy72d1smd3pk+/79coryezsPM/MzjzffWaeomOMMRBCCCGEEEIIEYRe7gwQQgghhBBCiJZQJYsQQgghhBBCBESVLEIIIYQQQggREFWyCCGEEEIIIURAVMkihBBCCCGEEAFRJYsQQgghhBBCBESVLEIIIYQQQggREFWyCCGEEEIIIURAVMkihBBCCCGEEAFRJYsQDWjbti3Gjx8vS9q5ubnQ6XRYsmSJLOkTQghRJopNpCGjShYhhJPvvvsO8+fPlyXtL7/8Evfddx/at2+Pxo0bo3PnzvjHP/6BsrIyWfJDCCFEGeSMTV999RXS09PRsmVLGI1GmEwm3HPPPThw4IAs+SHKQpUsQggn3333HZ577jlZ0p40aRIOHz6MBx98EK+//jpGjRqFN998E4MHD8bly5dlyRMhhBD5yRmb9u/fj/j4eDz55JN466238Nhjj2H37t0YMGAA9u7dK0ueiHKEyZ0BQggJ5IsvvsDw4cPdlqWlpWHcuHFYvXo1JkyYIE/GCCGENFhz586tt2zChAkwmUx4++238c4778iQK6IU9CSLNGjz58+HTqfDsWPH8OCDDyI2NhaJiYl49tlnwRhDXl4e7rjjDsTExCA5ORmvvPKK871VVVWYO3cu0tLSEBsbi6ioKAwdOhRbtmxxS2PevHnQ6/XYvHmz2/JJkyYhIiIiqLtdjDG88MILMJlMaNy4MW644QYcPHjQ67plZWV46qmn0KpVKxiNRnTs2BELFy6EzWZzruPaZv3VV19FmzZt0KhRIwwbNsytucP48eOxfPlyAIBOp3P+eHrvvffQoUMHGI1G9O/fH3/88QfnffPHs4IFAH/9618BAIcPHxYkDUIIUQqKTeqITd4kJSWhcePG1Jyd0JMsQgDgvvvuQ9euXfHyyy9j/fr1eOGFF9C0aVO8++67GDFiBBYuXIjVq1dj2rRp6N+/P66//npYrVasWLECY8eOxcSJE1FeXo6srCykp6dj586d6NOnDwBgzpw5+M9//oOMjAzs378fTZo0wcaNG/H+++/j+eefR+/evTnnc+7cuXjhhRdwyy234JZbbsGuXbtw0003oaqqym29S5cuYdiwYcjPz8fkyZPRunVrbN++HbNnz8a5c+ewbNkyt/U/+ugjlJeXY8qUKbhy5Qpee+01jBgxAvv370fz5s0xefJknD17Fj/++CM+/vhjr3lbs2YNysvLMXnyZOh0OixatAh33XUXTp48ifDwcABAZWUlysvLOe1rQkKC39cLCgo4rUcIIWpFsUkdsamsrAzV1dUoKCjAsmXLYLVaceONN3LaHtEwRkgDNm/ePAaATZo0ybmspqaGmUwmptPp2Msvv+xcfuHCBdaoUSM2btw453qVlZVu27tw4QJr3rw5e+SRR9yW79+/n0VERLAJEyawCxcusJSUFNavXz9WXV3NOa9FRUUsIiKC3XrrrcxmszmXP/PMMwyAM1+MMfb888+zqKgoduzYMbdtzJo1ixkMBnbmzBnGGGOnTp1iAFijRo2Y2Wx2rvf7778zAOzpp592LpsyZQrzVmQ4ttGsWTNWWlrqXP7NN98wAOw///mPc9nKlSsZAE4/gWRkZDCDwVBvHwkhRO0oNqkrNnXu3Nn5enR0NJszZw6rra3leASJVtGTLEIAtz49BoMB/fr1g9lsRkZGhnN5XFwcOnfujJMnTzrXMxgMAACbzYaysjLYbDb069cPu3btctt+jx498Nxzz2H27NnYt28fSkpK8MMPPyAsjPsluGnTJlRVVeHvf/+7W3OIp556Ci+99JLbup9//jmGDh2K+Ph4lJSUOJePHDkSL7/8Mn7++Wc88MADzuV33nknUlJSnP8PGDAAAwcOxHfffYelS5dyyt99992H+Ph45/9Dhw4FAOfxAoD09HT8+OOPHPfYtzVr1iArKwszZsxAamoq7+0RQogSUWxSR2xauXIlrFYrTp48iZUrV+Ly5cuora2FXk+9choyqmQRAqB169Zu/8fGxiIyMrJes4DY2FicP3/e+f+HH36IV155BUeOHEF1dbVzebt27eqlMX36dKxduxY7d+7ESy+9hG7dugWVx9OnTwNAvUpFYmKiWwABgJycHOzbtw+JiYlet1VUVOT2v7eKSqdOnfDZZ59xzp/nMXTk6cKFC85lLVq0QIsWLThv05tffvkFGRkZSE9Px4svvshrW4QQomQUm9QRmwYPHuz8+/7770fXrl0BgOboauCokkUI4LzrF2gZYO/gCwCffPIJxo8fjzvvvBPTp09HUlISDAYDFixYgBMnTtR738mTJ5GTkwPAPuyrmGw2G/7yl79gxowZXl/v1KmT4GkGOl4AcPnyZVgsFk7bS05Orrds7969uP3229GjRw988cUXQd1tJYQQtaHYxJ8UsclVfHw8RowYgdWrV1Mlq4GjbyiEhOiLL75A+/bt8eWXX7o1kZg3b169dW02G8aPH4+YmBhnE4p77rkHd911F+f02rRpA8B+J7B9+/bO5cXFxW535ACgQ4cOuHjxIkaOHMlp244A6+rYsWNo27at839vIzYFa926dXj44Yc5resaAAHgxIkTGDVqFJKSkvDdd98hOjqad34IIURrKDYFj09s8iaYShvRLqpkERIix90xxpizkP/999+xY8eOes0Tli5diu3bt+Pbb7/Frbfeiq1bt+Kxxx7D9ddfz3l0vJEjRyI8PBxvvPEGbrrpJmeanqMxAcCYMWMwf/58bNy4Eenp6W6vlZWVITo62u0p0Ndff438/Hxn2/edO3fi999/x1NPPeVcJyoqyvn+uLg4Tnn2FGq794KCAtx0003Q6/XYuHGjz6YmhBDS0FFsCl6osamoqAhJSUluy3Jzc7F582b069cvpLwQ7aBKFiEhGj16NL788kv89a9/xa233opTp07hnXfeQbdu3XDx4kXneocPH8azzz6L8ePH47bbbgMArFq1Cn369MHjjz/OuW15YmIipk2bhgULFmD06NG45ZZbsHv3bnz//ff1guH06dPx7bffYvTo0Rg/fjzS0tJQUVGB/fv344svvkBubq7bezp27IghQ4bgscceQ2VlJZYtW4ZmzZq5NelIS0sDADzxxBNIT0+HwWDA/fffH9QxC7Xd+6hRo3Dy5EnMmDED27Ztw7Zt25yvNW/eHH/5y1+C3iYhhGgRxSbpYlPPnj1x4403ok+fPoiPj0dOTg6ysrJQXV2Nl19+OejtEY2Rb2BDQuTnGCa3uLjYbfm4ceNYVFRUvfWHDRvGunfvzhhjzGazsZdeeom1adOGGY1Gds0117D//ve/bNy4caxNmzaMMftQuv3792cmk4mVlZW5beu1115jANi6des457e2tpY999xzrEWLFqxRo0Zs+PDh7MCBA6xNmzZuw+Qyxlh5eTmbPXs269ixI4uIiGAJCQns2muvZUuWLGFVVVWMsbohbhcvXsxeeeUV1qpVK2Y0GtnQoUPZ3r173bZXU1PD/v73v7PExESm0+mcQ9m6bsMTADZv3jzO++cL/AynO2zYMN7bJ4QQJaHYpI7YNG/ePNavXz8WHx/PwsLCWMuWLdn999/P9u3bx3vbRP10jHFoXEoI0aTc3Fy0a9cOixcvxrRp0+TODiGEEEKxiWgCDeBPCCGEEEIIIQKiPlmEKEBxcTFqa2t9vh4REYGmTZtKmCNCCCENHcUmQkJHlSxCFKB///7OCR29GTZsGLZu3SpdhgghhDR4FJsICZ2i+mQtX74cixcvRkFBAXr37o033ngDAwYM8Lru+++/j48++ggHDhwAYB9d5qWXXnJbf/z48fjwww/d3peeno4NGzaItxOEhODXX3/F5cuXfb4eHx/vHEGJEEIIkQLFJkJCp5hK1rp16/DQQw/hnXfewcCBA7Fs2TJ8/vnnOHr0aL05CADggQcewHXXXYdrr70WkZGRWLhwIb766iscPHjQOZ/C+PHjUVhYiJUrVzrfZzQaER8fL9l+EUIIIYQQQhoWxVSyBg4ciP79++PNN98EYJ+FvFWrVvj73/+OWbNmBXx/bW0t4uPj8eabb+Khhx4CYK9klZWV4euvvxYz64QQQgghhBDipIg+WVVVVcjOzsbs2bOdy/R6PUaOHIkdO3Zw2salS5dQXV1drwPm1q1bkZSUhPj4eIwYMQIvvPACmjVr5nM7lZWVqKysdP5vs9lQWlqKZs2aOWcxJ4QQIj7GGMrLy9GyZUvo9Q17MFyKTYQQogxcY5MiKlklJSWora1F8+bN3ZY3b94cR44c4bSNmTNnomXLlhg5cqRz2ahRo3DXXXehXbt2OHHiBJ555hncfPPN2LFjBwwGg9ftLFiwAM8991zoO0MIIURQeXl5MJlMcmdDVhSbCCFEWQLFJkU0Fzx79ixSUlKwfft2DB482Ll8xowZ+N///offf//d7/tffvllLFq0CFu3bkWvXr18rnfy5El06NABmzZtwo033uh1Hc+7hRaLBa1bt8bOnXkoLIxBhw7A1S5fDVaXLl1w7tw5tGjRgnMlmBBCPvoIeOIJgDFApwNefx242rrbK6vVilatWqGsrAyxsbHSZVSBhI5NpaWl+P7773HzzTdLNgS32GnKHZvkOKaENCRiXWNixSZFPMlKSEiAwWBAYWGh2/LCwkIkJyf7fe+SJUvw8ssvY9OmTX4rWADQvn17JCQk4Pjx4z4rWUajEUajsd7ygQNjwFgM9HrgvfeAjIwAO6Vhjkejer0eMTExMueGEKIGZjPw5JP2IAbYfz/1FHDnnUCgh1TUHE742HT8+HE8/vjjyM7ORtu2bYXNrExpyh2b5DimhDQkYlxjYsYmRVSyIiIikJaWhs2bN+POO+8EYG9vvnnzZkydOtXn+xYtWoQXX3wRGzduRL9+/QKmYzabcf78ebRo0SLoPDoOvs0GTJ4MpKcHPvhCyM7ORlVVlfMYEULUg67fOjk59vLTVW0tcPy4NGWpVoUam6655hpUV1f7bDofSCjnNt80lU7r+0e0Qc1xSYxrTMzYpIhKFgBkZmZi3Lhx6NevHwYMGIBly5ahoqICDz/8MADgoYceQkpKChYsWAAAWLhwIebOnYs1a9agbdu2KCgoAABER0cjOjoaFy9exHPPPYe7774bycnJOHHiBGbMmIGOHTsiPT2dV16l/GJwxx13ID8/HykpKTCbzeInSAgRDF2/dVJTAb3ePZgZDEDHjvLlSWuCiU06nQ5hYaF/BQjl3OabptJpff+INqg5LolxjYkZmxQzXNN9992HJUuWYO7cuejTpw/27NmDDRs2OAfDOHPmDM6dO+dc/+2330ZVVRXuuecetGjRwvmzZMkSAIDBYMC+fftw++23o1OnTsjIyEBaWhp++eUXr00ugkFfDAghJDgmk705m+MGpMEAvPsuPcUSUjCx6cSJE7j99ttx4sQJcTMlc5pS0vr+ESI3Ma4xMWOTom65TJ061WfzwK1bt7r9n5ub63dbjRo1wsaNGwXKWV0tl74YEEJIaDIy7M3Zjh+3VwaoHOWPYhMhhPAjVmxSVCVLyQ4cAAoL6YsBIYTwYTJRGSqkUGNThw4d8O2334qXMYWkKSWt7x8hchPzGhMjNlEli6OUFKBrV7lzQQghhNQJNTYxxlBbWwuDwSDZ6I1ypCklre8fIXJT2zWmmD5ZDZ3ZDGzZYv9NCCGEiGn37t0IDw/H7t27/a4nZGzimqZaaX3/CJGb2q4xepKlAFlZwKRJ9nb1NA+XstXU1KCmpkbubGheWFgYjdJFiIjatGmDlStXok2bNj7XETo2cUlTzbS+f4TITW3XGH2LkZnZXBfEAOnn4SLcXLp0CSUlJaioqJA7Kw1GVFQUEhIS0LhxY7mzQohyzZ1r/52UZP9dVAQYjUBlZd1vL68169wZ42+7DWjWzOtmxYhNzZo1w/jx40N7swpoff8IkZvarjGqZMmMJuhUvqqqKuTl5SE8PBwtWrSA0WhURVtgtWKMobKyEqWlpcjLy0O7du0QEREhd7YIUabXXgvpbRcAbHr8cYx87TXEP/FEvdfFiE0XLlzApk2bMHLkSMTHx4e2EQXT+v4RIje1XWNUyZIZTdCpfEVFRTAYDGjTpo2gs4wT3xo1aoQmTZrg1KlTKCoqgonuOBAiqFMAxgDIfuopxN91V72akxix6dSpUxgzZgyys7NV8QUpWFrfP0LkprZrjCpZMnNMgjZ5sv0uoedcJ4cPHwZjjJ6cyIQxhkuXLiE+Pp4qWBIzGAyIjY3FhQsXVHsN0PVLlKo3AAuAKMa8Pp4SIzb17t0bFosFUVFRwu2Igmh9/4g2qDkuqe0ao0oWR/n5QHa2/e6e0DfV/U2C1qRJE2ETI0Gprq5GbW0tGjVqJHdWGqRGjRqhpKQE1dXVqmwySNcvUSoDgBjA/rjKx+MpoWOTwWBATExMSPlVA63vH9EGNccltV1jNIQ7R927AyNGAG3a2EdcEprJBAwfTv2wlMZ2ta0MPcWSh+O42zw7hxBCeDkFYCyAUy++6DfwCBmbTp06hbFjx+LUqVP8N6ZAWt8/QuSmtmuMnmRxxJj9N43+1zCp8bG6FtBxJ8S/JXgKsdChCInQA5g9oRgxzSKAqiog4urvxET7ysXFQGkp8OGHqAFQnJyMmrvvliyvNTU1KC4u1uw0GFrfP0LkprZrjCpZIZBy9L+lS5fCarUiJiYGmZmZ4idICBEMXb9EbM/jOVxt+AcASH/A/uTJp337gA8/RCqATbfdZm8DH4JQzu3U1FRs2rQppPTUQOv7R7RBzXFJbdcYVbJCIOXof0uXLkV+fj5SUlJUdzEQ0tDR9evObLYPDS5G31bCMTa5TvLN424wnduEqBNdu/WJFZuoTxZH+qtHynOEJUIIIYFlZdn7tIrZt7UhCjo2Xa1k7QZg/PBD7N69W9T8udq9ezeMRqOkaUpJ6/tHiNzEuMbEjE30JIujAweAwsL6IywRQgjxz2wGJk2qm3OJ+rYKJ+jYdLWSZQKwtG9fSeegM5lMWLp0qWbnvdP6/hEiN6GvMbFjEz3J4iglhUb/IyQYo0ePhk6nw5NPPil3VojMcnLcJ7UF6vq2En6Cjk1XR+xMBDClfXskOgbFkEBiYiKmTJkiaZpS0vr+ESI3oa8xsWMTVbIIIYL78MMPsX79etx0001444038PPPP8udJSKj1NS6Zm0OUvZtJS6uPsmyAFiflweLxSJZ0haLBevXr5c0TSlpff8IkZvQ15jYsYkqWYQQQZ07dw5PP/00JkyYgO+//x4jR47EI488gkuXLsmdNSITkwl47z3nQxTq2yqnq5WsEwBG79iBEydOSJb0iRMnMHr0aEnTlJLW948QuQl9jYkdm6hPFiFEUJMnT4bJZMLrr78OvV6PTz75BH369MGsWbPw+uuvy509IpOMDHs79+PHqW+rrK5WsnoCOHvjjUjo2VOypHv27ImzZ88iISFBsjSlpPX9I0RuYlxjYsYmqmSJgIYpJg3Zt99+6/Z/UlISzp49K1NuiJKYTFQmyslsBk7uDsP1AMIBtAgLA8LDJUs/PDwcLVq0kCw9qWl9/wiRm1jXmFixiZoLCmzJEvehIJcsAbZssQc3QgghRA6O2HTL7fZ2MacB/O3P/fjtt9OS5eH06dOYMGECTp+WLk0paX3/CJGb2q4xepIloMWLgRkz6v632YDp0+1/6/X2dp8ZGcFts2/fvmjVqhWNVkSICtH1S5TANTbVXA37VwDsOl+Oa6+9gvfflyY2XblyBQcPHsSVK1eCS0wltL5/RBvUHJfUdo3pGGNM7kwomdVqRWxsLCwWC2JiYnyuZzYDrVsD/o6mwQDk5qq/uYzJZHLOFm7W+CO6K1eu4NSpU2jXrh0iIyPlzo5iff/997jlllvQrVs3HDx40Os6paWl6NKlC8rLy7F//3505DB8Dx1/dRKqyTTX8rchCjU2GVCDGtibCP6CIbgev1BsIoQ0CFLHJmouKJCcHP8VLIDmhSHadd1118FgMODw4cMoKyvzus60adNQXFyMf/7zn5wqWESdsrLcm0xnZcmdo4bNMzbVwuD8Oww19mUUmwghGidHbKJKlkC8jbXvieaFIUExm1XToS8mJga9e/cGYww7duyo9/rWrVuxcuVKdO3aFTNc29QSTTGbgUmT6iZ3tNmAyZNVcQprVv3YpEMt9NgLYBT+ALBXsti0d+9eNG3aFHv37hU/MRloff8IkVuo15hcsYkqWQLxHGtfrwfuuYfmhSEhUuHjgOuvvx4AsH37drfllZWVmDx5MnQ6Hd59911ERETIkT0igZycuiDmQE9J5OUtNtn0YUgG8DCaQ69Pliw2JScnY/bs2UhOThY/MRloff8IkVuo15hcsYkGvhCQt7H2zWZ+Y+/ffvvtKC4uRmJiYr2hsYlC9OsHFBQIt73aWvft2WzAhAnAnDl135SEkJwM/PmnYJu7/vrrsWzZsnqVrBdffBHHjh3DI488gqFDhwqWnho0tOvX8dTENZjRE3z5ecam8M5haH6pCi+0b4Zp/2suWWxq3rw5pjtGg9Igre8f0QY1x6VQrzG5YhNVsgTmOdY+37H3d+3a5ezISxSqoADIz5cmHQUbOnQodDoddu7cidraWhgMBhw6dAgLFy5EYmIiFi9eLHcWJdfQrl/HU5PJk+33CugJvnK4xaKwMJQDyK6xIC22HECToLcXyrldXl6O7OxspKWloUmT4NNUOq3vH9EGNcelUK8xuWITVbK4mjkTiIkBKiuBpCT7sqIiwGi0L3P8vvqa9UQRSiuMaBpViZgOgdd3e811WXm5/e/aWgl3lgRF6KYhnk+yXNMR+kmWgBISEtC1a1ccOnQI+/btQ58+fTB58mRUVVXhlVdeQdOmTQVNjyiTtyf6RDnMZqA5MyAHwA1nziA7Jwd9+/aVJO2cnBzccMMNyM7OlixNKWl9/wiRG59rTI7YRJUsrt55J6jVY67+CKagwN4vJ9jJTIj4BGxy55SVVf+Wiwo+++uvvx6HDh3C9u3b8ccff2Dbtm248cYb8be//U3urBEJ8X2CT8SRlWXv/H3OFoZuAHbFm9C1WzfJ0u/WrRtycnJg0ujJofX9I0RufK8xqWMTDXyhJjRMV8ORkWGfuGbLFvtvFVSwgLrBL7766ivMnDkTRqMRb7/9tsy5IoS4jq5VgzBEAmh2QYeSEunmn4uMjETHjh01O+ed1vePELmp7RqjSpaa0DBdDYvJBAwfrqpHAo5K1ubNm1FWVoZnnnkGqampMueKKI2KZifQDNfRtWoQhjwA81GK7dvzJMtDXl4ennjiCeTlSZemlLS+f4TITexrTOjYRJUsNaFhuojCpaSkoH379gCAzp07Y9asWTLniMjFV7BS4ewEmuA6X1YN7ANf/IYrSEgolywP5eXl2Lp1K8rLpUtTSlrfP0LkJsQ1JmVsokqWmtAwXUThKisrUVNTAwA0J1YD5itY0WTF8nGdL6sWBnufrKhYjBghbZ+sffv2oZuE/cCkpPX9I0RufK8xqWMTDXzB1eOPA02aAFVVQGKifVlxMRARYV/m+H31NevJYpRWRKBpVBVi2gde3+01x7L33rP3xwHsIw6qpF8OabgWLlyIM2fO4OGHH8awYcPkzg6Rga9glZ7uf0JIun8kPsfoWvHXhQFngEhDjdxZIoQQScgRm6iSxdWCBfYh3DkKNLqg2Wz/UFNT/XyAO3bUVbKEHLqbEBH8/PPPePHFF9GyZUu88sorcmeHyMRfsKLJiuVnMgGICcN+ADeXl+P7/fvRs2dP5+ucYlOI9u/fj5tvvhnff/+9W5paofX9I0RufK4xOWITVbJk4BhG12azf6jvvefjIZVej0wAVgAxkyZJnEtCAjtw4ACWLVuGwsJCbNiwAXq9HmvXrkV8fLzcWVOEzMxMWK1WxARxg0bt/AUrmqxYIcLCkABggl6PhIQE52LOsQmhndsJCQmYMGGCW5paovX9I9qg5rjE5xqTIzbpGGOM3ya0zWq1IjY2FhaLRZAT0my2twP1/JBzc718mHffDXz5pf3v/HygZUve6QvBZDI5Zws3a7wzxZUrV3Dq1Cm0a9dONUOGSmnx4sWYMWMGmjRpgr59+2L+/PkYPny4YNun469MgZ52BJrmzWzmNiGk0OWvlvA6Nv36AdnZ9g/nah/KoGKTQjWk2EQIccflKbzUsYkGvpCYv8eV9ej17isRojDTp08HYwxWqxVbt24VtIJFlInLCEyBpnlT4ewE2hIWhgoAf9TWouLiRQBBxqYQVVRU4I8//kBFRYVwG1UQre8fIXLzdY1xHRlQ6thEzQUlFlS7T9dKlmf0I4QQifnrOOwZlEwmqkQpVlgYjgIYACD78GH07d9fkv5yR48exYABA5CdnY2+ffsKt2GF0Pr+ESI35zU2YQL6tmsHALCeKELRB0YsRiWuwIhIWyVOTEhC2TEgrrIIMBqBykr7AHIATEVFMDmWebyGovrruy1z/ObYekBRlazly5dj8eLFKCgoQO/evfHGG29gwIABXtd9//338dFHH+HAgQMAgLS0NLz00ktu6zPGMG/ePLz//vsoKyvDddddh7ffflvWyVGDavdpMKAcAAOgs1rRROK8EkL4KS8vB2MMOp0OTZqo/wqm0QE1IiwMXQHsBZzxMNg+CaGc2127dsXevXs1O0G51vePaIOa41LXRYvs5daKFc5lMQBme1t5kUSZ8kMxzQXXrVuHzMxMzJs3D7t27ULv3r2Rnp6OoqIir+tv3boVY8eOxZYtW7Bjxw60atUKN910E/Lz853rLFq0CK+//jreeecd/P7774iKikJ6ejquXLki1W55FehxpZNej64AYgF0/ctfJMsfIUQYXbt2RWxsLLp27Sp3VgThOqGtQyhPO3xNBkkkYjCgEYBeABqFhzsXc45NCO3cbtSoEXr16oVGjRqFmnNF0/r+EW1QbVz64w80WrfOXm7JnReOFFPJWrp0KSZOnIiHH34Y3bp1wzvvvIPGjRvjgw8+8Lr+6tWr8fjjj6NPnz7o0qULVqxYAZvNhs2bNwOwP8VatmwZ5syZgzvuuAO9evXCRx99hLNnz+Lrr7+WcM+849Tu0/PbDCGEyMh1QlsgtBGYuLadJyIKC0M+7Hd/88+ccXtJzP5y+fn5mD17ttvNUC3R+v4RIqtffqkrt+TOC0eK+BZfVVWF7OxsjBw50rlMr9dj5MiR2LFjB6dtXLp0CdXV1WjatCkA4NSpUygoKHDbZmxsLAYOHMh5m1LyemeX5sYihChMME87XJnNwGefARMn1u/TRU+0JBYWhjIAnwMoKy31u6qQTx3Lysrw+eefo6ysjP/GFEjr+0eIrIYOrSu35M0JZ4rok1VSUoLa2lo0b97cbXnz5s1x5MgRTtuYOXMmWrZs6axUFRQUOLfhuU3Ha95UVlaisrLS+b/VauWUPh8+5yYJ4kmWmBNIEkIaJl/lSrCDWriWcZ6oTxc3gsamsDB0B3AcsH+4PgQzb5YvrudQ9+7dcVzI4QoVRuv7R4is+vdH95QUHFfRk2JFVLL4evnll7F27Vps3bqV91w6CxYswHPPPSdQzgLzO1qXayXLz3RmQgRCQghxJVS54lnGeRJ6BDutEjQ2hbmE/qvzZHkKZiRJXyg2EUKElBffE62uVrL+hTkY9dfGGNC2GIiIAKqq6n4nJtrfUFzsc5n1fBXeXBEBI6pQBPtrSShGJSIQiSo8PDMR8bEe23D8btIE+Ne/AuZXEZWshIQEGAwGFBYWui0vLCxEcnKy3/cuWbIEL7/8MjZt2oRevXo5lzveV1hYiBYtWrhts0+fPj63N3v2bGRmZjr/t1qtaNWqVTC7ExS/o3VxaC4oRCAkgdGc3fKg4y4PIcsVb2WcQyh9uhoqQWOTwYCDAO4C8OWhQ+juZaJ7viNJejuHJk06iJdeugvffvslunfvHlreFezgwYO466678OWX2tw/QuRkNgMbDliwBMCXAF7BNPzr29iQJ0zP3gL8c0X95Y4bQvH+bghZrZwqWYrokxUREYG0tDTnoBUAnINYDB482Of7Fi1ahOeffx4bNmxAv3793F5r164dkpOT3bZptVrx+++/+92m0WhETEyM24+Y/I7WxaG5oBQTSDZkhqsV3erqaplz0jA5jruB+idKSshyxVcZ99lnwfXpaugEjU1hYYgFcDuAWB8j4fEdSdLbOWSzxaJ//9sRGxsbdJbVIDY2Frffrt39I0ROOTlANPT2cgtALQy8vu96K+P0euC334SLS4qoZAFAZmYm3n//fXz44Yc4fPgwHnvsMVRUVODhhx8GADz00EOYPbtuJPyFCxfi2WefxQcffIC2bduioKAABQUFuHh19nqdToennnoKL7zwAr799lvs378fDz30EFq2bIk777xT0Lzz6Rjsd7QuDpUsoYZUJt6Fh4fDaDTCYrHQUxWJMcZgsVhgNBoR7jLMNBGfkOWKrzLu3nvpCZbYfMamsDCYACwGYHJMuOmB70iS3s8hE5YsWQyTRj94k8mExYu1u3+EyCk1FWiOCHu5BcAGPa/vu97KuPfeA/r3FyrHCmkuCAD33XcfiouLMXfuXBQUFKBPnz7YsGGDc+CKM2fOQO9SYr/99tuoqqrCPffc47adefPmYf78+QCAGTNmoKKiApMmTUJZWRmGDBmCDRs28O635UqINucZGfZmOMeP208WZ/nsevfexxf8YCeQJMFLSEhAfn4+zGYzYmNjER4eDp1OJ3e2NIsxhurqalgsFly8eBEpKSlyZ6nBCbVc8TVQhs8yjojGb2wKC8NlACcBtL940eecM3w+N2/n0BtvXIbFchLNmrXX5FxSly9fxsmTJ9G+vTb3jxA5mUxAVcdqHDwOtAeg0+s5f9+VKzbpGN2e98tqtSI2NhYWi6Ve8wyz2T7Pi2uTCIMBIbcPrefpp2Fatgz5AFISE2H2MTGzIy9SfYExmUzIz89HSkoKzA1k7GWr1YqSkhK30b2IuIxGIxISEkRvsismtV8rwZQrYgxy4K/8beh4xaaJE7FrxQqkAcheuxZ977sv6PS5ntuu51BR0S6kpaUhOzsbffv2DTpNofIkll27xN0/QoQg93XCx66+fZG2ezeyASSdqISpfUTA98gZmxTzJEuN+HYMDkivxzcAqgBELFjgd9Vgh1QmwXH0gaiurkZtba3c2dE8g8GgiSaC33zzDaqqqhARETgQKJG3csXbHcE//rDPf+W4ZUcD8MgrYGwKC0MnANsBdArxA+J6brueQ3FxnbB9+3Z06tQppDSVrlMnbe8f0QY1x6VORqO93AIQ3bqudZuvJ1VyxyaqZPHgaHPuebdQsP5QBgPSHH9Toa0I4eHhmvjyT6SRlpYWeCUF4DrPnrc7goB9mWebiFBuONF8f8IIGJvCwhANYDAAvPkm8O23QGUlYDTafzv6aRUV+VyWZjQCt90GBHGOR0dH+x14Su20vn9EG9QSl4D6McFZbgHOLjW+nlQ5lvONTbziEiN+WSwWBoBZLBavr69YwZjBwBhg/71ihYCJz5xp3zDA2JYtAm6Yn+TkFAaAJSenyJ0VQghPK1Ywptfbixm93ncZlpdXt57jR6+vv8zxYzDY38MnH4HK34aMV2waOZKdBdjzADvr7cML5mfcOM55Pnv2LHv++efZ2bNn+e28D3LHJrH3j5CGxFtMONunT125xbzHJYOBsZ07hYlNvuIj19ikmNEF1Sojw97OfcsWEYYjdh2ayddEMxLLygIKCux/FxTY/yeEqJOv+bC8NdP3PiS396JJrw9uAB5f+bg65yQJgc/YZDYDmzahGMByAMV8E/rwQ3ubHA6Ki4uxfPlyFBfzTrUeJcQmMfePkIbEV0w4W1FlL7euDj7mq2n0tm38Y1Mw8dEXai4oANH6QxkM+C+AywAabd+O0SNGiJAId44TzhX1uyDEt//+97+4fPkyGjVqhNGjR8udnXqC6VfqrQma4z6Q57LffgtuGFxf+Th5kvs2SH1eY1NODgCgF4BzPLbtjE0ARv/6K6cPvFevXjh3jk+q3iklNom1f4QISelxCfAdEzoh0l5uXW0q6Ktp9JAh3uNVMLHJX3zkOq4NPclSMr0ejwIYA+DR116TOzc08TEhQXr00UcxZswYPProo3Jnxatg5sPyNaeIEPOM+MpH+/bBbYdwkJoqyGacsQkArrtOkG2GimITIdwpPS4BvmOCMeLqhX71RV/z+fXvzz82CTFfJFWylMxlnixbrfwj7dPEx4RoS6AJZz0ns/XWBE2IJtO+8kFTpInAZAJWrMBhAH0BHOa7vcaNOX9zOXz4MPr27YvDh3mn6kYpsUms/SOkofEVE05UVaAvgEOsLjb5ikF8YxPfCdkBai4oOn+jkgQasSR7d13UuHDB3sZc0D5fQXKccBMm1C2jiY8JUTdfkzH6GrHJWxM0IZpMe8uH1cpvm8SHjAxcTOqBLnNmwNYtFWiTAFRVARERQFUVLkQk4nwJkKQrRkwz+zIkJgIA9v1UjI6b3gJgnzPwkjGec7JRUVEYPHgwoqKiBN0dpcQmsfaPkIbIW0w4s9A+uqCu2oARI+rHJk98YxPvyYr5jPzREPAZ3crfqF2BRvTKy2PsGd2LLAVgAFgkmgU9WpdY5B7BiRC1SEmxXyspKeq6VnyN2CR1+UOjC/omR2xynBfn0NwZm4AURcQlxig2EcKFWuMSY4xVpXZlDGAWNFFFbKLmgiLxNyoJlxFLcnKAGmZw26ZS2pi7PjolhGgP9XHRrrr4UwkgFzZbJefY5DgvquA+iSnX86KyshK5ubmorKwUbodcyB2bxN4/Qhq68ss1yAVwGTrnMiXHJqpkicTflxQuX2D+/BOwuX08jPo/EUIkoZQ+LkR4dfHnIIB2AA5yjk2pqYBOB1TDfUJ2rufFwYMH0a5dOxw8eJDnXiiT1vePELkdt11BOwCHUDdOgZJjE1WyROLtS4peD0RFBf4CYzYDs2a5V7J0AF5+mfo/EULEJ0SHX6JMdfGnI4AfAXTkHJscPJ9kcdWxY0f8+OOP6KjUb0Q8aX3/CJFbF2MYfgTQ9uqQEkqPTVTJEonnlxTAfodw0CBg40b/X2AcdxNr4d7moV8/iTJPCGnwRJ1oncimLjbFABgJICao2MRY/SdZXJvqxMTEYOTIkYiJiRFsf5RE6/tHiNxiGMNIAK0TwlURm2h0QRFlZAC9etmDl2cb99xc+4+3EUvqJldzby5IN8cIIVLyNzJToNFRiXJlZAApKYW45ZbVYOwBAM2Dik1VNvcnWVxjU2FhIVavXo0HHngAzZs3F2p3FEPr+0eI3Aqrq7EawAM6huHDva+jpNhET7JEdvGi7zbuJhMwfHj9OWkA+91E6PSIBtAEQEJspOwnCyEkONHR0WjSpAmio6PlzkpIPOfJcsjKAtq0AUaMsP/OypInfyR0eXnnwNh8AOecy7jGphqEO2NTcnI059h07tw5zJ8/H+fOnQu8sgppff+INqg5Lp2rrsZ8AOZadcQmqmSJjGsbd88TAwCee8GAIwCsAM68/rLb+r6+/BBClOPIkSOwWq04cuSI3FkJmq9gxWV0VKJ8N9/cB3q9FUAf5zKusemagRHO2HTu9D639f3Fpj59+sBqtaJPnz71X9QAre8f0QY1x6U+YWGwAkgqNaoiNlElS2RcOpD7OjHcame1tc4/lVZTJ4Roi79gRcO7awOf2MTCXfpkVVU5/6TYRAgRU22NvTByDAyn9NhElSwJBOpA7uvEKCpx+XiurqDEmjohRFv8BSsa3l0bjh49ipUrh2Dz5qNBx6aKapc+WdXVALjFpqNHj2LIkCE4evSo8DukAFrfP0LkduhKFYYAyEHdgwclxyaqZEnEs427K18nRmKyx9CEUGZNnRCiLf6CFQ3vrg1GoxEdO3ZEmzbGoGNT45j6T7K4xCZHmkajUZidUBit7x8hcmsMGzoCCHMZfVvJsYkqWQrg68RomqDHdAATAEz/7DMAyqup63Q6t5/w8HAkJCSgZ8+eGD9+PP7973+jpqZGnswFoW3bttDpdIFXFMnly5cxd+5cdOrUCZGRkWjZsiUeeeQR5Ofny5Ynwt/06dMxYcIETJ8+Xe6sBCVQsKLh3dWvbdu2WLVqFdq2betzHV/nQaPYiLrYNG8eAG6xiUuaaqb1/SPaoNa4BAAdDDqsAtAS9hsZSo9NOsYYC7xaw2W1WhEbGwuLxSL63Bdms8ewuR99BNO4ccgH0Dw6Dn8evgCTyd7OffJk+11Cxwkm5YlkMpmQn5+PlJQUZyVg3LhxAACbzQaLxYJjx47h6NGjYIyhY8eOWL16NQYMGCBdJj3odDq0adMGubm5Xl9v27YtTp8+DTkuhytXruCGG27Ab7/9hhYtWmDo0KHIzc3Fzp07kZiYiN9++w3t27eXPF+EP9drxazCNr31yiSJSVn+qg3fY1NdXY2ysjLExcUhPDzc77r1zoP/+z+YPv0U+QBaJCbjbJF9NL1AsSmYNEMh9/Um9v4RIgS5rxM+qmNjUWa1Iqp9J+zMOqr42ERPshSkXpNCl5mML16s60istJo6AKxatQqrVq3CRx99hG+++QaHDx9GTk4OxowZg+PHj+OGG27Anj175M6mIr3wwgv47bffMHjwYBw7dgzr1q3D77//jldeeQXFxcV45JFH5M4iaaD8NXMm6rZ//34kJSVh//79Adf1PA+Ona7rk1VczJwDXASKTcGkqUZa3z9C5La/pgZJAI7YqlQRm6iSpWDnL7hPRuzakVgNX346dOiAdevWISMjA5cuXaLKghdVVVV48803AQDLly93m7ciMzMTvXr1wv/+9z9kZ2fLlUVCiAa1b98e33zzTdBPyc1m4Oftrk9pmNsAF/5iU6hpqoXW948QubUH8A2A9o0ayZ0VTqiSpWAFRXUfj6O3kBoHuXjllVcQFRWF3bt3Y9u2bfVez8vLw9SpU9GhQwdERkaiadOmGD16NLZv315v3a1bt0Kn02H8+PE4d+4cxo8fj+bNm6NRo0bo27cvPvroI7f1V61a5exrdfr0abf+Y8N9TBe+YsUK9OrVC40aNUJycjImT56MsrIy3sfBm19//RUWiwUdOnTANddcU+/1e+65BwDwn//8R5T0CSENU1xcHG6//XbExcUF9b6cHKASdU+ydOAel0JNUy20vn+EyC3OZsPtAOJU0hyXKlkKlpxiqLdM7uEoQxEbG4ubb74ZALBlyxa313bs2IHevXtj+fLlCA8Px6233ooePXpg48aNuP7667Fu3Tqv2ywtLcWgQYOwYcMGDB8+HEOHDsX+/fsxbtw4zJ8/37lex44dnf3FoqKiMG7cOOfPqFGj6m13xowZmDJlClq0aIGbb74ZjDG89957uP3220Xpr7V3714AQN++fb2+7li+b98+r68TQkgoiouL8e6776K4uDio96WmAjVwf5LFNS6FmqZaaH3/CJFbcW0t3gVQ7DmUqUJRJUvBmiW6NxdUwnCUoerTpw8A4PDhw85lVqsVd999N6xWKz755BMcOXIE//73v/Hzzz9jx44diImJwYQJE7wGrP/85z/o3LkzTpw4gXXr1uGHH37A9u3bER0djeeffx67du0CAAwZMgSrVq0CACQkJDj7jq1atQqzZs2qt92PP/4Y+/btw8aNG/Hll1/i4MGD6NixI3755Zd6FUTHU7Jgfjyfnp05cwaAvSOqN47lp0+fDnyQCSGEo7y8PEyZMgV5eXlBvc9kAm5Id3+SxTUuhZqmWmh9/wiRW57NhikA8lQwajUAhMmdgYbGMSt1aiqHoOQyHm50FHD4iDorWIC9ggMAFy5ccC774IMPcO7cOfzjH//AAw884LZ+v3798OyzzyIzMxOffPIJnn76abfX9Xo93njjDURFRTmX9e/fH1OmTMHChQvx1ltvYcWKFUHn8/nnn0fnzp3d8v3oo49i2rRp+PnnnzFixAjna65Pybjq0qWL2/8XL14EADRu3Njr+o79Ky8vDyodQgjxp2/fvm7TawQTm3qlhQMb7X83a8o4D77kmabWaH3/CJFbX8ZQAwAu/deVjCpZEsrKAiZNsk/YqNfb5x/xG5xcRhc0GISvYAVV4ePJ0dTOdS6qH374AQBw1113eX3P0KFDAQA7d+6s91qfPn3cKkMOY8eOxcKFC/HLL7+ElM+bbrqp3rJOnToBAM6dO+e2fMiQIRgyZEhI6RBCiFIEHZsi6p5k6XXCN6OWMjYRQlTE0UzQc1I+hVJHLjXAbK4LYgDcRgr0KcBJZDbbh8oNZZqDrCz7kPAjRtQNDS+mkpISAEDTpk2dyxxzVl133XVem9b179/f7b2u2rRp4zUdxySQZ8+eDSmf3prtNWnSBABQWVkZ0jb9cYwmeOnSJa+vV1RUuOWBEEKEkJOTg/T0dPzyS07wsclPp3N/ccmRZk5Ojs/3Sx2bhMRl/wghIWIMOQDSAeRUVcmdG07oSZZEcnLqgpiDY0Qmn3fq9HrcCqAUQNPUVLeXgr7z6MJXhS89Xby7hrt37wYAdOvWzbnMdjUD99xzj1uzP0+eTezEpA/i7si2bduCbpLYpUsXt75grVu3BgCfEwI6lvuqVBJlu/XWW1FaWup2c4EQJTAYDIiJicGZM4bgY1NERF1schkVNVBccqRpMNQf1AmQJzYJKdD+EaIEqo1LtbUwAIgBVHONUSVLIqmp9qDjGswCjshkMOBdx9+33upczDcQhVTh48FisWDjRnsD/htuuMG53GQy4ejRo5g1axbS0tKC2qavgSAcy1u2bBlibrk7fvw4Pvzww6DeM2zYMLdKVu/evQHAOVCHJ8fyXr16hZhLIqd333038EqEyKB9+/b4/PPPYTaHEJvCw+ti09X5D7nEJUeavkgdm4QWaP8IUQLVxiWbDe0BfA4APvqxKw01F5SIyWS/q+eofHMaKdD1qUptrfNPf4GIC0eFz5WYQ8P/4x//QEVFBfr374/Bgwc7l//lL38BAHz11VdBb3PPnj1em2SsXbsWAOr1lQoPDxe8Q/L48ePBGAvqZ+vWrW7buO666xAbG4sTJ05gz5499dL44osvAAC33XaboHknhDRstbW1qKioQIsWtcHHJpc+WbjabIdLXHKkWesSz1xJHZuEFmj/1KBt27Y+55AkRFY2G2oBVACodenfr2RUyZJQRgaQm2tvr56by6F5n2u0cYlefANRSBW+EJw8eRL33XcfsrKyEBUVhSyPxvWTJ09GUlISFi1ahPfee8/ZfNChpqYGGzduxIEDB+pt22az4e9//7tbX6bs7Gy8+eab0Ol0eOyxx9zWb9myJQoLC0WbVDhUERERmDp1KgBgypQpzj5YALB06VLs27cPw4YNC/pJHyGE+LN3715ER0dj7969wccm1z5Z1dUAuMUl1zS9kSo2icXX/m3dutWtv7HBYEB8fDx69OiBcePGYcOGDaLMw+jLsmXLnFObyGXXrl2YNm0a+vbti/j4eMTHx6N///546623UH31nCLEjc2GvQCiAez10Y9daai5oMRMpiAChmubU5cKiCMQTZ5sv1MYSiDKyLA34zh+3B4E+Qax8ePHX82mDVarFceOHcORI0fAGENqairWrFmDnj17ur0nLi4O33zzDW677TZMnjwZL7zwAnr06IH4+HgUFBRg165dKCsrw1dffYUePXq4vXf06NHYu3cvOnTogOuvvx4WiwU//fQTqqurMWfOHPTr189t/dtvvx1vvPEG+vbti2uvvRaRkZHo3Lkzpk+fzm/HBTBnzhxs2rQJ27dvR2pqKoYOHYrTp0/j999/R2JiIj744AO5s0hIQDQinLq0bdsWa9ascQ4WFFRscn2S9e23wIEDMBmN2HNDJdZvNiIClShBEu68DTAtKQKMRqCyEm2jo7HmiSfQ1s/AGULHJil5HlNPY8eOxS233ALGGMrLy3H06FF8/fXX+OijjzBy5Eh8/vnniIuLEz2fy5YtQ9u2bZ1xWw6LFi3Cpk2bcOedd2LixImora3Ff//7X0yZMgXffPMNNmzY4DYaMSGw2dAWwBoAbf304/cka2xixC+LxcIAMIvFIn3iP//M0gCWArC05s3rvZyXx9iWLfbfeXmM/fST/bfYUlJSGADnb9efsLAw1rRpU9ajRw82btw49uWXX7Kamhq/2zt37hybMWMG6969O2vcuDFr3Lgx69ChA7vjjjvYqlWrWHl5uXPdLVu2MABs3LhxLD8/nz344IMsMTGRGY1G1rt3b7Zy5UqvaVy8eJFNnTqVtWrVioWFhTEAbNiwYc7X27Rpw3xdDq5piuXSpUvs2WefZR06dGAREREsOTmZjR8/nuVJ8YES0aSlpbGUlBSWlpYmd1a8EqrcWLGCMb2eMcD+e8UKYfIna/mrcLIem4kT62ITYP/gg/nR6YQ7SVy4xiYlccSQxYsX13utpqaGZWZmMgBs1KhRkuSnTZs2bvGP62tC2rZtG7t8+XK95Q888AADwP7zn/+InoeGSulxiTEfsamsrK4MuekmTtuROzZRJSsAWQPZtm0s5WrlJSU62udqYp1EvsgZyKSo8BAiFKV+6WPMe7kRSqUrL69uO44fg0GYGz5UyfKN77E5f/48+/jjj9n58+eDe2NeHmNXK1i4+ptr5eo8wD6++luwk8SF3Nebr2Pqr5LlMGTIEAaA/fLLL85lZWVlbMaMGc6bbwkJCez+++9nJ06ccHvvypUrGQD2448/snnz5rHWrVuziIgI1rNnT/bpp5+6ret5Y9Txc+rUKcZYXSXr8OHD7JZbbmHR0dEsJiaG3X333ezcuXM8j1Bg3377LQPAFixYIHpaDZXc10kgnrFp8WJ7XDLvL60rQ0aMCLgdJcQmai6oZByGqFT7kLeEEOl5KzcmTaoLRcFMC6H2EeEaqtzcXPztb39DdnZ2cEM585gDKhfA3wBkA2iqwZMk5GMKICMjA9u2bcP69esxZMgQWCwWXHvttThz5gweeeQRdO/eHefOncNbb72FgQMH4s8//6w3tcfMmTNRUVGBxx9/HACwcuVKjB07FleuXHE2Dfz444/x9NNPIyEhAf/85z+d701MTHT+nZ+fj+HDh+Ovf/0rFi9ejL179+Ldd9+F1WrFDz/84FyvuroaFouF8z4mJCQEXMcxbUnz5s05b5doh7fY5OjVkaCzYSOuliGVlQh0hSkhNlElS8lcexH76BSrhJOIEKIu3soN1/+DuVkT0vQURHZ9+vTBlStXEO6nf5RXqamATuczJvlNE8AVAOGAJk+SkI8p6qbpOHbsGABg7ty5OHnyJH777TfnVB+Avf9zz549MW/evHqDV5SUlGDfvn2IjY0FADz66KPo1asXMjMzcd9996FRo0Z48MEHMWfOHDRv3hwPPvig17wcP34c69atw5gxY5zL9Ho93nrrLRw9ehSdO3cGAPz6669u07IEwgKcMxcvXsTixYsRGxuLO+64g/N2iXZ4i01OzFZXhnDou6iE2ESVLCXjMDGuEk4iQogPjqGcKyqAl14CiuoGAXD+Tkqyr+P6mrdlAq7fx5iEWQASUYQrMCISlc7fRbCvn1RbhJrpRqC5/+2bkpKw43bg16+LcBlGNEIlrrstCaaPBMi/1SraR9PQ6fV6GI3G4N9oMgHvvw9MmBB8mgCMgL2SpqZhAzkK+ZgCiImJAQBYrVYwxrB69Wpcf/31SElJQUlJiXO9qKgoDBo0yO2JksNjjz3mrGABQGxsLB599FE888wz2Lp1K26++WZOeWnZsqVbBQsARowYgbfeegs5OTnOSlbv3r3x448/Br2v3tTW1uLBBx/EqVOnsGbNGvVNlEsE4e07rYMetroyJCxw9UWIQeL4okqWknFoLmgyAQsXAjNn2k9KtQ15G6zhw4dLOtQtISHLygIKCux/l5UBLk1z5BYPYAGXFddy296Aqz9OX1/9IYp18uRJTJs2DUuWLEH79u2De3NGBjBnjv38jomx/x8RYZ8z6+rvCxGJ2PErcGRbMa4gAiOxCQnIxjQAS15/He25tEVVGT7H1Hr1hkJMTAyKi4tx/vx5/PDDD27N+FzpvdyE7dq1a71l3bp1c+aNK295b9asGQDg/PnzzmXx8fEYOXIk5+36YrPZ8Mgjj+Cbb77Biy++iLFjx/LeJlEnz4qRKz1sOAnYy5DLl8HlCpN7tFJFVbKWL1+OxYsXo6CgAL1798Ybb7yBAQMGeF334MGDmDt3LrKzs3H69Gm8+uqreOqpp9zWmT9/Pp577jm3ZZ07d8aRI0fE2gVhcXiSlZVVV8HS6YAFC7j1oyCEiMhsBiZOlDsXhPhks9lQWVlZb35Czhw3AZs0AZYudXspK6uuX4VOB/zjH0CX6rkoei0blQBs0dH8Mq9QfI7pvn37ANi/ozhuJI4cORIzZ84UNI9cGPzc4HW9yVlVVYXS0lLO201OTq63zGazYcKECfjoo48wb948PPPMM8FllmiOa8Xojz+A2bPtFa5wXS1sDPYyJIjh/YOankJgiqlkrVu3DpmZmXjnnXcwcOBALFu2DOnp6Th69CiSHE1JXFy6dAnt27fHvffei6efftrndrt3745NmzY5/w/j8IhRMQL0yfLsIMiY/WQcO1a7T7IIUYWcnJD6rBAilY4dO2L9+vWCb9dbXHr1VWDOU+HoCGA9UNc8VGP4HNOsrCwAwK233orExETExcXBarUG9aTo8OHD9foyHTp0CID70ymh5p/avn07rz5ZjgrWypUrMWfOHMyfP1+QfBF18Dd/laNiNHy4/UbNzJkAbLa6MuRq81qlU0yNY+nSpZg4cSIefvhhAMA777yD9evX44MPPsCsWbPqrd+/f3/0798fALy+7hAWFub17okqBGguSINeEKJQPAYHIETNfMWl89ZwOHsLVVVJnS3Fqq2txcyZM7Ft2zbccsstuO666wAADzzwAJYvX44vvvgC99xzT733FRUV1bsB/fbbb7v1y7JYLHjnnXcQFxeHYcOGOdeLjo4O6gmUL3z6ZDHGMHHiRKxcuRLPPPMMnn/+ed75Ierh+rTb32i2ZnNday096gqWS1f0aCxhfkOliEpWVVUVsrOzMXv2bOcyvV6PkSNHYseOHby2nZOTg5YtWyIyMhKDBw/GggUL0Lp1a75ZlkaA5oI06AUhCmUyAe+8Y29YTogC7dq1C2lpacjOzkbfvn0F266vuNQ0OQK7AKQByD56FMKlqByBjumuXbvwySefAADKy8tx9OhRfP311zh9+jRuuukmrFmzxrnuiy++iF9//RVjxozBmDFjMGjQIEREROD06dP47rvvkJaWVm90wYSEBAwcONB5s3rlypU4c+YMVqxYgcaN676SDho0CFlZWXj22WfRtWtX6PV63HbbbYiKigpqf/n0yZo+fTo++OAD9O7dG127dnUeF4cOHTpg8ODBIW2bKFswUw+53rTRw+YsQ34oKMNfpMx0iBRRySopKUFtbW29eRGaN2/Oq//UwIEDsWrVKnTu3Bnnzp3Dc889h6FDh+LAgQNo0qSJ1/dUVlaisrLS+b9VztGt9HosAnAJQGMvBba3kVP8tJwMib/HuYQQP8aPx6LJk+3Xb1IS8NRTQHFxvQEC4OjY7vqat2VBrp97LgL/XluFIthfS0IxKhGBB++pQuu+7utbz1fhzRURMMJ9/WuHRWBwX2HyE9L65eXAW28J/9mokNCxqXXr1nj//fdDvum4aNEiXLp0ye3LO+A9Lr38MlB0OhytAbwPoLXLCHihUmJsCnRMP/30U3z66afQ6/WIjo6GyWTCsGHDMHbsWIwaNcpt3djYWPz666945ZVX8Nlnn+Gbb75BWFgYTCYThgwZggleRndcuHAhfvnlFyxfvhyFhYXo1KkTVq9ejf/7v/9zW+/FF19EaWkpli9fjrKyMjDGcOrUqaArWXz8+eefAIC9e/fib3/7W73Xx40bR5Uskfi6dqUSTCss15s2eticZUijSu/f4RWH/7zH/OXn5zMAbPv27W7Lp0+fzgYMGBDw/W3atGGvvvpqwPUuXLjAYmJi2IoVK3yuM2/ePK+zoQea1VkUx4/XTVP9wAM+V8vLY2zaNMZ0uroZsv3sImees247tqn02cIJUYTLl+uu3+uvlzz5YGe7X7zYfd1A60vBYrHIV/4qjKJiEwd5eYxt2cLYokX283AS3qk7sVau5LVtik3uVq5cyQCwLVu2yJ0VQgIKNTal4qjzDR/qHlJFbAo8fJ0EEhISYDAYUFhY6La8sLBQ0P5UcXFx6NSpE44fP+5zndmzZ8NisTh/8vLyBEs/aK7NBT3HsvSwdGld9w/Ho9erE6eHxNfjXD7bJKRBce2PxWGkUKE5nig4unYGmt4hLa3+MsfdRSI/oWPThQsX8OWXX+LChQsC5dCdyWRvuj5rlj1+VCMcFwB8CeDUqdDTVHJsEvuYEqIFocYmA2qdZYiVVasiNimikhUREYG0tDRs3rzZucxms2Hz5s2CPi6+ePEiTpw4gRYtWvhcx2g0IiYmxu1HNq5fzPwMCevv0WuoxNgmIQ2K6wUk0GhewcrIAHJzgS1b7L/9Te/gaJbhivp4KofQsenUqVO4++67cerUKYFyWJ9rHKlCBE4BuBvAnmOF/t7GeZsOSolNUhxTQrQglNikh81ZhhSjQhWxSRF9sgAgMzMT48aNQ79+/TBgwAAsW7YMFRUVzg6cDz30EFJSUrBggX0KzaqqKufQpFVVVcjPz8eePXsQHR2NjleP/LRp03DbbbehTZs2OHv2LObNmweDwaCeie4MBhwFUAMgzGJBZx+riTEABg2qQQhPjNVdv5cu+bx+xcZ1jpCNG+s/fNPyxOYNXa9evXD+/PmQK2tHjx5FTU0NwsLC0Lmz97PbNY5UIxy9AJwHYOuYEHK+lRyb+B5TQqTA5dqVQrCxSQ+bsww5NyRBFbFJEU+yAOC+++7DkiVLMHfuXPTp0wd79uzBhg0bnINhnDlzBufOnXOuf/bsWVxzzTW45pprcO7cOSxZsgTXXHONW2dQs9mMsWPHonPnzhgzZgyaNWuG3377zecM6oqj1+NGAD0A3Pi///lcLdhHr1yIsU1CGhTG6q7fPXtkzox/jiZYniPOp6fLkx8ivrCwMDRt2jTkuSNvvPFG9OjRAzfeeKPPdVzjSDXCEQagKYCEJiFOgAxlxya+xzRU48ePB2MMw4cPlzRdok5crl2lcI1NeticZUibtuFyZ40TxTzJAoCpU6di6tSpXl/bunWr2/9t27atN7Gdp7Vr1wqVNXkEmCfLlesM2R07ChNwxNgmIQ2Gnya+SuOtCZbNRnPuadmpU6fw7LPP4vnnn0e7du1ES8cRRy58HIFTzwDPAni+sBB8UlRqbJLqmBLSUHgO4X4K9jJkyrmLUMPYk4qqZBEPQXaW5/roVe5tEtIgqGgiYiU3wSLiqK6uhtlsRnV1tehpmUyAqW84jgEwA6h2GYqe1zYVFpukPKaENASuscmAWlTDXoZEqmQEd8U0FyReyDAiGSFEICqqZCm5CRYRR6dOnbB161Z06tRJmgQjItAJwFYAnTTaZ0nyY0qIxrnGJj1szjLkmlZNZc4ZN/QkS8lcmwuq6AsbIQSqai4IKLcJFtGIcJc+FFVV8uWDEKIqjthU9K0NmHJ1oUoeQlAlS8lcT6KaGuCpp4CkJPv/RUWA0QhUVnpf5vJamQWwHi9CTHMj4iICrx9w+xaL/f8Ac3cR0qC53hiRaQh3b8xmezv31NT6FSklNsEi4tizZw+uu+46/Prrr+jTp4/4CUZEYA+A6wD8evYs+sD/uahGkh9TQjQkYGzqY6srQ86fRx/psxg0qmQp2ccf1/1dWwu89lpIm4m7+iO4ggIgK8v/BAeENFQKfJKVlVU3kateb2+GQZdvw9SyZUssWLAALVu2lCbB8HC0BLAAQMuICE2ei5IfU0I0glN5YLPVlSFN1NEpSx3P2xoisxn4+9/lzkVgkybZ80oIcaewJr6OoXAddT+bDZg8mS7fhiopKQlPPPEEkhwtFcQWHo4kAE8AiKoM0+S5KPkxJUQDOMcmm81ZhiRRJYvwkpOjuC9pXjnGeSaEuFPY9ettmPbaWrp8Gyqr1YqNGzfCarVKk2BEBKwANgI4W3hRk+ei5MeUEA3gHJtqa51liFUlI3hSJUupUlMV1Y/DJ72exnkmxBuRmguazcCWLcHf9XcMheuKhmlvuI4fP45Ro0bhuFQ1m/BwHAcwCkBR7QVNnouSH1NCFET02GSzOcuQ446xARSOKllKZTIB77+PPwDkAfhD7vz48t572uixTIjQGKu7fm+4QZBNZmUBbdoAI0bYf2dl+V/fNejRMO3EVY8ePZCXl4cePXqE9P4//vgDeXl5+OMPjtEpIgI9YL8e+jdtrMlzke8xJUQKQV+7HEgSm2w2ZxnSIzlZsLyLiQa+ULKMDLRITwd27LA/Ny0uBhIT7a8VF9ubX5yvQjFLRLMEIK7KvgxVVXW/ExPtowueKEZMYgTiIqvqbcNz/XqvuS7LzQW++sr+f3S0+nsqEyIWmw0tHH83bsx7c77araene/9y6qsjsa9h2rU20hvxLyIiAiYeH3SLFi38vl7vfAoPRwQAEwDU1mpyygC+x5QQKQS6doMlVWzqUWhDIq6WIa5TQigYVbKUzmQC7r3X60tZWcCkxYFHZ4qDgKMLrl9fV8lSyTwFhMjCtU+WANeKv3brnoEsUNDzXF+LI70R/86cOYMXX3wR//znP9G6dWtBt+31fLorHGcAvAjgnxYLWkN7UwaIeUwJUSqpYtMtOhvextUyxGqFGq4w+pasUkKMFBZS+1k19BMjRAkEnicrmD5VXDsSm83AZ58BEydqb6Q34t+lS5ewa9cuXLp0SdDt+opN+cURuARgF4BLlZUBtxFK3w65iXVMCVEyqWITmK2uDFHJPK30JEvh3nvvPVy8eBHR0dGYNGmSc3kwdw68CfnOteuVpLDR0whRFJsN7wG4CCD65ElMCrR+AI5265Mn2691f/1YHEHPtYzwDHquZYCnYMoSok5dunTh1Scj6Nh0OhzDcLV/cUSEz+2q+akq32NKiBR8Xbuhkio2GVCLLrhahjRvzjvfkmDEL4vFwgAwi8UiS/opKSkMAEtJSXFbnpfHmF7PmL2mY/8xGOzLA+HzXrZxI0sB7Hlq0iS0nSKkIThxou5aadRIsM3m5TG2ZUvg63XFCvt17bi+V6xw34ZnGRBSeSAyuctfJZP72AQdm87Y6hYMHOh1m7xik588EULqiHWdiB2b7sBXdf8sWCBo3oPFtfyl5oIqxWekMF7z5VBzQUK4EWkId5MJGD488LWekWEfp2bLFvtv16cB3soAB62M9Eb827dvHxITE7Fv3z5Bt+szNrXSYZ/BgEQA+3wMv6z2udzEOqaEqIHYsSlMb8M+wF6GFBUJkmexUXNBFQt1dCYuj2t9osEuCOGGZ58sIUb78zWwgK8y4NNPgcGDqYLVECQlJSEzMxNJSUmCb9tXbEqKiEDm5ctI8tHUnFdsUgAxjykhSiFXbBpRakP1o0AmgKQmTUJLWGJUyVK5YEZncr0w/LWf9XsB0ZMsQrjh0WdR7H4pvtrQ+xjIlGhQcnIyZs+eLdr2vcWm5IgIzL582T4dyMyZQGUlYDTCWlKJYiShWQKw54YirN9sRAQqUYIk3HkbYFpS5FyvtMKIplGViOlwtSJTZH8NlZVAebl9mUyd4sU+poTITdbY9Jm95jUbAOLihEtURFTJaiC8XRi5ufXvNAa8gGjgC0K4CbG5YLBzjoRKi/MUEe4uXryIPXv2oE+fPoiOjpYmzcuXsQdAn9JSRC9a5Fwec/UHsE830tP1TV/D63p+FRTYg5nEI2bIcUwJkYrssam2FhcBexlSXQ01XGHU9kslamtDH9LW14UBuLef5TQsPDUXJISbEG5COIatlapfCtc29ER7jh07hqFDh+LYsWO8tsM5Nv3xB45VVWEoAH4pciTDPARCHVNClEYRsclmwzHAXoaUlAifqAjoG7NKFBQAI0YAbdrYb9AFg2tnYk7rUXNBQrgJ8klWVpb9+v7HP+q/pqZ+KUQdunXrhiNHjqBbt268tsM5Nv3yC7oBOAKAX4ocyTBihlDHlBAlUUxsstnqypCUFIkS5YeaCyqcZ9PyUB7Pcu1M7Gu9oiL7XQyTCfQkixCugniS5fkU2RWN9kfEEBkZic6dO4f8/qBj09ChiAQQeopBkuHOBN9jSoikLl8GXnrJvV+j4/fVwVusJ4pQ9IERi1GJItiXJaEIV2BEI1Ri+EgjTEvq1nfblrdl/l4LtH52dl0ZopIJv6mSpXA1NfWXBTtRKNeJ4kwmYOFCe39kR58smw247z6X/lld6UkWIZwEUcnyNWztq68C99wTWgVLiBGgiHaZzWYsXboUmZmZMIVwggQdm/r3h/mee7D0iy+QCUD0U1KGOxN8jykhkqiosP8uLQX++U+/q8bg6kATvmy8+iMRM4ClADKfeQampCTFz1ROlSyF69SpE0pKYgHUzW4dyg06Lp3cs7LqKlg6nfuXPsddytu+pCdZhHBis6ETgFgAzWNj/a7q6ylyqBUssUeAIupntVqxceNGTJgwIaT3hxKbrM89h43Z2Zhw881AkyZAVRUQEQHr+SoUIxHNmgFxVcVARIT9tcREAMC+n4rx300RMKIKlbD/LoL9tSQUowoRyOy2ETi0x55Qs2aynPB8jykhojOb0amsDO5XrnpYYa/TTQDsQU7oUTcERpUshfv115+QlRX4KRQX/oZ792yu5O0mfG0tkJdPowsSwglj+Mnx9+23+12V69NmLqQaAYqoW7du3XDw4MGQ3x9KbOrWrRsOnjxZb7m/UQPNZuCaOUCgHo4P9qipq2TJ1Kyd7zElRHQ5OXVxSYW6AXBeYTZbcM26ZECVLBWQYqhlf7NsOxgMgKkVNRckhBMvkxH7a8In1HXubwAbBcciokJKik1xzQx1C+gGICHepabKnQPh6PWKHxGK2n6pBN+hls1m/8PsOporudLr65Y57lI2b0GnDCGcuH4z1OudIzT5G4lNiCHVvV3LNDoh8XTgwAG0bdsWBw4c4LWdYM5ZzzQDxSWAe2xqEit/bBLqmBIiGpMJ+Pvf5c5FyA4AaHv1N957T/F3DulJVgPApX+Gr+ZK9e5S7qInWYRw4nI3/WKFTrImfEI2PSTa1bRpUzz44INo2rSpLGly7TfIOTbNMdR/s8TkOKaEBO0vfwHeeAMAcHnoTXj7l+64gro+j5GowsMzExEfC6C4fh9Jt2X+XvOybPfhCGzZUIVCJEIPYOzIYvRK4779pqdO4cG8PDR9/nmgXz/pjlmIdIzRc3V/rFYrYmNjYbFYEBPDaa55QT3wwAMoKSlBQkICVq9eHfT7zWb7XXPPDvW5ud6/dJnNAZp+7NkD0zXXIB9ASlQUzBcvBp0nQhqEnTvxwMCBKAHQ2NQJX5uP1ltlyxb7UwAxBLyWVUDu8lfJ5D42fGJTsHHJ8R6/5/O8eTD961/22JSQAHNxcVB5IqSheGDYMJT8/DMSAMwcuwC9P51Vbx2KTf5xLX/pSZbC/e9//0N+fj5SQpx4Ldj+Gf4GxwBAkxETwhVj+B+AfABJRfn1Xha7CV/Aa5k0aJcuXcKRI0fQpUsXNG7cOOj3hxKbHGkWFnaBzeaeZqB+gwHPZ738gzLxPaaESOF/Bw7Yb0YAMK2t38xWybFJbdeY/I2YiaiE6J/h1m5eAYGMEFVwuT4qq+rfnFiwgCpBRD5HjhxBWloajhw5InmawBFB+g26xSaD/M0F5TimhPBRy9wvRL1e2c3L1XaNUSVL4xzt2R3xJ9j+GZ6d9f/9FZ0yhHASYEi0/v0lygchXnTp0gXZ2dno0qWL5GkOG9aFV1wC6semP3bLX8mS45gSwofNoxqwdq2y51RU2zVGzQUbgFCH2fU2387856i5ICGc+HnSy/Wuvb8h3wnho3Hjxujbt69safIZ/t1bbPryK/lbWchxTAnhg6HuO53BAAweHPg9csYltV1jQT+WMJlMeOutt9yWbd++HY0bN8bp06cFyxgRVihDQ3vrz1VtoydZhHDicvEYjcHftecy5DshoTp79izmzJmDs2fPypZmqFMWeI1NTP4nWXIcU0J40dm/06klLqntGgv6G/PAgQPxxx9/OP9njOGpp57C008/jTZt2giaOcIfl3lIfPE+Pwk9ySKEE5e76eHh9pHTtmyx/w7UHMPbnfrJk0O7jgnxprS0FJ988glKS0tlSVPo2MR08ley5DimhATNJTbNfU6vqriktmss6ErWoEGD3CpZH3/8MfLy8jB79mxBM0b443vHwVt/rueepydZhHDi0WQpmLv2/kYFJUQIPXr0QG5uLnr06CF5mr//3kPw2HTvffLHJjmOKSF8xDfVqyouqe0aC6mSdfjwYVy8eBEVFRV45pln8MILLyA6OlqM/JEQCXXHISPD/Q68WyCj0QUJ8c01GgU59YEQo4ISokRixaZB17k8yaLYRAg3QcQmikvBC7qSlZaWBr1ej127dmHhwoVITEzEww8/LEbeCA9C3nFwuwNP82QRwg2PL3p8RwUlJJBDhw6he/fuOHTokKRpDhnSHTabe5qCxCYFDOEuxzElhBfPWpMfSohLarvGgh5dsHHjxujZsyf+/e9/4/3338d3330HfRAfEgnOxIkTYbFYEBsbG9T7HHccXCtagtxxoM+aEG5sNkwEYAEQ26dP0G/nM/oaIYHExMQgPT0dMTExIb0/lNgUExODv/wlHVlZMW73ILQSm/geU0KkMPH662H55hvEAkFfN3LHJbVdYyEN4T5o0CC88cYbuOOOOzB8+HCBs0RczZs3L6T3Oe44TJ5sv0so2B0HepJFCDeMwXn1DhsW0iZMJu/XLA3tTvgymUxYunRpyO8PJTaZTCa8//5SDBokQmwyyN9ckO8xJUQK826/HfjmG/s/Idyc8BWXAPFjk9qusZBu/fTu3Rvh4eFYvHix0PkhAvJssy7IBHMKuFtIiCq4ftEL8eaEtxHY5B5Cl2jDlStXcPToUVy5ckXyNB944IomY5Mcx5SQoPHoL+wgV2xS2zUWUqm0du1aTJ06FR2pt5vihToPiU8KCGSEqIJLIDt1Wh90x35vAYvPoAF8hswm2nPo0CF06dJF8j5ZjjQFj00K6ZMl9TElJGgusenwUXXFJrVdY5y/MdtsNhQWFuKll15CTk5OyM3Y/Fm+fDnatm2LyMhIDBw4EDt37vS57sGDB3H33Xejbdu20Ol0WLZsGe9tEt8cF8HZcy53PWgEJ0J8c7k+Vn2kC+rOnq+AtX17aAPa0NMv4qlTp0745Zdf0KlTJ9Wm6fblTAGVLDmOKSFBcwkiLy/Wqyo2qe0a41zJ+vnnn9GiRQt88skn+Pe//y14p7N169YhMzMT8+bNw65du9C7d2+kp6ejqKjI6/qXLl1C+/bt8fLLLyM5OVmQbSqRyWSCTqeDScaOF64XQb8B9CSLEC5KimwwAdABeBVLg7qz52t00NLS4IfQVcIEkkR5oqOjMWTIkJCnXwklNvFN05Xnl7MtP8s/vYiQ+0eIWFrOnA0dABMAG/Sqik1qu8Y4f2MePnw4bDYbDh06hIEDBwqekaVLl2LixIl4+OGH0a1bN7zzzjto3LgxPvjgA6/r9+/fH4sXL8b9998Po9EoyDZJfZ4XQS2jgS8I4eJsfv0velyHqvY2H4leDzz+eP0RQwMNGqCECSSJ8hQUFGDBggUoKChQXZrevpy9t0L+J1lyHFNCgmWrrfubwf6dTi2xSW3XmCIeS1RVVSE7OxsjR450LtPr9Rg5ciR27Ngh6TYrKythtVrdfhoyz4vApoxThhDFa9mifiWL61DV3uYjYcz9Br1eD+zYEXjQAJpAUhuEjk1FRUVYunSppC07hErT25ezaiZ/JUuOY0pIsPSGukDi+E6nltiktmtMEd+YS0pKUFtbi+bNm7stb968eci11VC3uWDBAsTGxjp/WrVqFVL6WuF5ETjuehBC/Eto6v4tMNihql1HB12zpn4LKJsNqKgIvB0lTCBJ+BM6NvXq1QvFxcXo1auXQDmULk1vX87cA5U8zQXlOKaEBEvv8jXOBr2qYpParjFFVLKUZPbs2bBYLM6fvLw8ubMkK8+LQKeAQEaIKrhcH1FRoQ1V7RiB7dpr+T2NEmU6ByIpik11vH05m/yY/E+yCFGbZ+fqKTaJSBGVrISEBBgMBhQWFrotLyws9DmohVjbNBqNiImJcftp6DIy7I9+ly4FNv6oiFOGEOVzqWQZDPyeHAnxNErwIbOJpISOTUeOHEH//v1x5MgRgXIobZqucWnHDuAvo+SvZMlxTAnho3sPnapik9quMUV8Y46IiEBaWho2b97sXGaz2bB582YMHjxYMdtsqLKygEGDgMxM4MaR1FyQEE4EmPDRldbv+BFpNW7cGH379kXjxo1VmaZrXBo0SBk3AOU4poTwIsDcp1LGJrVdY2FyZ8AhMzMT48aNQ79+/TBgwAAsW7YMFRUVePjhhwEADz30EFJSUrBgwQIA9oEtHJORVVVVIT8/H3v27EF0dLRzkuRA2ySBeY7iVMN8X5Bms71Dcmoq3S0nJJjmtK7XDuD7OjKZ6NoiwmjdujXeffddVabpbXTB1950eZLlcu1JGZfkOKaEBM1zlAo/HNdPdDRw8aLv60iq2KS2a0wxlaz77rsPxcXFmDt3LgoKCtCnTx9s2LDBOXDFmTNnoHc5Gc6ePYtrrrnG+f+SJUuwZMkSDBs2DFu3buW0TRKY5yhOvga+yMqqC3p6vf3xMd1pJw2a5/BnPrheO44HXozRdUTEVVVVhaKiIiQlJSEiIkJVaXobXbDKVr+5oNRxSY5jSggvflpZuF4/DnLHJbVdY/I/X3cxdepUnD59GpWVlfj999/d5uPaunUrVq1a5fy/bdu2YIzV+3FUsLhskwTmOYqTtyHcabJTQrxwuVvIfNS3PK8d16Fw6ToiYjpw4ABatWqFAwcOqC5Nb6ML6jwWyBGX5DimhPBRUuq9GuB5/TjIHZfUdo0pqpJF6vvkk0+wYcMGfPLJJ7Kkz2V0QZrslJD6tvzE8AmADQAGlz+IrKz663i7dlzRdUTE0rFjR2zYsMHZvD5YocQmvmk6eOts/9Q/3JsLyhGXhNo/QsT0bM97sAHAJwAemaAPOjbJGZfUdo3pGKNxuP2xWq2IjY2FxWLR7EiDXNqsm832i6pjq0oM6hiJfAApEREwV1bCbAbatKk/23duLvUfIQ2T2QzMar0Gn7AHAABP4DW8ZXii3jXh7dpx1dCvo4ZQ/oZK68cmqLjUETCd+gWm66+3x6boaPx2uJziEiEezGbgvdbP419sLgDgFqzHD4ZbgopNdB1xL3/pSVYDl5Vlv5BGjLD/9nZHA3AZYrN1/VOGJjslxF1ODuB6/4pB5/Xun+e1o9fXNZGn64iIqaioCK+//jqKiooUl2bQccmEuovI5TWp45Icx5SQYOTkwL0pO8fY5CB3XFLbNUaVrAYspDbrPjpJ0vDShNRJTQXCdHW3AG3Q+5yg0fXaOX0aOHOGriMivrNnz2L27Nk4e/asotIMuS+Vof7oglLHJTmOKSHBSE0FDAg+Nu3cqYy4pLZrTDGjCxLvtm7disrKShiNRgwfPlzQbftrs+7zLoWf4T5peGlC7Ewm4JGHGbZ+AFQCOIscv3f/PK8duo6I2Pr06YOKioqQ3x9KbOKSZkhxCfAZm6SMS3yPKSFiM5mAloNOYuNvgBH2fvbBxCa5qe0ao0qWwj344IPIz89HSkoKzAIP5+IYocmzzbrf/oQCTKpKSEMw7HoG0wdAPoAWMR/iq4xlcmeJEMGIFZtCikuOlQghAT138Gt730UAu1frkXi/3DnSLmou2ICF1GbdtZJFY6YQAsDelGnLFo8mTS7fEgPM90iI5I4dO4bhw4fj2LFjikoz5L5UCqhkyXFMCfHHa2xykZikrhvnarvGKPQ3cHzarDNGc/gQ4rOTPt2EIAoWHh4Ok8mE8PBwxaUZUlxyuZNRW8tkiU1yHFNCfOE0gIzK7gCq7RpT19ElonAboSkINTX+R37yxttdlUB3WghRKr+d9P1NgEWIzNq1a4dPPvkE7dq1U2SaQccllydZlVf4x6ZQ4pIcx5QQb/zGJtcbgCqrZKntGlPX0SWy8ww4wcz+7e2uCteheoVCFToiJL8TnroGMurLSBSmpqYGpaWlqKmp0USaBcXuzQX5xKbx40OLS3z2j2ITERLnybhVVsmSo9ziQ11Hl8guJwfwbATFZfZvX3dVJk4MYajeEEldoSPa5+ik78rZST/I5oL0JYtIad++fWjWrBn27duniTRzz9T/OhNqbPrww9DiUqj7R7GJCM1vbHLF4QagkmKTHOUWH1TJIkFJTa2/jMvIT77uqnh+D+USFEMR8twrpEELFFz8dtIPorkgfckiUmvXrh3+/e9/S95cUKw027R3fZJlDyyhxiZPXONSKPtHsYmEgldschXgSZbSYpMc5RYfVMkiQTGZAB0cdz5YwJGfHAVBdLT3uyqeN1E4DdUbAs6Pzgm5imtw8dlJn+OTLPqSReQQHx+Pu+66C/Hx8ZpIs4WprpKlQ+BRCf3FJk9c41Io+0exiQSLd2xy5efkV2JskqPc4oMqWSR4VytGYQb/Iz+5FgSDBgF/+1v9uyrvvx/CUL0h4PzonBAEH1y8dtLnWMmiL1lEDiUlJVixYgVKSkq0kaZLAW+M4Bebxo0LLS6Fsn8Um0gwBI9NfipZSoxNcpRbfFAli4RMp/N/l9CzIPjkE2DHDve7KnyGkA9GyHOvkAZJkODCsbkgfckicjhz5gwmTpyIM2fOaCNNl9EF9XrGKzatWhVaXApl/yg2kWAIXvHx0ydLibFJjnKLjzC5M0D8M6u0zZCvgqCiwn5XxZXJJE1AycgA0tPthVHHjhTEiG+O4OJ6DgcdXBiD8+p94w2fqzm+ZE2ebL9G6EsWkULfvn3BeMzlFkps4pumXxwnI+Yam0KJS6HuH8UmwpUQsck8bhzw1lv2f/w8yVJibBK1DBEBVbKIKAT5kioCqSp0RN0ECS6uJ3+ATh/0JYsQnjgORU2xiagZxSZ1oeaCRBTUBIKoHe+mrEHOkxXqpOCEhOL48eO49dZbcVzCDhaipsnxSZaYsUmOY0oaHt6xKYhKFqCs2KS2a4yeZJHg6XT2L5ABHtn6ugNiNtubbKSmKuOiJcQXXneXOXYuJkQOer0eRqMRegnPTVHTdN2mTLFJjmNKGibBYhOHG4BKorZrjCpZCvfcc8/BYrEgNjYW8+bNkzs7QfMsCLKy6jod6/X2O4piDXZBiKxsNjwHwAIg9osvMG/MGLlzRIhT+/bt8eWXX4b8/lBiE980/eL4JMtBjNgk6v4RIpDn/vzTHpcAzFNJZcVBbdeYjqmpB5kMrFYrYmNjYbFYEBMTI3n6JpMJ+fn5SElJUcwgGCaDAfk2G1IMBphraji/z2y2D5vr2RY+N5eeaBENWrIEpunTkQ8gpWlTmM+flztHqiN3+atkfI+NzWZDdXU1wsPDQ7orHEps4pumX5cuwRQVZb/ejEaYr1zh/FahYpOo+0eIQExRUci/dAkpAMyHDgFdu8qdJc6Uco1xLX+pFCCSUeKcC4SIhuMQ7oTIYc+ePYiMjMSePXu0kWYQzQU9CRWb5DimhPCisuaCarvGqJJFJCPEnAtms72zp0Ie6hHiGzUSIArWtm1bfPzxx2jbtq020gyyuaAroWJTXl5bvPaatMeUEF5U9sRVjnKLD3UdXaIsQX6JDHZUJ88KVVaWvUnHiBH231lZPPLegFFFVSIq7lxMtK9p06Z48MEH0bRpU22kyaOSJVRsuvPOpnj66Qfx1VfSHVMtodgkA5VVsuQot/hQ19ElysDjCyPXoUc9K1SLF9d1SgbsvydPpsI4WFRR9U3wAE/NBYmClZaW4tNPP0Vpaak20nSNSyFOCMw/NpXCZvsUkyaVUmwKEsUm3wSPTSoe+VaOcosPdR1dogmB5lwwm+tXqGbNov5cfHk7rlRRtRMlwFNzQaJgubm5+L//+z/k5uZqI00Bnhbzj025AP4PNlsuxaYgUGzyTfTKp8paWchRbvFBlSyiON46ITuG1XUVbJv5ho4GHvFOtABPT7KIgvXu3RsXL15E7969NZ2mkALHpt4ALkKv702xKQgUm7yTpPKpsidZaitD1HV0SYPgqxPywoXc28yT+oTo3K1FogV4epJFFMxgMCAqKgoGHn2Z1JCmkALHJgMMhii8956BYlMQKDZ5J0nlU2WVLLWVIeo6ukT1uLQt9tUJedo0bm3miXfBdu5uKKKjhQvwbud3iJUs6vxNpHDy5Ence++9OHnypHbSdDR9CvLa43rNBYpNq1efxKhR9+KGG6Q7plpAsck70WKTK46VLKXEJTnKLT7C5M4A8W/YsGEoKSlBQkKC3FmpE2Igy8qqe/St19sLVV8VpYwMID3dfsemY8e6wtZkooKXD1/HtaFyPScdQg3wnuf3H7faMAxACYCEXr1C2oa/a4QQPmpra2G1WlFbWxvS+0OJTXzTFEOw15y/2NS/fy0+/FBZ+6cWFJvciRmbbotthu6XzUgAOPXJUlJcUmIZ4o+OMWrT4g/XWZ0bElN4OPJrapCi08HMsd+J2WzvtOlZYOTmUmFK5OHtnNTrgd9+A/r3r79uTo69WYu389Xbtp7XzcUc9rz9n40bgZtuCjo/Df0aofLXNzo29Zn0euQzhpSwMJirqwOuT9ccUSKusSlQXPK1ra/wV9yJr+3/nDsHJCcHlRe6RriXv9RckEiCOrYSpfHVib2iwn0Zl9GdvG0LrG7B9xsDF7V0jRAiLbrmiBJxiU1cRx30ti0d6has/tR/bKJrhB9qLkgk4ejY6nk3pKF3bCXy4XJO+hrdKT3d/S6et23pUNdI4NVlOvR82v+dP7pGiJR27dqFAQMGYOfOnejbt6820tTp7M3Ya2uBmTOBykrAaLT/Tkqyr1NU5FzWx5iEWQASUYQrMCISlaiCEX0/rgS211/f2zZcX9tVXo4BixZh53//i7433yz8/pEGIVAs4BqXfG1L71LJypymx7B7fccmpcUlOcotPqiSRSTh6Ng6ebI9/kndsZXLY3WiLGJ/ZlzOSX938VzX89wW4F7Jqrbp670nlPwQIpRWrVph+fLlaNWqlXbSdFysjAGLFgVcPR7AAm8vfBBa8q0ALAfQ6pZbgBUrOHVcodikPnLHJq5xydu2APdKVo1N5zc2KS0uyVFu8UF9sgKQu937iBEjUFhYiObNm+Onn36SPH1vQumT5WA2S9+xVUmdNgk3Un5m/s7JYNujm83Ajh3AffcBL7FZ+AELUQjgLK7B/rxdnM55Oa4RpZK7/FUyuY+N4mLTH3/ANGAA8gGkAJB9cE4OHVcoNqmPEmJTKP2kXGNTT9YMNShFcwD79KXYczo+YKyhuOSOc/nLiF8Wi4UBYBaLRZb0U1JSGACWkpIiS/repISH2/OkgtMnL48xvZ4x+61N+4/BYF9OlElpn9mKFfb0HflYsYLbexbpZrAUgAFgzaITxM+oBsld/ioZ32Nz4cIF9s0337ALFy6E9P5QYhPfNP165RXn9ZbiWnhI+HMBYN9c/c0AxrZs8ZldpZVzJDAlfWahxCXH+xJgdF4nH75eJm5GBSZqGRIEruUvDXxBNI06baqP0j6zjIzg52fLyAAenVzXSCAiIvAwuYRI6eTJk7jjjjsknydLtDSHDhV+m0E6CeCOq78DdVxRWjlHAlPSZxZKXHK8LyK87v+HxqurGiBHucUH9ckimqa0TpskMCV+ZqHMz5Z7sm4Hzpfam5lQUyCiFD179kRRURHi4uK0kWb//kDjxsClS8Jvm6OeAIoAxOl0ATuuKLGcI/4p7TMLJS5lZQFVLrMbfPiRDuOmCJsvMclRbvFBlSyiaUrrtEkC08JnZjYDP/7g3t3V1+hPhMghPDwciYmJ2kozPt5eyWrSBHj8caCqCoiIsP92pFtc7H9ZqOt/8QXC8/KQCNgfLbRu7TerWijnGhq1f2aOUQmbuiyb+oQeN96hnn2Qo9zigypZRPNoJnn1UftnlpPjPhcJ4Hv0J0LkkJubi/nz52P+/Plo27atttKMiQFeflm87Xuzaxdy8/IwH8D8y5fRlsNb1F7ONURq/szqmjsGN/KtkshRbvGhqMaYy5cvR9u2bREZGYmBAwdi586dftf//PPP0aVLF0RGRqJnz5747rvv3F4fP348dDqd28+oUaPE3AXCgdlsb0dslnD4J5MJGD5cPQUJEf4zk/K8S00F9C6BjIGaAhFlqaysxPHjx1FZWanpNIPBq4wIC0MlgOMAKj1nNPeDYpP6qDU2OZo7utLp9aqKS0ovQzwpppK1bt06ZGZmYt68edi1axd69+6N9PR0FBUVeV1/+/btGDt2LDIyMrB7927ceeeduPPOO3HgwAG39UaNGoVz5845fz799FMpdkfbdHWd+M15/mcA8Cw8uM5SToiQgUfq885kAkbeWHdt6KCuZiVE+zp37oxt27ahc+fOmkuzsjJwuSF4bAoPR2cA2wB0bt8+hFwTtVBrbHI0d3Qdhmn5WzpVxSU5yi1eJBrtMKABAwawKVOmOP+vra1lLVu2ZAsWLPC6/pgxY9itt97qtmzgwIFs8uTJzv/HjRvH7rjjDl75knsIYSUO4d7CEOEc/tOgq/U5dOiKFXXDner1jC1aJOzwp3l5jP30Ew15q0We586KFaF/3rINuzt1qnNI6RbNkkROTJvkLn+VTO5jo8TYFB9vzxOQ4iw3vBElNt12m/PN+XuKKDZplBZiU0p43Xc4VlMjbmIapaoh3KuqqpCdnY2RI0c6l+n1eowcORI7duzw+p4dO3a4rQ8A6enp9dbfunUrkpKS0LlzZzz22GM4f/688DvQgJjNQE2tywJmw+TJ9e/oODpYOkbhsdmAWbOEG/6Unohpl7dzZ+LE0D9v2YbddZnnXW8QOS1CgrRnzx7ExMRgz549mkjTbAYuXKj732aDtLEpLAx7AMQA6NNnN8UmDdJMbHJ9lOXZflDh5Ci3+FDEwBclJSWora1F8+bN3ZY3b94cR44c8fqegoICr+sXFBQ4/x81ahTuuusutGvXDidOnMAzzzyDm2++GTt27IDB4P1bT2VlpVtbT6vVGupuCWLu3Lm4ePEioqOjZc2HQ06O+/86MNR46dDvrfBwzJDOd/hTbwUdjdymHd7OHcd9PiD4z1u2YXdtNswFcBFA9MSJIidGtE7o2NSiRQvMnz8fLVq0COn9ocQmvmn64xmbAO+DzYgWm8LD0QLAfACvoJlzuxSbtEMrsWluSgounjqFaMCt+4caiFmGiEERlSyx3H///c6/e/bsiV69eqFDhw7YunUrbrzxRq/vWbBgAZ577jmpshjQpEmT5M6Cm9RU13FpAD1sXguF6Gj7tetyMx8Gg33Ap1mz+A1/6u/uj5IDmdlsz3tqqrLzKTdvgcdTMJ+3bMPuMgbn1Xv33SInRrRO6NjUvHlzZGZmhvz+UGIT3zT9SU2tv0zS2BQejuYAMgEsR5xzMcUm7dBKbJqUkACcOqW6ChYgbhkiBkU8J0xISIDBYEBhYaHb8sLCQiQnJ3t9T3JyclDrA0D79u2RkJCA436exc6ePRsWi8X5k5eXF8SeaJ/JBIS5VM3D9KxeoZCVBQwaVD+IvfsuMG1aaLOUu/I2Qk6guz9yjGjoipo3cucIPI6HzXp9/VgQ7N2+jAz+513QXC8AlTXJIMojdGyyWq3YtGmTpK01xEzTZLJPk+Xg7QurqLEpLAxWAJsA6FHXbtFfWSV3XAIoNgVDc7FJhXFJjnKLF4n6iAU0YMAANnXqVOf/tbW1LCUlxe/AF6NHj3ZbNnjwYLeBLzzl5eUxnU7HvvnmG875krtzsRKlGI3OTpPmYxVur3nryKnTMfaf/3jfVqgdRlessHcQdXQU9dXB2bGuZ0dVKck28ALHvCm1g3ZeHmOffcbYunWMLV7M/fMOJR1RjsHEiXUf+N69Am+8YaDy1ze+xyY7O5sBYNnZ2QLnTL40HYNxJCSk1LuefcWmnTvrbyekMmHCBJZ9daCbjrp1AcsqueMSYxSbQqX62NS3rz3D4eECb1h8cpRb3nAtfxVTyVq7di0zGo1s1apV7NChQ2zSpEksLi6OFRQUMMYY+9vf/sZmzZrlXP/XX39lYWFhbMmSJezw4cNs3rx5LDw8nO3fv58xxlh5eTmbNm0a27FjBzt16hTbtGkT69u3L0tNTWVXrlzhnC+5g/zZs2dZXl4eO3v2rCzpe+NayWLl5W6v/fSTe4Ht+PEWRPgGmbw8xrZs8V8AKSGI+DomW7ZIlwdvlBDk/fE2Aligz5tvGoIegwkT2FmA5QHs7ObNAm644ZC7/FUyvsfmypUr7NSpU0HFQ1ehxCa+aQbib8RDX+XwtGnu64VcJjz6KLsCsFMAO/3Nb37LKiXEJcYoNoVK7bHpbPfu9rikwkqW2GUIV6qrZDHG2BtvvMFat27NIiIi2IABA9hvv/3mfG3YsGFs3Lhxbut/9tlnrFOnTiwiIoJ1796drV+/3vnapUuX2E033cQSExNZeHg4a9OmDZs4caKz0saV3EFeicPkulWyrFa317wFD29BRKogo4QgopSAqvQ8ueKTP653/0Q/Bo884hzCPaV5c4E22rDIXf4qmdzHRpGxyU+e8vLsT65Ei0t//3vdm37/3e+qSohLjCkzDigxT65CzV8wT6XEPgYp4eF13+FISFQ1hLvD1KlTcfr0aVRWVuL333/HwIEDna9t3boVq1atclv/3nvvxdGjR1FZWYkDBw7glltucb7WqFEjbNy4EUVFRaiqqkJubi7ee++9eiMSEp48eoA62ix7a+rrOjSpVEOXhtJ/S2ie7bglG3jBD9mGjuUo1PwF079A9GPg2vGDEIU5c+YMpkyZgjNnzmg6TQeTCfjHP+ovFywuhYfjDIApAM4E6GSlhLgEUGwKRSj5C7bfm2SxSYUDX8hZhoRCUZUsokJevkhmZAC//eY/iEgVZJQSRGTp3OqHUoK8L6EObuJtaH9f33dEPwZUySIKVlFRgR07dqCiokLTabp68kkR41JYGCoA7ABQcfGi31WVEpcAik3BCjZ/wcalUNJoSOQuQ4JFlSzCj4+xTPv39x9EpAwy6enAmjXAZ59xDyJijPpkMgHDhytjiFwlBXlvQslfsHf/RD8G/sb5JURmXbt2xa5du9C1a1dNp+kq0DXPq0wID0dXALsAdG3ZMuDqGRnAjh3A0qX233LFJYBiUzCCzV8oT6WUfgzkJHcZEixNz5NFROL6iNnPF8mMDHsF5/hx+x0YzwIi0OtCyMqqu4uk19sLrkDBzPM9CxcCaWnam0NEiuPPl+vdv0BCmdhR1GNAT7IIURzR4lJ4eN3fNTUBVw82NnlbPz1dm/NbaSk2hTrhsBqOAeFAoj5iqkWdi+tLiYys6zRZVCTYdoUerjSUzqP+Bu7Q6ZQ3ypFWeeukzqXjbzBD+4vB7Rx+4AEa+IInuctfJeN7bPbu3cuSk5PZ3hCnFwglNvFNU4w8BcI5Lr3wAtsLsGSA7X3ttYDbDCY2eVvf9X+KTdLZuTP42CR3XGLM/TxOCQuzXyc6nfQZ4UnsMoQrVQ58QVRIoCZRQk+IaDbbmwcG+5je26N9B8aAiRPlnTiyoXjttfoPgrh0/JWzf4HnOXz8OD3JIsqVmJiIKVOmIDExUdNp8hFUXAoPRyLsA18kRkV5XcXR3G/79uBik7e45Po/xSZpeJvMGggcm+Tu9+Z5HjObege+UFsZQs0FCT8CNIny1TE0PT20R+SuzSo8BXpM7+3RvivG7O3n773XPf9abLIhF7PZ3k/Bk17PreOvySTs58Dl8/V2Dv+x0+XaUGEwI9rWokULzJkzR/NphirouBQejhYA5gBAkyb1XnaNSzqd/cc1fPqLTYHiEkCxSWye54MrLs3/hI5LjjyFEptqhc2GpNRUhgA08AXhS4AnWVw6hnLt8OuvINTrgZdf9l/QeXY4DUToJ3DE99PEzEzpvyhw/Xy95VnHaOALolwXL17Ejh07cDHASHhqTzNUvuLSa6+5L3PEpgsXw3AR9tEFL5aX11vHNS45KleOOBMoNnnGJW9TpHii2CQsX3FJr5dnUApesUn87IlGTWUIQJUsEgrXu/ICPMmKjva+3NHigu/cR4A9yzYbMHNm4GDjeLT/2Wf1H0Do9cDgwfa/QxmalQTma/jaJ5+UNh/BfL7e8qzXUXNBolzHjh3Dtddei2PHjmk6zVB5u6YB+1N2RxngGpv+OS8cxwBcC+BYXp7be7zFJcaACRO4xybXJme//UaxSWpey3i9/bOQuvkf39ikZmoqQwCqZCne5s2bceDAAWzevFnurHgnwJMsXzckKiqEmfsIqKsLcg02JpO92cX777sPo/ree3V3rJQ+aaJaKWX42mA+X2957pfGsBnAAQCbP/1U7OwSEpSuXbviwIEDIQ+FHEps4pumlEwm+9NzTzabvQzwjE1VLAxdYb/euyYlub3H1xf0998PLjY5hlrv359ik9S8lfHvvWf/LKQWTJ8+b/leH5toj0sJCaLnVWhqKkMAqmQpXufOndG9e3d07txZ7qx4J0Aly9/Ee8EEC0f75IUL/TerCCbY+OuwShMG8uOvCajcHYUBIDu7/jJ/n69nntu3saEzgO4AOtNJQRSmUaNG6N69Oxo1ahTS+0OJTXzTlNqTT9Z/YuQrNlUjHI1gv94beemDmZnp/kU3M5NfRYhikziUHpeysoCxY+svDyY29Y022OOS67QDKqG2MoQqWYQfAZoL+ntywTVYuDbbmDkTWLCgrlkF32Dja6JGpTxxUSMuTUDlnCDTbLafR54WLAjcp8+ZZ9drQ0vtNYgmmM1mTJ8+HWYJ25DJkSYfJlP9J0a+YlM1wmEGMB2AubjYudxR1i1ZYq9UTZtm/6L75JMUm5RGDXHJW59zLv3C3PLt2IAK45LayhD1HWGiLAIN4e7rDhGXYOGtSeHs2fZg1b+/uMFGCXe21Ebo/gJcB0UJhq++fUE1DXHdAI0uSBTGYrHg22+/hcVi0XSafHGNTTZdGCwAvgVgudoG3tuAF6++6v39FJvkJUY/NqFjk6+4tHZtkJ+v4wagCitZaitDaAh3hVuzZg0uXbqExo0b4//+7//kzk59AjzJcvA1xGmgmc/9NSk0mcSfOV2MoVm1LNDnFQzXYZH1evuXFiG+THgbMjno5jaMYQ2ASwAaf/UV/u/xx/lnjBCBdO/eHUePHg35/aHEJr5pyoVLbOqRE46EScBRAGjWDADFJjURMi4B4sQmX3HJMeAJV2suXbLHpUuXoMBvlX6prQxRXzW2gZkxYwYmTpyIGTNmyJ2VOq535QV6khWIyVTXDt7zrhCXJoViPeIX4ymK1gnVX0DMEbQEucvMGGYAmAhgxgsv8M8UIQqiyNgkA0dsOlvs0r+luhoAxSY1EbIfm1ixSainnzMuXrTHpdJSfhkiAdGTLMLPW28BiYlAURFgNAKVlYBjZCXXZf5e47D+/p+K8N/NRhhRiR+QhL/eCQxoY1/fVFmJP/9ixNaNlShEEvQAxg4vgumNwNu/YEzC+RIgCUWISQgu/7uP1KX5A4D/u7EIPdOE2V8tr8/n83J9rabAiMW2ShTBviwJRbhSa4TtyUogjf8+ZRiNuHdcJYp1SWjWDIjbXwTMDCKP+/bVXSe1ap7+kWjRwYMHcccdd+Cbb75B9+7dNZum2BxPLP5iC8MrAO4A8M3Zs+iOui/Fkyfbi4BgvhTzmUhYrCf8Wsbns/Ik9FMxV4I8/RSwBZLU1FaGUCWLBK+ysu7vZcskSbLn1R+nr91fv+bqj9Pmqz8BxF/9CUWoaRJhjl1bAF5GWAa+vPojgJirP7wVFdm/+dA3HaIQcXFxuPfeexEXF6fpNMXk+sSiGuGIA3AvAMPluq9WoXwp5lNJ8vUUJT2dmg4GIlTzTUGam/shWDNQFfYVVlsZQs0FSXD++IPuyhMSCpoNlChISkoKFixYgJSUFE2nKSbXJxbVCEcKgAUAGl+OdFsvmCaBfJuaiTFHVkNqeihE802+zfq4HO+G9Jm4UlsZQpUsEpxffpE7B4SoE80GShTk8uXL2LdvHy5fvqzpNMXk2o+nBmG4DGAfgMhGV0LeJt9KktBzZHEZ1lyNxK6khDq6I5fjrdXPhAu1lSFUySLBGTpU7hwQok40GyhRkMOHD6N37944fPiwptMUk+sTi2qE4zCA3gCycwtD/vLOt5Ik5NDwYg4uJKesLKB1a3slpXXr4CspXCtowT4V43K8tfqZcKW2MoQqWSQ4/fsDjRvLnQtC1IdmAyUK0rlzZ+zcuROdO3fWdJpiczyxePu9MHQGsBPAsa3RzicMwT4xEaKSJNQcWWI0PZSb2QxMnFg39gNj9v//+IPb+4V8iuR5bnA53lr8TIKhtjKEBr4gwYuPBy5dApo0sc/6CwDFxUBEBFBVZR9t0HOZv9eCWN96vgovrUgEA5CEYlQiAkZUOX8XIxF3/RUY0Jbb9i9EJKL0PJCIYsQ0Ez//vvbpzRX2/BfB/lpzFOORxyIQFyl9foRY33qyuN4+OT6vSFThhlERuKar921diEjEwoX2z8TxuVYhAlMmVCGmPcf8XF2276di/HdT3TkyemQVeo1IRJkF+GBhMa6gLo96ALMn1J0HZVci8MHbVSh0yb/ffHjLT1YWYLUCyck06AVRlKioKPQPanZtdaYpBZMJCGsXjigA/QHYsAeLbU8hZ0ISjgNIRBF+gxGjb6ysG4HWzyinGUVFuHuSEdbCSsQ0NyJufyXwUnAjvZqSkmACgK9DHxm2jzEJs67m/wqMiEQlqmBE348rge3yj1QbyvrWU0YsYR6j0jIjfhlQCXZnEgb09739MgtgXVSE568eiyJbEk5MAKzbXUYn5pgfx+jEl2HEHlTiujuTcE0nYCmKcNmx/auj7/b9uAj43r6Na6qMWAr7yLyO/Pv9TLzlx7WGqTKqK0MY8ctisTAAzGKxyJJ+SkoKA8BSUlJkSd8bOfP000+M2UsG3z8GA2N5eZJnjZcVK+z5duR/xQq5c8RPoM/J32fk671btgSXh7w8xvR63+lyOeZ8PxclXr9qInf5q2R8j83Zs2fZvHnz2NmzZ0N6fyjnNt80xciTUI4//AI7C7B5ADsbKEjRD/3I+JMCMFz9zcaNk/xa4UPsMoQrruUvNRdUuOTkZKSkpCA5OVnurCiCt/bqnpT06JxrUxGhmncoRaDPyd9nJFTH7UDNKrgcc76fC12/RKlKSkqwYsUKlJSUhPT+UM5tvmkqltmM9queRQmAFQA0tndEY5IBpFz9jQ8/5N5WUgHUVoZQc0GF+/PPP+XOgqJ4Thio19fdnnFQyvgCwc51ItjcFwrg+Tl58vcZCTUpJJe5Srgccz6fC12/RKl69uwJM4/e8qGc23zTVKycHOgYQ08AGtw7ojH1rtxff7X3t1cBtZUh9CSLqI7r04XTp4H33xdmJCUhNfQRgAD3z2nRouA+I65PkPw9KRRylC1CCPEpNVWVE7sSAgC47jq5c6BZVMkiquQ6NKrjC/lnnwFr1thnbJebUkcAknoCQ8fnNH168M3uAg1/y2WUJ601wyREKIcOHUKvXr1w6NAhTacpCZMJeP99HALQC4DG9o5o2bhxqnmKBaivDKHmgkQTNm4Mrmme2AI1VTOb7RWx1FTxn6w40vrzT2DWLPmOkZDNIX09KUxPr5+GlpphEiKUJk2aYPjw4WjSpImm05RMRgaa9OyJ4fPmoUmvXkB1NZCYiJ1/ANu+so9kGmhkVSlGhvUcWdUx4uvDMxMRH2sfGbb0YgSaRgcximqI+bkQkYjzJUDp0WJs2lZ/FFg5jg/f9QUZlVaK/EdGArfeqqoKFqC+MkTHmGtvFuLJarUiNjYWFosFMTExkqc/efJklJaWomnTpnj33XclT98bk8mE/Px8pKSkKKJtrNlsf5LhWaHJzZX3y3VWVv1+RRkZwffV4psH18qIKyUco1Bt2WJ/guVt+fDhkmfHJyVev2oid/mrZHIfGyWe2xSbuKHYJB41xCYlXrtqw7X8pSdZCrd+/Xpn0CDe+Wuax7eQ5vPEKSPD/mTl+HH7EyyTKbgnMHx5puVJqGMkBy6DWigBXb9Eqa5cuQKz2QyTyYTIyMig3x/Kuc03TaXz3D+KTb7zTrFJPmqOS2orQ6hPFlE9oYb89iTEzO6e/Yqk7KvlLS1XSiv4g0GDWhDCz6FDh5Camip5nyyp05SS5/5RbPKOYhMJldrKEKpkEdUTo1ATa3RAsYIu17Rc01R7wU+DWhASutTUVGzZsgWpqamaTlNKnvtHsYl7Wq5pUmwivqitDKFKFtEEoQs1se7qSXmXy1taixcHd4zEHI1QiG37G4FQ6pEUCVETGvhCeN72j2ITt7QaSmyiuMSP2soQqmQRzQg05HcwxLyrJ+VdLs+0pk3jfoyEaJIix7al2D4haldYWIjFixejsLBQ02lKydf+UWwKnFZDiE0Ul/hTWxlClSyiaaHeNRL7rp6QQdcXx74Dwacl5mTKYk/UTBNBExJYQUEBFixYgIKCAk2nKaVg9o9ik/3vhhKbKC4JQ21lCFWyiGbxvWuk5nbVXPbdX5AXsxO02B2slToRNCFK0rt3b5SWlqJ3796aTlNKXPePYpPvfQ9U+VRrbKK4JAy1lSFUySKaJNRdIynu6gmNy74HCnRiNkkRu4O1lB24CSEkGBSbfO87l8qnWmMTxaWGiSpZRJMa8l2jQPvOJciL2SRFiuYuNIQuIf4dPXoUgwcPxtGjRzWdppS47B/FJvdljn3nWvlUa2yiuCQMtZUhNBmxwo0dOxYXLlxAfHy83FlRFTVMCCiWQPvOdYJMbxNWCkXMbUuxfa7o+iVKFRkZie7du4c8oWco5zbfNJWOy/5RbPK+78FM3KzW2ERxiT+1lSE6xhiTOxNKZrVaERsbC4vFgpiYGLmzowgmk8k5W7hZwb02s7Lsd8Jqa+vuGqmp7Tof/vbdbLY3xfAMdLm5dFeNKAuVv77RsamPYpPy+dp3iktETbiWv/Qki2iWUu4aSclstt8RTE+3Bydv++5otuAZ6BrC8SGE2FVXV6OkpAQJCQkIDw/XbJpS4rp/FJvq7zvFJcKF2soQRfXJWr58Odq2bYvIyEgMHDgQO3fu9Lv+559/ji5duiAyMhI9e/bEd9995/Y6Ywxz585FixYt0KhRI4wcORI5OTli7gJRmFA7BytpwkCuefHsNLxxo+999zc6lZL2nRAijv3796Nly5bYv3+/ptOUUjD7p/bYFEw+uMYmikskENWVIUwh1q5dyyIiItgHH3zADh48yCZOnMji4uJYYWGh1/V//fVXZjAY2KJFi9ihQ4fYnDlzWHh4ONu/f79znZdffpnFxsayr7/+mu3du5fdfvvtrF27duzy5cuc82WxWBgAZrFYeO+jWNq0acOGDRsmWXopKSkMAEtJSZEsTSmtWMGYXs8YYP+9YoXy85KXV7ee48dgsC8XIz3ScEhdvrhSQ/krF77HpqysjP33v/9lZWVlAudMvjTljk1i759Syudg8iFEbFLKfhP5yVFuecO1/FVMJWvAgAFsypQpzv9ra2tZy5Yt2YIFC7yuP2bMGHbrrbe6LRs4cCCbPHkyY4wxm83GkpOT2eLFi52vl5WVMaPRyD799NOA+dmyZQsD4PzR6/UsLi6Ode/enT300EPs+++/ZzabLZRdDUrnzp1ZkyZNWGJiIlu5cqXXdaT6ElReXs7mz5/PIiMjncdFri9fYhGqsiJ1Xn76yX09x8+WLeKkR7hxXL+dO3d2W66U8sXh1VdfVUz5cttttzm/LA8ZMoQqWT7IXQH1dW7LSe5KlpiUUj4Hmw++sUkp+60lSrx21YZr+auI5oJVVVXIzs7GyJEjncv0ej1GjhyJHTt2eH3Pjh073NYHgPT0dOf6p06dQkFBgds6sbGxGDhwoM9tenPPPfcAAN5++228+OKLuPHGG7F161bcfPPNuOmmm1BWVsZ5W6G4ePEiysvLUVpailWrVomaViAlJSWYP38+qqqqZM2HmLgMfy5Vk4VghvoVYg4OuYcW1mJzEMf1e/HiRa+vjx07Fh9//DFWrVolS/nisGzZMsWULzt37kTv3r0RFkZdhsVUXFyM5cuXo7i4OKT3Bzq3xUhT6cTcP6XEpmDjBN/YJHdcArQXm0K5dpVCbWWIIipZJSUlqK2tRfPmzd2WN2/eHAUFBV7fU1BQ4Hd9x+9gtgkAlZWVsFqtqKioAAB07twZAHD//ffj8ccfx2uvvYaTJ08iMzMTmzZtwtixY4PYU3Vr0aIF8vLy0KJFCwCATqeTOUfC8xcQuEyUKFVePAt9IebgkHOyRKmPrVL07dsXDz74IP72t79R+XK1fCkoKMD69ethNBrlzpKiOGKT6w8fZrMZmZmZko7CJ0eaUhJz/5QSmwLFCaFjk9yT+DbU2KRUaitDFFHJUpIFCxYgNjYWo0ePBgC8+OKL9dYxGAx45ZVXMGTIEGzYsAHbtm1zvmaxWDBz5kx07NgRRqMRiYmJGDt2LE6ePOm2jVWrVkGn02HTpk2YP38+2rRpA6PRiF69emHt2rX10qytrcX//vc/6HQ6509ubq7bOkeOHMGtt96KJk2aIDY2Fvfcc4/fCmWwjEYjTBof6sdXQAC4TZQoRV42bvRe6PvrNMwnPbE/cq6TUDYUUpcvOp0Op0+fpvJF4RyxyfHTqlUrXtu75pprUFlZiWuuuUagHCozTSmJuX9KiU3+4oSvCgmf2CTnJL4Um5RHbWWIItpjJCQkwGAwoLCw0G15YWEhkpOTvb4nOTnZ7/qO34WFhc4nL47/+/Tp4zMvs2fPRmZmJn755ReMHj0a//znP71WtAAgIyMD27Ztw/r16zFkyBBYLBZce+21OHPmDB555BF0794d586dw1tvvYWBAwfizz//RJs2bdy2MXPmTFRUVODxxx8HAKxcuRJjx47FlStXMH78eOd6er0enTp1wj//+U/nssTEROff+fn5GD58OP76179i8eLF2Lt3L959911YrVb88MMPzvWqq6thsVh87r+nhIQEzusqjWPI2NTU4Apkb8PrbtnCfaJEIXnmBXCfS8RR6Ken2/Ph+BEivago4OJF+3EUcx+DmYSyIZGqfPn444/x9NNPIyEhgcoXBXPEJger1cq7okWkF2pcApQTm7zlw1eFRIjY5C0ObtkS2jEMBsUmwptEfcQCGjBgAJs6darz/9raWpaSkuJ34IvRo0e7LRs8eHC9gS+WLFnifN1isQQ98MXzzz/vs3NbdnY2A8DuuusuxhhjTzzxBIuMjGR79uxxWy83N5c1adKEjRs3zrls5cqVDABr3bq12ygpZWVlrHXr1iw+Pp5dunTJ2ZHXYDD47Hzepk0bBoCtW7fObfnjjz/OALAjR47U2y+uP9448qTT6RQ78IXQoxEppfOtEANccBHo+OXl2fMixP4r5diKwVdHfMd16DowjycpyhcHf4NbSF2+OERFRdHAF37wHfji2LFj7MYbb2THjh0L6f2hDDLBN00x8iSkQPsnxih5Sik/lRCbhIxLju0p4dgKTe7rhA+xyxCuuJa/iniSBQCZmZkYN24c+vXrhwEDBmDZsmWoqKjAww8/DAB46KGHkJKSggULFgAAnnzySQwbNgyvvPIKbr31VqxduxZ//vkn3nvvPQD2JjBPPfUUXnjhBaSmpqJdu3Z49tln0bJlS9x5552C5Nkxy7PVagVjDKtXr8b111+PlJQUlJSUONeLiorCoEGD3O74Ojz22GOIjY11/h8bG4tHH30UzzzzDLZu3co5Ly1btsSYMWPclo0YMQJvvfUWcnJynH3LevfujR9//DGY3VSdQHfUQqGUiRId7dNd764J3T490PHLyqp7Xa+3H5dgmya6UsqxVRopypebb76ZU16ofNGesLAwJCYmSjrAiBxpSsnf/okRlwDllJ9yx6aNG4WNS4Byji2po7YyRDG5vO+++1BcXIy5c+eioKAAffr0wYYNG5wDV5w5cwZ6l96P1157LdasWYM5c+bgmWeeQWpqKr7++mv06NHDuc6MGTNQUVGBSZMmoayszNnHITIyUpA8Ozoex8TEoLi4GOfPn8cPP/zg1szGld6z9yaArl271lvWrVs3AKjXz8Kf9u3b11vWrFkzAMD58+edy+Lj4+uNyqg1Yj3i99ZEQmpSFPqBRnMS44uCEo6t0lD5QsTUrl07fPrpp5pPU0r+9k/MpmdKKD/ljE07dogTlwBlHFtSR21liGIqWQAwdepUTJ061etr3p7q3Hvvvbj33nt9bk+n0+Ff//oX/vWvfwmVRTf79u0DYB+BkDEGABg5ciRmzpwpSnr+GBy9Qr1w5A2wD5dfWlrKebu++sQpmZh31Pj2eRKC2IW+v+PH54tCoL4ISji2SkLlCxFTbW0tKioqEBUV5ffzVXuaUvK3f2I/6VFC+SlXbGKMXwWWYpN6qK0MUVQlS22yrg6dc+uttyIxMRFxcXGwWq1B3ck9fPgw7rjjDrdlhw4dAuD97jFf27dvxw033MB5fdcvUGrREB7xi1noBzp+oXxRELqJYUMgZfki1HQMDaF80Yq9e/ciLS0N2dnZ6Nu3r2bTlJK//WsIcQmQJzZde23oFViKTeqitjKEKlkhqK2txcyZM7Ft2zbccsstuO666wAADzzwAJYvX44vvvjCOYmxq6KiIiQlJbkte/vtt936TVgsFrzzzjuIi4vDsGHD8M477+Dy5cuYPn16UHeIfWkofSaU/oifzwhTUvB1/EL5oiBWXwQ1cFy/jRo14vweKcsXh+joaCpfGph27drhs88+Q7t27UJ6fyjnNt80lS7Q/ik9LgHqjU2hVGAbamwK5dpVCrWVIVTJCmDv3r0AgHXr1qGmpgZHjx7F119/jdOnT+Omm27CmjVrnOu++OKL+PXXXzFmzBiMGTMGgwYNQkREBE6fPo3vvvsOaWlpWLVqldv2ExISMHDgQOcAHytXrsSZM2ewYsUKNG7c2Dlf1w8//ICsrCw8++yz6Nq1K/R6PW677TZERUUFtT98+0y8+eabzr4ijDGcPn0aL7zwAgD7F6zbbrst5G0LLdAdNbmCiVrunHkeP8fxSk+3z3XC9YtCQx4G13H9+rJr1y588sknAIDy8nLJyxeHQYMGKaZ8KSsrA2AfDv7MmTMAgMWLF2PAgAGKKl/ULj4+3m9z+0ACndtipKl0XPaPy5Meik3+eYtN7dvb+2ZVVHCvwDbU2BTKtasUqitDxB3kUL08hyLW6/UsJiaGdevWjT300EPs+++/9/q+iooK9q9//Yv16NGDRUZGsujoaNalSxc2YcIE9ttvvznXcwyx/OOPP7K5c+eyVq1asYiICNajRw+2evXqetstLCxkd911F4uPj2c6nY4BYKdOnWKM+R5+2bEPK1euFOKQONNyPS6uP65DSCudGEPpciHnkLB8hrflc7y0OgwuH1S+eKeV8kUKfIdwLykpYStXrmQlJSUC50y+NOUemlqI/aPYFByKTQ2LHOWWN1zLX6pkBcA3kPni+BK0RehJJCQgdyDjy1fBunOnsHNseCPVXCKe5A5EK1bY3+N4r1RfHBoqNZcvrsQqf7WA77FxzMOWnZ0tcM7kS1Pu2MR3/yg2UWwi/slRbnmjunmyiHfZ2dmoqqpCREQE0tLS5M6OJvhqIjBokPjNJKSYS8QT33bnQjSpUENfBDHQ9UuU6pprrkF1dXXII3SFcm7zTVPp+O4fxSaKTVJQc1xSWxlSf2IVoih33HEHrr322nojhBHfzGZgyxb7b28cwcSTZ0Hv6/18OAaOcJQPUowwFWjeq0C8Ha9Qgq/JVDcMvBjHVono+iVKpdPpEBYWFvLIkqGc23zTVLpA+0exyZ1SYhNgfw7WUKg5LqmtDKFKFtGUrCygTRtgxAj776ujYLvxDCbeglowBX2wMjLsA0ds2WL/LXbHYr6BSKjg6/rZtG4NTJ/ecCpbhCjNiRMncPvtt+PEiROaTlNK/vaPYlN9SohNnp/L4sX+K8JEXmorQ6iSJZPx48eDMYbhw4fLnRXN8NX0wFth6RpMfvtNuLthXJlMwPDh4t0ldL1jKkQg4ht8PT8bxoAlS3x/2SD8UPlCiHJQbKqjpNjk7XOZMcN/RZiQYFCfLKIZwbbPdh0GVq5JIsUYqtfXMLx8253zmWTS22cDNJx5SQhRmg4dOuDbb7/VfJpS8rV/FJvslBabfMUlgGKTUqmtDKEnWUQz+DQ9kLqZBMCt+Uiw/N0x9XWHMlA/ASH46msAiNv8hRDiHWMMNTU1YBJ2RpEjTSn52j+KTcHHJrnjEkCxSYnUVoZQJYtoBt+mB2I3k3AVTPORYATbkdizn9TixfzS98Xx2XgLaGI3fyGE1Ld7926Eh4dj9+7dmk5TSr72j2JTcLFJqv67np+LJ4pNyqO2MoQqWURT5LjrFwq+oyr5EswdU2/9pGbMsPeVEkNGBnD6NDBtmrQjWBFC6mvTpg1WrlyJNm3aaDpNKfnbP4pN3GKT1P13XT+XxYspNimd2soQ6pNFNIdP3yGpiDUniePOHJc2/L7ao8+cCdx/vzjH0GSyB7Inn2xY85IQojTNmjXD+PHjNZ+mlALtH8WmwLFJjv67js9l+HB77KPYpFxqK0PoSRZpsKRo8+2LmHOScL1jmpoKeJtqwmYTvx26lM1fCCH1XbhwAZ9//jkuXLig6TSlJNT+NeTYJHf/XYpNyqa2MoQqWaRBEqNjb7DEbD7CJVCYTMDChfWXUzt0QrTv1KlTGDNmDE6dOqXpNKUkxP419NhE/XeJP2orQ3RMLUN0yMRqtSI2NhYWiwUxMTGSp19eXg7GGHQ6HZo0aSJ5+t6YTCbk5+cjJSUFZhXO2Gc224OXZ3OI3NyGefdqyRJ7E0Gbre6upVL7C6iNEq9fNZG7/FUyvsemtrYWFRUViIqKgsFXz38/Qjm3+aYZiNyxie/+UWyqYzYDr70GvPqqe/NCik38qTkuiV2GcMW1/KU+WQqntgtADYKds0RLvM19Mm0atUMXC12/RKkMBgOvimso5zbfNJWO7/5RbKqLTdR/VzxqjktqK0OouSBpcPjMWaJm/pqhUDt0QhqWU6dOYezYsZI3F5Q6TSnx3T+KTRSbiH9qK0OokkUaHDE79iqVWHOfEELUqaamBsXFxaipqdF0mlLiu38Umyg2Ef/UVoZQc0GFW7p0KaxWK2JiYpCZmSl3djQjI8M+FGxDaYbgqxnK558D996r/f2XC12/RKlSU1OxadOmkN8fyrnNN02lE2L/KDZRbBKbmuOS2soQGvgiALk7XsvdkdcbJeaJ+OetQ7WDXm+/e0odioVH1wo/cpe/Sib3sVHiua3EPBH/KDZJj64T/riWv9RckJAGwLMZiitqnkFIw7N7924YjUbs3r1b02lKSev7JwaKTSQYarvGqJJFiAZ5m8zSMffJ0qX115dikkdCiHKYTCYsXboUJgnbY8mRppS0vn9CoNhE+FDbNUaVLEI0JtBITffe2zBHsCKE1ElMTMSUKVOQmJio6TSlpPX944tiE+FLbdcYVbII0RAuIzU1xBGsCCHuLBYL1q9fD4vFouk0paT1/eODYhMRgtquMapkEaIh/iazdOVonrFli/23Z8dib006CCHaceLECYwePRonTpzQdJpS0vr+8UGxiQhBbdcYDeFOiIY4JrN0DWa+mluYTN7vEGZl1d1xpNGdCNGmnj174uzZs0hISNB0mlLS+v7xQbGJCEFt1xg9ySJEQ/g2t6CJIQlpGMLDw9GiRQuEh4drOk0paX3/+KDYRISgtmuMKllElajJgG+Bmlv4w7VJByFE3U6fPo0JEybg9OnTmk5TSqdPn8bYsRPw6aenKTZ5QbGJ8KW2MoSaCypc37590apVK9WMpCIFajIQmK/mFoEE06SDBEbXL1GqK1eu4ODBg7hy5UpI7w/l3OabptJ98skVrF17EGvXXqHY5APFJvmpOS6prQzRMcaY3JlQMq6zOjckcs4W7m12eIPBfleMRiASRlaWvRlGbW1dkw76okDkQOWvb3Rs6qPYpG0Um4hScC1/6UkWURV/TQYokAkjIwNIT7cf044d6bgSQkggFJvER7GJqA31ySKq4mgy4IqaDAjPZAKGD6cgRohW7d27F02bNsXevXs1naZUUlMBnW4vgKYA7PtHsUl4FJsaNrWVIVTJIqpCkxUSQgh/ycnJmD17NpKTkzWdplRMJmDJkmTodLMBJFNsIkQEaitDqE9WAHK3e7/99ttRXFyMxMREfPvtt5Kn742c7d4dzGZqMkCUT4nXr5rIXf4qmdzHRonnNsUmQgJT4rWrNtQnSyN27drlDBqkTqgjFBEiJbp+iVKVl5cjOzsbaWlpaNKkSdDvD+Xc5pum0pWXl+P4ce3uH9EGNccltZUh1FyQEEIIaWBycnJwww03ICcnR9NpSknr+0eI3NR2jdGTLEIIIaSB6datG3JycmCSsEmAHGlKSev7R4jc1HaNUSWLEEIIaWAiIyPRUeKh7+RIU0pa3z9C5Ka2a4yaCxJCCCENTF5eHp544gnk5eVpOk0paX3/CJGb2q4xRVSySktL8cADDyAmJgZxcXHIyMjAxYsX/b7nypUrmDJlCpo1a4bo6GjcfffdKCwsdFtHp9PV+1m7dq2Yu0IIIYQoXnl5ObZu3Yry8nJNpyklre8fIXJT2zWmiOaCDzzwAM6dO4cff/wR1dXVePjhhzFp0iSsWbPG53uefvpprF+/Hp9//jliY2MxdepU3HXXXfj111/d1lu5ciVGjRrl/D8uLk6s3SCEEEJUoVu3bti3b5/m05SS1vePELmp7RqTvZJ1+PBhbNiwAX/88Qf69esHAHjjjTdwyy23YMmSJWjZsmW991gsFmRlZWHNmjUYMWIEAHtlqmvXrvjtt98waNAg57pxcXGqmbSMEEIIIYQQon6yV7J27NiBuLg4ZwULAEaOHAm9Xo/ff/8df/3rX+u9Jzs7G9XV1Rg5cqRzWZcuXdC6dWvs2LHDrZI1ZcoUTJgwAe3bt8ejjz6Khx9+GDqdzmd+KisrUVlZ6fzfYrEAsE88Jgebzeb8LVcePCkxT4QoEV0r/DiOGWNM5pzIT+jYdPDgQdx9993497//je7duwf9/lDObb5pipEnIYm9f4QIQe7rhA+lXGOcYxOT2Ysvvsg6depUb3liYiJ76623vL5n9erVLCIiot7y/v37sxkzZjj//9e//sW2bdvGdu3axV5++WVmNBrZa6+95jc/8+bNYwDoh37oh37oRyE/eXl5QUYW7aHYRD/0Qz/0o6yfQLFJtCdZs2bNwsKFC/2uc/jwYbGSBwA8++yzzr+vueYaVFRUYPHixXjiiSd8vmf27NnIzMx0/m+z2VBaWopmzZr5fQImBavVilatWiEvLw8xMTGy5kVp6Nh4R8fFOzouvinp2DDGUF5e7rXZeENDsUl96Lj4RsfGOzouvinp2HCNTaJVsv7xj39g/Pjxftdp3749kpOTUVRU5La8pqYGpaWlPvtSJScno6qqCmVlZW4DWRQWFvrtfzVw4EA8//zzqKyshNFo9LqO0Wis95rSBsuIiYmR/QRTKjo23tFx8Y6Oi29KOTaxsbFyZ0ERKDapFx0X3+jYeEfHxTelHBsusUm0SlZiYiISExMDrjd48GCUlZUhOzsbaWlpAICffvoJNpsNAwcO9PqetLQ0hIeHY/Pmzbj77rsBAEePHsWZM2cwePBgn2nt2bMH8fHxPitYhBBCCCGEEMKX7ANfdO3aFaNGjcLEiRPxzjvvoLq6GlOnTsX999/vfAyXn5+PG2+8ER999BEGDBiA2NhYZGRkIDMzE02bNkVMTAz+/ve/Y/Dgwc5BL/7zn/+gsLAQgwYNQmRkJH788Ue89NJLmDZtmpy7SwghhBBCCNE42StZALB69WpMnToVN954I/R6Pe6++268/vrrzterq6tx9OhRXLp0ybns1Vdfda5bWVmJ9PR0vPXWW87Xw8PDsXz5cjz99NNgjKFjx45YunQpJk6c+P/t3VlIVP8bx/Fn1CaDtEkiJaGi5VcUbRKaghk1kGJhG4FFlER10QLRRSKFUUQWEYV41UJdJSUYQistQoXaHuISWHZRqFHCZAul9vwvoqmpM/4bO3POOPN+wVz49XvkmYcZPzyeOUdLn5uZBg8eLMXFxZyJM0BvjNEXY/TFP3qDQPGaMUZf/KM3xuiLfwOxNw5V7o0LAAAAAGaJsrsAAAAAAAgnDFkAAAAAYCKGLAAAAAAwEUMWAAAAAJiIISvEdXZ2yurVqyU+Pl5cLpesX79ePnz48FfHqqrk5OSIw+GQCxcuBLdQiwXal87OTtm6datMmjRJhgwZIqNHj5Zt27aJx+OxsOrgKCsrk7Fjx0psbKykpaXJvXv3+tx//vx5mTx5ssTGxsq0adPk0qVLFlVqrUD6cvz4ccnMzJThw4fL8OHDxe12/98+DmSBvmZ+KC8vF4fDIUuWLAlugQh5ZJMxsuknsskY2eRf2GWTIqRlZ2frjBkztLa2Vm/fvq0TJkzQ/Pz8vzr2yJEjmpOToyKilZWVwS3UYoH2pb6+XpctW6ZVVVXa0tKiN27c0IkTJ+ry5cstrNp85eXl6nQ69dSpU9rQ0KAbNmxQl8ulHR0dhvvv3r2r0dHReujQIW1sbNRdu3bpoEGDtL6+3uLKgyvQvqxatUrLysr08ePH2tTUpOvWrdNhw4bpq1evLK48+ALtzQ+tra2anJysmZmZmpeXZ02xCFlkkzGy6TuyyRjZ5F84ZhNDVghrbGxUEdH79+971y5fvqwOh0Nfv37d57GPHz/W5ORkbWtrC7sg+5e+/OrcuXPqdDq1u7s7GGVaIjU1VTdv3uz9ure3V0eNGqUHDhww3L9y5UrNzc31WUtLS9NNmzYFtU6rBdqX3/X09GhcXJyeOXMmWCXapj+96enp0YyMDD1x4oSuXbs25IIM1iKbjJFNP5FNxsgm/8Ixm/i4YAirqakRl8sls2fP9q653W6JioqSuro6v8d9+vRJVq1aJWVlZZKUlGRFqZbqb19+5/F4JD4+XmJiQuJ/cgfs69ev8vDhQ3G73d61qKgocbvdUlNTY3hMTU2Nz34RkYULF/rdPxD1py+/+/Tpk3R3d0tCQkKwyrRFf3uzd+9eGTlypKxfv96KMhHiyCZjZNN3ZJMxssm/cM2mgfkOjhDt7e0ycuRIn7WYmBhJSEiQ9vZ2v8dt375dMjIyJC8vL9gl2qK/ffnV27dvZd++fbJx48ZglGiJt2/fSm9vryQmJvqsJyYmSnNzs+Ex7e3thvv/tm8DQX/68rudO3fKqFGj/gj9ga4/vblz546cPHlSnjx5YkGFGAjIJmNk03dkkzGyyb9wzSbOZNmgsLBQHA5Hn4+/fcP9rqqqSm7evClHjx41t2gLBLMvv3r//r3k5ubKlClTZM+ePf9eOMJKSUmJlJeXS2VlpcTGxtpdjq26urpkzZo1cvz4cRkxYoTd5SDIyCZjZBNCAdn000DJJs5k2WDHjh2ybt26PveMGzdOkpKS5M2bNz7rPT090tnZ6fejFjdv3pTnz5+Ly+XyWV++fLlkZmZKdXX1P1QeXMHsyw9dXV2SnZ0tcXFxUllZKYMGDfrXsm0zYsQIiY6Olo6ODp/1jo4Ov31ISkoKaP9A1J++/HD48GEpKSmR69evy/Tp04NZpi0C7c3z58/l5cuXsnjxYu/at2/fROT7X+ifPXsm48ePD27RsAzZZIxsCgzZZIxs8i9ss8nui8Lg34+LaB88eOBdu3r1ap8X0ba1tWl9fb3PQ0T02LFj+uLFC6tKD6r+9EVV1ePx6Jw5czQrK0s/fvxoRalBl5qaqlu2bPF+3dvbq8nJyX1eXLxo0SKftfT09LC8uDiQvqiqHjx4UOPj47WmpsaKEm0TSG8+f/78x++TvLw8nT9/vtbX1+uXL1+sLB0hgmwyRjb9RDYZI5v8C8dsYsgKcdnZ2Tpr1iytq6vTO3fu6MSJE31uB/vq1SudNGmS1tXV+f0ZEmZ3cFINvC8ej0fT0tJ02rRp2tLSom1tbd5HT0+PXU/jn5WXl+vgwYP19OnT2tjYqBs3blSXy6Xt7e2qqrpmzRotLCz07r97967GxMTo4cOHtampSYuLi8P2NrmB9KWkpESdTqdWVFT4vDa6urrsegpBE2hvfheKd3CC9cgmY2TTd2STMbLJv3DMJoasEPfu3TvNz8/XoUOHanx8vBYUFPi8uVpbW1VE9NatW35/RjgGWaB9uXXrloqI4aO1tdWeJ2GS0tJSHT16tDqdTk1NTdXa2lrv97KysnTt2rU++8+dO6f//fefOp1OnTp1ql68eNHiiq0RSF/GjBlj+NooLi62vnALBPqa+VUoBhmsRzYZI5t+IpuMkU3+hVs2OVRVg/uBRAAAAACIHNxdEAAAAABMxJAFAAAAACZiyAIAAAAAEzFkAQAAAICJGLIAAAAAwEQMWQAAAABgIoYsAAAAADARQxYAAAAAmIghCwAAAABMxJAFhImzZ8/KkCFDpK2tzbtWUFAg06dPF4/HY2NlAIBIRTYhUjlUVe0uAsC/U1WZOXOmzJ07V0pLS6W4uFhOnToltbW1kpycbHd5AIAIRDYhUsXYXQAAczgcDtm/f7+sWLFCkpKSpLS0VG7fvu0NsaVLl0p1dbUsWLBAKioqbK4WABAJyCZEKs5kAWEmJSVFGhoa5Nq1a5KVleVdr66ulq6uLjlz5gxBBgCwFNmESMM1WUAYuXLlijQ3N0tvb68kJib6fG/evHkSFxdnU2UAgEhFNiESMWQBYeLRo0eycuVKOXnypCxYsEB2795td0kAgAhHNiFScU0WEAZevnwpubm5UlRUJPn5+TJu3DhJT0+XR48eSUpKit3lAQAiENmESMaZLGCA6+zslOzsbMnLy5PCwkIREUlLS5OcnBwpKiqyuToAQCQimxDpOJMFDHAJCQnS3Nz8x/rFixdtqAYAALIJ4O6CQIRwu93y9OlT+fjxoyQkJMj58+clPT3d7rIAABGMbEK4YsgCAAAAABNxTRYAAAAAmIghCwAAAABMxJAFAAAAACZiyAIAAAAAEzFkAQAAAICJGLIAAAAAwEQMWQAAAABgIoYsAAAAADARQxYAAAAAmIghCwAAAABMxJAFAAAAACZiyAIAAAAAE/0PO3Pfw+50UZIAAAAASUVORK5CYII=",
|
|
"text/plain": [
|
|
"<Figure size 1000x400 with 2 Axes>"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"import matplotlib.pyplot as plt\n",
|
|
"\n",
|
|
"def plot_regression_predictions(tree_reg, X, y, axes=[-0.5, 0.5, -0.05, 0.25]):\n",
|
|
" x1 = np.linspace(axes[0], axes[1], 500).reshape(-1, 1)\n",
|
|
" y_pred = tree_reg.predict(x1)\n",
|
|
" plt.axis(axes)\n",
|
|
" plt.xlabel(\"$x_1$\")\n",
|
|
" plt.plot(X, y, \"b.\")\n",
|
|
" plt.plot(x1, y_pred, \"r.-\", linewidth=2, label=r\"$\\hat{y}$\")\n",
|
|
"\n",
|
|
"fig, axes = plt.subplots(ncols=2, figsize=(10, 4), sharey=True)\n",
|
|
"plt.sca(axes[0])\n",
|
|
"plot_regression_predictions(tree_reg, X_quad, y_quad)\n",
|
|
"\n",
|
|
"th0, th1a, th1b = tree_reg.tree_.threshold[[0, 1, 4]]\n",
|
|
"for split, style in ((th0, \"k-\"), (th1a, \"k--\"), (th1b, \"k--\")):\n",
|
|
" plt.plot([split, split], [-0.05, 0.25], style, linewidth=2)\n",
|
|
"plt.text(th0, 0.16, \"Depth=0\", fontsize=15)\n",
|
|
"plt.text(th1a + 0.01, -0.01, \"Depth=1\", horizontalalignment=\"center\", fontsize=13)\n",
|
|
"plt.text(th1b + 0.01, -0.01, \"Depth=1\", fontsize=13)\n",
|
|
"plt.ylabel(\"$y$\", rotation=0)\n",
|
|
"plt.legend(loc=\"upper center\", fontsize=16)\n",
|
|
"plt.title(\"max_depth=2\")\n",
|
|
"\n",
|
|
"plt.sca(axes[1])\n",
|
|
"th2s = tree_reg2.tree_.threshold[[2, 5, 9, 12]]\n",
|
|
"plot_regression_predictions(tree_reg2, X_quad, y_quad)\n",
|
|
"for split, style in ((th0, \"k-\"), (th1a, \"k--\"), (th1b, \"k--\")):\n",
|
|
" plt.plot([split, split], [-0.05, 0.25], style, linewidth=2)\n",
|
|
"for split in th2s:\n",
|
|
" plt.plot([split, split], [-0.05, 0.25], \"k:\", linewidth=1)\n",
|
|
"plt.text(th2s[2] + 0.01, 0.15, \"Depth=2\", fontsize=13)\n",
|
|
"plt.title(\"max_depth=3\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "subslide"
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"### Training regression models\n",
|
|
"\n",
|
|
"CART algorithm works in the same way except the cost function is based on the mean squared error (MSE):\n",
|
|
"\n",
|
|
"$$\n",
|
|
"J(k,t_k)=\\frac{m_\\text{left}}{m}\\text{MSE}_\\text{left}+\\frac{m_\\text{right}}{m}\\text{MSE}_\\text{right}.\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "subslide"
|
|
},
|
|
"tags": [
|
|
"exercise_pointer"
|
|
]
|
|
},
|
|
"source": [
|
|
"**Exercises:** *You can now complete Exercise 2 in the exercises associated with this lecture.*"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"## Limitations"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": ""
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"### Sensitivity to axis orientation"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": ""
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"Decision trees are sensitive to the axis orientation.\n",
|
|
"\n",
|
|
"Consider same data but with axis rotated by 45${}^\\circ$."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 11,
|
|
"metadata": {
|
|
"editable": true,
|
|
"execution": {
|
|
"iopub.execute_input": "2025-02-27T23:21:21.490962Z",
|
|
"iopub.status.busy": "2025-02-27T23:21:21.490688Z",
|
|
"iopub.status.idle": "2025-02-27T23:21:21.495775Z",
|
|
"shell.execute_reply": "2025-02-27T23:21:21.495198Z"
|
|
},
|
|
"slideshow": {
|
|
"slide_type": "skip"
|
|
},
|
|
"tags": []
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"def plot_decision_boundary(clf, X, y, axes, cmap):\n",
|
|
" x1, x2 = np.meshgrid(np.linspace(axes[0], axes[1], 100),\n",
|
|
" np.linspace(axes[2], axes[3], 100))\n",
|
|
" X_new = np.c_[x1.ravel(), x2.ravel()]\n",
|
|
" y_pred = clf.predict(X_new).reshape(x1.shape)\n",
|
|
" \n",
|
|
" plt.contourf(x1, x2, y_pred, alpha=0.3, cmap=cmap)\n",
|
|
" plt.contour(x1, x2, y_pred, cmap=\"Greys\", alpha=0.8)\n",
|
|
" colors = {\"Wistia\": [\"#78785c\", \"#c47b27\"], \"Pastel1\": [\"red\", \"blue\"]}\n",
|
|
" markers = (\"o\", \"^\")\n",
|
|
" for idx in (0, 1):\n",
|
|
" plt.plot(X[:, 0][y == idx], X[:, 1][y == idx],\n",
|
|
" color=colors[cmap][idx], marker=markers[idx], linestyle=\"none\")\n",
|
|
" plt.axis(axes)\n",
|
|
" plt.xlabel(r\"$x_1$\")\n",
|
|
" plt.ylabel(r\"$x_2$\", rotation=0)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 12,
|
|
"metadata": {
|
|
"editable": true,
|
|
"execution": {
|
|
"iopub.execute_input": "2025-02-27T23:21:21.497513Z",
|
|
"iopub.status.busy": "2025-02-27T23:21:21.497340Z",
|
|
"iopub.status.idle": "2025-02-27T23:21:21.688609Z",
|
|
"shell.execute_reply": "2025-02-27T23:21:21.687963Z"
|
|
},
|
|
"slideshow": {
|
|
"slide_type": ""
|
|
},
|
|
"tags": []
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"Text(0, 0.5, '')"
|
|
]
|
|
},
|
|
"execution_count": 12,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
},
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA1MAAAFzCAYAAADbi1ODAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAYaRJREFUeJzt3Xt8VNW5P/7PnoQEUEKIkGQCsYj1hiIIHCKotV+IEkGPcqDeOFo5CJYDrYj1VbAJWG9oRY0iP6kUfrXnV6sVDv48BlE6yimXEC7a1lpAUSmUMIBGEi5KYGZ9/5gLM5O57NmzL2vv/Xm/Xnkhw57J2gFnzbPWs55HEUIIEBERERERUVY8Vg+AiIiIiIjIjhhMERERERERacBgioiIiIiISAMGU0RERERERBowmCIiIiIiItKAwRQREREREZEGDKaIiIiIiIg0yLd6ALIIBoNobm5Gt27doCiK1cMhInINIQSOHDmCiooKeDxc44vFuYmIyBpq5yYGU2HNzc2orKy0ehhERK61d+9e9OnTx+phSIVzExGRtTLNTQymwrp16wYA+McH21DU7UyLR0OkTrCoO3Z+9jlWr16NhoYG5OXl4bXXXkNRUZHVQyNSra2tDeecc070fZhO49xERGSNtiNH8Z3BQzLOTQymwiLpE0XdzkQRJ3SyiWBREc4880x07twZeXl5yM/PR1FREYMpsiWmsXXEuYmIyFqZ5iYmpxMREREREWnAYIqIiIiIiEgDBlNEREREREQaMJgiIiIiIiLSgMEUERERERGRBgymiIiIiIiINGAwRUREREREpAGDKSIiIiIiIg0YTBEREREREWnAYIqIiIiIiEgDBlNEREREREQaMJgiIiIiIiLSgMEUERERERGRBgymiIiIiIiINGAwRUREREREpIG0wdSiRYvQt29fdO7cGVVVVdi8eXPa6w8fPozp06fD6/WisLAQ559/PlatWmXSaImIiIiIyG3yrR5AMq+99hpmzZqFxYsXo6qqCvX19Rg9ejR27tyJ0tLSDte3t7fjmmuuQWlpKZYvX47evXvjH//4B4qLi80fPBERERERuYKUwdQzzzyDKVOmYNKkSQCAxYsXo6GhAcuWLcPs2bM7XL9s2TK0tLRg48aN6NSpEwCgb9++Zg6ZiIiIiHIVCACbmoCDB4DSMuDyKiAvz+pREaUkXZpfe3s7tm3bhurq6uhjHo8H1dXVaGxsTPqcN998E8OHD8f06dNRVlaGSy65BI8//jgCgUDK73PixAm0tbXFfREREVmJcxO5WsMqKEOHwTN+AjzTpsMzfgKUocOABh7bIHlJF0x9+eWXCAQCKCsri3u8rKwMfr8/6XM+//xzLF++HIFAAKtWrUJdXR2efvppPProoym/z/z589G9e/foV2Vlpa73QURElC3OTeRaDaug3D0F2L8//nG/P/Q4AyqSlHTBlBbBYBClpaV46aWXMGTIENxyyy34+c9/jsWLF6d8zpw5c9Da2hr92rt3r4kjJiIi6ohzE7lSIACltg4QAkrCHylChH6tmxtKASSSjHRnpnr27Im8vDwcOHAg7vEDBw6gvLw86XO8Xi86deqEvJic2osuugh+vx/t7e0oKCjo8JzCwkIUFhbqO3giIqIccG4iV9rUBCVxRyqGIgTQ3AyxqQm4YoSJAyPKTLqdqYKCAgwZMgQ+ny/6WDAYhM/nw/Dhw5M+54orrsCuXbsQDAajj33yySfwer1JAykiIiIiksTBA5mvyeY6IhNJF0wBwKxZs7BkyRK8/PLL2L59O6ZNm4Zjx45Fq/vdeeedmDNnTvT6adOmoaWlBffeey8++eQTNDQ04PHHH8f06dOtugUiIiIiUqO0LPM12VxHZCLp0vwA4JZbbsGhQ4cwd+5c+P1+DBo0CKtXr44WpdizZw88ntNxYGVlJd555x3cd999uPTSS9G7d2/ce++9+NnPfmbVLRARERGRGpdXQXi9oWIT4TNSsYSiAF5vqEw6kWSkDKYAYMaMGZgxY0bSP1u7dm2Hx4YPH45NmzYZPCoiIiIi0lVeHsSjj0C5ewqEosQFVEIJlaQQjzzMflMkJSnT/IiIiIjIIIEAsGEjsHJl6FcZquSNHQPx6yVAYrExrzf0+Ngx1oyLKANpd6aIiIiISGcNq6DU1sVVzxNeL8Sjj1gfsIwdA1EzOlS17+CB0Bmpy6u4I0VSYzBFRERE5AaRxriJ55LCjXGl2AHKy2P5c7IVpvkREREROZ3bG+PKmNpIjsCdKSIiIiKnc3NjXJlTG8n2uDNFRERE5HRubYwbSW1MDCTDqY1oWGXNuMgxGEwREREROZ0bG+O6PbWRTMFgioiIiMjpwo1xI32bEglFgaiocFZj3HBqY/I7DgVUSnMzsKnJ1GGRszCYIiIiInK6cGNcAB0CKsc2xnVraiOZisEUERERkRs4sTFuuip9bkxtJNOxmh8RERGRWzipMW6mKn3h1Eb4/dEzUrGEogBer7NSG8l03JkiIiIicpNIY9xx40K/2jWQylSlz42pjWQ6BlNEREREZB/ZVOlzYmojSYXBFJHRXNp13edTMGBAHny+VHWUzH0dIiJyiGyr9I0dA7F1M4IrliP44iIEVyyH2NLEQIp0wTNTREZyadd1IYDaWg927FBQW+vByJEBpKjGa8rrEBGRg2ip0hdJbSTSGXemyF7stMvjgK7rWneF1qxRsHVr6DlbtypYs0ZbBKTX6xARkYOwSh9JhMEU2UfDKihDh8EzfgI806bDM34ClKHD5AxKHNB1PXFXKEkhpJTPmzfPg7y80BPy8gTmzVP/fL1fxyxMRyQiMokbGxCTtBhMkT3YbZfHAV3Xte4KRZ4XCISuDwQUTbtKer2OGbQGnkREmtgpS8MIrNJHEmEwRfKz4y6Pzbuua90VSnxeRLa7Snq9jlmYjkhEprFTloaRWKWPJMFgiuRnx10em+dza90VSnxeRLa7Snq9jhnslo5IRDZmtywNo7FKH0mAwRTpx6i0Azvu8tg4n1vrrlDkeR5P8gs8nux2t3J9HbPYKR2RiGzMjlkaekv2OcMJDYjJ1hhMkT6MTDuw4y6PjfO5te4KtbcDe/YAwWDyPw8GFezdG7ouHb1exwx2S0ckIhuzY5aGnpjeSJJinynKXSTtIPGTYzjtIOfc5fAuD/z+6OpbLKEogNcr3y5POJ9bqa2LT8nwekOBlIRpCLG7QsmCmciu0DXXdOz3VFgIbNoUwKFDqV+/tDR0XTp6vY4ZYs9KxQoFnqE/v/ZaRlRErhUIhIKbgwdCC36XV2lfRLNjloZejP6cQZQDBlOUmwxpB0JRoNTNhagZrX0CCe/yKHdPCb1ezJup7Ls8GDsGomY0hF6TqcHU7QoJtLcnD2YqK0NfudLrdYyUS+BJRC6gd9N2O2Zp6MGMzxlEOWAwRbkJpx2koggBNDeHgolcOo/bcJcnykZd1+20K2S1XANPInIwI3ZS7JqlkSuzPmcQacRginJjZtqBzXZ57MoOu0IyYOBJREkZtZNi5yyNXLg5vZFsgcEU5caMtAM9c87J8Xw+BTNnelBfH8SoUcaeV2LgSUQdGLmTIluWhhnzs1vTG8k2GExRboxOO9A755wcTQigttaDHTsU1NZ6MHIkzysRkcmM3kmRJUvDrPk58jkjRSVDx6Y3km2wNDrlxsgS4GxOSFmKra7HXk9EZImevdRd98kn2nsyqu2tZFT/R7Pm50AAqH8OaG1NHUjBoemNZBsMpih34bQDlJfHP+71ai9XyuaElKXEnk/s9UREpmtYBeXHP0l7SeQtyfPsc8b2SjKqL5NZ83PDKiiXXArPUwugHD+e/JoexSyLTpZjMEX6GDsGYutmBFcsR/DFRQiuWA6xpUn7G5zbmxNS1hKbDWdqMmwGn0/BgAF58Pm4Q0bkeJHdGr8/5SVJ13aMyLYwcufIjPk5Mv6vv055iQCAws5AzWjt38cKRu0WkmUYTJF+1KYdqMHqPZSFxF2pCCt3pxLPb3GHjMjB0uzWJDI828LonSOj52eVP0sFCJ3XstOiqlG7hWQpBlMkJ1bvoSwk7kpFWLk7xfNbRC6SYbcmwpRsC6N3joyen1X+LKPssqjKc+COJW0wtWjRIvTt2xedO3dGVVUVNm/erOp5r776KhRFwU033WTsAMlY4eo9iUUtIoSiQFRUsHoPRXelPJ7kWz8ej/m7Uzy/ReQyen2g1+N11L6GP3X59rSMnp+z/RnYYVGV58AdTcpg6rXXXsOsWbMwb948fPDBBxg4cCBGjx6NgwcPpn3e7t278dOf/hRXXXWVSSMlwxhZJZAcpb0d2LMHCAaTT+zBoIK9e0PXmUXG81tEZCC9PtDr8ToqX0Opm6dtN8To+Vnl+AVgn0VVngN3NCmDqWeeeQZTpkzBpEmT0L9/fyxevBhdu3bFsmXLUj4nEAhg4sSJ+MUvfoF+/fqZOFoyjBFVAslxCguBTZsCaGo6lfJr06YACgvNGY+M57eIyGBqdms8nuQFKKBztkWGsUS1tGhPLzNyflYx/sjP0TaLqjwH7mjSNe1tb2/Htm3bMGfOnOhjHo8H1dXVaGxsTPm8hx9+GKWlpZg8eTLWrVuX8fucOHECJ06ciP6+ra0tt4GTMdI1JzSj8zrZQmVl6EsGsWelYoV2p0J/fu21jKgoOc5NNhXerVHungKhKHFN7KO7NT+6B8qLiyGA5H+uV2CQZiyxFISCEqVuLkTN6Oy/t1HNg9WMv0cPiAVP2WdRlefAHU26nakvv/wSgUAAZWXx/6DKysrgT1FudP369Vi6dCmWLFmi+vvMnz8f3bt3j35VyvJJjDpKViWQFXFIQjKe3yJ74dwkIbWlrDPt1sytMy/bIjKWkpK0l2lOL4v8TN58M/T7f/3X3Kv4xkrxsxQ9eiD4wE8h/vZX+wRSAM+BO5x0O1PZOnLkCO644w4sWbIEPXv2VP28OXPmYNasWdHft7W1cdKyi0hFnMRPpOGKOEwBJKuoO78l0N4O09IOyV44N0mmYRWU2rpQCe4w4fWGzgwlm2cy7dYYtZuTzNgxEI2NUH69NPO12aSXZfsz0crMn5XR1Oxc2iVlkTqQLpjq2bMn8vLycOBA/P/YBw4cQHniag6Azz77DLt378YNN9wQfSwYDAIA8vPzsXPnTpx77rkdnldYWIhCfpqxnwwVcYSiaE9ZIMpR5PzWoUOpryktZSBFqXFukojWhbtINkUqmf5cLw2r1AVSgPr0MrMXM836WZkhvNum1NbFl0f3ekOBFBeBbUu6YKqgoABDhgyBz+eLljcPBoPw+XyYMWNGh+svvPBCfPTRR3GP1dbW4siRI3juuee4ouc04Yo4qShCAM3NoZUsp7wBk63IdH6LiDSy+8JdZPwZCABQm15m95+JDJy020ZR0gVTADBr1iz88Ic/xNChQzFs2DDU19fj2LFjmDRpEgDgzjvvRO/evTF//nx07twZl1xySdzzi4uLAaDD4+QArIhDRERGy3XhzuoCSRnGH0t1epmei5lW/3ys5KTdNgIgaTB1yy234NChQ5g7dy78fj8GDRqE1atXR4tS7NmzBx6PdLUzyAysiENEREbLZeHOrDNF2Y4rCTF1ivox6bWYKcPPh0hHUgZTADBjxoykaX0AsHbt2rTP/c1vfqP/gEgO4Yo48PuTlksVANCjR2jVKxBwz0oXERHpR+vCnSwFktSOf/Ro/V8z3XWy/HyIdMTtHbKXdJ3XEeqboXz9NTw33wJlyL+wVDoREWVPSynrDGeKgFBPp5Sl1fVkRCnuXF9Tpp8PkY4YTJH9pOrlkcjvhzL5bgZURESUnXQLd6lKWYfPFCUPNTL0dFLby0qN8HkkccP1gBBIzOHQXIpby88kVi4/HyKJMZgiexo7BmLrZgRf/wNE167RXalYkd8rP32AK10W8/kUDBiQB58v1TRKRCSZTE14E9PRtJ4p0rMJfexrvbQkNA8mnjHPpUlwtj+TWCwgRQ4l7Zkpoozy8kLpAsePp7xEAYCvv4bYsBH43lWmDY1OEwKorfVgxw4FtbUejBwZQIosESIiuWRTylrLmSI9zxCleq3w7pSYOiV0RirXynlay3uzgBQ5FIMpsrfGjeqvYzBliTVrFGzdGoqetm5VsGaNgmuv7Vg8hIhISmpLWWcqkKQogNd7+kyRnn2b1LzWWw0Q8+bqU5hJS3nvbH8+OgkWFOJUpwJ0THgkSq/9pLqsJgZTZG9C5RaH2utIV0IA8+Z5kJcnEAgoyMsTmDfPg2uu4e4UETlM+EyRcveUUPASEzAkPVOkZ98mOzS0z/bno4NTnbtg31ct2NfcjADT/SlLJ0+eVHUdgymytytGAPX16q4j08XuSgFAIKBg61Y4anfK51Mwc6YH9fVBjBrljHsiIo3CZ4qU2jogNrjxekOBQmzKnp5niOxyHimbn0+OTnXpil3/3If//d//xc6dO3Hq1CndXpvcoVOnTqquYzDlVk7pPj5iOESPHsDXXyetECQAoKQHMGK4yQOjxF2pCCftTvE8GBF1oPZMkZ5niOx0HknrmassBM7ohs/27sX777+P1atX4+DBgymrCBKl0vWMM1Rdx2DKjZzUfTwvD2LBU1Am392hol9kj0A89ZQ9A0WbS9yVinDS7hTPgxFRUmrOFP3LUIiSEqClJfliYDZniCw6j6SZljNXank8OBEM4uuWr9Hc3IyWlhYUFhTgjDPPREFBgTHfkxypV69eeP/99zNex2DKbZzYfXzsGIilv+6YNlBRoXvaAKkT2ZXyeASCwY4fEzwe++9O8TwYEWkWWdRsaUn6x1mfIbLgPJLsogUnhEBhYSF+9atfoaKiwtpBka20tbVh6dKlGa9jnyk3cXL38UjfqRXLEXxxEYIrlkNsaWIgZZH2dmDPHiQNpIDQ43v3hq6zq8iuVCSFMbTjFtqdIiJKKbKomaZghKZeULn0gCIizbgz5SZ2qPaTCyPTBigrhYXApk0BHDqU+prS0tB1duSG82BEZIA0i5pAOD39rLMgNm0EtKSkmXAeiYjiMZhyE7tU+yFHqKwMfTmRG86DEZEBMi1qAsBXX0Fs2ap9cdCIhUWnFK0iMgCDKTexU7UfIkm54TwYERnEjouaTipaRWQAnplyk3C1H5HiE55QFIiKCnOr/QQCwIaNwMqVoV/teF6LXEXP82A+n4IBA/Lg8zHqInIFuy1qpjrfFS5ahYZV+n0vfh4gm+LOlJvIVu2Hq11kQ3qdB2OPKiIXslMJ8wxFq4SiQKmbC1EzOvfPDfw8QDbGnSm3kaXaj9GrXVzhIgNVVgKDB6f+6tMn82sk61FFRA4XXtQE0CFLRLoS5uHzXanemRQhoDQ3h85S5cLM3S9iRoQBGEy5kdVlxI0u0d6wCsrQYfCMnwDPtOnwjJ8AZegw89+QGdBRCrHVAIHTVQCTLFQTkdPIsqiZiRnnu5zcskVCiRkRnHP0wTQ/t7KyjLiRJdplaUrMlAVKI7EaIKsAErmMHUqYm3G+y4jPA4EA8tatQ88tW3DOP/6BPzFiiEqWEcE5J3cMpsh8Rq12mZnfnY4sAR1JiT2qiAiA/L0RzTjfpffngYZVUOrq0KV5Py4AcAGAGwoKsPiCC7SO0DES5x7OOfphmh+Zz6jVLrPyu9NhygJlEFkZjA2kgMjuFM9OEZEkzDjfpefngchCZnP8Tlev9nbM/egjdHn7bQ0DdI7EuYdzjn4YTJH5jCrRLkP/DhkCOpJWbI+qZCI9qpiVQkRSMPp8l16fB9IsZHoACAA9fvEL1y5kJp7TjeB5XX0wmCLzqV3tArIr4CBD/w4ZAjqSlp49qoiITGFk0Sq9dr8yLGR6AOTv3w9l/focB2xPzIgwFs9MkTXCq11KbV18OVSvNxpIKUOHZVfAQYb+HTIEdCQtvXpUERGZysjzXZk+D6gJ2tQuUKYpduFUsRkRyRbyIhkRPDulHYMpsk6qakar39FWwEGGpsQZAjoAEB4P0NJy+oFAIJT2J2tFJ9JVZWXoi4iIwnKpbhgIAAfTrFDF8npzG6cNqcuIEGhvT76Q5/MpmDnTg/r6IEaNYj5gMgymyFqJq125VuTTY4UrZixZBzmxAR2QPOUgGIQy9Z5QUAiwhLoFODkQEUlGy+5XkjYkyQQBBL1e4MortY/PpnLJiEjsSzVyJHevkmEwRXLRo+eEHv07cukTNXYMxEu/gnLPj4BgsOM9IHQYVnngAaDl647PZwl1Q3FyICJygBRtSBIXMoPh3389bx56uDTrQ2tGBPtSqcMCFCQXvQo4RFa4xo0L/ZptIHX3lI651eEgBw2rMr9GSQmUJIFUhCIElHAgxRLq5ko2ORARkY2ky2JJ+P2hggI8PGAAvrnuOrNG5wiJFQBZ+S81BlMU+sCeTdU8I1ldwEGvPlEqg0KWUM+Nz6dgwIA8+HzqAiJODkREDpChel9EQ3U1JgwejPWlpaYMy0nYl0o9BlNu17AKytBh8IyfAM+06fCMnwBl6DB1uy9GMKoHlVp69YnSK9hjCfWUEtP11AREnByIiBxA5dx49IwzEGQed9bYlyo7DKbcTI90NkDfnS0zOq6no1eaYaagUO14JCqhnu0ukNGyTdfj5EBE5BAq58ZeX36Jy1pb4eEbfFbYlyo7DKbcSq90NiN2tozuuJ6OXmmGmYJCRYEoLrZuBy5LWnaBjB5Ptul6nByIiBxC5YLlyA0b8MLf/47/b8MGdHn7bfPGZ2OxfamSifSlsvpzgEwYTLmVHulsmXa2nn5G+26VkR3X09EzzTBDUCieXhB9zcTvAZjQEysLshVtyDZdz66Tg2y7gUSkUaoMDpnOLGfLyrGnW7BMcvlZJ06g57RpUFauNGFw9qauL1XoOgqRtjT6okWL8NRTT8Hv92PgwIFYuHAhhg0blvTaJUuW4Le//S3+9re/AQCGDBmCxx9/POX1hNzT2TL1gwLgeWpB9DFNvZOM7Lie5nvq2vg3Q5l23XpiGSh2FygQUKK7QFZ1S08cT0S6ceXatNAKLOFOruPUBuapWm2MuwnKyje0teDI9LMy+meZS/sQvaTqK4mOxZ08CAVZnlmzEPjXf3XGvyuD5NKXyq2kDKZee+01zJo1C4sXL0ZVVRXq6+sxevRo7Ny5E6VJKrKsXbsWt912G0aMGIHOnTvjySefxLXXXouPP/4YvXv3tuAObCDXdLZM/aASH7BT7yQ9G/8C6YNCPXpiGSx2VwqI7ALBsn4TieNRMy47Tg7s70GuIsOHcyOk6IWE/fuh/D8vdrxezVyZ6Wdl9M8y1T1ZMc/HzqHr1sNTX5824wb//CeU9eshrr7anPHZlNa+VBE+n4KZMz2orw9i1Cjnz1uKELIltgBVVVX4l3/5F7zwwgsAgGAwiMrKSvz4xz/G7NmzMz4/EAigR48eeOGFF3DnnXeq+p5tbW3o3r07vv50J4q6dctp/LYQCITONvn9cbsvEUJRQsHDlqbkH+xXroRn2vSsvmXG15SNDVZJg92Lsf3TXXjrrbfwxhtvID8/H//zP/+DoqIiXV5fCGDEiDx8+CE67AJddhmwcaO5uyWR8XzwQfJdJo9HYPBg88elt8Sfu1U/b7O0tbXhrLPOQmtrq27/dp3CFXNTzIfz2H/e0WwAOyzCJROZZ1Ok1Cc2l40+nm6uzPSzmvYjKC8uNu5nmemerJznVX4uCfzXf0HceqsJA3KnyPy1dauCoUOFrecttXOTdGem2tvbsW3bNlRXV0cf83g8qK6uRmNjo6rXOH78OE6ePImSkpKU15w4cQJtbW1xX66Sa9U8DVXmbNc7KZfGvw4hW9EGt+Rys4S7e7lubtKrGJKMMp1NTvV4qrky089KCCiLf2Xsz1Kv9iFGUPu5xOs1dhwuJ9sZazNIF0x9+eWXCAQCKCuL/5+irKwMfr9f1Wv87Gc/Q0VFRVxAlmj+/Pno3r179Ksyl/1Mu8qlal6GQg1pvfNO9s8h08lYtCGSrtfUdCrl16ZNAc3pejIUfGAJd3dz3dwk84fzXOXaJzDx+SqCMyUYNPZnqVf7ECOoKSDVpw/ElVeaPDD30FJp1wmkC6Zy9cQTT+DVV1/FypUr0blz55TXzZkzB62trdGvvXv3mjhKiWitmpdmZysT5aUl1jUFJtVy2QUyMiiprAQGD0791aePtteVpfy7bLuBZC7XzU0yfzjPVa59AhOfr9fPIJfX0at9iBHSfC4JRn595hlXZpmYxa1ZFdIVoOjZsyfy8vJw4ED8/+wHDhxAeeIOSoIFCxbgiSeewB//+Edceumlaa8tLCxEoUynza2ktWpeikINqfLAoxQFSt1ciJrR+r6p2eCMk51oLdpg1yp0MhR8iN0NTHUmzMpKimQ8181NMn84z1V4pyTl2WSkPzPVoQWHXj+DXF4n0z2lGrtZxo6BWPproLYutAsX9mVhIZTnnkOPceOsGZcLaKm06xTS7UwVFBRgyJAh8Pl80ceCwSB8Ph+GDx+e8nm//OUv8cgjj2D16tUYOnSoGUMloOPO1gM/zfgUQ9I2jGge7BLpdpG07ALZMV9altQEt5wJI4rSs7efbFT0Qkp8i0l7ZllFo1rh8Rj7s8z1vLUZrh+L43/7Gz56/nm8duONmNG/P+644gp8c9111o3JBdycVSFdMAUAs2bNwpIlS/Dyyy9j+/btmDZtGo4dO4ZJkyYBAO68807MmTMnev2TTz6Juro6LFu2DH379oXf74ff78fRo0etugV3iS3UcP8siKlT1D1Pr5SFTM2DGVClpHdqmyxBSbZkSU0w+kwYkanUNHW1w4fzXKQ6m1xRAfGf0zoWQ0h3ZjnTz0pRIH50T+o/h04/y1zOW5slLw9tgwfjrxdfjA+7d0fQqVsikpDxjLWZpEvzA4BbbrkFhw4dwty5c+H3+zFo0CCsXr06WpRiz5498HhOx4Evvvgi2tvbMWHChLjXmTdvHh566CEzh04AMHo08NKSzNfpkbKQqXmwUSmFDqF3aptsPanUkC01Idf+HkRSyKbXkd69/WSTpp+g+PmD2fUZVPGzEkOGGP+ztEGPRDKPuqwKgfZ2ufo56kXKPlNWcEUvD7Pk2sMqGxs2wjN+QsbLgiuWazsXJrlc+kzp3ctItp5Uar37roKxY1P/O2xoCFgSCLqp6SH7TKVmy7lJa98onntVL9PPys0/S48Hxwu74C8ffYRVq1Zh7fvvo2vXrli6bBkqKiqsHp1j7d2LjGestRaIsorauUnKnSmyuXAqgnL3lNDOUExAJQBACIjrx4be6HN9g3dyJSiD6b2LlPh6er2ukWQt+GDXIh5EOWULaC2G5EaZflb8WZLJ3JxVIeWZKcqRmjx1o6XKqfZ4oADwvLREnyIRTq4EZSC9exnZNV9a1oIPdiziQQTA2X2jiIiS4M6U02STp2602Jzqd94J9ZcKBuOvCReJ0HxoVfYyrZLSexfJrvnSWsu/GynxDJcbysqSgzBbgIhchsGUk8TkqcdJDFjMzKXOywMur4Iy48cAOvbUyLlIRLqUQidUgjKAEaltMgYlasmWmmDHIh5EUcwWICKXYTDlFGrz1INBKHPnmbtzFU77SEURAmhuDu1g6dg82DGVoHRm1C6SbEGJHclWWZAoa8wWICKXYTDlFCoDFkyZ2vEPc021y8SMtA+jyrTKWBEpZkxKv3OzXuG18y6S09mxiAdRHGYLEJnKTZVfZcVgyimyCERM78dkVtqH3tWLZDp/lmZMF5SV4ZNRo7J6Ge4iyUfWyoJEWWO2AJEpWPlVDqzm5xQqAxFLKiyF0z4SO7JHCEWBqKiQK+0jcv4scbcvvIuXUwVCncfU6eBB/Nsrr+D7LS3mj4l0I2tlQSJNxo6B2LoZwRXLEXxxEYIrlod6CzKQItKNbJVffT4FAwbkwedzV0THYMopMgQsqhlRYSmc9gGgw/ikTPvIcP4MAJS6ueaWnFcxpvv27IFHttrjDmDW5BBJv2xqOpXya9OmANMvyT4i2QLjxoV+leU9nsgBElucaG1toud4YnfJ3PRxhMGUU6QJWLJiVIWlVH2nvF7jzmppJWOflExjAlDe3o6BbW3mjckFzJ4cKiuBwYNTf9mtezwRQY7ej+Q4kV2pSLGi0Nla63anZNslMxPPTDlJqjx1FUypsGRUkQi9ydgnReX3Oos5YLpKNjmwAASRwxhZaEjGs7dke7JVfnV7f0QGU04zdgxEt25Qbr5F9VM0pdppnXz0LhJhBBn7pKj8Xl8VFBg8EPdw++RA5ApGBjtqez9SZjJW1rWQbJVf3d4fkWl+TvTVl9ldn22qXcMqKEOHwTN+AjzTpsMzfgKUocOsKcpgBBkLZmQaEwB/QQH+UlRk3pgcTrYUCiJp2TWNzchCQzKevbUrp3/myFJs5ddkIpVfzTqzlHh2K8LqM1xmYjDlRCp3MYL33Zt9hSUZq9zpTcaCGSrG9OzZZyPILRNdcHIgUsmuH3SNDnZkPHtrR274zJEl2Sq/Ji48RrhpAZLBlBOp3Vn56U+zq7DkppU2GQtmpBjTybIy/Pftt2NtSYnhQ3BL2VNODkQq2PmDrtHBjoxnb+3GTZ85siBT5VfZdsmswjNTTmRUB/rw5JOKIgTQ3BwqMCH7uSg1ZCyYkTAm0e9c7Cwtwydvvw3s2mXot3ZLc0A2zyVSIcMHXUMbwevB6GBHxrO3dmODzxw+n4KZMz2orw9i1CjzIobKytCX1dTtkgm0t8PRbT0YTDmVlg70mQ54unGlTcaCGTFjEt2LgU87BlFGvMG7pbIdJwciFWzwQTcto4OdcIYI/P64Bc0IUyro2p3knzncssCYTmSX7NCh1NeUljp/rmQw5WTZ7KyoqWjElTZbMOIN3k2V7Tg5EKkg+QfdjIwOdozKEHETEz5z5LLw6JYFxkxk2SWzEs9MOZ2aDvRq895lrHJHHbz/fifdG+e5rbIdm+cSZWD3xTUzCg3JePbWTgz+zJFLU/bEIkUsTuRuDKbcLpsDnpkmHyEgJt4OvPmmvcrjOogQwOOPd9H1DZ6V7YhszojS5U5YXDMj2Bk7BmLrZgRXLEfwxUXZV9B1M4MD3mQ7S9k+1y0LjDKToTAWgym3y7aiUarJp7gY6NEDnqcW2Ks8rsMcPjwMH36Yr+sbPCvbxZPhjZtINaNKl8vYQkILM4IdNRkilJxBAW8uO0tcYJRHLruLemIw5WRqViO15L0nTj4P/BQ4fBj4+uv459ihPK6DCAHs2TNF1zd4lj2NJ8sbN5EqRpcud0oaG4MduRkQ8Oays8QFRnnksruoJxagcCo1BSUA7XnvkcknEAitcspYHjdTdUKHaW2twrFj/Ts8HnqDh6bDsaxsF48Hjsk2zCpdLmMLCUor2LkLhB3/fq69VuWFCk6dPAkRFAgGg0h8h04sqBShprASW2fIQ6bCWAymnCiyGpm4bB5ejYxbMcy1opGs5XHVBpMOIQSwb9+PAASRbMNZ6xs8K9udJtMbN1FGZr43y9hCgjpSFLQXdsHegwdx7Ngxq0djqG+//RbrN6zHhx9+iGAwiE6dCtCtWzcA8YtisdQsPHKBUR6Jf4+5LBznisGU02S7Gplr+VYZy+NmE0w6RCCQhxMnypAqczeXN3iryp5a1QwxFZneuIkykvG9mazj8eDbgkL8dft2NDU1obW11eoRqfbZZ+dg1aoajBmzGuee+4Wq5xw9ehTbtm3D0aNH0bVLF9w7815069Yt550lLjDKIZfdRSNkHUz9/ve/x3/8x3/g888/h9frBQBMmjQJ27Ztw7p169C9e3fdB0lZ0LIaqaXBb4Rs5XHNSm2RTH5+ABdffBeE6In6+nqcccYZHa6x0xu8bM0QZXvjJspItvdmso4nD8c6FeAvH/0N77zzDhobG/HNN99YPSpVhAD++tclOHasF1asGIJLL12s6r02GAxCCIHi4mI89thjGDRoEAB9dpbYV8l6uewuGiHrYOrWW2/FE088gccffxwLFy7EvHnz8Mc//hGbNm1iICUDrauRWvPeZevyLmvaoQkKCw8iP78FAwcGUFRk9WhyI9vZJNneuIkyku29mSwjCgvx9eFW7N69Gzt37sTRo0fRubAQikf+GmQtLf8SPQt87Fh/HDt2BUpKtqh6bllZGR599FH07ds3+hh3luxPxnNrWQdTiqLgsccew4QJE1BeXo6FCxdi3bp16N27N/bu3Ys77rgDBw8eRH5+Purq6vCDH/zAiHFTKrmsRmrJe5ehy3tsoYmdn6p7DlNbpCXb2SQZ37iJMpLhvZnkoCgIBoMIBAIIBAJQFAWXDhyIWbNmWT2ytIQAbrrJG33vDVWVfQTLlu1X9V5bVlaGTp06dXicO0v2JuO5NU1npq6//nr0798fDz/8MN59911cfPHFoRfLz0d9fT0GDRoEv9+PIUOGYMyYMUlTjsggVqxG5pImmKskhSZUYWqLtGQ7myTjGzeRKla+NzuVQ6rElpeXo0+fPlYPI61331Xw17+e/tkGgwr++tdCbN9eKVUmgGzne51Oxt1FTcHU6tWrsWPHDgQCAZSVnf5Q6vV6o+eoysvL0bNnT7S0tDCYMpNVq5FWlMdNUWgi8rtkH32Z2iI3Gc8myfjGTaQaS5frx2VVYq0k41yQjGzne91Ctt3FrIOpDz74ADfffDOWLl2K3/zmN6irq8Prr7/e4bpt27YhEAigUqa7dQurViPNLI+brtAEQgGVQHxAxdQW+cl6Nkm2N26irLB0ee5cWCXWSrLOBYlkO99L1sgqmNq9ezfGjh2LBx98ELfddhv69euH4cOH44MPPsDgwYOj17W0tODOO+/EkiVLdB8wqeT01chMhSaSPcjUFqnxbBIRScmlVWKtYpe5QLbzvWQd1aVcWlpaUFNTgxtvvBGzZ88GAFRVVeG6667Dgw8+GL3uxIkTuOmmmzB79myMGMGVMEtFViPHjQv96qQ3eZUFJIL33Yvgi4sQXLEcYksTAymJqTubFLqOiMg04cW7VJ+PFSGgNDeHzlJRzuwyF0R2pSJpiKFds9DuFOnL51MwYEAefD45f7aqd6ZKSkqwY8eODo83NDRE/1sIgbvuugsjR47EHXfckdPAFi1ahKeeegp+vx8DBw7EwoULMWzYsJTXv/7666irq8Pu3btx3nnn4cknn8SYMfzg7FhqC0hceRXTW2yCZ5OISEpsgGwqO8wFdjnT5QR2OJemqQBFKhs2bMBrr72GSy+9FG+88QYA4L/+678wYMCArF7ntddew6xZs7B48WJUVVWhvr4eo0ePxs6dO1FaWtrh+o0bN+K2227D/Pnzcf311+OVV17BTTfdhA8++ACXXHKJHrdGVkhXNYk9VBxJy9kkVlIiIkOxAbLpZD+napczXU5gh3NpunZsu/LKKxEMBvHnP/85+pVtIAUAzzzzDKZMmYJJkyahf//+WLx4Mbp27Yply5Ylvf65555DTU0NHnjgAVx00UV45JFHMHjwYLzwwgu53hJZpWEVlKHD4Bk/AZ5p0+EZPwHK0GFAw6rQn4erFgKnC0tEsNCEeySuWCWJq4mIchNevEucayKEokBUVHDxziViz3QlEznTxfkod7E7gMDpnT/ZfrbStb9ub2/Htm3bUF1dHX3M4/GguroajY2NSZ/T2NgYdz0AjB49OuX1QOhsV1tbW9wXSSJSNSmxwES4alI0oApXLUR5efx1Xi8rK7lEshUrIjvj3CQhLt5lRcbzLXqOyS5nupzALufSpAumvvzyyw79q4BQJ2u/35/0OX6/P6vrAWD+/Pno3r179Isl3CWRoWoSACh1c0MpgEAooNq6GcEVy11daGLXrr74619fxeHDQ60eimnssmJFlA3OTZLi4p0qMmYL6D2myJmupqZTKb82bQrofqZLxiDVSIlzfISMc72uZ6bsZM6cOZg1a1b0921tbZy0ZJCp5LkQQHNzqOR7pLCEy3uoCAG8887V+PbbCvzjH/dI9QZjpMScdeaqkxNwbpKY01uO6EDG8y1GjMnsM112KMKgNzudS5NuZ6pnz57Iy8vDgQPxVXEOHDiA8sQVobDy8vKsrgeAwsJCFBUVxX2RxQIBYN16ddeyalLUxo1nYt++CgDAsWP98f77nUz73latlNlpxYooG5ybJOfkliM5kjFbQMYxaeG2lHa7nUuTLpgqKCjAkCFD4PP5oo8Fg0H4fD4MHz486XOGDx8edz0ArFmzJuX1JKFIwYn6enXXs2oSgNAbzgsvlEFRguFHAnj88S6mvMFYmc6RmEcdIWs+NRGR08l4vkXGMWXLKQFhNux2Lk26YAoAZs2ahSVLluDll1/G9u3bMW3aNBw7dgyTJk0CANx5552YM2dO9Pp7770Xq1evxtNPP40dO3bgoYcewtatWzFjxgyrboGykargRBKsmhRvzR89+PjjrhAi8r9yHj78MN+UicKqlTK7rVgRETmdjNkCMo5JCzsFhHplq1h1Lk0rKc9M3XLLLTh06BDmzp0Lv9+PQYMGYfXq1dEiE3v27IHHczoOHDFiBF555RXU1tbiwQcfxHnnnYc33niDPabsIE3BiUSsmhRPCGDeLzrB4xFxqzdmNA1MbFhoZqNCdStWAu3tbPBLRGQGGc+3yDimbNmpObDe57pk7zUWS8pgCgBmzJiRcmdp7dq1HR77wQ9+gB/84AcGj4p0l6HgRByvNxRIsWoSAODdtQXYuq3j5rIZE4WVxR8iK1aHDqW+prSUgRQRkRliswWSLXJFsgXM/OAv45i0sFNAKGPxEbNImeZHLqKykETwvntdWfI8FSGAuU92syTVTYbUicpKYPDg1F99+hg/BiIiqQUCwIaNUF5fjs6bNkEJBjM/RwMZz7fIOKZs5ZrSbmaBKDee64ol7c4UuYTaQhJXXsXUvhjt7cDefXmWpLrZaaWMiMiVGlZBqa2LZn6UAxjfowf855+PlTpvxciYLSDjmLKVS0q72aXU3d6qhMEUWevyKgivF/D7o015YwlFAbxeFpxIUFgINK3+EgdOdMcXe/Zi3bp1WLt2LfLy8lBfX48zzjjDkInCKakTRESOFSnqlDCndv36azzQ1ISW88/X/VvKeL5FxjFlI5eA0MyUOzud6zIKgymyVl4exKOPQLl7CoSixAVULDiRXmXvIHp3F+hy5rf4/PMDOOOMncjPz8fAgQEY1ZqGxR+IiCSWpqiTAiAIYObu3fh/DUr5I31pCQjNLhDFbBUGUySDsWMgfr0kNAHEFqNgwQnpOCF1gojIsTIUdfIAKG9vR+Xu3aYNicxlZsods1VCGEyRHMaOgagZDbGpKVSUorQslNrHHSnp2D11gojIsVQWdTrzyBGDB0JWMDvljtkqIQymSB55ecAVI6weBRGRswUCgJ4LV3q/HmmnsqjT0W7dDB4IWcHslDtmq4QwmCJyOJ9PwcyZHtTXBzFqlLPzlokog4QqbwAgvF6IRx/RllKt9+slYqCWnQxFnYIADhYUYG/fvjl/K84tcrEq5Y7ZKuwzReRoieVR3dLzIVtm9uMgskykylvimRq/P/R4wyprXy/Z6w8dBs/4CfBMmw7P+AlQhg4D/uctYMNGYOXK0K+BQG7fx0nCRZ2A00WcIgRCRSjq+/aF8OT28Y9zi3yc0FvLrrgzReRgbu5IrpbZ/TiILJGuypsQoWqqdXMhakar2/nR+/USpSjvjf37oUyZGvc9dd0Jc4IURZ2Ol5Rg0Xnn4X8VBTfk+C04t8iHKXfW4c4UOUe42ztXK0Pc3pFcrWQfCozEXTCyRLjKW6p/dYoQUJqbQyl1VrxerAzlvTvQayfMScaOgdi6GcEVyxFcuhT+3/8eKxYsQFPv3jm/NOcWeVVWAoMHp/7q08fqEToTgylyhlTpIC6eXCNBQqSiT+gAqvHBgkwyBS5mfyhgagxZRmWVN8uui5UpUEv8ffh/JKVurusX0eKEizqJH0zAt5dfnnNqX4SZcwsXn8gOGEyR/Rmdt29DiUFChJtWENUELmYHnGbvghFFqazyZtl1sTQEYDnthJFqZs4tXHxSj0GntRhMkb1lyNsH3LlamRgkRLhpdypT4GJ2wMnUGLJUuMpbYlGCCKEoEBUVoWp5VrxeLC0BWISWnTBSzcy5hYtP6sgedLoh0GMwRfZmZN6+TcWWR00mUh5VtjdcPakJXMwOOJl2SZZKV+Ut/HvxyMPqi0Xo/XqxMgRqaeUSiKXDM7mmzi1cfFJP5qBT9kBPLwymyN6MzNu3KZZHzRy4mB1wMu2SpBCu8oby8vjHvd7Q49lWw9P79SIylPdOJqedsEx4JheAuXOLVYtPdttFkT3olDnQ0xNLo5O9GZm3bwUdGlS6vTxq7OQSu+sUmWSuuSag8kOBQHu7Pj8ns7vSE6U0dgxEzWgIvRrh6v16sa+bpLw3cLpfUvT3ue6EpZOqRHv4TG5OQaOFdu3qiwED8rJquGvW3KLmPdyI9hV2bJOROLfINKck/j0a/fdnJQZTZG8Zur0LRQG8XmNWK/XWsApKbR2UmA8OWvunuLkjudrAxayA06qu9EQphau8Sft6EckCtZYWKHPnxQdYXm8okNI7qMm2l5YOi2Fax5nN9xUCePfd72PfvsxBg8+nYOZMTzToMmNusWrxyW69s6wKOtWSOdDTG4MpsrdwOohy95TQxBYTUBm6Wqk3k1c/EydIp8gmcDEr4DR7F4zIUZIEamLMdfrvhCUTPpObiiIE0NwcGsvhw7othmVFwyJca2sV9u2rAJA+aLBip8aqxSc77qLInPEge6CnN56ZIvszKm/fLCZXJHTygVAZz4tFUmOamk6l/Nq0KcBAikitSIA1blzoV6MWy9SetX3nHWvac2hoCyIEsHfvVChKEED6MzZWnHex6j3cbgWCZC805baKwtyZImcwKm/fDNmsfuqQSmO3VIZsyHpezM1pl0S2pfKsrbJ8hfpUQL1kk4IY4+DBQTh2rH/MyyTfxbBqp8aK93A77qLInPHgxtR2BlPkHEbl7RvNxIqEdkxlyJbTAhenpmSSi1h1lihXas7klpRA+eqrlC+h92JYVDaLcNdcExqvAD7++FYAAQCnf/7J5gErz7uY/R4uc7pcKrIuHAJyB3pGYTBFZDUTKxK66UCoHWQKlOxYXYoojo6FdUyn5kzu+H+D8tKSzK+ld3sODYtwf/tbBQ4fPq/DJYnzgB13arSy8y6KrAuHMgd6RuGZKSKrZWhQqVf/FPY6kouas2tu6dFBDpHY2PZ/3rLmLJGeMp3JHT06+fMS6d2eI8tFOCGA//7vywAEk14We8bGTeddZDxn6wSVlcDgwam/+vSxeoT64s4UkdV0rki4dm0+fv7zjv1D7JjK4GSZzq65ISWTHCTZDpTHY/5ZIiOkO5MbCFjTniPLtiDt7cBXX52BVGvokdSrEyfsu1OjhRt3UUh/3JkieSSuaupUvc4WdKpIKATw6KNdO+x2yF75x23UdK23W3UpcrEUVeWUYLBDIBX9MyGgNDeHzlLFknUeSFVBMLwYBqBDdoGh7Tmy/L6FhcC8eW/h//yfn2LAgB/iP/9zadKqoorivp0a2XdRfD4FAwbkwefje7+suDNFcrBzXr1edKhIePjwMGzfHvrfOna3w40HQq2ipmBEprNrbjqzQDaXpqqcKrFnf+w6D4QXw5TaOnOaCWv8vmeddRw9enyBb75pRu/eF2Dw4OQvy50aefDcrD0wmCLrmdywVmo5VCQUAtizZ0rStDCmMphDzcSnJlBiSibZRoaqchlFzv7YfR6wqj2HAd9X1sIGbuTkViZOwmCKrJVNrwzZ8+ot1tpalbZ/CCdI46mZ+DIFSu++q+Chh9xzZoFsTmOVurgzPU6ZB6xqz2HXtiCUFs/N2gfPTLmNbPno4VXNrPPqKY4QwL59P0Kof8hprNRnHjXnoNScXZs71+O6MwtkYxqq1HU408N5gCQh0/kknpu1DwZTbtKwCsrQYfCMnwDPtOnwjJ8AZegwa8vTmtiw1sk+/fSc8K5U/Kot33zNo2biU3N2bd8+4E9/CnQ4HJ54UJwpmSSFTK0dEK7qFyuxsA7nAZKAmnYVZo7FDq1MZAo+rcQ0P7eQNR/dxIa1ThXqC/I9hPqHdFwfYVqY8dQWjFB7ds3q6lFEqqlp7fCrxRAlJanP9HAeIAnIdD7JDudmWRzjNAZTbiBzPnqWvTI0CQRC6SFmHgo2Usz9KP3OxcnicrS2FiFT/xBW6jNONhMfz66R4+Razc6MeYAoDZnOJ8Wmg8t8blam4NNq0qX5tbS0YOLEiSgqKkJxcTEmT56Mo0ePpr3+xz/+MS644AJ06dIFZ599Nn7yk5+gtbXVxFFLTuZ8dKN7dMiY2piLhPvJG12DAf86GgtGzcHFF9+JSy+dhPfea2VamInYw4sIoYBq62YEVyxH8MVFCK5YDrGlSV3Gg1W9mojCZDqfpK6VibXnZtWcEXYT6XamJk6ciP3792PNmjU4efIkJk2ahKlTp+KVV15Jen1zczOam5uxYMEC9O/fH//4xz/wox/9CM3NzVi+fLnJo5eU7PnoRvXokDW1UasU99Pp4EFM/f9fwOff/S7Wl5Zi4MAAioosGqMLsYcXUVguVeWs6tVEridbXz87tDLJ1CvRbaQKprZv347Vq1djy5YtGDp0KABg4cKFGDNmDBYsWICKiooOz7nkkkuwYsWK6O/PPfdcPPbYY/j3f/93nDp1Cvn5Ut2iNeyQj653rwyZUxu1yHQ/AO7bswcbe/WyYnSuZoeJj8gWrOrVlMhpqeGUloznk2ROB5ct+JSBVJFGY2MjiouLo4EUAFRXV8Pj8aCpqQnjxo1T9Tqtra0oKipKG0idOHECJ06ciP6+ra1N+8BlZ5d8dD17ZWRoJKkIATQ3hyZtO/TnyHQ/AMrb2zHQyf+OJSbzxEf24qq5KRk184CRwU7DKii1dXHvt8LrDaUhcnfMcexyPkkmMgafVpPqzJTf70dpaWncY/n5+SgpKYHf71f1Gl9++SUeeeQRTJ06Ne118+fPR/fu3aNflU7+JOTGfHTZUxuzpXKcZ7H5EEu1kq25am7SwshzsJFU6sSFq3BquG3P2lJKMp1PkmnuSjUWnhFOzpRgavbs2VAUJe3Xjh07cv4+bW1tGDt2LPr374+HHnoo7bVz5sxBa2tr9Gvv3r05f3+phfPRUV4e/3hivw+nsENqYzZUjvOrggKDByI3mfqEEGnhurkpG0YGOxlSqQFAqZtrfaN70lUkTdvqvn4yzV3pxiJT8CkTU9L87r//ftx1111pr+nXrx/Ky8tx8ODBuMdPnTqFlpYWlCcGAQmOHDmCmpoadOvWDStXrkSnTp3SXl9YWIhCtx1ikCUf3Qx2SW1UK9P9ADhQUIC/uLzyhB6lWn0+BTNnelBfH8SoUYzGyFyunJvUMPocrNNSw0k1GdK0ZSoznm4sPCOcnCnBVK9evdBLxcH44cOH4/Dhw9i2bRuGDBkCAHjvvfcQDAZRVZX6Q29bWxtGjx6NwsJCvPnmm+jcubNuY3ccPc8lyUxNI0k7pTZmuh8h8OzZZyNog6Ruo4IVPfqEsAkhkaSMDnaclhpOtiFjj6t0Y5Eh+JSNVGemLrroItTU1GDKlCnYvHkzNmzYgBkzZuDWW2+NVvLbt28fLrzwQmzevBlAKJC69tprcezYMSxduhRtbW3w+/3w+/0IcDve3ZyW2pjifk6WleG/b78da0tKLBqYekamMujRJyTZihwRScDoYMdpqeFkGzL1uJJpLHYiVTU/APjd736HGTNmYNSoUfB4PBg/fjyef/756J+fPHkSO3fuxPHjxwEAH3zwAZqaQs1mv/vd78a91hdffIG+ffuaNnapubXUq9NSGxPuR/Q7FztLy/DJ228Du3ZZPbqMjEpl0KNUq0yrg0SUwOhgR0UqNc46C/DvBzZstPc8QtKQqcy4TGOxG+mCqZKSkpQNegGgb9++EDFvdN///vfjfk9JuL3Uq9NSG2PuR3QvBj6VP4gCjA1W9CjVyiaERBIz+hxsulTq8K/KV19Bmf7j0GNumkPJMDKVGZdpLHYjVZofGYClXkkSRqUP6FGqNTbQixUJ+LheQ2QxM1p8pEoNT4ZzKOVIpjLjMo3FjhhMORlLvZIkjAxW9CjVmhjoRZidLy5TnxEi6ZhxDnbsGIitmxFcsRzBRQshzjoLADiHku5kKjMu01jsSLo0P9IRS72SJIxMH8i1VGvsilyyiSSyImd0vjgrCRKpYMY52Egq9YaNUL76KuVlnEMpFzKVGZdpLHbEYMrJWOqVJGBGsJJLqVZ1K3IC7e3GTiQy9RkhkppZ52A5h5LBZCozLtNY7IbBlJOx1CtJQJZgJRUZVuRYSZBIQpxDbY9N2MkMDKaczOjqR0QqyBCsZGL1ihwrCVJGeXksxW22K0ZAVHiB/Wnm0ApvaJdM5r8bl67IMHWazMJgysnSlXrVq/oRkQpWBysyY28PUuMwPAiyZpS58jzo9NjjOPM//iPlHHr00cdwMq+TVSNURXx7AkePHsXXX38d7dHpBkydJrMwmHK6cPUjpbYuvjy61xsKpNgjw3hubZhMqrC3B6mxYeNGdO3a1ephuE/37qh44AEMWLoUXWOKUXxTUoKPJk9Gc/fuwLp1Fg5QnZ07d2Lt2rX4+uuv0bmwM6644gqrh2Qopk6TmRhMuYEZ1Y8oObc3TKa0ZKkkSPJ744030KmT3DsgTqZUV+PCQ4dQ/M03ONylC3b06gWxdy+wd6/VQ1PlwIEDOHr0KLp26YKfzZ6dUzBlh3NITJ0mMzGYcguzqh/RaZGGyYm59uFmj7r1RSHbkr04B8njiy++QH4+p2wrfRb5j2PHQl82ogAoLi7GY489hkGDBml+HTucQ8o2ddoOwSHJje/MRFqlS9/L0DBZKAqUurkQNaO5Q+hidijOQXK44oorUMh/CKRRt27dMHHiRJx99tk5vY4dziFlkzpth+CQ5MdgikiLTOl7bJhMKrE4B6kxb948FBUVWT0McjE7nEPKNnXaDsEhyY+lgSgkEAA2bARWrgz9GghYPSJ5RdL3EoOlcPoeGlax2SMRETlKJPCIpM6FdnpCAYgs1KVOh66LDQ6B02mASargE6XFnSlikYRsqE3fq39W3eux2SMREUnOLi0cskmdfvddFqkgfTCYcjsWSciO2vQ9RWHDZCIicgQ7tXBQkzptl+CQ7IFpfm6WYZcFAJS6uUz5i6U2Le/LQ6GdPZxu7hjBhslERGQXseeQkomcQ7JTelxiymKEjKmLJD8GU24W3mVJ9ZahCAGluTlUsY5C1KbllZZFGyajvDz+z7xe7vjZgM+nYMCAPPh8nFSJSH92eY/J5hySHTgxOCRrMc3PzVgkIXuXV2WXvseGybbEcrlEZCQ7vcc4rYUD+/uR3hhMuVk2uywUkpcH8egjofNkihIXUKVM32PDZNthuVwiMpLd3mOc1MLBacEhWY/BlJtlu8tCIeH0PaW2Lr48utcbCqSYvmdrduilQkT2xfcY6zkpOCTr8cyUm4V3WQAWScja2DEQWzcjuGI5gi8uQnDFcogtTQykHMAOvVSIyL74HnOaXc6NEaXDYMrtWCRBu0j63rhxoV8ZdNpeYhPHiMRmjvwAQERaqH2PcYPEc2NuundyFgZTxF0WojA15XL5AYCItGJJ7tOSnRsjsiOemaIQpxVJCARCJd1ZQY9Uii2Xm6zK0+lyuUFbHRwnIjmofY9xw9kpnhsjJ+HOFDlPwyooQ4fBM34CPNOmwzN+ApShw4CGVVaPjCSmplzunj3A3LmnU3TcmJpDRNo4rV9TLnhujJyEO1PkLA2roNw9BR0+3fr9oXLmPAdGKagpl7t9u4K77jq9wxn6AADuThFRRizJHZK4KxXB3SmyKwZT5AyBALBhI5T7fwoIgcT3YUWIUF+ourkQNaOZ8kdJpSuXKwQwfTo/ABCRdizJHX9WKpadF6d8PgUzZ3pQXx/EqFH2Gjvljml+ZH+RtL6bb4Fy+HCHQCpCEQJKc3PoLBVRlnhwnIgoN7HnxpI5fTbV5IHlgEWJiMEU2VskrS+2eW4mBw8YNx41wrtoWLky9GsgYO14KCMnfgAgIjKbE8+NsSohMc2P7CsQgFJblzStL63SMqNGlFnDKii1dVBigj/h9YaaJ8ee5WI1Qqmo+wAg0N7u/PMORERaOe3cGKsSEsBgiuxsU1NcUJKJUBTA6w0FJlZQWxxDbcClM+Z8p+a0DwBERFZx0rmxxPNfdj73RdoxmCL7yiJdT4SXiMQjD1uzw5NmFy2uOEYwCGXqPaZXI0zM+R45kqtqiZz0AYCIyE5kXOxjVUKK4Jkpsq9s0vW8XmvLood30TIVx1Bmz0kZcAGAUjfXkDNWzPkmIiIZyVrggUWJKILBlJM5vdDB5VWh9LcUSz8CgCguRvD1P0BsabK2v5TKXTTlq69Mr0YYu7oGsBEtEZGefD4FAwbkwefjh2stjFzs0/p3w6JEFEu6YKqlpQUTJ05EUVERiouLMXnyZBw9elTVc4UQuO6666AoCt544w1jByq7SLnw8RPgmTYdnvEToAwdBjSssnpk+snLC50jAjoEVEJRAEWBeHoBcNWV1hdv0LPohc7VCNmJnojIGLLuqtiFkYt9ufzdOLEqIWknXTA1ceJEfPzxx1izZg3eeust/OlPf8LUqVNVPbe+vh4KE1RTlwsPn7txVEA1dkwofa+8PP5xq9P6EmXaRVMUiLPOUvdaOgZmiRNVBHeniIhyxxTq3Bi52JfL302kKFFT06mUX5s2BViUyCWkCqa2b9+O1atX49e//jWqqqpw5ZVXYuHChXj11VfR3Nyc9rl//vOf8fTTT2PZsmWqvteJEyfQ1tYW9+UIGQodAMadu7HM2DEQWzcjuGI5gi8uQnDFcuvT+hJl2kUDIJ6YnzngqqjQtRohc76J5OLYucmFmEKdGyMX+7T83SSmBFZWAoMHp/7q00f7+MhepAqmGhsbUVxcjKFDh0Yfq66uhsfjQVNT6nMix48fx+23345FixahPHGHIoX58+eje/fu0a9Kp5TpUlnoQO9zN5bLywOuGAGMGxf61eq0vmQy7aLdcH3mgEvHaoSy53zznAG5kWPnJhdiCnVujFzsy/bvhumalI5UwZTf70dpaWncY/n5+SgpKYHf70/5vPvuuw8jRozAjTfeqPp7zZkzB62trdGvvXv3ah63VNSep9H53A2plGkXzcS0RZlzvjlxkVs5dm5yGaZQ58bIxT4tfzdM16R0TOkzNXv2bDz55JNpr9m+fbum137zzTfx3nvv4cMPP8zqeYWFhSh0YjKr2vM0ehZEoOxEdtFSGTsGomY0xKamUNBbWhZK7dN5t03mRrTJJi42QCQ3cOzc5DKJzVwj2NRVHXWLfQLt7dnPUdn+3ST2k2IfKUpkSjB1//3346677kp7Tb9+/VBeXo6DBw/GPX7q1Cm0tLSkTN9777338Nlnn6G4uDju8fHjx+Oqq67C2rVrcxi5DYULHcDvj56RiiUUBfB6dT13k5NAIJRyaGDQYEuZAi6dyNiIlhMXEdlZ7K5KsmAgsqvC97TUjFrs0/J3kxh8MSCmRKYEU7169UKvXr0yXjd8+HAcPnwY27Ztw5AhQwCEgqVgMIiqquQf/mfPno2777477rEBAwbg2WefxQ033JD74O0mXOhAuXsKhKLEBVRGnLvJScMqKLV1UGKqDgqvN3RuyOjiEQzipOX2icvnUzBzpgf19UGMGuX8+yVyGiN3VdzEiMW+bP9uEhf3IrjIR7FMCabUuuiii1BTU4MpU6Zg8eLFOHnyJGbMmIFbb70VFRUVAIB9+/Zh1KhR+O1vf4thw4ahvLw86a7V2WefjXPOOcfsW5BD+NyNUlsXXx7d6w0FUjJUuYuUb0/cPQuXbze0rLmVQRyl5faJK/Gs2MiRzr5fIieSOYXa7bL9u1GbEshFMHeTKpgCgN/97neYMWMGRo0aBY/Hg/Hjx+P555+P/vnJkyexc+dOHD9+3MJR2oBJ5240yVC+XSgKlLq5EDWj9R+vlUEcZeT2cwY8K0bkDFakUPMDvTpq/27UpgRWVwe4COZy0gVTJSUleOWVV1L+ed++fSEylG/J9OeuYdK5m6yFy7enoggBNDeHAkE9x69XEMcUQUO4/ZwBz4oRkVbc1daf2pTAt9/mIpjbSRdMkQsYUb5dTYCjRxDHFEHDuP2cgdvPihGRdtzV1p+alMBevYCbb87jIpjLMZgi8+ldvl1tgJNrEMcUQQDGpZJYfc7AyhQZt58VIyLtuKttnEwpge++y0UwkqxpL7lEuHy7SPEuLxQFoqJCXfn2SICTuOMUDnDQsOr0Y7kEcRlSBAFAqZsb2iFzMKOb6VZWAoMHp/7q00ff7xdhdZPgyKpybCAFRCZmNogkotQS3z9kfd/w+RQMGJAHn0+ucWnFxswUwWCKzBcu3w6gQ0CVVfn2bAOcXIK4cIpgqilAEQJKc3Mo1dDBnNoF3sr7ij0rlkzkrBgnZiJKZJcP9FYvWBmBi2AUwWCKrBEu347EsvZer/p0uWwDnFyCOCPOedlM4qQt22StldX3pe6sWOg6IqJYdvlA77SFOC6CUSyemSLr5Fq+XUuAo7UHl97nvGzIqQUSrL4vq8+KEZE92aUCqhPPdLm9YBLFYzBF1sqlfPvnX6i7LjHA0RLEhVME4fdHUwhjCUUBvF5157xsyKkFEmS5Lyt60hCRvdnlA73VC1ZG4CIYxWIwRfowu/dSwyooC56GAFKm+aUNcLIN4sIpgsrdU0L9qGICqqzOedmUU5vpOvW+iMj57PCBXpYFKyNwEYwiGExR7szuvZSm8ET0+wOAEPoGOFpTBG3OLqkk2XLqfRGRe8j+gZ4LVuQGDKYoN1b0XsrQfBcI7VYFH/ip/t8713NeNmSXVJJsOfW+iIhkwAUrcgsGU6RdhtLkQlGg1M2FqBmtb7ChtvBEv3P0+56xcjnnZUN2SCXRwqn3RUQkA9kWrKxszE7OxmCKtMuwQ6QIATQ3h3Zx9Aw+WFnPdLKnkmjl1PsiIrJa4oLV5s0Knn7ag/vvD2LYsFAwY9aCVWKfq5EjuRtG+mGfKdLOqt5LuTTfJSIiIlNUVgKDBwOXXQa8/LIHu3crePllDy67LPR4nz7mjMNpfa5ILgymSDurdohyab5LREREprIymLG6MTs5H4Mp0s7KHaJwZT2Ul8c/7vUaU/SCiIiIsmZ1MBMJ5CKl2UOVBLk7RfphMEXaWb1DNHYMxNbNCK5YjuCLixBcsRxiSxMDKSIiIklYGcwkBnIR3J0iPTGYotxYvUMUqaw3blzoV6b2ERERScHqYCYxkIvg7hTpidX8KHcu7L1ERERE6VnZtJd9rsgsDKZIHy7rvURERESpWR3MyNbnipyLwRQRERER6crqYIaN2cksDKaIiIiISFcyBDNszE5mYDBFRERERLpjMENuwGp+RERERC7j8ykYMCAPPh+rLxDlgsEUERERkYsIAdTWerBjh4LaWvZbIsoFgykiIiIiF4ktWc5+S0S5YTBFRERE5BKJjXTNaqBL5FQMpoiIiIhcIrIrFQiEdqNCDXS5O0WkFYMpIiIiIhdI3JWK4O4UkXYMpoiIiIhcIHFXKoK7U0TaMZgiIiIicrjIrpTHk3z7yePh7hSRFgymiIiIiByuvR3YswcIBpPvPgWDCvbuDV1HROrlWz0AIiIiIjJWYSGwaVMAhw6lvqa0NHQdEanHYIqIiIjIBSorQ19EpB/p0vxaWlowceJEFBUVobi4GJMnT8bRo0czPq+xsREjR47EGWecgaKiInzve9/DN998Y8KIiYiIiIjIjaQLpiZOnIiPP/4Ya9aswVtvvYU//elPmDp1atrnNDY2oqamBtdeey02b96MLVu2YMaMGfB4pLs9IiIiIiJyCKnS/LZv347Vq1djy5YtGDp0KABg4cKFGDNmDBYsWICKioqkz7vvvvvwk5/8BLNnz44+dsEFF6T9XidOnMCJEyeiv29ra9PhDoiIiLTj3EREZC9Sbd00NjaiuLg4GkgBQHV1NTweD5qampI+5+DBg2hqakJpaSlGjBiBsrIyXH311Vi/fn3a7zV//nx07949+lXJJGIiIrIY5yYiInuRKpjy+/0oLS2Neyw/Px8lJSXw+/1Jn/P5558DAB566CFMmTIFq1evxuDBgzFq1Ch8+umnKb/XnDlz0NraGv3au3evfjdCRESkAecmIiJ7MSWYmj17NhRFSfu1Y8cOTa8dDAYBAPfccw8mTZqEyy67DM8++ywuuOACLFu2LOXzCgsLUVRUFPdFRERkJc5NRET2YsqZqfvvvx933XVX2mv69euH8vJyHDx4MO7xU6dOoaWlBeXl5Umf5/V6AQD9+/ePe/yiiy7Cnj17tA+aiIiIiHTj8ymYOdOD+vogRo0SVg+HSBemBFO9evVCr169Ml43fPhwHD58GNu2bcOQIUMAAO+99x6CwSCqqqqSPqdv376oqKjAzp074x7/5JNPcN111+U+eCIiIiLKiRBAba0HO3YoqK31YOTIABTF6lER5U6qM1MXXXQRampqMGXKFGzevBkbNmzAjBkzcOutt0Yr+e3btw8XXnghNm/eDABQFAUPPPAAnn/+eSxfvhy7du1CXV0dduzYgcmTJ1t5O0REREQEYM0aBVu3hqKnrVsVrFnDSIqcQarS6ADwu9/9DjNmzMCoUaPg8Xgwfvx4PP/889E/P3nyJHbu3Injx49HH5s5cya+/fZb3HfffWhpacHAgQOxZs0anHvuuVbcAhERERGFCQHMm+dBXp5AIKAgL09g3jwPrrmGu1Nkf9IFUyUlJXjllVdS/nnfvn0hRMc829mzZ8f1mSIiIiIi68XuSgFAIKBg69bQ49dey7NTZG9SpfkRERERkXPE7krFiuxOJVkfJ7IVBlNEREREZIjIrlQgEJ/PF9qd4tkpsj8GU0RERESku8iulMeTfPvJ4+HuFNkfgykiIiIi0l17O7BnDxAMJt99CgYV7N0buo7IrqQrQEFERERE9ldYCGzaFMChQ6mvKS0NXUdkVwymiIiIiMgQlZWhLyKnYpofERERERGRBgymiIiIiIiINGAwRUREREREpAGDKSIiIiIiIg0YTBEREREREWnAYIqIiIiIiEgDBlNEREREREQasM9UmBACANB25KjFIyFSL6h4cPToUXz77bcIBAIAgLa2NotHRZSdyL/ZyPswnRadm/j/NRGRqdTOTQymwo4cOQIA+M7gIRaPhCg355xzjtVDINLkyJEj6N69u9XDkEpkbuL/10RE1sg0NymCS4EAgGAwiObmZnTr1g2KohjyPdra2lBZWYm9e/eiqKjIkO9hBqfcB+Cce+F9yMcp92LGfQghcOTIEVRUVMDjYfZ5LM5N6vE+5OOUe+F9yEemuYk7U2Eejwd9+vQx5XsVFRXZ/h8x4Jz7AJxzL7wP+TjlXoy+D+5IJce5KXu8D/k45V54H/KRYW7iEiAREREREZEGDKaIiIiIiIg0YDBlosLCQsybNw+FhYVWDyUnTrkPwDn3wvuQj1PuxSn3Qak55e+Y9yEfp9wL70M+Mt0LC1AQERERERFpwJ0pIiIiIiIiDRhMERERERERacBgioiIiIiISAMGU0RERERERBowmCIiIiIiItKAwZTBWlpaMHHiRBQVFaG4uBiTJ0/G0aNHMz6vsbERI0eOxBlnnIGioiJ873vfwzfffGPCiJPTeh8AIITAddddB0VR8MYbbxg70AyyvY+Wlhb8+Mc/xgUXXIAuXbrg7LPPxk9+8hO0traaOOqQRYsWoW/fvujcuTOqqqqwefPmtNe//vrruPDCC9G5c2cMGDAAq1atMmmk6WVzH0uWLMFVV12FHj16oEePHqiurs5432bK9u8k4tVXX4WiKLjpppuMHaBK2d7H4cOHMX36dHi9XhQWFuL888+X5t8XZeaUeQng3MS5ST9OmZucMi8BNpqbBBmqpqZGDBw4UGzatEmsW7dOfPe73xW33XZb2uds3LhRFBUVifnz54u//e1vYseOHeK1114T3377rUmj7kjLfUQ888wz4rrrrhMAxMqVK40daAbZ3sdHH30k/u3f/k28+eabYteuXcLn84nzzjtPjB8/3sRRC/Hqq6+KgoICsWzZMvHxxx+LKVOmiOLiYnHgwIGk12/YsEHk5eWJX/7yl+Lvf/+7qK2tFZ06dRIfffSRqeNOlO193H777WLRokXiww8/FNu3bxd33XWX6N69u/jnP/9p8sg7yvZeIr744gvRu3dvcdVVV4kbb7zRnMGmke19nDhxQgwdOlSMGTNGrF+/XnzxxRdi7dq14s9//rPJIyetnDIvCcG5iXOTPpwyNzllXhLCXnMTgykD/f3vfxcAxJYtW6KPvf3220JRFLFv376Uz6uqqhK1tbVmDFEVrfchhBAffvih6N27t9i/f7/lE1Yu9xHrD3/4gygoKBAnT540YphJDRs2TEyfPj36+0AgICoqKsT8+fOTXn/zzTeLsWPHxj1WVVUl7rnnHkPHmUm295Ho1KlTolu3buLll182aoiqabmXU6dOiREjRohf//rX4oc//KEUk1a29/Hiiy+Kfv36ifb2drOGSDpyyrwkBOemRJybtHPK3OSUeUkIe81NTPMzUGNjI4qLizF06NDoY9XV1fB4PGhqakr6nIMHD6KpqQmlpaUYMWIEysrKcPXVV2P9+vVmDbsDLfcBAMePH8ftt9+ORYsWoby83IyhpqX1PhK1traiqKgI+fn5Rgyzg/b2dmzbtg3V1dXRxzweD6qrq9HY2Jj0OY2NjXHXA8Do0aNTXm8GLfeR6Pjx4zh58iRKSkqMGqYqWu/l4YcfRmlpKSZPnmzGMDPSch9vvvkmhg8fjunTp6OsrAyXXHIJHn/8cQQCAbOGTTlwyrwEcG5KxLlJG6fMTU6ZlwD7zU0Mpgzk9/tRWloa91h+fj5KSkrg9/uTPufzzz8HADz00EOYMmUKVq9ejcGDB2PUqFH49NNPDR9zMlruAwDuu+8+jBgxAjfeeKPRQ1RF633E+vLLL/HII49g6tSpRgwx5fcMBAIoKyuLe7ysrCzluP1+f1bXm0HLfST62c9+hoqKig6Tsdm03Mv69euxdOlSLFmyxIwhqqLlPj7//HMsX74cgUAAq1atQl1dHZ5++mk8+uijZgyZcuSUeQng3BSLc5N2TpmbnDIvAfabmxhMaTB79mwoipL2a8eOHZpeOxgMAgDuueceTJo0CZdddhmeffZZXHDBBVi2bJmet2Hofbz55pt47733UF9fr+uYkzHyPmK1tbVh7Nix6N+/Px566KHcB05ZeeKJJ/Dqq69i5cqV6Ny5s9XDycqRI0dwxx13YMmSJejZs6fVw8lJMBhEaWkpXnrpJQwZMgS33HILfv7zn2Px4sVWD83VnDIvAZybssW5yVp2nZucNC8B1s5N5uwFO8z999+Pu+66K+01/fr1Q3l5OQ4ePBj3+KlTp9DS0pIytcDr9QIA+vfvH/f4RRddhD179mgfdBJG3sd7772Hzz77DMXFxXGPjx8/HldddRXWrl2bw8jjGXkfEUeOHEFNTQ26deuGlStXolOnTrkOW7WePXsiLy8PBw4ciHv8wIEDKcddXl6e1fVm0HIfEQsWLMATTzyBP/7xj7j00kuNHKYq2d7LZ599ht27d+OGG26IPhb5gJqfn4+dO3fi3HPPNXbQSWj5O/F6vejUqRPy8vKij1100UXw+/1ob29HQUGBoWOm5JwyLwGcmwDOTWZyytzklHkJsOHcZPopLReJHCrdunVr9LF33nkn7aHSYDAoKioqOhz0HTRokJgzZ46h401Fy33s379ffPTRR3FfAMRzzz0nPv/8c7OGHkfLfQghRGtrq7j88svF1VdfLY4dO2bGUDsYNmyYmDFjRvT3gUBA9O7dO+0h3+uvvz7useHDh0txyDeb+xBCiCeffFIUFRWJxsZGM4aoWjb38s0333T4/+HGG28UI0eOFB999JE4ceKEmUOPk+3fyZw5c8R3vvMdEQgEoo/V19cLr9dr+Fgpd06Zl4Tg3MS5ST9OmZucMi8JYa+5icGUwWpqasRll10mmpqaxPr168V5550XV+70n//8p7jgggtEU1NT9LFnn31WFBUViddff118+umnora2VnTu3Fns2rXLilsQQmi7j0SQpPxsNvfR2toqqqqqxIABA8SuXbvE/v37o1+nTp0ybdyvvvqqKCwsFL/5zW/E3//+dzF16lRRXFws/H6/EEKIO+64Q8yePTt6/YYNG0R+fr5YsGCB2L59u5g3b5405WezuY8nnnhCFBQUiOXLl8f97I8cOWLVLURley+JZKmalO197NmzR3Tr1k3MmDFD7Ny5U7z11luitLRUPProo1bdAmXJKfOSEJybODfpwylzk1PmJSHsNTcxmDLYV199JW677TZx5plniqKiIjFp0qS4/9m++OILAUC8//77cc+bP3++6NOnj+jatasYPny4WLdunckjj6f1PmLJMGFlex/vv/++AJD064svvjB17AsXLhRnn322KCgoEMOGDRObNm2K/tnVV18tfvjDH8Zd/4c//EGcf/75oqCgQFx88cWioaHB1PGmks19fOc730n6s583b575A08i27+TWDJNWtnex8aNG0VVVZUoLCwU/fr1E4899pipH+AoN06Zl4Tg3MS5ST9OmZucMi8JYZ+5SRFCCOOSCImIiIiIiJyJ1fyIiIiIiIg0YDBFRERERESkAYMpIiIiIiIiDRhMERERERERacBgioiIiIiISAMGU0RERERERBowmCIiIiIiItKAwRQREREREZEGDKaIiIiIiIg0YDBFZDO///3v0aVLF+zfvz/62KRJk3DppZeitbXVwpEREZFbcW4it1KEEMLqQRCRekIIDBo0CN/73vewcOFCzJs3D8uWLcOmTZvQu3dvq4dHREQuxLmJ3Crf6gEQUXYURcFjjz2GCRMmoLy8HAsXLsS6deuik9W4ceOwdu1ajBo1CsuXL7d4tERE5Aacm8ituDNFZFODBw/Gxx9/jHfffRdXX3119PG1a9fiyJEjePnllzlhERGRqTg3kdvwzBSRDa1evRo7duxAIBBAWVlZ3J99//vfR7du3SwaGRERuRXnJnIjBlNENvPBBx/g5ptvxtKlSzFq1CjU1dVZPSQiInI5zk3kVjwzRWQju3fvxtixY/Hggw/itttuQ79+/TB8+HB88MEHGDx4sNXDIyIiF+LcRG7GnSkim2hpaUFNTQ1uvPFGzJ49GwBQVVWF6667Dg8++KDFoyMiIjfi3ERux50pIpsoKSnBjh07Ojze0NBgwWiIiIg4NxGxmh+Rw1RXV+Mvf/kLjh07hpKSErz++usYPny41cMiIiIX49xETsVgioiIiIiISAOemSIiIiIiItKAwRQREREREZEGDKaIiIiIiIg0YDBFRERERESkAYMpIiIiIiIiDRhMERERERERacBgioiIiIiISAMGU0RERERERBowmCIiIiIiItKAwRQREREREZEGDKaIiIiIiIg0+L+DBb8pKcM9swAAAABJRU5ErkJggg==",
|
|
"text/plain": [
|
|
"<Figure size 1000x400 with 2 Axes>"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"np.random.seed(6)\n",
|
|
"X_square = np.random.rand(100, 2) - 0.5\n",
|
|
"y_square = (X_square[:, 0] > 0).astype(np.int64)\n",
|
|
"\n",
|
|
"angle = np.pi / 4 # 45 degrees\n",
|
|
"rotation_matrix = np.array([[np.cos(angle), -np.sin(angle)],\n",
|
|
" [np.sin(angle), np.cos(angle)]])\n",
|
|
"X_rotated_square = X_square.dot(rotation_matrix)\n",
|
|
"\n",
|
|
"tree_clf_square = DecisionTreeClassifier(random_state=42)\n",
|
|
"tree_clf_square.fit(X_square, y_square)\n",
|
|
"tree_clf_rotated_square = DecisionTreeClassifier(random_state=42)\n",
|
|
"tree_clf_rotated_square.fit(X_rotated_square, y_square)\n",
|
|
"\n",
|
|
"fig, axes = plt.subplots(ncols=2, figsize=(10, 4), sharey=True)\n",
|
|
"plt.sca(axes[0])\n",
|
|
"plot_decision_boundary(tree_clf_square, X_square, y_square,\n",
|
|
" axes=[-0.7, 0.7, -0.7, 0.7], cmap=\"Pastel1\")\n",
|
|
"plt.sca(axes[1])\n",
|
|
"plot_decision_boundary(tree_clf_rotated_square, X_rotated_square, y_square,\n",
|
|
" axes=[-0.7, 0.7, -0.7, 0.7], cmap=\"Pastel1\")\n",
|
|
"plt.ylabel(\"\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "subslide"
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"Could mitigate with principle component analysis (PCA) or with ensemble methods (see upoming lectures)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "subslide"
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"### Decision trees are not highly stable"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": ""
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"- Small changes to hyperparameters or to data may produce very different models.\n",
|
|
"- Even retraining with a different random seed can produce very different models."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": ""
|
|
},
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"Can leverage this property by averaging over many trees, which gives rises to *ensemble methods* (as discussed in the next lecture)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"editable": true,
|
|
"slideshow": {
|
|
"slide_type": "subslide"
|
|
},
|
|
"tags": [
|
|
"exercise_pointer"
|
|
]
|
|
},
|
|
"source": [
|
|
"**Exercises:** *You can now complete Exercises 1-2 in the exercises associated with this lecture.*"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"celltoolbar": "Slideshow",
|
|
"kernelspec": {
|
|
"display_name": "Python 3 (ipykernel)",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.11.11"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 4
|
|
}
|