add week 6 material

week4/exercises/Lecture13_TrainingDeepNNs_Exercises_no_solutions.ipynb

@@ -0,0 +1 @@
+{"cells": [{"cell_type": "markdown", "metadata": {}, "source": ["# Exercises for Lecture 13 (Training deep neural networks)"]}, {"cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": ["import numpy as np\n", "\n", "# To plot pretty figures\n", "%matplotlib inline\n", "import matplotlib as mpl\n", "import matplotlib.pyplot as plt\n", "mpl.rc('axes', labelsize=14)\n", "mpl.rc('xtick', labelsize=12)\n", "mpl.rc('ytick', labelsize=12)"]}, {"cell_type": "markdown", "metadata": {"slideshow": {"slide_type": "subslide"}}, "source": ["## Exercise 1: Build a deep neural network with ELU activation functions and batch normalisation and apply it to fashion MINST."]}, {"cell_type": "markdown", "metadata": {}, "source": ["How good of an accuracy can you achieve on the test set?"]}], "metadata": {"celltoolbar": "Tags", "kernelspec": {"display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3"}, "language_info": {"codemirror_mode": {"name": "ipython", "version": 3}, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.7"}}, "nbformat": 4, "nbformat_minor": 4}

week4/exercises/Lecture14_CNN_Exercises_no_solutions.ipynb

@@ -0,0 +1 @@
+{"cells": [{"cell_type": "markdown", "metadata": {}, "source": ["# Exercises for Lecture 14 (Convolutional neural networks)"]}, {"cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": ["import numpy as np\n", "import tensorflow as tf\n", "from tensorflow import keras\n", "\n", "# To make this notebook's output stable across runs\n", "def reset_state(seed=42):\n", "    tf.keras.backend.clear_session()\n", "    tf.random.set_seed(seed)\n", "    np.random.seed(seed)"]}, {"cell_type": "markdown", "metadata": {"slideshow": {"slide_type": "subslide"}}, "source": ["## Exercise 1: Build a deeper CNN architecture for fashion MNIST than considered in the corresponding lecture to achieves a better classification accuracy."]}, {"cell_type": "markdown", "metadata": {}, "source": ["Following the guiding CNN design principles covered in the lecture to construct a better CNN architecture."]}, {"cell_type": "markdown", "metadata": {}, "source": ["Load and set up data."]}, {"cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": ["# Load fashion MNIST data\n", "(X_train_full, y_train_full), (X_test, y_test) = keras.datasets.fashion_mnist.load_data()\n", "X_train, X_valid = X_train_full[:-30000], X_train_full[-30000:]\n", "y_train, y_valid = y_train_full[:-30000], y_train_full[-30000:]\n", "\n", "# Standardize\n", "X_mean = X_train.mean(axis=0, keepdims=True)\n", "X_std = X_train.std(axis=0, keepdims=True) + 1e-7\n", "X_train = (X_train - X_mean) / X_std\n", "X_valid = (X_valid - X_mean) / X_std\n", "X_test = (X_test - X_mean) / X_std\n", "\n", "# Add final channel axis (one channel)\n", "X_train = X_train[..., np.newaxis]\n", "X_valid = X_valid[..., np.newaxis]\n", "X_test = X_test[..., np.newaxis]"]}, {"cell_type": "markdown", "metadata": {}, "source": ["Build and fit new model, and evaluate its performance."]}], "metadata": {"celltoolbar": "Tags", "kernelspec": {"display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3"}, "language_info": {"codemirror_mode": {"name": "ipython", "version": 3}, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.12"}}, "nbformat": 4, "nbformat_minor": 4}

week4/exercises/Lecture15_DeepCNN_Exercises_no_solutions.ipynb

@@ -0,0 +1 @@
+{"cells": [{"cell_type": "markdown", "id": "c1c2c049-4d1c-4a46-b035-b3f477aace8d", "metadata": {}, "source": ["# Exercises for Lecture 15 (Deep CNN architectures)"]}, {"cell_type": "code", "execution_count": null, "id": "ea8f9065-2320-422b-8a8c-a55d7fd726a3", "metadata": {}, "outputs": [], "source": ["import numpy as np\n", "import tensorflow as tf\n", "from tensorflow import keras\n", "from functools import partial\n", "\n", "# To make this notebook's output stable across runs\n", "def reset_state(seed=42):\n", "    tf.keras.backend.clear_session()\n", "    tf.random.set_seed(seed)\n", "    np.random.seed(seed)"]}, {"cell_type": "markdown", "id": "c5dc69f6-efea-4b71-9746-25ca8d347132", "metadata": {"slideshow": {"slide_type": "subslide"}}, "source": ["## Exercise 1: Build a ResNet CNN architecture for fashion MNIST."]}, {"cell_type": "markdown", "id": "9fa998f5-7e8e-4cd6-a335-13a146c49de3", "metadata": {}, "source": ["Load and set up data."]}, {"cell_type": "code", "execution_count": null, "id": "b7fd0854-0e6f-4d4a-838b-506c4a26268c", "metadata": {}, "outputs": [], "source": ["# Load fashion MNIST data\n", "(X_train_full, y_train_full), (X_test, y_test) = keras.datasets.fashion_mnist.load_data()\n", "X_train, X_valid = X_train_full[:-30000], X_train_full[-30000:]\n", "y_train, y_valid = y_train_full[:-30000], y_train_full[-30000:]\n", "\n", "# Standardize\n", "X_mean = X_train.mean(axis=0, keepdims=True)\n", "X_std = X_train.std(axis=0, keepdims=True) + 1e-7\n", "X_train = (X_train - X_mean) / X_std\n", "X_valid = (X_valid - X_mean) / X_std\n", "X_test = (X_test - X_mean) / X_std\n", "\n", "# Add final channel axis (one channel)\n", "X_train = X_train[..., np.newaxis]\n", "X_valid = X_valid[..., np.newaxis]\n", "X_test = X_test[..., np.newaxis]"]}, {"cell_type": "markdown", "id": "4362c522-3172-4fc9-9265-761cccad73cc", "metadata": {}, "source": ["Use the subclassing API to define a `ResidualUnit` layer."]}, {"cell_type": "markdown", "id": "dd33a8d1-af2f-49cc-ad73-8b447c07955f", "metadata": {}, "source": ["Buid a ResNet model using the layer you defined above."]}, {"cell_type": "markdown", "id": "561c8d32-b8e4-426b-8092-7ec79432dde0", "metadata": {}, "source": ["Compile your model and train it."]}, {"cell_type": "markdown", "id": "1159fce2-0749-4170-ab4d-f4b8e0ed6014", "metadata": {}, "source": ["Evaluate the model performance on the test set."]}], "metadata": {"kernelspec": {"display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3"}, "language_info": {"codemirror_mode": {"name": "ipython", "version": 3}, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.12"}}, "nbformat": 4, "nbformat_minor": 5}

week6/exercises/Lecture17_EnsembleRFs_Exercises_no_solutions.ipynb

@@ -0,0 +1 @@
+{"cells": [{"cell_type": "markdown", "metadata": {"editable": true, "slideshow": {"slide_type": ""}, "tags": []}, "source": ["# Exercises for Lecture 17 (Ensemble Learning and Random Forests)"]}, {"cell_type": "code", "execution_count": null, "metadata": {"editable": true, "slideshow": {"slide_type": ""}, "tags": []}, "outputs": [], "source": ["import numpy as np\n", "import matplotlib.pyplot as plt\n", "from sklearn.ensemble import GradientBoostingRegressor \n", "from sklearn.model_selection import train_test_split"]}, {"cell_type": "markdown", "metadata": {"editable": true, "slideshow": {"slide_type": ""}, "tags": []}, "source": ["## Exercise 1: Early stopping"]}, {"cell_type": "markdown", "metadata": {"editable": true, "slideshow": {"slide_type": ""}, "tags": []}, "source": ["### Set up mock data"]}, {"cell_type": "code", "execution_count": null, "metadata": {"editable": true, "slideshow": {"slide_type": ""}, "tags": []}, "outputs": [], "source": ["# Training set: a noisy quadratic function\n", "np.random.seed(42)\n", "X = np.random.rand(100, 1) - 0.5\n", "y = 3*X[:, 0]**2 + 0.05 * np.random.randn(100)\n", "\n", "# First create test and train data\n", "X_train, X_val, y_train, y_val = train_test_split(X, y)"]}, {"cell_type": "markdown", "metadata": {"editable": true, "slideshow": {"slide_type": ""}, "tags": []}, "source": ["### Train with many trees"]}, {"cell_type": "code", "execution_count": null, "metadata": {"editable": true, "slideshow": {"slide_type": ""}, "tags": []}, "outputs": [], "source": ["from sklearn.metrics import mean_squared_error\n", "\n", "n_estimators = 300\n", "gbrt = GradientBoostingRegressor(\n", "    max_depth=2, \n", "    n_estimators=n_estimators, \n", "    learning_rate=0.1, # Set a low learning rate here\n", "    random_state=42)\n", "\n", "gbrt.fit(X_train, y_train)"]}, {"cell_type": "markdown", "metadata": {"editable": true, "slideshow": {"slide_type": ""}, "tags": []}, "source": ["### Compute and plot validation error for intermediate number of trees"]}, {"cell_type": "code", "execution_count": null, "metadata": {"editable": true, "slideshow": {"slide_type": ""}, "tags": []}, "outputs": [], "source": ["# measure MSE validation error at each stage\n", "errors = [mean_squared_error(y_val, y_pred) for y_pred in gbrt.staged_predict(X_val)]"]}, {"cell_type": "code", "execution_count": null, "metadata": {"editable": true, "slideshow": {"slide_type": ""}, "tags": []}, "outputs": [], "source": ["plt.figure(figsize=(11, 4))\n", "plt.plot(errors, \"b.-\")\n", "plt.axis([0, 300, 0, 0.01])\n", "plt.xlabel(\"Number of trees\")\n", "plt.title(\"Validation error\", fontsize=14)"]}, {"cell_type": "markdown", "metadata": {"editable": true, "slideshow": {"slide_type": "slide"}, "tags": []}, "source": ["### Training a better model with fewer trees\n", "\n", "- Find the best number of trees from the validation error.  Show this on a plot.\n", "- Train a new GBRT using the optimal number of trees from above.\n", "- Plot predictions of the original and best models.\n"]}, {"cell_type": "code", "execution_count": null, "metadata": {"editable": true, "slideshow": {"slide_type": ""}, "tags": []}, "outputs": [], "source": ["def plot_predictions(\n", "    regressors, X, y, axes, \n", "    label=None, \n", "    style=\"r-\", \n", "    data_style=\"b.\", \n", "    data_label=None):\n", "    \n", "    x1 = np.linspace(axes[0], axes[1], 500)\n", "    \n", "    y_pred = sum(\n", "        regressor.predict(x1.reshape(-1, 1)) for regressor in regressors)\n", "            \n", "    plt.plot(X[:, 0], y, data_style, label=data_label)\n", "    plt.plot(x1, y_pred, style, linewidth=2, label=label)\n", "    if label or data_label:\n", "        plt.legend(loc=\"upper center\", fontsize=16)\n", "    plt.axis(axes)"]}], "metadata": {"celltoolbar": "Tags", "kernelspec": {"display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3"}, "language_info": {"codemirror_mode": {"name": "ipython", "version": 3}, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.11"}}, "nbformat": 4, "nbformat_minor": 4}

week6/exercises/Lecture18_DimensionalityReduction_Exercises_no_solutions.ipynb

@@ -0,0 +1 @@
+{"cells": [{"cell_type": "markdown", "metadata": {"editable": true, "slideshow": {"slide_type": ""}, "tags": []}, "source": ["# Exercises for Lecture 18 (Dimensionality Reduction)"]}, {"cell_type": "markdown", "metadata": {"editable": true, "slideshow": {"slide_type": "slide"}, "tags": []}, "source": ["## Exercise 1\n", "\n", "- Load the MNIST dataset and split it into a training set and a test set (take the first 60,000 instances for training, and the remaining 10,000 for testing).\n", "- Train a Random Forest classifier on the dataset and time how long it takes.\n", "- Then evaluate the resulting model on the test set. \n", "- Next, use PCA to reduce the dataset\u2019s dimensionality, with an explained variance ratio of 95%. \n", "- Train a new Random Forest classifier on the reduced dataset and see how long it takes. \n", "- Was training much faster?\n", "- Next evaluate the classifier on the test set.\n", "- How well does it perform compare to the previous classifier?"]}, {"cell_type": "code", "execution_count": null, "metadata": {"editable": true, "slideshow": {"slide_type": ""}, "tags": []}, "outputs": [], "source": ["from sklearn.datasets import fetch_openml\n", "mnist = fetch_openml('mnist_784', version=1, cache=True)"]}], "metadata": {"celltoolbar": "Tags", "kernelspec": {"display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3"}, "language_info": {"codemirror_mode": {"name": "ipython", "version": 3}, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.11"}}, "nbformat": 4, "nbformat_minor": 4}