## File: README.md Machine Learning Notebooks ========================== # ⚠ THE [THIRD EDITION OF MY BOOK](https://github.com/ageron/handson-ml3) IS NOW AVAILABLE, AS WELL A [PYTORCH VERSION](https://github.com/ageron/handson-mlp) AND [MANY TRANSLATIONS](https://homl.info/). This project is for the first edition, which is now outdated (it came out in 2017). This project aims at teaching you the fundamentals of Machine Learning in python. It contains the example code and solutions to the exercises in my O'Reilly book [Hands-on Machine Learning with Scikit-Learn and TensorFlow](https://learning.oreilly.com/library/view/hands-on-machine-learning/9781491962282/): [](https://learning.oreilly.com/library/view/hands-on-machine-learning/9781491962282/) ## Quick Start ### Want to play with these notebooks online without having to install anything? Use any of the following services. **WARNING**: Please be aware that these services provide temporary environments: anything you do will be deleted after a while, so make sure you download any data you care about. * **Recommended**: open this repository in [Colaboratory](https://colab.research.google.com/github/ageron/handson-ml/blob/master/): [](https://colab.research.google.com/github/ageron/handson-ml/blob/master/) * Or open it in [Binder](https://mybinder.org/v2/gh/ageron/handson-ml/master): [](https://mybinder.org/v2/gh/ageron/handson-ml/master) * _Note_: Most of the time, Binder starts up quickly and works great, but when handson-ml is updated, Binder creates a new environment from scratch, and this can take quite some time. * Or open it in [Deepnote](https://beta.deepnote.com/launch?template=data-science&url=https%3A//github.com/ageron/handson-ml/blob/master/index.ipynb): [](https://beta.deepnote.com/launch?template=data-science&url=https%3A//github.com/ageron/handson-ml/blob/master/index.ipynb) ### Just want to quickly look at some notebooks, without executing any code? Browse this repository using [jupyter.org's notebook viewer](https://nbviewer.jupyter.org/github/ageron/handson-ml/blob/master/index.ipynb): [](https://nbviewer.jupyter.org/github/ageron/handson-ml/blob/master/index.ipynb) _Note_: [github.com's notebook viewer](index.ipynb) also works but it is slower and the math equations are not always displayed correctly. ### Want to run this project using a Docker image? Read the [Docker instructions](https://github.com/ageron/handson-ml/tree/master/docker). ### Want to install this project on your own machine? Start by installing [Anaconda](https://www.anaconda.com/distribution/) (or [Miniconda](https://docs.conda.io/en/latest/miniconda.html)), [git](https://git-scm.com/downloads), and if you have a TensorFlow-compatible GPU, install the [GPU driver](https://www.nvidia.com/Download/index.aspx), as well as the appropriate version of CUDA and cuDNN (see TensorFlow's documentation for more details). Next, clone this project by opening a terminal and typing the following commands (do not type the first `$` signs on each line, they just indicate that these are terminal commands): $ git clone https://github.com/ageron/handson-ml.git $ cd handson-ml Next, run the following commands: $ conda env create -f environment.yml $ conda activate tf1 $ python -m ipykernel install --user --name=python3 Finally, start Jupyter: $ jupyter notebook If you need further instructions, read the [detailed installation instructions](INSTALL.md). # FAQ **Which Python version should I use?** I recommend Python 3.7. If you follow the installation instructions above, that's the version you will get. Most code will work with other versions of Python 3, but some libraries do not support Python 3.8 or 3.9 yet, which is why I recommend Python 3.7. **I'm getting an error when I call `load_housing_data()`** Make sure you call `fetch_housing_data()` *before* you call `load_housing_data()`. If you're getting an HTTP error, make sure you're running the exact same code as in the notebook (copy/paste it if needed). If the problem persists, please check your network configuration. **I'm getting an SSL error on MacOSX** You probably need to install the SSL certificates (see this [StackOverflow question](https://stackoverflow.com/questions/27835619/urllib-and-ssl-certificate-verify-failed-error)). If you downloaded Python from the official website, then run `/Applications/Python\ 3.7/Install\ Certificates.command` in a terminal (change `3.7` to whatever version you installed). If you installed Python using MacPorts, run `sudo port install curl-ca-bundle` in a terminal. **I've installed this project locally. How do I update it to the latest version?** See [INSTALL.md](INSTALL.md) **How do I update my Python libraries to the latest versions, when using Anaconda?** See [INSTALL.md](INSTALL.md) ## Contributors I would like to thank everyone [who contributed to this project](https://github.com/ageron/handson-ml/graphs/contributors), either by providing useful feedback, filing issues or submitting Pull Requests. Special thanks go to Haesun Park and Ian Beauregard who reviewed every notebook and submitted many PRs, including help on some of the exercise solutions. Thanks as well to Steven Bunkley and Ziembla who created the `docker` directory, and to github user SuperYorio who helped on some exercise solutions. --- ## File: datasets/lifesat/README.md # Life satisfaction and GDP per capita ## Life satisfaction ### Source This dataset was obtained from the OECD's website at: http://stats.oecd.org/index.aspx?DataSetCode=BLI ### Data description Int64Index: 3292 entries, 0 to 3291 Data columns (total 17 columns): "LOCATION" 3292 non-null object Country 3292 non-null object INDICATOR 3292 non-null object Indicator 3292 non-null object MEASURE 3292 non-null object Measure 3292 non-null object INEQUALITY 3292 non-null object Inequality 3292 non-null object Unit Code 3292 non-null object Unit 3292 non-null object PowerCode Code 3292 non-null int64 PowerCode 3292 non-null object Reference Period Code 0 non-null float64 Reference Period 0 non-null float64 Value 3292 non-null float64 Flag Codes 1120 non-null object Flags 1120 non-null object dtypes: float64(3), int64(1), object(13) memory usage: 462.9+ KB ### Example usage using python Pandas >>> life_sat = pd.read_csv("oecd_bli_2015.csv", thousands=',') >>> life_sat_total = life_sat[life_sat["INEQUALITY"]=="TOT"] >>> life_sat_total = life_sat_total.pivot(index="Country", columns="Indicator", values="Value") >>> life_sat_total.info() Index: 37 entries, Australia to United States Data columns (total 24 columns): Air pollution 37 non-null float64 Assault rate 37 non-null float64 Consultation on rule-making 37 non-null float64 Dwellings without basic facilities 37 non-null float64 Educational attainment 37 non-null float64 Employees working very long hours 37 non-null float64 Employment rate 37 non-null float64 Homicide rate 37 non-null float64 Household net adjusted disposable income 37 non-null float64 Household net financial wealth 37 non-null float64 Housing expenditure 37 non-null float64 Job security 37 non-null float64 Life expectancy 37 non-null float64 Life satisfaction 37 non-null float64 Long-term unemployment rate 37 non-null float64 Personal earnings 37 non-null float64 Quality of support network 37 non-null float64 Rooms per person 37 non-null float64 Self-reported health 37 non-null float64 Student skills 37 non-null float64 Time devoted to leisure and personal care 37 non-null float64 Voter turnout 37 non-null float64 Water quality 37 non-null float64 Years in education 37 non-null float64 dtypes: float64(24) memory usage: 7.2+ KB ## GDP per capita ### Source Dataset obtained from the IMF's website at: http://goo.gl/j1MSKe ### Data description Int64Index: 190 entries, 0 to 189 Data columns (total 7 columns): Country 190 non-null object Subject Descriptor 189 non-null object Units 189 non-null object Scale 189 non-null object Country/Series-specific Notes 188 non-null object 2015 187 non-null float64 Estimates Start After 188 non-null float64 dtypes: float64(2), object(5) memory usage: 11.9+ KB ### Example usage using python Pandas >>> gdp_per_capita = pd.read_csv( ... datapath+"gdp_per_capita.csv", thousands=',', delimiter='\t', ... encoding='latin1', na_values="n/a", index_col="Country") ... >>> gdp_per_capita.rename(columns={"2015": "GDP per capita"}, inplace=True) --- ## File: datasets/housing/README.md # California Housing ## Source This dataset is a modified version of the California Housing dataset available from [Luís Torgo's page](http://www.dcc.fc.up.pt/~ltorgo/Regression/cal_housing.html) (University of Porto). Luís Torgo obtained it from the StatLib repository (which is closed now). The dataset may also be downloaded from StatLib mirrors. This dataset appeared in a 1997 paper titled *Sparse Spatial Autoregressions* by Pace, R. Kelley and Ronald Barry, published in the *Statistics and Probability Letters* journal. They built it using the 1990 California census data. It contains one row per census block group. A block group is the smallest geographical unit for which the U.S. Census Bureau publishes sample data (a block group typically has a population of 600 to 3,000 people). ## Tweaks The dataset in this directory is almost identical to the original, with two differences: * 207 values were randomly removed from the `total_bedrooms` column, so we can discuss what to do with missing data. * An additional categorical attribute called `ocean_proximity` was added, indicating (very roughly) whether each block group is near the ocean, near the Bay area, inland or on an island. This allows discussing what to do with categorical data. Note that the block groups are called "districts" in the Jupyter notebooks, simply because in some contexts the name "block group" was confusing. ## Data description >>> housing.info() RangeIndex: 20640 entries, 0 to 20639 Data columns (total 10 columns): longitude 20640 non-null float64 latitude 20640 non-null float64 housing_median_age 20640 non-null float64 total_rooms 20640 non-null float64 total_bedrooms 20433 non-null float64 population 20640 non-null float64 households 20640 non-null float64 median_income 20640 non-null float64 median_house_value 20640 non-null float64 ocean_proximity 20640 non-null object dtypes: float64(9), object(1) memory usage: 1.6+ MB >>> housing["ocean_proximity"].value_counts() <1H OCEAN 9136 INLAND 6551 NEAR OCEAN 2658 NEAR BAY 2290 ISLAND 5 Name: ocean_proximity, dtype: int64 >>> housing.describe() longitude latitude housing_median_age total_rooms \ count 16513.000000 16513.000000 16513.000000 16513.000000 mean -119.575972 35.639693 28.652335 2622.347605 std 2.002048 2.138279 12.576306 2138.559393 min -124.350000 32.540000 1.000000 6.000000 25% -121.800000 33.940000 18.000000 1442.000000 50% -118.510000 34.260000 29.000000 2119.000000 75% -118.010000 37.720000 37.000000 3141.000000 max -114.310000 41.950000 52.000000 39320.000000 total_bedrooms population households median_income count 16355.000000 16513.000000 16513.000000 16513.000000 mean 534.885112 1419.525465 496.975050 3.875651 std 412.716467 1115.715084 375.737945 1.905088 min 2.000000 3.000000 2.000000 0.499900 25% 295.000000 784.000000 278.000000 2.566800 50% 433.000000 1164.000000 408.000000 3.541400 75% 644.000000 1718.000000 602.000000 4.745000 max 6210.000000 35682.000000 5358.000000 15.000100 --- ## File: docker/README.md # Hands-on Machine Learning in Docker This is the Docker configuration which allows you to run and tweak the book's notebooks without installing any dependencies on your machine!OK, any except `docker` and `docker-compose`.And optionally `make`.And a few more things if you want GPU support (see below for details). ## Prerequisites Follow the instructions on [Install Docker](https://docs.docker.com/engine/installation/) and [Install Docker Compose](https://docs.docker.com/compose/install/) for your environment if you haven't got `docker` and `docker-compose` already. Some general knowledge about `docker` infrastructure might be useful (that's an interesting topic on its own) but is not strictly *required* to just run the notebooks. ## Usage ### Prepare the image (once) The first option is to pull the image from Docker Hub (this will download about 1.9 GB of compressed data): ```bash $ docker pull ageron/handson-ml ``` **Note**: this is the CPU-only image. For GPU support, read the GPU section below. Alternatively, you can build the image yourself. This will be slower, but it will ensure the image is up to date, with the latest libraries, except for TensorFlow 1.15 (instead of the latest TensorFlow 2.x). For this, assuming you already downloaded this project into the directory `/path/to/project/handson-ml`: ```bash $ cd /path/to/project/handson-ml/docker $ docker-compose build ``` This will take quite a while, but is only required once. After the process is finished you have an `ageron/handson-ml:latest` image, that will be the base for your experiments. You can confirm that by running the following command: ```bash $ docker images REPOSITORY TAG IMAGE ID CREATED SIZE ageron/handson-ml latest 4fcbafcd715b 3 minutes ago 4.85GB ``` ### Run the notebooks Still assuming you already downloaded this project into the directory `/path/to/project/handson-ml`, run the following commands to start the Jupyter server inside the container, which is named `handson-ml`: ```bash $ cd /path/to/project/handson-ml/docker $ docker-compose up ``` Next, just point your browser to the URL printed on the screen (or go to if you enabled password authentication inside the `jupyter_notebook_config.py` file, before building the image) and you're ready to play with the book's code! The server runs in the directory containing the notebooks, and the changes you make from the browser will be persisted there. You can close the server just by pressing `Ctrl-C` in the terminal window. ### Using `make` (optional) If you have `make` installed on your computer, you can use it as a thin layer to run `docker-compose` commands. For example, executing `make rebuild` will actually run `docker-compose build --no-cache`, which will rebuild the image without using the cache. This ensures that your image is based on the latest version of the `continuumio/miniconda3` image which the `ageron/handson-ml` image is based on. If you don't have `make` (and you don't want to install it), just examine the contents of `Makefile` to see which `docker-compose` commands you can run instead. ### Run additional commands in the container Run `make exec` (or `docker-compose exec handson-ml bash`) while the server is running to run an additional `bash` shell inside the `handson-ml` container. Now you're inside the environment prepared within the image. One of the useful things that can be done there would be starting TensorBoard (for example with simple `tb` command, see bashrc file). Another one may be comparing versions of the notebooks using the `nbdiff` command if you haven't got `nbdime` installed locally (it is **way** better than plain `diff` for notebooks). See [Tools for diffing and merging of Jupyter notebooks](https://github.com/jupyter/nbdime) for more details. You can see changes you made relative to the version in git using `git diff` which is integrated with `nbdiff`. You may also try `nbd NOTEBOOK_NAME.ipynb` command (custom, see bashrc file) to compare one of your notebooks with its `checkpointed` version. To be precise, the output will tell you *what modifications should be re-played on the **manually saved** version of the notebook (located in `.ipynb_checkpoints` subdirectory) to update it to the **current** i.e. **auto-saved** version (given as command's argument - located in working directory)*. ## GPU Support on Linux (experimental) ### Prerequisites If you're running on Linux, and you have a TensorFlow-compatible GPU card (NVidia card with Compute Capability ≥ 3.5) that you would like TensorFlow to use inside the Docker container, then you should download and install the latest driver for your card from [nvidia.com](https://www.nvidia.com/Download/index.aspx?lang=en-us). You will also need to install [NVidia Docker support](https://github.com/NVIDIA/nvidia-docker): if you are using Docker 19.03 or above, you must install the `nvidia-container-toolkit` package, and for earlier versions, you must install `nvidia-docker2`. Next, edit the `docker-compose.yml` file: ```bash $ cd /path/to/project/handson-ml/docker $ edit docker-compose.yml # use your favorite editor ``` * Replace `dockerfile: ./docker/Dockerfile` with `dockerfile: ./docker/Dockerfile.gpu` * Replace `image: ageron/handson-ml:latest` with `image: ageron/handson-ml:latest-gpu` * If you want to use `docker-compose`, you will need version 1.28 or above for GPU support, and you must uncomment the whole `deploy` section in `docker-compose.yml`. ### Prepare the image (once) If you want to pull the prebuilt image from Docker Hub (this will download over 3.5 GB of compressed data): ```bash $ docker pull ageron/handson-ml:latest-gpu ``` If you prefer to build the image yourself: ```bash $ cd /path/to/project/handson-ml/docker $ docker-compose build ``` ### Run the notebooks with `docker-compose` (version 1.28 or above) If you have `docker-compose` version 1.28 or above, that's great! You can simply run: ```bash $ cd /path/to/project/handson-ml/docker $ docker-compose up [...] or http://127.0.0.1:8888/?token=[...] ``` Then point your browser to the URL and Jupyter should appear. If you then open or create a notebook and execute the following code, a list containing your GPU device(s) should be displayed (success!): ```python import tensorflow as tf tf.config.list_physical_devices("GPU") ``` To stop the server, just press Ctrl-C. ### Run the notebooks without `docker-compose` If you have a version of `docker-compose` earlier than 1.28, you will have to use `docker run` directly. If you are using Docker 19.03 or above, you can run: ```bash $ cd /path/to/project/handson-ml $ docker run --name handson-ml --gpus all -p 8888:8888 -p 6006:6006 --log-opt mode=non-blocking --log-opt max-buffer-size=50m -v `pwd`:/home/devel/handson-ml ageron/handson-ml:latest-gpu /opt/conda/envs/tf1/bin/jupyter notebook --ip='0.0.0.0' --port=8888 --no-browser ``` If you are using an older version of Docker, then replace `--gpus all` with `--runtime=nvidia`. Now point your browser to the displayed URL: Jupyter should appear, and you can open a notebook and run `import tensorflow as tf` and `tf.config.list_physical_devices("GPU)` as above to confirm that TensorFlow does indeed see your GPU device(s). Lastly, to interrupt the server, press Ctrl-C, then run: ```bash $ docker rm handson-ml ``` This will remove the container so you can start a new one later (but it will not remove the image or the notebooks, don't worry!). Have fun!