Goals:
First, let's mount our Google drive! (Note: please wait! this will take a moment!)
from google.colab import drive
drive.mount('/myDrive/')
Go to this URL in a browser: https://accounts.google.com/o/oauth2/auth?client_id=947318989803-6bn6qk8qdgf4n4g3pfee6491hc0brc4i.apps.googleusercontent.com&redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&scope=email%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdocs.test%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive.photos.readonly%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fpeopleapi.readonly&response_type=code Enter your authorization code: ·········· Mounted at /myDrive/
I assumed that you have a folder named "Colab Notebooks" in your Google Drive. You may want to use a different folder. Then, run the following command to clone the project into the folder.
!git clone https://github.com/pytorch/examples "/myDrive/My Drive/Colab Notebooks/pytorch-examples"
Cloning into '/myDrive/My Drive/Colab Notebooks/pytorch-examples'... remote: Enumerating objects: 9, done. remote: Counting objects: 100% (9/9), done. remote: Compressing objects: 100% (8/8), done. remote: Total 1724 (delta 1), reused 1 (delta 1), pack-reused 1715 Receiving objects: 100% (1724/1724), 38.82 MiB | 8.37 MiB/s, done. Resolving deltas: 100% (905/905), done.
Let's go into mnist directory in pytorch-examples. This folder contains a python file named main.py that implements a convolution network to classify MNIST digits using Pytorch.
root = "/myDrive/My Drive/Colab Notebooks/pytorch-examples"
import os
os.chdir(root+'/mnist')
!ls
main.py README.md requirements.txt
We need Pytorch. However, Google Colab does not have Pytorch. So, we need to install it. Note that if you go to the official site of Pytorch, you will see that you need to specify your machine's configuration. Here, we are using an instance of Google Colab's virtual machine. Colab works on Linux. Run the following codes to install pytorch. (Or try this !pip3 install torch torchvision)
from os import path
from wheel.pep425tags import get_abbr_impl, get_impl_ver, get_abi_tag
platform = '{}{}-{}'.format(get_abbr_impl(), get_impl_ver(), get_abi_tag())
accelerator = 'cu80' if path.exists('/opt/bin/nvidia-smi') else 'cpu'
!pip install -q http://download.pytorch.org/whl/{accelerator}/torch-0.4.0-{platform}-linux_x86_64.whl torchvision
tcmalloc: large alloc 1073750016 bytes == 0x555920bd4000 @ 0x7fd9b0d992a4 0x5558c4f25b68 0x5558c501192d 0x5558c4f3901a 0x5558c4f3dd72 0x5558c4f368ca 0x5558c4f3e24e 0x5558c4f368ca 0x5558c4f3e24e 0x5558c4f368ca 0x5558c4f3e24e 0x5558c4f368ca 0x5558c4f3e7d3 0x5558c4f368ca 0x5558c4f3e24e 0x5558c4f368ca 0x5558c4f3e24e 0x5558c4f3dd72 0x5558c4f3dd72 0x5558c4f368ca 0x5558c4f3e7d3 0x5558c4f3dd72 0x5558c4f368ca 0x5558c4f3e7d3 0x5558c4f368ca 0x5558c4f3e7d3 0x5558c4f368ca 0x5558c4f3e24e 0x5558c4f368ca 0x5558c4f361e9 0x5558c4f66bdf
import torch
print(torch.__version__)
import torchvision
print(torchvision.__version__)
0.4.0 0.2.1
Before running the following code to train an mnist classifier, edit the file main.py and remove the reduction argument. I am not a Pytorch guy! So I don't know why this keyword does not work! (Note that removing this keyword will result in performing averaging instead of summation in the loss function)
By running the following command, the MNIST dataset is downloaded and saved in ../data (you can check it in your Google Drive!) Then, the network is trained.
!python main.py
Train Epoch: 1 [0/60000 (0%)] Loss: 2.373651 Train Epoch: 1 [640/60000 (1%)] Loss: 2.310517 Train Epoch: 1 [1280/60000 (2%)] Loss: 2.281828 Train Epoch: 1 [1920/60000 (3%)] Loss: 2.315808 Train Epoch: 1 [2560/60000 (4%)] Loss: 2.235439 Train Epoch: 1 [3200/60000 (5%)] Loss: 2.234249 Train Epoch: 1 [3840/60000 (6%)] Loss: 2.226109 Train Epoch: 1 [4480/60000 (7%)] Loss: 2.228646 Train Epoch: 1 [5120/60000 (9%)] Loss: 2.132811 Train Epoch: 1 [5760/60000 (10%)] Loss: 2.113179 Train Epoch: 1 [6400/60000 (11%)] Loss: 2.030113 Train Epoch: 1 [7040/60000 (12%)] Loss: 1.877119 Train Epoch: 1 [7680/60000 (13%)] Loss: 1.894014 Train Epoch: 1 [8320/60000 (14%)] Loss: 1.725610 Train Epoch: 1 [8960/60000 (15%)] Loss: 1.739437 Train Epoch: 1 [9600/60000 (16%)] Loss: 1.533461 Train Epoch: 1 [10240/60000 (17%)] Loss: 1.549235 Train Epoch: 1 [10880/60000 (18%)] Loss: 1.498123 Train Epoch: 1 [11520/60000 (19%)] Loss: 1.502510 Train Epoch: 1 [12160/60000 (20%)] Loss: 1.326281 Train Epoch: 1 [12800/60000 (21%)] Loss: 1.236888 Train Epoch: 1 [13440/60000 (22%)] Loss: 1.243505 Train Epoch: 1 [14080/60000 (23%)] Loss: 0.932353 Train Epoch: 1 [14720/60000 (25%)] Loss: 0.943364 Train Epoch: 1 [15360/60000 (26%)] Loss: 1.114814 Train Epoch: 1 [16000/60000 (27%)] Loss: 1.122974 Train Epoch: 1 [16640/60000 (28%)] Loss: 1.039028 Train Epoch: 1 [17280/60000 (29%)] Loss: 1.034557 Train Epoch: 1 [17920/60000 (30%)] Loss: 0.837697 Train Epoch: 1 [18560/60000 (31%)] Loss: 1.096114 Train Epoch: 1 [19200/60000 (32%)] Loss: 1.017020 Train Epoch: 1 [19840/60000 (33%)] Loss: 1.125287 Train Epoch: 1 [20480/60000 (34%)] Loss: 0.972650 Train Epoch: 1 [21120/60000 (35%)] Loss: 0.840127 Train Epoch: 1 [21760/60000 (36%)] Loss: 0.832300 Train Epoch: 1 [22400/60000 (37%)] Loss: 0.779812 Train Epoch: 1 [23040/60000 (38%)] Loss: 0.668639 Train Epoch: 1 [23680/60000 (39%)] Loss: 0.954052 Train Epoch: 1 [24320/60000 (41%)] Loss: 0.666618 Train Epoch: 1 [24960/60000 (42%)] Loss: 0.823918 Train Epoch: 1 [25600/60000 (43%)] Loss: 0.777713 Train Epoch: 1 [26240/60000 (44%)] Loss: 0.479919 Train Epoch: 1 [26880/60000 (45%)] Loss: 0.742106 Train Epoch: 1 [27520/60000 (46%)] Loss: 0.760975 Train Epoch: 1 [28160/60000 (47%)] Loss: 0.641163 Train Epoch: 1 [28800/60000 (48%)] Loss: 0.729141 Train Epoch: 1 [29440/60000 (49%)] Loss: 0.809032 Train Epoch: 1 [30080/60000 (50%)] Loss: 0.807876 Train Epoch: 1 [30720/60000 (51%)] Loss: 0.660017 Train Epoch: 1 [31360/60000 (52%)] Loss: 1.038869 Train Epoch: 1 [32000/60000 (53%)] Loss: 0.503807 Train Epoch: 1 [32640/60000 (54%)] Loss: 0.485682 Train Epoch: 1 [33280/60000 (55%)] Loss: 0.477208 Train Epoch: 1 [33920/60000 (57%)] Loss: 0.638251 Train Epoch: 1 [34560/60000 (58%)] Loss: 0.835420 Train Epoch: 1 [35200/60000 (59%)] Loss: 0.453797 Train Epoch: 1 [35840/60000 (60%)] Loss: 0.878027 Train Epoch: 1 [36480/60000 (61%)] Loss: 0.486478 Train Epoch: 1 [37120/60000 (62%)] Loss: 0.753955 Train Epoch: 1 [37760/60000 (63%)] Loss: 0.654475 Train Epoch: 1 [38400/60000 (64%)] Loss: 0.479842 Train Epoch: 1 [39040/60000 (65%)] Loss: 0.550988 Train Epoch: 1 [39680/60000 (66%)] Loss: 0.619028 Train Epoch: 1 [40320/60000 (67%)] Loss: 0.507909 Train Epoch: 1 [40960/60000 (68%)] Loss: 0.670269 Train Epoch: 1 [41600/60000 (69%)] Loss: 0.431634 Train Epoch: 1 [42240/60000 (70%)] Loss: 0.467366 Train Epoch: 1 [42880/60000 (71%)] Loss: 0.463084 Train Epoch: 1 [43520/60000 (72%)] Loss: 0.654702 Train Epoch: 1 [44160/60000 (74%)] Loss: 0.604984 Train Epoch: 1 [44800/60000 (75%)] Loss: 0.447935 Train Epoch: 1 [45440/60000 (76%)] Loss: 0.448044 Train Epoch: 1 [46080/60000 (77%)] Loss: 0.472945 Train Epoch: 1 [46720/60000 (78%)] Loss: 0.701945 Train Epoch: 1 [47360/60000 (79%)] Loss: 0.424748 Train Epoch: 1 [48000/60000 (80%)] Loss: 0.454524 Train Epoch: 1 [48640/60000 (81%)] Loss: 0.477744 Train Epoch: 1 [49280/60000 (82%)] Loss: 0.618845 Train Epoch: 1 [49920/60000 (83%)] Loss: 0.441788 Train Epoch: 1 [50560/60000 (84%)] Loss: 0.538795 Train Epoch: 1 [51200/60000 (85%)] Loss: 0.660148 Train Epoch: 1 [51840/60000 (86%)] Loss: 0.431412 Train Epoch: 1 [52480/60000 (87%)] Loss: 0.613825 Train Epoch: 1 [53120/60000 (88%)] Loss: 0.465359 Train Epoch: 1 [53760/60000 (90%)] Loss: 0.628077 Train Epoch: 1 [54400/60000 (91%)] Loss: 0.370612 Train Epoch: 1 [55040/60000 (92%)] Loss: 0.586689 Train Epoch: 1 [55680/60000 (93%)] Loss: 0.390014 Train Epoch: 1 [56320/60000 (94%)] Loss: 0.327559 Train Epoch: 1 [56960/60000 (95%)] Loss: 0.507480 Train Epoch: 1 [57600/60000 (96%)] Loss: 0.370664 Train Epoch: 1 [58240/60000 (97%)] Loss: 0.388799 Train Epoch: 1 [58880/60000 (98%)] Loss: 0.523447 Train Epoch: 1 [59520/60000 (99%)] Loss: 0.598093 Test set: Average loss: 0.0002, Accuracy: 9405/10000 (94%) Train Epoch: 2 [0/60000 (0%)] Loss: 0.470916 Train Epoch: 2 [640/60000 (1%)] Loss: 0.210711 Train Epoch: 2 [1280/60000 (2%)] Loss: 0.356534 Train Epoch: 2 [1920/60000 (3%)] Loss: 0.506210 Train Epoch: 2 [2560/60000 (4%)] Loss: 0.237110 Train Epoch: 2 [3200/60000 (5%)] Loss: 0.278420 Train Epoch: 2 [3840/60000 (6%)] Loss: 0.475663 Train Epoch: 2 [4480/60000 (7%)] Loss: 0.474693 Train Epoch: 2 [5120/60000 (9%)] Loss: 0.291893 Train Epoch: 2 [5760/60000 (10%)] Loss: 0.511268 Train Epoch: 2 [6400/60000 (11%)] Loss: 0.396071 Train Epoch: 2 [7040/60000 (12%)] Loss: 0.400062 Train Epoch: 2 [7680/60000 (13%)] Loss: 0.570561 Train Epoch: 2 [8320/60000 (14%)] Loss: 0.280094 Train Epoch: 2 [8960/60000 (15%)] Loss: 0.389267 Train Epoch: 2 [9600/60000 (16%)] Loss: 0.415635 Train Epoch: 2 [10240/60000 (17%)] Loss: 0.509592 Train Epoch: 2 [10880/60000 (18%)] Loss: 0.323109 Train Epoch: 2 [11520/60000 (19%)] Loss: 0.315953 Train Epoch: 2 [12160/60000 (20%)] Loss: 0.425256 Train Epoch: 2 [12800/60000 (21%)] Loss: 0.477006 Train Epoch: 2 [13440/60000 (22%)] Loss: 0.512797 Train Epoch: 2 [14080/60000 (23%)] Loss: 0.768390 Train Epoch: 2 [14720/60000 (25%)] Loss: 0.373473 Train Epoch: 2 [15360/60000 (26%)] Loss: 0.403764 Train Epoch: 2 [16000/60000 (27%)] Loss: 0.436422 Train Epoch: 2 [16640/60000 (28%)] Loss: 0.377981 Train Epoch: 2 [17280/60000 (29%)] Loss: 0.309124 Train Epoch: 2 [17920/60000 (30%)] Loss: 0.335952 Train Epoch: 2 [18560/60000 (31%)] Loss: 0.449727 Train Epoch: 2 [19200/60000 (32%)] Loss: 0.629582 Train Epoch: 2 [19840/60000 (33%)] Loss: 0.369373 Train Epoch: 2 [20480/60000 (34%)] Loss: 0.421323 Train Epoch: 2 [21120/60000 (35%)] Loss: 0.362353 Train Epoch: 2 [21760/60000 (36%)] Loss: 0.602461 Train Epoch: 2 [22400/60000 (37%)] Loss: 0.332647 Train Epoch: 2 [23040/60000 (38%)] Loss: 0.434016 Train Epoch: 2 [23680/60000 (39%)] Loss: 0.309780 Train Epoch: 2 [24320/60000 (41%)] Loss: 0.439433 Train Epoch: 2 [24960/60000 (42%)] Loss: 0.244156 Train Epoch: 2 [25600/60000 (43%)] Loss: 0.320011 Train Epoch: 2 [26240/60000 (44%)] Loss: 0.290765 Train Epoch: 2 [26880/60000 (45%)] Loss: 0.641062 Train Epoch: 2 [27520/60000 (46%)] Loss: 0.275221 Train Epoch: 2 [28160/60000 (47%)] Loss: 0.330871 Train Epoch: 2 [28800/60000 (48%)] Loss: 0.447913 Train Epoch: 2 [29440/60000 (49%)] Loss: 0.417425 Train Epoch: 2 [30080/60000 (50%)] Loss: 0.439291 Train Epoch: 2 [30720/60000 (51%)] Loss: 0.339246 Train Epoch: 2 [31360/60000 (52%)] Loss: 0.292304 Train Epoch: 2 [32000/60000 (53%)] Loss: 0.505118 Train Epoch: 2 [32640/60000 (54%)] Loss: 0.425261 Train Epoch: 2 [33280/60000 (55%)] Loss: 0.426719 Train Epoch: 2 [33920/60000 (57%)] Loss: 0.364924 Train Epoch: 2 [34560/60000 (58%)] Loss: 0.406070 Train Epoch: 2 [35200/60000 (59%)] Loss: 0.636075 Train Epoch: 2 [35840/60000 (60%)] Loss: 0.460837 Train Epoch: 2 [36480/60000 (61%)] Loss: 0.400673 Train Epoch: 2 [37120/60000 (62%)] Loss: 0.287236 Train Epoch: 2 [37760/60000 (63%)] Loss: 0.322719 Train Epoch: 2 [38400/60000 (64%)] Loss: 0.254464 Train Epoch: 2 [39040/60000 (65%)] Loss: 0.345899 Train Epoch: 2 [39680/60000 (66%)] Loss: 0.362181 Train Epoch: 2 [40320/60000 (67%)] Loss: 0.308415 Train Epoch: 2 [40960/60000 (68%)] Loss: 0.338849 Train Epoch: 2 [41600/60000 (69%)] Loss: 0.738289 Train Epoch: 2 [42240/60000 (70%)] Loss: 0.407792 Train Epoch: 2 [42880/60000 (71%)] Loss: 0.252234 Train Epoch: 2 [43520/60000 (72%)] Loss: 0.234799 Train Epoch: 2 [44160/60000 (74%)] Loss: 0.236716 Train Epoch: 2 [44800/60000 (75%)] Loss: 0.373811 Train Epoch: 2 [45440/60000 (76%)] Loss: 0.295831 Train Epoch: 2 [46080/60000 (77%)] Loss: 0.443412 Train Epoch: 2 [46720/60000 (78%)] Loss: 0.372460 Train Epoch: 2 [47360/60000 (79%)] Loss: 0.459296 Train Epoch: 2 [48000/60000 (80%)] Loss: 0.222927 Train Epoch: 2 [48640/60000 (81%)] Loss: 0.194848 Train Epoch: 2 [49280/60000 (82%)] Loss: 0.289654 Train Epoch: 2 [49920/60000 (83%)] Loss: 0.349498 Train Epoch: 2 [50560/60000 (84%)] Loss: 0.310730 Train Epoch: 2 [51200/60000 (85%)] Loss: 0.430103 Train Epoch: 2 [51840/60000 (86%)] Loss: 0.480240 Train Epoch: 2 [52480/60000 (87%)] Loss: 0.236234 Train Epoch: 2 [53120/60000 (88%)] Loss: 0.307188 Train Epoch: 2 [53760/60000 (90%)] Loss: 0.279072 Train Epoch: 2 [54400/60000 (91%)] Loss: 0.240781 Train Epoch: 2 [55040/60000 (92%)] Loss: 0.209316 Train Epoch: 2 [55680/60000 (93%)] Loss: 0.266671 Train Epoch: 2 [56320/60000 (94%)] Loss: 0.248095 Train Epoch: 2 [56960/60000 (95%)] Loss: 0.418421 Train Epoch: 2 [57600/60000 (96%)] Loss: 0.461359 Train Epoch: 2 [58240/60000 (97%)] Loss: 0.353400 Train Epoch: 2 [58880/60000 (98%)] Loss: 0.401144 Train Epoch: 2 [59520/60000 (99%)] Loss: 0.346248 Test set: Average loss: 0.0001, Accuracy: 9586/10000 (96%) Train Epoch: 3 [0/60000 (0%)] Loss: 0.511905 Train Epoch: 3 [640/60000 (1%)] Loss: 0.301071 Train Epoch: 3 [1280/60000 (2%)] Loss: 0.469065 Train Epoch: 3 [1920/60000 (3%)] Loss: 0.375524 Train Epoch: 3 [2560/60000 (4%)] Loss: 0.216433 Train Epoch: 3 [3200/60000 (5%)] Loss: 0.369009 Train Epoch: 3 [3840/60000 (6%)] Loss: 0.278727 Train Epoch: 3 [4480/60000 (7%)] Loss: 0.427139 Train Epoch: 3 [5120/60000 (9%)] Loss: 0.233043 Train Epoch: 3 [5760/60000 (10%)] Loss: 0.310233 Train Epoch: 3 [6400/60000 (11%)] Loss: 0.337354 Train Epoch: 3 [7040/60000 (12%)] Loss: 0.344165 Train Epoch: 3 [7680/60000 (13%)] Loss: 0.395338 Train Epoch: 3 [8320/60000 (14%)] Loss: 0.310753 Train Epoch: 3 [8960/60000 (15%)] Loss: 0.419105 Train Epoch: 3 [9600/60000 (16%)] Loss: 0.286377 Train Epoch: 3 [10240/60000 (17%)] Loss: 0.194102 Train Epoch: 3 [10880/60000 (18%)] Loss: 0.197192 Train Epoch: 3 [11520/60000 (19%)] Loss: 0.403153 Train Epoch: 3 [12160/60000 (20%)] Loss: 0.274780 Train Epoch: 3 [12800/60000 (21%)] Loss: 0.220420 Train Epoch: 3 [13440/60000 (22%)] Loss: 0.347923 Train Epoch: 3 [14080/60000 (23%)] Loss: 0.273567 Train Epoch: 3 [14720/60000 (25%)] Loss: 0.253692 Train Epoch: 3 [15360/60000 (26%)] Loss: 0.199628 Train Epoch: 3 [16000/60000 (27%)] Loss: 0.212681 Train Epoch: 3 [16640/60000 (28%)] Loss: 0.246277 Train Epoch: 3 [17280/60000 (29%)] Loss: 0.371214 Train Epoch: 3 [17920/60000 (30%)] Loss: 0.352649 Train Epoch: 3 [18560/60000 (31%)] Loss: 0.329059 Train Epoch: 3 [19200/60000 (32%)] Loss: 0.520884 Train Epoch: 3 [19840/60000 (33%)] Loss: 0.321161 Train Epoch: 3 [20480/60000 (34%)] Loss: 0.173570 Train Epoch: 3 [21120/60000 (35%)] Loss: 0.213814 Train Epoch: 3 [21760/60000 (36%)] Loss: 0.365140 Train Epoch: 3 [22400/60000 (37%)] Loss: 0.550874 Train Epoch: 3 [23040/60000 (38%)] Loss: 0.315863 Train Epoch: 3 [23680/60000 (39%)] Loss: 0.407930 Train Epoch: 3 [24320/60000 (41%)] Loss: 0.293472 Train Epoch: 3 [24960/60000 (42%)] Loss: 0.202417 Train Epoch: 3 [25600/60000 (43%)] Loss: 0.229407 Train Epoch: 3 [26240/60000 (44%)] Loss: 0.127372 Train Epoch: 3 [26880/60000 (45%)] Loss: 0.231388 Train Epoch: 3 [27520/60000 (46%)] Loss: 0.252991 Train Epoch: 3 [28160/60000 (47%)] Loss: 0.323540 Train Epoch: 3 [28800/60000 (48%)] Loss: 0.538975 Train Epoch: 3 [29440/60000 (49%)] Loss: 0.322273 Train Epoch: 3 [30080/60000 (50%)] Loss: 0.183331 Train Epoch: 3 [30720/60000 (51%)] Loss: 0.253175 Train Epoch: 3 [31360/60000 (52%)] Loss: 0.324845 Train Epoch: 3 [32000/60000 (53%)] Loss: 0.251241 Train Epoch: 3 [32640/60000 (54%)] Loss: 0.336655 Train Epoch: 3 [33280/60000 (55%)] Loss: 0.313223 Train Epoch: 3 [33920/60000 (57%)] Loss: 0.454837 Train Epoch: 3 [34560/60000 (58%)] Loss: 0.170750 Train Epoch: 3 [35200/60000 (59%)] Loss: 0.244435 Train Epoch: 3 [35840/60000 (60%)] Loss: 0.195073 Train Epoch: 3 [36480/60000 (61%)] Loss: 0.345108 Train Epoch: 3 [37120/60000 (62%)] Loss: 0.278598 Train Epoch: 3 [37760/60000 (63%)] Loss: 0.218793 Train Epoch: 3 [38400/60000 (64%)] Loss: 0.287310 Train Epoch: 3 [39040/60000 (65%)] Loss: 0.274616 Train Epoch: 3 [39680/60000 (66%)] Loss: 0.097904 Train Epoch: 3 [40320/60000 (67%)] Loss: 0.341571 Train Epoch: 3 [40960/60000 (68%)] Loss: 0.315088 Train Epoch: 3 [41600/60000 (69%)] Loss: 0.365613 Train Epoch: 3 [42240/60000 (70%)] Loss: 0.293647 Train Epoch: 3 [42880/60000 (71%)] Loss: 0.196777 Train Epoch: 3 [43520/60000 (72%)] Loss: 0.088222 Train Epoch: 3 [44160/60000 (74%)] Loss: 0.392760 Train Epoch: 3 [44800/60000 (75%)] Loss: 0.097802 Train Epoch: 3 [45440/60000 (76%)] Loss: 0.242959 Train Epoch: 3 [46080/60000 (77%)] Loss: 0.233115 Train Epoch: 3 [46720/60000 (78%)] Loss: 0.308320 Train Epoch: 3 [47360/60000 (79%)] Loss: 0.230800 Train Epoch: 3 [48000/60000 (80%)] Loss: 0.321063 Train Epoch: 3 [48640/60000 (81%)] Loss: 0.375167 Train Epoch: 3 [49280/60000 (82%)] Loss: 0.139469 Train Epoch: 3 [49920/60000 (83%)] Loss: 0.272992 Train Epoch: 3 [50560/60000 (84%)] Loss: 0.340776 Train Epoch: 3 [51200/60000 (85%)] Loss: 0.355403 Train Epoch: 3 [51840/60000 (86%)] Loss: 0.155090 Train Epoch: 3 [52480/60000 (87%)] Loss: 0.179023 Train Epoch: 3 [53120/60000 (88%)] Loss: 0.303225 Train Epoch: 3 [53760/60000 (90%)] Loss: 0.127476 Train Epoch: 3 [54400/60000 (91%)] Loss: 0.305985 Train Epoch: 3 [55040/60000 (92%)] Loss: 0.334631 Train Epoch: 3 [55680/60000 (93%)] Loss: 0.361212 Train Epoch: 3 [56320/60000 (94%)] Loss: 0.289112 Train Epoch: 3 [56960/60000 (95%)] Loss: 0.313953 Train Epoch: 3 [57600/60000 (96%)] Loss: 0.548181 Train Epoch: 3 [58240/60000 (97%)] Loss: 0.134213 Train Epoch: 3 [58880/60000 (98%)] Loss: 0.338072 Train Epoch: 3 [59520/60000 (99%)] Loss: 0.276611 Test set: Average loss: 0.0001, Accuracy: 9697/10000 (97%) Train Epoch: 4 [0/60000 (0%)] Loss: 0.383451 Train Epoch: 4 [640/60000 (1%)] Loss: 0.203270 Train Epoch: 4 [1280/60000 (2%)] Loss: 0.238818 Train Epoch: 4 [1920/60000 (3%)] Loss: 0.322409 Train Epoch: 4 [2560/60000 (4%)] Loss: 0.375902 Train Epoch: 4 [3200/60000 (5%)] Loss: 0.164150 Train Epoch: 4 [3840/60000 (6%)] Loss: 0.212402 Train Epoch: 4 [4480/60000 (7%)] Loss: 0.251358 Train Epoch: 4 [5120/60000 (9%)] Loss: 0.216792 Train Epoch: 4 [5760/60000 (10%)] Loss: 0.321498 Train Epoch: 4 [6400/60000 (11%)] Loss: 0.297037 Train Epoch: 4 [7040/60000 (12%)] Loss: 0.182845 Train Epoch: 4 [7680/60000 (13%)] Loss: 0.246318 Train Epoch: 4 [8320/60000 (14%)] Loss: 0.477240 Train Epoch: 4 [8960/60000 (15%)] Loss: 0.179840 Train Epoch: 4 [9600/60000 (16%)] Loss: 0.195521 Train Epoch: 4 [10240/60000 (17%)] Loss: 0.253881 Train Epoch: 4 [10880/60000 (18%)] Loss: 0.277786 Train Epoch: 4 [11520/60000 (19%)] Loss: 0.226974 Train Epoch: 4 [12160/60000 (20%)] Loss: 0.236465 Train Epoch: 4 [12800/60000 (21%)] Loss: 0.134410 Train Epoch: 4 [13440/60000 (22%)] Loss: 0.143215 Train Epoch: 4 [14080/60000 (23%)] Loss: 0.286758 Train Epoch: 4 [14720/60000 (25%)] Loss: 0.213467 Train Epoch: 4 [15360/60000 (26%)] Loss: 0.173835 Train Epoch: 4 [16000/60000 (27%)] Loss: 0.199458 Train Epoch: 4 [16640/60000 (28%)] Loss: 0.285372 Train Epoch: 4 [17280/60000 (29%)] Loss: 0.280726 Train Epoch: 4 [17920/60000 (30%)] Loss: 0.313475 Train Epoch: 4 [18560/60000 (31%)] Loss: 0.294834 Train Epoch: 4 [19200/60000 (32%)] Loss: 0.207383 Train Epoch: 4 [19840/60000 (33%)] Loss: 0.219744 Train Epoch: 4 [20480/60000 (34%)] Loss: 0.275060 Train Epoch: 4 [21120/60000 (35%)] Loss: 0.468476 Train Epoch: 4 [21760/60000 (36%)] Loss: 0.249823 Train Epoch: 4 [22400/60000 (37%)] Loss: 0.473506 Train Epoch: 4 [23040/60000 (38%)] Loss: 0.174089 Train Epoch: 4 [23680/60000 (39%)] Loss: 0.181804 Train Epoch: 4 [24320/60000 (41%)] Loss: 0.225974 Train Epoch: 4 [24960/60000 (42%)] Loss: 0.214781 Train Epoch: 4 [25600/60000 (43%)] Loss: 0.360860 Train Epoch: 4 [26240/60000 (44%)] Loss: 0.292142 Train Epoch: 4 [26880/60000 (45%)] Loss: 0.212564 Train Epoch: 4 [27520/60000 (46%)] Loss: 0.421580 Train Epoch: 4 [28160/60000 (47%)] Loss: 0.145797 Train Epoch: 4 [28800/60000 (48%)] Loss: 0.186721 Train Epoch: 4 [29440/60000 (49%)] Loss: 0.150159 Train Epoch: 4 [30080/60000 (50%)] Loss: 0.378528 Train Epoch: 4 [30720/60000 (51%)] Loss: 0.144539 Train Epoch: 4 [31360/60000 (52%)] Loss: 0.226689 Train Epoch: 4 [32000/60000 (53%)] Loss: 0.164089 Train Epoch: 4 [32640/60000 (54%)] Loss: 0.304837 Train Epoch: 4 [33280/60000 (55%)] Loss: 0.265629 Train Epoch: 4 [33920/60000 (57%)] Loss: 0.287068 Train Epoch: 4 [34560/60000 (58%)] Loss: 0.285403 Train Epoch: 4 [35200/60000 (59%)] Loss: 0.269815 Train Epoch: 4 [35840/60000 (60%)] Loss: 0.085518 Train Epoch: 4 [36480/60000 (61%)] Loss: 0.420862 Train Epoch: 4 [37120/60000 (62%)] Loss: 0.136135 Train Epoch: 4 [37760/60000 (63%)] Loss: 0.299046 Train Epoch: 4 [38400/60000 (64%)] Loss: 0.195031 Train Epoch: 4 [39040/60000 (65%)] Loss: 0.193720 Train Epoch: 4 [39680/60000 (66%)] Loss: 0.295083 Train Epoch: 4 [40320/60000 (67%)] Loss: 0.329599 Train Epoch: 4 [40960/60000 (68%)] Loss: 0.221367 Train Epoch: 4 [41600/60000 (69%)] Loss: 0.352944 Train Epoch: 4 [42240/60000 (70%)] Loss: 0.220328 Train Epoch: 4 [42880/60000 (71%)] Loss: 0.427901 Train Epoch: 4 [43520/60000 (72%)] Loss: 0.191625 Train Epoch: 4 [44160/60000 (74%)] Loss: 0.222399 Train Epoch: 4 [44800/60000 (75%)] Loss: 0.089040 Train Epoch: 4 [45440/60000 (76%)] Loss: 0.169473 Train Epoch: 4 [46080/60000 (77%)] Loss: 0.160626 Train Epoch: 4 [46720/60000 (78%)] Loss: 0.215215 Train Epoch: 4 [47360/60000 (79%)] Loss: 0.158428 Train Epoch: 4 [48000/60000 (80%)] Loss: 0.158811 Train Epoch: 4 [48640/60000 (81%)] Loss: 0.154177 Train Epoch: 4 [49280/60000 (82%)] Loss: 0.264236 Train Epoch: 4 [49920/60000 (83%)] Loss: 0.363211 Train Epoch: 4 [50560/60000 (84%)] Loss: 0.147806 Train Epoch: 4 [51200/60000 (85%)] Loss: 0.141175 Train Epoch: 4 [51840/60000 (86%)] Loss: 0.275700 Train Epoch: 4 [52480/60000 (87%)] Loss: 0.275807 Train Epoch: 4 [53120/60000 (88%)] Loss: 0.471929 Train Epoch: 4 [53760/60000 (90%)] Loss: 0.334664 Train Epoch: 4 [54400/60000 (91%)] Loss: 0.189815 Train Epoch: 4 [55040/60000 (92%)] Loss: 0.380074 Train Epoch: 4 [55680/60000 (93%)] Loss: 0.143756 Train Epoch: 4 [56320/60000 (94%)] Loss: 0.210418 Train Epoch: 4 [56960/60000 (95%)] Loss: 0.074293 Train Epoch: 4 [57600/60000 (96%)] Loss: 0.281315 Train Epoch: 4 [58240/60000 (97%)] Loss: 0.431379 Train Epoch: 4 [58880/60000 (98%)] Loss: 0.234782 Train Epoch: 4 [59520/60000 (99%)] Loss: 0.443469 Test set: Average loss: 0.0001, Accuracy: 9744/10000 (97%) Train Epoch: 5 [0/60000 (0%)] Loss: 0.526699 Train Epoch: 5 [640/60000 (1%)] Loss: 0.146328 Train Epoch: 5 [1280/60000 (2%)] Loss: 0.267061 Train Epoch: 5 [1920/60000 (3%)] Loss: 0.197870 Train Epoch: 5 [2560/60000 (4%)] Loss: 0.365284 Train Epoch: 5 [3200/60000 (5%)] Loss: 0.405225 Train Epoch: 5 [3840/60000 (6%)] Loss: 0.393590 Train Epoch: 5 [4480/60000 (7%)] Loss: 0.301428 Train Epoch: 5 [5120/60000 (9%)] Loss: 0.172054 Train Epoch: 5 [5760/60000 (10%)] Loss: 0.321626 Train Epoch: 5 [6400/60000 (11%)] Loss: 0.132384 Train Epoch: 5 [7040/60000 (12%)] Loss: 0.233974 Train Epoch: 5 [7680/60000 (13%)] Loss: 0.377445 Train Epoch: 5 [8320/60000 (14%)] Loss: 0.168215 Train Epoch: 5 [8960/60000 (15%)] Loss: 0.131074 Train Epoch: 5 [9600/60000 (16%)] Loss: 0.324022 Train Epoch: 5 [10240/60000 (17%)] Loss: 0.171283 Train Epoch: 5 [10880/60000 (18%)] Loss: 0.090830 Train Epoch: 5 [11520/60000 (19%)] Loss: 0.382126 Train Epoch: 5 [12160/60000 (20%)] Loss: 0.131575 Train Epoch: 5 [12800/60000 (21%)] Loss: 0.201085 Train Epoch: 5 [13440/60000 (22%)] Loss: 0.260417 Train Epoch: 5 [14080/60000 (23%)] Loss: 0.209992 Train Epoch: 5 [14720/60000 (25%)] Loss: 0.284642 Train Epoch: 5 [15360/60000 (26%)] Loss: 0.217786 Train Epoch: 5 [16000/60000 (27%)] Loss: 0.237606 Train Epoch: 5 [16640/60000 (28%)] Loss: 0.222922 Train Epoch: 5 [17280/60000 (29%)] Loss: 0.181022 Train Epoch: 5 [17920/60000 (30%)] Loss: 0.223055 Train Epoch: 5 [18560/60000 (31%)] Loss: 0.251572 Train Epoch: 5 [19200/60000 (32%)] Loss: 0.305420 Train Epoch: 5 [19840/60000 (33%)] Loss: 0.134483 Train Epoch: 5 [20480/60000 (34%)] Loss: 0.146125 Train Epoch: 5 [21120/60000 (35%)] Loss: 0.343406 Train Epoch: 5 [21760/60000 (36%)] Loss: 0.265331 Train Epoch: 5 [22400/60000 (37%)] Loss: 0.228911 Train Epoch: 5 [23040/60000 (38%)] Loss: 0.142105 Train Epoch: 5 [23680/60000 (39%)] Loss: 0.202304 Train Epoch: 5 [24320/60000 (41%)] Loss: 0.246190 Train Epoch: 5 [24960/60000 (42%)] Loss: 0.240228 Train Epoch: 5 [25600/60000 (43%)] Loss: 0.192894 Train Epoch: 5 [26240/60000 (44%)] Loss: 0.173004 Train Epoch: 5 [26880/60000 (45%)] Loss: 0.163632 Train Epoch: 5 [27520/60000 (46%)] Loss: 0.146411 Train Epoch: 5 [28160/60000 (47%)] Loss: 0.426880 Train Epoch: 5 [28800/60000 (48%)] Loss: 0.123550 Train Epoch: 5 [29440/60000 (49%)] Loss: 0.302521 Train Epoch: 5 [30080/60000 (50%)] Loss: 0.125200 Train Epoch: 5 [30720/60000 (51%)] Loss: 0.138874 Train Epoch: 5 [31360/60000 (52%)] Loss: 0.073579 Train Epoch: 5 [32000/60000 (53%)] Loss: 0.245884 Train Epoch: 5 [32640/60000 (54%)] Loss: 0.272967 Train Epoch: 5 [33280/60000 (55%)] Loss: 0.331716 Train Epoch: 5 [33920/60000 (57%)] Loss: 0.282765 Train Epoch: 5 [34560/60000 (58%)] Loss: 0.087803 Train Epoch: 5 [35200/60000 (59%)] Loss: 0.217498 Train Epoch: 5 [35840/60000 (60%)] Loss: 0.137978 Train Epoch: 5 [36480/60000 (61%)] Loss: 0.173309 Train Epoch: 5 [37120/60000 (62%)] Loss: 0.279590 Train Epoch: 5 [37760/60000 (63%)] Loss: 0.263498 Train Epoch: 5 [38400/60000 (64%)] Loss: 0.181718 Train Epoch: 5 [39040/60000 (65%)] Loss: 0.255031 Train Epoch: 5 [39680/60000 (66%)] Loss: 0.097662 Train Epoch: 5 [40320/60000 (67%)] Loss: 0.288037 Train Epoch: 5 [40960/60000 (68%)] Loss: 0.351782 Train Epoch: 5 [41600/60000 (69%)] Loss: 0.247443 Train Epoch: 5 [42240/60000 (70%)] Loss: 0.121090 Train Epoch: 5 [42880/60000 (71%)] Loss: 0.247940 Train Epoch: 5 [43520/60000 (72%)] Loss: 0.095442 Train Epoch: 5 [44160/60000 (74%)] Loss: 0.327494 Train Epoch: 5 [44800/60000 (75%)] Loss: 0.210919 Train Epoch: 5 [45440/60000 (76%)] Loss: 0.308306 Train Epoch: 5 [46080/60000 (77%)] Loss: 0.283116 Train Epoch: 5 [46720/60000 (78%)] Loss: 0.322720 Train Epoch: 5 [47360/60000 (79%)] Loss: 0.141194 Train Epoch: 5 [48000/60000 (80%)] Loss: 0.155493 Train Epoch: 5 [48640/60000 (81%)] Loss: 0.279040 Train Epoch: 5 [49280/60000 (82%)] Loss: 0.143175 Train Epoch: 5 [49920/60000 (83%)] Loss: 0.186716 Train Epoch: 5 [50560/60000 (84%)] Loss: 0.148668 Train Epoch: 5 [51200/60000 (85%)] Loss: 0.187844 Train Epoch: 5 [51840/60000 (86%)] Loss: 0.330497 Train Epoch: 5 [52480/60000 (87%)] Loss: 0.350030 Train Epoch: 5 [53120/60000 (88%)] Loss: 0.300682 Train Epoch: 5 [53760/60000 (90%)] Loss: 0.188495 Train Epoch: 5 [54400/60000 (91%)] Loss: 0.231396 Train Epoch: 5 [55040/60000 (92%)] Loss: 0.180557 Train Epoch: 5 [55680/60000 (93%)] Loss: 0.145958 Train Epoch: 5 [56320/60000 (94%)] Loss: 0.422280 Train Epoch: 5 [56960/60000 (95%)] Loss: 0.135156 Train Epoch: 5 [57600/60000 (96%)] Loss: 0.201596 Train Epoch: 5 [58240/60000 (97%)] Loss: 0.251806 Train Epoch: 5 [58880/60000 (98%)] Loss: 0.355534 Train Epoch: 5 [59520/60000 (99%)] Loss: 0.289681 Test set: Average loss: 0.0001, Accuracy: 9757/10000 (98%) Train Epoch: 6 [0/60000 (0%)] Loss: 0.241769 Train Epoch: 6 [640/60000 (1%)] Loss: 0.204252 Train Epoch: 6 [1280/60000 (2%)] Loss: 0.337168 Train Epoch: 6 [1920/60000 (3%)] Loss: 0.192640 Train Epoch: 6 [2560/60000 (4%)] Loss: 0.167889 Train Epoch: 6 [3200/60000 (5%)] Loss: 0.206232 Train Epoch: 6 [3840/60000 (6%)] Loss: 0.158683 Train Epoch: 6 [4480/60000 (7%)] Loss: 0.264739 Train Epoch: 6 [5120/60000 (9%)] Loss: 0.226367 Train Epoch: 6 [5760/60000 (10%)] Loss: 0.064817 Train Epoch: 6 [6400/60000 (11%)] Loss: 0.244415 Train Epoch: 6 [7040/60000 (12%)] Loss: 0.208979 Train Epoch: 6 [7680/60000 (13%)] Loss: 0.157422 Train Epoch: 6 [8320/60000 (14%)] Loss: 0.271373 Train Epoch: 6 [8960/60000 (15%)] Loss: 0.200890 Train Epoch: 6 [9600/60000 (16%)] Loss: 0.223573 Train Epoch: 6 [10240/60000 (17%)] Loss: 0.272707 Train Epoch: 6 [10880/60000 (18%)] Loss: 0.106059 Train Epoch: 6 [11520/60000 (19%)] Loss: 0.234719 Train Epoch: 6 [12160/60000 (20%)] Loss: 0.153302 Train Epoch: 6 [12800/60000 (21%)] Loss: 0.286034 Train Epoch: 6 [13440/60000 (22%)] Loss: 0.205256 Train Epoch: 6 [14080/60000 (23%)] Loss: 0.225036 Train Epoch: 6 [14720/60000 (25%)] Loss: 0.177635 Train Epoch: 6 [15360/60000 (26%)] Loss: 0.355354 Train Epoch: 6 [16000/60000 (27%)] Loss: 0.131433 Train Epoch: 6 [16640/60000 (28%)] Loss: 0.120960 Train Epoch: 6 [17280/60000 (29%)] Loss: 0.204652 Train Epoch: 6 [17920/60000 (30%)] Loss: 0.229926 Train Epoch: 6 [18560/60000 (31%)] Loss: 0.174806 Train Epoch: 6 [19200/60000 (32%)] Loss: 0.183799 Train Epoch: 6 [19840/60000 (33%)] Loss: 0.239440 Train Epoch: 6 [20480/60000 (34%)] Loss: 0.385592 Train Epoch: 6 [21120/60000 (35%)] Loss: 0.417520 Train Epoch: 6 [21760/60000 (36%)] Loss: 0.220968 Train Epoch: 6 [22400/60000 (37%)] Loss: 0.175077 Train Epoch: 6 [23040/60000 (38%)] Loss: 0.141992 Train Epoch: 6 [23680/60000 (39%)] Loss: 0.153215 Train Epoch: 6 [24320/60000 (41%)] Loss: 0.212310 Train Epoch: 6 [24960/60000 (42%)] Loss: 0.133514 Train Epoch: 6 [25600/60000 (43%)] Loss: 0.261122 Train Epoch: 6 [26240/60000 (44%)] Loss: 0.283612 Train Epoch: 6 [26880/60000 (45%)] Loss: 0.394510 Train Epoch: 6 [27520/60000 (46%)] Loss: 0.078062 Train Epoch: 6 [28160/60000 (47%)] Loss: 0.190539 Train Epoch: 6 [28800/60000 (48%)] Loss: 0.164226 Train Epoch: 6 [29440/60000 (49%)] Loss: 0.191821 Train Epoch: 6 [30080/60000 (50%)] Loss: 0.201779 Train Epoch: 6 [30720/60000 (51%)] Loss: 0.123816 Train Epoch: 6 [31360/60000 (52%)] Loss: 0.080671 Train Epoch: 6 [32000/60000 (53%)] Loss: 0.139634 Train Epoch: 6 [32640/60000 (54%)] Loss: 0.216230 Train Epoch: 6 [33280/60000 (55%)] Loss: 0.210472 Train Epoch: 6 [33920/60000 (57%)] Loss: 0.089774 Train Epoch: 6 [34560/60000 (58%)] Loss: 0.188745 Train Epoch: 6 [35200/60000 (59%)] Loss: 0.281396 Train Epoch: 6 [35840/60000 (60%)] Loss: 0.220271 Train Epoch: 6 [36480/60000 (61%)] Loss: 0.268656 Train Epoch: 6 [37120/60000 (62%)] Loss: 0.523529 Train Epoch: 6 [37760/60000 (63%)] Loss: 0.378742 Train Epoch: 6 [38400/60000 (64%)] Loss: 0.280642 Train Epoch: 6 [39040/60000 (65%)] Loss: 0.296551 Train Epoch: 6 [39680/60000 (66%)] Loss: 0.205661 Train Epoch: 6 [40320/60000 (67%)] Loss: 0.201317 Train Epoch: 6 [40960/60000 (68%)] Loss: 0.110486 Train Epoch: 6 [41600/60000 (69%)] Loss: 0.112068 Train Epoch: 6 [42240/60000 (70%)] Loss: 0.174746 Train Epoch: 6 [42880/60000 (71%)] Loss: 0.241495 Train Epoch: 6 [43520/60000 (72%)] Loss: 0.303549 Train Epoch: 6 [44160/60000 (74%)] Loss: 0.142237 Train Epoch: 6 [44800/60000 (75%)] Loss: 0.148537 Train Epoch: 6 [45440/60000 (76%)] Loss: 0.169703 Train Epoch: 6 [46080/60000 (77%)] Loss: 0.295839 Train Epoch: 6 [46720/60000 (78%)] Loss: 0.077247 Train Epoch: 6 [47360/60000 (79%)] Loss: 0.156170 Train Epoch: 6 [48000/60000 (80%)] Loss: 0.221277 Train Epoch: 6 [48640/60000 (81%)] Loss: 0.313294 Train Epoch: 6 [49280/60000 (82%)] Loss: 0.226801 Train Epoch: 6 [49920/60000 (83%)] Loss: 0.179619 Train Epoch: 6 [50560/60000 (84%)] Loss: 0.325818 Train Epoch: 6 [51200/60000 (85%)] Loss: 0.122626 Train Epoch: 6 [51840/60000 (86%)] Loss: 0.110798 Train Epoch: 6 [52480/60000 (87%)] Loss: 0.142486 Train Epoch: 6 [53120/60000 (88%)] Loss: 0.464635 Train Epoch: 6 [53760/60000 (90%)] Loss: 0.331677 Train Epoch: 6 [54400/60000 (91%)] Loss: 0.210523 Train Epoch: 6 [55040/60000 (92%)] Loss: 0.273049 Train Epoch: 6 [55680/60000 (93%)] Loss: 0.148325 Train Epoch: 6 [56320/60000 (94%)] Loss: 0.261807 Train Epoch: 6 [56960/60000 (95%)] Loss: 0.083502 Train Epoch: 6 [57600/60000 (96%)] Loss: 0.143875 Train Epoch: 6 [58240/60000 (97%)] Loss: 0.171255 Train Epoch: 6 [58880/60000 (98%)] Loss: 0.261712 Train Epoch: 6 [59520/60000 (99%)] Loss: 0.249309 Test set: Average loss: 0.0001, Accuracy: 9785/10000 (98%) Train Epoch: 7 [0/60000 (0%)] Loss: 0.215102 Train Epoch: 7 [640/60000 (1%)] Loss: 0.131275 Train Epoch: 7 [1280/60000 (2%)] Loss: 0.069194 Train Epoch: 7 [1920/60000 (3%)] Loss: 0.104071 Train Epoch: 7 [2560/60000 (4%)] Loss: 0.179451 Train Epoch: 7 [3200/60000 (5%)] Loss: 0.248513 Train Epoch: 7 [3840/60000 (6%)] Loss: 0.173441 Train Epoch: 7 [4480/60000 (7%)] Loss: 0.243722 Train Epoch: 7 [5120/60000 (9%)] Loss: 0.198971 Train Epoch: 7 [5760/60000 (10%)] Loss: 0.101612 Train Epoch: 7 [6400/60000 (11%)] Loss: 0.232090 Train Epoch: 7 [7040/60000 (12%)] Loss: 0.315571 Train Epoch: 7 [7680/60000 (13%)] Loss: 0.239443 Train Epoch: 7 [8320/60000 (14%)] Loss: 0.340601 Train Epoch: 7 [8960/60000 (15%)] Loss: 0.050898 Train Epoch: 7 [9600/60000 (16%)] Loss: 0.194004 Train Epoch: 7 [10240/60000 (17%)] Loss: 0.136828 Train Epoch: 7 [10880/60000 (18%)] Loss: 0.258005 Train Epoch: 7 [11520/60000 (19%)] Loss: 0.409187 Train Epoch: 7 [12160/60000 (20%)] Loss: 0.131922 Train Epoch: 7 [12800/60000 (21%)] Loss: 0.178241 Train Epoch: 7 [13440/60000 (22%)] Loss: 0.068387 Train Epoch: 7 [14080/60000 (23%)] Loss: 0.417622 Train Epoch: 7 [14720/60000 (25%)] Loss: 0.141857 Train Epoch: 7 [15360/60000 (26%)] Loss: 0.475218 Train Epoch: 7 [16000/60000 (27%)] Loss: 0.331239 Train Epoch: 7 [16640/60000 (28%)] Loss: 0.218433 Train Epoch: 7 [17280/60000 (29%)] Loss: 0.087861 Train Epoch: 7 [17920/60000 (30%)] Loss: 0.185781 Train Epoch: 7 [18560/60000 (31%)] Loss: 0.123277 Train Epoch: 7 [19200/60000 (32%)] Loss: 0.116893 Train Epoch: 7 [19840/60000 (33%)] Loss: 0.167921 Train Epoch: 7 [20480/60000 (34%)] Loss: 0.205522 Train Epoch: 7 [21120/60000 (35%)] Loss: 0.136235 Train Epoch: 7 [21760/60000 (36%)] Loss: 0.284596 Train Epoch: 7 [22400/60000 (37%)] Loss: 0.226520 Train Epoch: 7 [23040/60000 (38%)] Loss: 0.158972 Train Epoch: 7 [23680/60000 (39%)] Loss: 0.201735 Train Epoch: 7 [24320/60000 (41%)] Loss: 0.196866 Train Epoch: 7 [24960/60000 (42%)] Loss: 0.348597 Train Epoch: 7 [25600/60000 (43%)] Loss: 0.282424 Train Epoch: 7 [26240/60000 (44%)] Loss: 0.118476 Train Epoch: 7 [26880/60000 (45%)] Loss: 0.158547 Train Epoch: 7 [27520/60000 (46%)] Loss: 0.187790 Train Epoch: 7 [28160/60000 (47%)] Loss: 0.107565 Train Epoch: 7 [28800/60000 (48%)] Loss: 0.105622 Train Epoch: 7 [29440/60000 (49%)] Loss: 0.355571 Train Epoch: 7 [30080/60000 (50%)] Loss: 0.269021 Train Epoch: 7 [30720/60000 (51%)] Loss: 0.091620 Train Epoch: 7 [31360/60000 (52%)] Loss: 0.129190 Train Epoch: 7 [32000/60000 (53%)] Loss: 0.192322 Train Epoch: 7 [32640/60000 (54%)] Loss: 0.290810 Train Epoch: 7 [33280/60000 (55%)] Loss: 0.167235 Train Epoch: 7 [33920/60000 (57%)] Loss: 0.086800 Train Epoch: 7 [34560/60000 (58%)] Loss: 0.135100 Train Epoch: 7 [35200/60000 (59%)] Loss: 0.227101 Train Epoch: 7 [35840/60000 (60%)] Loss: 0.248887 Train Epoch: 7 [36480/60000 (61%)] Loss: 0.294561 Train Epoch: 7 [37120/60000 (62%)] Loss: 0.230433 Train Epoch: 7 [37760/60000 (63%)] Loss: 0.150416 Train Epoch: 7 [38400/60000 (64%)] Loss: 0.104069 Train Epoch: 7 [39040/60000 (65%)] Loss: 0.106915 Train Epoch: 7 [39680/60000 (66%)] Loss: 0.372378 Train Epoch: 7 [40320/60000 (67%)] Loss: 0.207565 Train Epoch: 7 [40960/60000 (68%)] Loss: 0.065137 Train Epoch: 7 [41600/60000 (69%)] Loss: 0.347302 Train Epoch: 7 [42240/60000 (70%)] Loss: 0.172198 Train Epoch: 7 [42880/60000 (71%)] Loss: 0.109692 Train Epoch: 7 [43520/60000 (72%)] Loss: 0.080982 Train Epoch: 7 [44160/60000 (74%)] Loss: 0.080288 Train Epoch: 7 [44800/60000 (75%)] Loss: 0.151189 Train Epoch: 7 [45440/60000 (76%)] Loss: 0.151043 Train Epoch: 7 [46080/60000 (77%)] Loss: 0.120681 Train Epoch: 7 [46720/60000 (78%)] Loss: 0.073592 Train Epoch: 7 [47360/60000 (79%)] Loss: 0.282301 Train Epoch: 7 [48000/60000 (80%)] Loss: 0.146001 Train Epoch: 7 [48640/60000 (81%)] Loss: 0.269991 Train Epoch: 7 [49280/60000 (82%)] Loss: 0.242981 Train Epoch: 7 [49920/60000 (83%)] Loss: 0.067744 Train Epoch: 7 [50560/60000 (84%)] Loss: 0.283833 Train Epoch: 7 [51200/60000 (85%)] Loss: 0.322808 Train Epoch: 7 [51840/60000 (86%)] Loss: 0.268276 Train Epoch: 7 [52480/60000 (87%)] Loss: 0.141551 Train Epoch: 7 [53120/60000 (88%)] Loss: 0.196744 Train Epoch: 7 [53760/60000 (90%)] Loss: 0.332197 Train Epoch: 7 [54400/60000 (91%)] Loss: 0.309745 Train Epoch: 7 [55040/60000 (92%)] Loss: 0.149245 Train Epoch: 7 [55680/60000 (93%)] Loss: 0.177063 Train Epoch: 7 [56320/60000 (94%)] Loss: 0.076496 Train Epoch: 7 [56960/60000 (95%)] Loss: 0.098253 Train Epoch: 7 [57600/60000 (96%)] Loss: 0.323857 Train Epoch: 7 [58240/60000 (97%)] Loss: 0.147353 Train Epoch: 7 [58880/60000 (98%)] Loss: 0.205689 Train Epoch: 7 [59520/60000 (99%)] Loss: 0.228839 Test set: Average loss: 0.0001, Accuracy: 9795/10000 (98%) Train Epoch: 8 [0/60000 (0%)] Loss: 0.292745 Train Epoch: 8 [640/60000 (1%)] Loss: 0.120792 Train Epoch: 8 [1280/60000 (2%)] Loss: 0.135363 Train Epoch: 8 [1920/60000 (3%)] Loss: 0.164676 Train Epoch: 8 [2560/60000 (4%)] Loss: 0.114712 Train Epoch: 8 [3200/60000 (5%)] Loss: 0.287538 Train Epoch: 8 [3840/60000 (6%)] Loss: 0.117525 Train Epoch: 8 [4480/60000 (7%)] Loss: 0.151503 Train Epoch: 8 [5120/60000 (9%)] Loss: 0.068311 Train Epoch: 8 [5760/60000 (10%)] Loss: 0.173197 Train Epoch: 8 [6400/60000 (11%)] Loss: 0.277148 Train Epoch: 8 [7040/60000 (12%)] Loss: 0.074069 Train Epoch: 8 [7680/60000 (13%)] Loss: 0.364987 Train Epoch: 8 [8320/60000 (14%)] Loss: 0.072494 Train Epoch: 8 [8960/60000 (15%)] Loss: 0.136759 Train Epoch: 8 [9600/60000 (16%)] Loss: 0.296852 Train Epoch: 8 [10240/60000 (17%)] Loss: 0.103341 Train Epoch: 8 [10880/60000 (18%)] Loss: 0.150845 Train Epoch: 8 [11520/60000 (19%)] Loss: 0.081442 Train Epoch: 8 [12160/60000 (20%)] Loss: 0.156578 Train Epoch: 8 [12800/60000 (21%)] Loss: 0.177557 Train Epoch: 8 [13440/60000 (22%)] Loss: 0.250457 Train Epoch: 8 [14080/60000 (23%)] Loss: 0.136040 Train Epoch: 8 [14720/60000 (25%)] Loss: 0.134271 Train Epoch: 8 [15360/60000 (26%)] Loss: 0.276332 Train Epoch: 8 [16000/60000 (27%)] Loss: 0.218526 Train Epoch: 8 [16640/60000 (28%)] Loss: 0.222499 Train Epoch: 8 [17280/60000 (29%)] Loss: 0.077347 Train Epoch: 8 [17920/60000 (30%)] Loss: 0.151819 Train Epoch: 8 [18560/60000 (31%)] Loss: 0.219843 Train Epoch: 8 [19200/60000 (32%)] Loss: 0.190889 Train Epoch: 8 [19840/60000 (33%)] Loss: 0.099739 Train Epoch: 8 [20480/60000 (34%)] Loss: 0.285653 Train Epoch: 8 [21120/60000 (35%)] Loss: 0.120615 Train Epoch: 8 [21760/60000 (36%)] Loss: 0.179479 Train Epoch: 8 [22400/60000 (37%)] Loss: 0.217448 Train Epoch: 8 [23040/60000 (38%)] Loss: 0.125785 Train Epoch: 8 [23680/60000 (39%)] Loss: 0.154384 Train Epoch: 8 [24320/60000 (41%)] Loss: 0.103312 Train Epoch: 8 [24960/60000 (42%)] Loss: 0.116148 Train Epoch: 8 [25600/60000 (43%)] Loss: 0.226390 Train Epoch: 8 [26240/60000 (44%)] Loss: 0.098650 Train Epoch: 8 [26880/60000 (45%)] Loss: 0.202164 Train Epoch: 8 [27520/60000 (46%)] Loss: 0.100702 Train Epoch: 8 [28160/60000 (47%)] Loss: 0.292449 Train Epoch: 8 [28800/60000 (48%)] Loss: 0.085550 Train Epoch: 8 [29440/60000 (49%)] Loss: 0.132873 Train Epoch: 8 [30080/60000 (50%)] Loss: 0.140347 Train Epoch: 8 [30720/60000 (51%)] Loss: 0.104647 Train Epoch: 8 [31360/60000 (52%)] Loss: 0.081841 Train Epoch: 8 [32000/60000 (53%)] Loss: 0.156113 Train Epoch: 8 [32640/60000 (54%)] Loss: 0.318736 Train Epoch: 8 [33280/60000 (55%)] Loss: 0.085701 Train Epoch: 8 [33920/60000 (57%)] Loss: 0.088705 Train Epoch: 8 [34560/60000 (58%)] Loss: 0.342402 Train Epoch: 8 [35200/60000 (59%)] Loss: 0.166592 Train Epoch: 8 [35840/60000 (60%)] Loss: 0.065936 Train Epoch: 8 [36480/60000 (61%)] Loss: 0.288784 Train Epoch: 8 [37120/60000 (62%)] Loss: 0.344001 Train Epoch: 8 [37760/60000 (63%)] Loss: 0.125484 Train Epoch: 8 [38400/60000 (64%)] Loss: 0.148977 Train Epoch: 8 [39040/60000 (65%)] Loss: 0.086165 Train Epoch: 8 [39680/60000 (66%)] Loss: 0.211954 Train Epoch: 8 [40320/60000 (67%)] Loss: 0.328222 Train Epoch: 8 [40960/60000 (68%)] Loss: 0.369507 Train Epoch: 8 [41600/60000 (69%)] Loss: 0.061260 Train Epoch: 8 [42240/60000 (70%)] Loss: 0.102083 Train Epoch: 8 [42880/60000 (71%)] Loss: 0.154937 Train Epoch: 8 [43520/60000 (72%)] Loss: 0.136727 Train Epoch: 8 [44160/60000 (74%)] Loss: 0.241387 Train Epoch: 8 [44800/60000 (75%)] Loss: 0.115584 Train Epoch: 8 [45440/60000 (76%)] Loss: 0.161828 Train Epoch: 8 [46080/60000 (77%)] Loss: 0.145702 Train Epoch: 8 [46720/60000 (78%)] Loss: 0.189414 Train Epoch: 8 [47360/60000 (79%)] Loss: 0.219590 Train Epoch: 8 [48000/60000 (80%)] Loss: 0.297140 Train Epoch: 8 [48640/60000 (81%)] Loss: 0.058468 Train Epoch: 8 [49280/60000 (82%)] Loss: 0.333354 Train Epoch: 8 [49920/60000 (83%)] Loss: 0.143101 Train Epoch: 8 [50560/60000 (84%)] Loss: 0.219106 Train Epoch: 8 [51200/60000 (85%)] Loss: 0.124515 Train Epoch: 8 [51840/60000 (86%)] Loss: 0.080556 Train Epoch: 8 [52480/60000 (87%)] Loss: 0.220966 Train Epoch: 8 [53120/60000 (88%)] Loss: 0.066355 Train Epoch: 8 [53760/60000 (90%)] Loss: 0.171026 Train Epoch: 8 [54400/60000 (91%)] Loss: 0.161316 Train Epoch: 8 [55040/60000 (92%)] Loss: 0.153295 Train Epoch: 8 [55680/60000 (93%)] Loss: 0.348095 Train Epoch: 8 [56320/60000 (94%)] Loss: 0.129968 Train Epoch: 8 [56960/60000 (95%)] Loss: 0.259951 Train Epoch: 8 [57600/60000 (96%)] Loss: 0.173553 Train Epoch: 8 [58240/60000 (97%)] Loss: 0.305707 Train Epoch: 8 [58880/60000 (98%)] Loss: 0.099230 Train Epoch: 8 [59520/60000 (99%)] Loss: 0.212924 Test set: Average loss: 0.0001, Accuracy: 9816/10000 (98%) Train Epoch: 9 [0/60000 (0%)] Loss: 0.079323 Train Epoch: 9 [640/60000 (1%)] Loss: 0.286593 Train Epoch: 9 [1280/60000 (2%)] Loss: 0.085763 Train Epoch: 9 [1920/60000 (3%)] Loss: 0.265807 Train Epoch: 9 [2560/60000 (4%)] Loss: 0.145771 Train Epoch: 9 [3200/60000 (5%)] Loss: 0.112922 Train Epoch: 9 [3840/60000 (6%)] Loss: 0.200996 Train Epoch: 9 [4480/60000 (7%)] Loss: 0.142001 Train Epoch: 9 [5120/60000 (9%)] Loss: 0.171615 Train Epoch: 9 [5760/60000 (10%)] Loss: 0.219111 Train Epoch: 9 [6400/60000 (11%)] Loss: 0.121241 Train Epoch: 9 [7040/60000 (12%)] Loss: 0.165811 Train Epoch: 9 [7680/60000 (13%)] Loss: 0.107933 Train Epoch: 9 [8320/60000 (14%)] Loss: 0.469319 Train Epoch: 9 [8960/60000 (15%)] Loss: 0.155425 Train Epoch: 9 [9600/60000 (16%)] Loss: 0.260249 Train Epoch: 9 [10240/60000 (17%)] Loss: 0.249099 Train Epoch: 9 [10880/60000 (18%)] Loss: 0.108027 Train Epoch: 9 [11520/60000 (19%)] Loss: 0.295349 Train Epoch: 9 [12160/60000 (20%)] Loss: 0.229297 Train Epoch: 9 [12800/60000 (21%)] Loss: 0.170714 Train Epoch: 9 [13440/60000 (22%)] Loss: 0.130689 Train Epoch: 9 [14080/60000 (23%)] Loss: 0.160045 Train Epoch: 9 [14720/60000 (25%)] Loss: 0.096786 Train Epoch: 9 [15360/60000 (26%)] Loss: 0.343305 Train Epoch: 9 [16000/60000 (27%)] Loss: 0.089230 Train Epoch: 9 [16640/60000 (28%)] Loss: 0.087048 Train Epoch: 9 [17280/60000 (29%)] Loss: 0.222524 Train Epoch: 9 [17920/60000 (30%)] Loss: 0.196789 Train Epoch: 9 [18560/60000 (31%)] Loss: 0.139464 Train Epoch: 9 [19200/60000 (32%)] Loss: 0.364201 Train Epoch: 9 [19840/60000 (33%)] Loss: 0.184264 Train Epoch: 9 [20480/60000 (34%)] Loss: 0.358588 Train Epoch: 9 [21120/60000 (35%)] Loss: 0.117137 Train Epoch: 9 [21760/60000 (36%)] Loss: 0.145588 Train Epoch: 9 [22400/60000 (37%)] Loss: 0.168730 Train Epoch: 9 [23040/60000 (38%)] Loss: 0.209654 Train Epoch: 9 [23680/60000 (39%)] Loss: 0.086675 Train Epoch: 9 [24320/60000 (41%)] Loss: 0.239534 Train Epoch: 9 [24960/60000 (42%)] Loss: 0.212524 Train Epoch: 9 [25600/60000 (43%)] Loss: 0.254639 Train Epoch: 9 [26240/60000 (44%)] Loss: 0.229474 Train Epoch: 9 [26880/60000 (45%)] Loss: 0.203544 Train Epoch: 9 [27520/60000 (46%)] Loss: 0.193628 Train Epoch: 9 [28160/60000 (47%)] Loss: 0.208155 Train Epoch: 9 [28800/60000 (48%)] Loss: 0.141017 Train Epoch: 9 [29440/60000 (49%)] Loss: 0.094801 Train Epoch: 9 [30080/60000 (50%)] Loss: 0.296131 Train Epoch: 9 [30720/60000 (51%)] Loss: 0.159564 Train Epoch: 9 [31360/60000 (52%)] Loss: 0.100721 Train Epoch: 9 [32000/60000 (53%)] Loss: 0.229705 Train Epoch: 9 [32640/60000 (54%)] Loss: 0.429516 Train Epoch: 9 [33280/60000 (55%)] Loss: 0.332226 Train Epoch: 9 [33920/60000 (57%)] Loss: 0.168803 Train Epoch: 9 [34560/60000 (58%)] Loss: 0.173486 Train Epoch: 9 [35200/60000 (59%)] Loss: 0.135947 Train Epoch: 9 [35840/60000 (60%)] Loss: 0.127091 Train Epoch: 9 [36480/60000 (61%)] Loss: 0.297337 Train Epoch: 9 [37120/60000 (62%)] Loss: 0.156834 Train Epoch: 9 [37760/60000 (63%)] Loss: 0.405716 Train Epoch: 9 [38400/60000 (64%)] Loss: 0.134249 Train Epoch: 9 [39040/60000 (65%)] Loss: 0.082479 Train Epoch: 9 [39680/60000 (66%)] Loss: 0.122936 Train Epoch: 9 [40320/60000 (67%)] Loss: 0.137343 Train Epoch: 9 [40960/60000 (68%)] Loss: 0.108289 Train Epoch: 9 [41600/60000 (69%)] Loss: 0.098429 Train Epoch: 9 [42240/60000 (70%)] Loss: 0.067356 Train Epoch: 9 [42880/60000 (71%)] Loss: 0.110135 Train Epoch: 9 [43520/60000 (72%)] Loss: 0.096491 Train Epoch: 9 [44160/60000 (74%)] Loss: 0.253454 Train Epoch: 9 [44800/60000 (75%)] Loss: 0.157230 Train Epoch: 9 [45440/60000 (76%)] Loss: 0.169301 Train Epoch: 9 [46080/60000 (77%)] Loss: 0.141514 Train Epoch: 9 [46720/60000 (78%)] Loss: 0.212609 Train Epoch: 9 [47360/60000 (79%)] Loss: 0.108132 Train Epoch: 9 [48000/60000 (80%)] Loss: 0.128629 Train Epoch: 9 [48640/60000 (81%)] Loss: 0.163764 Train Epoch: 9 [49280/60000 (82%)] Loss: 0.119731 Train Epoch: 9 [49920/60000 (83%)] Loss: 0.154034 Train Epoch: 9 [50560/60000 (84%)] Loss: 0.152267 Train Epoch: 9 [51200/60000 (85%)] Loss: 0.053886 Train Epoch: 9 [51840/60000 (86%)] Loss: 0.108673 Train Epoch: 9 [52480/60000 (87%)] Loss: 0.124303 Train Epoch: 9 [53120/60000 (88%)] Loss: 0.204565 Train Epoch: 9 [53760/60000 (90%)] Loss: 0.187487 Train Epoch: 9 [54400/60000 (91%)] Loss: 0.133514 Train Epoch: 9 [55040/60000 (92%)] Loss: 0.137428 Train Epoch: 9 [55680/60000 (93%)] Loss: 0.050837 Train Epoch: 9 [56320/60000 (94%)] Loss: 0.205850 Train Epoch: 9 [56960/60000 (95%)] Loss: 0.175952 Train Epoch: 9 [57600/60000 (96%)] Loss: 0.305810 Train Epoch: 9 [58240/60000 (97%)] Loss: 0.101633 Train Epoch: 9 [58880/60000 (98%)] Loss: 0.105160 Train Epoch: 9 [59520/60000 (99%)] Loss: 0.315622 Test set: Average loss: 0.0001, Accuracy: 9834/10000 (98%) Train Epoch: 10 [0/60000 (0%)] Loss: 0.097688 Train Epoch: 10 [640/60000 (1%)] Loss: 0.118289 Train Epoch: 10 [1280/60000 (2%)] Loss: 0.113985 Train Epoch: 10 [1920/60000 (3%)] Loss: 0.199488 Train Epoch: 10 [2560/60000 (4%)] Loss: 0.348783 Train Epoch: 10 [3200/60000 (5%)] Loss: 0.130896 Train Epoch: 10 [3840/60000 (6%)] Loss: 0.387860 Train Epoch: 10 [4480/60000 (7%)] Loss: 0.228294 Train Epoch: 10 [5120/60000 (9%)] Loss: 0.221053 Train Epoch: 10 [5760/60000 (10%)] Loss: 0.187114 Train Epoch: 10 [6400/60000 (11%)] Loss: 0.017330 Train Epoch: 10 [7040/60000 (12%)] Loss: 0.115623 Train Epoch: 10 [7680/60000 (13%)] Loss: 0.091393 Train Epoch: 10 [8320/60000 (14%)] Loss: 0.150872 Train Epoch: 10 [8960/60000 (15%)] Loss: 0.123215 Train Epoch: 10 [9600/60000 (16%)] Loss: 0.277278 Train Epoch: 10 [10240/60000 (17%)] Loss: 0.249497 Train Epoch: 10 [10880/60000 (18%)] Loss: 0.314627 Train Epoch: 10 [11520/60000 (19%)] Loss: 0.148233 Train Epoch: 10 [12160/60000 (20%)] Loss: 0.253433 Train Epoch: 10 [12800/60000 (21%)] Loss: 0.165765 Train Epoch: 10 [13440/60000 (22%)] Loss: 0.095515 Train Epoch: 10 [14080/60000 (23%)] Loss: 0.196772 Train Epoch: 10 [14720/60000 (25%)] Loss: 0.154946 Train Epoch: 10 [15360/60000 (26%)] Loss: 0.097302 Train Epoch: 10 [16000/60000 (27%)] Loss: 0.075378 Train Epoch: 10 [16640/60000 (28%)] Loss: 0.103841 Train Epoch: 10 [17280/60000 (29%)] Loss: 0.542589 Train Epoch: 10 [17920/60000 (30%)] Loss: 0.193030 Train Epoch: 10 [18560/60000 (31%)] Loss: 0.058140 Train Epoch: 10 [19200/60000 (32%)] Loss: 0.164775 Train Epoch: 10 [19840/60000 (33%)] Loss: 0.116774 Train Epoch: 10 [20480/60000 (34%)] Loss: 0.098373 Train Epoch: 10 [21120/60000 (35%)] Loss: 0.152201 Train Epoch: 10 [21760/60000 (36%)] Loss: 0.158482 Train Epoch: 10 [22400/60000 (37%)] Loss: 0.181521 Train Epoch: 10 [23040/60000 (38%)] Loss: 0.127969 Train Epoch: 10 [23680/60000 (39%)] Loss: 0.141418 Train Epoch: 10 [24320/60000 (41%)] Loss: 0.131132 Train Epoch: 10 [24960/60000 (42%)] Loss: 0.094523 Train Epoch: 10 [25600/60000 (43%)] Loss: 0.250731 Train Epoch: 10 [26240/60000 (44%)] Loss: 0.196437 Train Epoch: 10 [26880/60000 (45%)] Loss: 0.198200 Train Epoch: 10 [27520/60000 (46%)] Loss: 0.124787 Train Epoch: 10 [28160/60000 (47%)] Loss: 0.143860 Train Epoch: 10 [28800/60000 (48%)] Loss: 0.130015 Train Epoch: 10 [29440/60000 (49%)] Loss: 0.128928 Train Epoch: 10 [30080/60000 (50%)] Loss: 0.108551 Train Epoch: 10 [30720/60000 (51%)] Loss: 0.063688 Train Epoch: 10 [31360/60000 (52%)] Loss: 0.089477 Train Epoch: 10 [32000/60000 (53%)] Loss: 0.166978 Train Epoch: 10 [32640/60000 (54%)] Loss: 0.081646 Train Epoch: 10 [33280/60000 (55%)] Loss: 0.247325 Train Epoch: 10 [33920/60000 (57%)] Loss: 0.114443 Train Epoch: 10 [34560/60000 (58%)] Loss: 0.146846 Train Epoch: 10 [35200/60000 (59%)] Loss: 0.156275 Train Epoch: 10 [35840/60000 (60%)] Loss: 0.204058 Train Epoch: 10 [36480/60000 (61%)] Loss: 0.389685 Train Epoch: 10 [37120/60000 (62%)] Loss: 0.106072 Train Epoch: 10 [37760/60000 (63%)] Loss: 0.122947 Train Epoch: 10 [38400/60000 (64%)] Loss: 0.158832 Train Epoch: 10 [39040/60000 (65%)] Loss: 0.212625 Train Epoch: 10 [39680/60000 (66%)] Loss: 0.190883 Train Epoch: 10 [40320/60000 (67%)] Loss: 0.119547 Train Epoch: 10 [40960/60000 (68%)] Loss: 0.138365 Train Epoch: 10 [41600/60000 (69%)] Loss: 0.159457 Train Epoch: 10 [42240/60000 (70%)] Loss: 0.115805 Train Epoch: 10 [42880/60000 (71%)] Loss: 0.059752 Train Epoch: 10 [43520/60000 (72%)] Loss: 0.233170 Train Epoch: 10 [44160/60000 (74%)] Loss: 0.207679 Train Epoch: 10 [44800/60000 (75%)] Loss: 0.068984 Train Epoch: 10 [45440/60000 (76%)] Loss: 0.265697 Train Epoch: 10 [46080/60000 (77%)] Loss: 0.204301 Train Epoch: 10 [46720/60000 (78%)] Loss: 0.085270 Train Epoch: 10 [47360/60000 (79%)] Loss: 0.129904 Train Epoch: 10 [48000/60000 (80%)] Loss: 0.154879 Train Epoch: 10 [48640/60000 (81%)] Loss: 0.333942 Train Epoch: 10 [49280/60000 (82%)] Loss: 0.134290 Train Epoch: 10 [49920/60000 (83%)] Loss: 0.173393 Train Epoch: 10 [50560/60000 (84%)] Loss: 0.165202 Train Epoch: 10 [51200/60000 (85%)] Loss: 0.162377 Train Epoch: 10 [51840/60000 (86%)] Loss: 0.083045 Train Epoch: 10 [52480/60000 (87%)] Loss: 0.079579 Train Epoch: 10 [53120/60000 (88%)] Loss: 0.093638 Train Epoch: 10 [53760/60000 (90%)] Loss: 0.197897 Train Epoch: 10 [54400/60000 (91%)] Loss: 0.141999 Train Epoch: 10 [55040/60000 (92%)] Loss: 0.165381 Train Epoch: 10 [55680/60000 (93%)] Loss: 0.099536 Train Epoch: 10 [56320/60000 (94%)] Loss: 0.251508 Train Epoch: 10 [56960/60000 (95%)] Loss: 0.065102 Train Epoch: 10 [57600/60000 (96%)] Loss: 0.431946 Train Epoch: 10 [58240/60000 (97%)] Loss: 0.183199 Train Epoch: 10 [58880/60000 (98%)] Loss: 0.407611 Train Epoch: 10 [59520/60000 (99%)] Loss: 0.314210 Test set: Average loss: 0.0000, Accuracy: 9839/10000 (98%)
Here, is another simple pytorch example which I downloaded from here. This example fits a two-layer network to random data. However, before running the example, first remove the reduction argument from the expression defining loss function! (as we did it for the Example 2).
# -*- coding: utf-8 -*-
import torch
# N is batch size; D_in is input dimension;
# H is hidden dimension; D_out is output dimension.
N, D_in, H, D_out = 64, 1000, 100, 10
# Create random Tensors to hold inputs and outputs
x = torch.randn(N, D_in)
y = torch.randn(N, D_out)
# Use the nn package to define our model and loss function.
model = torch.nn.Sequential(
torch.nn.Linear(D_in, H),
torch.nn.ReLU(),
torch.nn.Linear(H, D_out),
)
loss_fn = torch.nn.MSELoss()
# Use the optim package to define an Optimizer that will update the weights of
# the model for us. Here we will use Adam; the optim package contains many other
# optimization algoriths. The first argument to the Adam constructor tells the
# optimizer which Tensors it should update.
learning_rate = 1e-4
optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)
for t in range(500):
# Forward pass: compute predicted y by passing x to the model.
y_pred = model(x)
# Compute and print loss.
loss = loss_fn(y_pred, y)
print(t, loss.item())
# Before the backward pass, use the optimizer object to zero all of the
# gradients for the variables it will update (which are the learnable
# weights of the model). This is because by default, gradients are
# accumulated in buffers( i.e, not overwritten) whenever .backward()
# is called. Checkout docs of torch.autograd.backward for more details.
optimizer.zero_grad()
# Backward pass: compute gradient of the loss with respect to model
# parameters
loss.backward()
# Calling the step function on an Optimizer makes an update to its
# parameters
optimizer.step()
(0, 1.0399448871612549) (1, 1.0127382278442383) (2, 0.9862440228462219) (3, 0.9604975581169128) (4, 0.9357196688652039) (5, 0.9117013216018677) (6, 0.8884192705154419) (7, 0.8658498525619507) (8, 0.8438955545425415) (9, 0.8226255178451538) (10, 0.8019316792488098) (11, 0.781884491443634) (12, 0.7623889446258545) (13, 0.743523120880127) (14, 0.7253612875938416) (15, 0.7077023983001709) (16, 0.6906373500823975) (17, 0.6740445494651794) (18, 0.6579065918922424) (19, 0.6421589255332947) (20, 0.626873791217804) (21, 0.6119142770767212) (22, 0.5973230600357056) (23, 0.5831117033958435) (24, 0.5692239999771118) (25, 0.555722713470459) (26, 0.5425689220428467) (27, 0.5296918749809265) (28, 0.5170916318893433) (29, 0.5048167705535889) (30, 0.4928136467933655) (31, 0.4810701012611389) (32, 0.4696010947227478) (33, 0.4583776891231537) (34, 0.44743603467941284) (35, 0.4367571771144867) (36, 0.4263865351676941) (37, 0.41625022888183594) (38, 0.4062841832637787) (39, 0.396533727645874) (40, 0.3870033621788025) (41, 0.3776722550392151) (42, 0.3685188889503479) (43, 0.3595620095729828) (44, 0.35076025128364563) (45, 0.3421204686164856) (46, 0.3336712718009949) (47, 0.3254075348377228) (48, 0.3173092305660248) (49, 0.30938681960105896) (50, 0.3016127943992615) (51, 0.29397842288017273) (52, 0.28651413321495056) (53, 0.2791885435581207) (54, 0.2720264196395874) (55, 0.2650095820426941) (56, 0.25813716650009155) (57, 0.25142258405685425) (58, 0.2448357790708542) (59, 0.23837760090827942) (60, 0.2320663034915924) (61, 0.2258930504322052) (62, 0.21984103322029114) (63, 0.21393270790576935) (64, 0.20817610621452332) (65, 0.2025449275970459) (66, 0.19701847434043884) (67, 0.19160166382789612) (68, 0.18629172444343567) (69, 0.1811296045780182) (70, 0.17608778178691864) (71, 0.1711496263742447) (72, 0.16631850600242615) (73, 0.16159312427043915) (74, 0.15698324143886566) (75, 0.1524806171655655) (76, 0.14808949828147888) (77, 0.14379936456680298) (78, 0.1396062672138214) (79, 0.13550740480422974) (80, 0.13150236010551453) (81, 0.12758885324001312) (82, 0.12377242743968964) (83, 0.12005691230297089) (84, 0.11643408238887787) (85, 0.11290253698825836) (86, 0.10945405811071396) (87, 0.10608841478824615) (88, 0.10281498730182648) (89, 0.09962429851293564) (90, 0.0965193659067154) (91, 0.0934956818819046) (92, 0.09055440127849579) (93, 0.0876845195889473) (94, 0.08488873392343521) (95, 0.08216538280248642) (96, 0.07952049374580383) (97, 0.07694515585899353) (98, 0.07444293797016144) (99, 0.07200442254543304) (100, 0.06963548064231873) (101, 0.06733433902263641) (102, 0.06509789079427719) (103, 0.0629168301820755) (104, 0.06079805642366409) (105, 0.05873542279005051) (106, 0.05673372000455856) (107, 0.05478786304593086) (108, 0.05290016531944275) (109, 0.05106498673558235) (110, 0.04928257316350937) (111, 0.04755670577287674) (112, 0.045883335173130035) (113, 0.04425622150301933) (114, 0.04267922788858414) (115, 0.04115071892738342) (116, 0.039668008685112) (117, 0.03823353722691536) (118, 0.036843568086624146) (119, 0.035499945282936096) (120, 0.03419632092118263) (121, 0.032935597002506256) (122, 0.03171474486589432) (123, 0.03053225949406624) (124, 0.029387827962636948) (125, 0.02828199230134487) (126, 0.02721169963479042) (127, 0.026175931096076965) (128, 0.02517428994178772) (129, 0.02420603297650814) (130, 0.023271452635526657) (131, 0.022367211058735847) (132, 0.021494904533028603) (133, 0.02065267041325569) (134, 0.019839514046907425) (135, 0.01905370131134987) (136, 0.018297377973794937) (137, 0.017566243186593056) (138, 0.016860060393810272) (139, 0.016179433092474937) (140, 0.015523483045399189) (141, 0.014891056343913078) (142, 0.014280972070991993) (143, 0.0136934919282794) (144, 0.01312757283449173) (145, 0.012582285329699516) (146, 0.012057144194841385) (147, 0.011550987139344215) (148, 0.011064061895012856) (149, 0.010595316998660564) (150, 0.010144194588065147) (151, 0.009710493497550488) (152, 0.009293089620769024) (153, 0.008891928009688854) (154, 0.008506153710186481) (155, 0.008135931566357613) (156, 0.007779931183904409) (157, 0.007437871303409338) (158, 0.007109491620212793) (159, 0.006794381886720657) (160, 0.006491917185485363) (161, 0.0062016271986067295) (162, 0.00592260155826807) (163, 0.00565518019720912) (164, 0.005398910958319902) (165, 0.00515322620049119) (166, 0.00491737574338913) (167, 0.00469172140583396) (168, 0.004474855028092861) (169, 0.0042671700939536095) (170, 0.004068297334015369) (171, 0.0038779042661190033) (172, 0.003695521969348192) (173, 0.003520999802276492) (174, 0.003354029729962349) (175, 0.0031944066286087036) (176, 0.003041766118258238) (177, 0.0028958707116544247) (178, 0.0027564503252506256) (179, 0.0026232278905808926) (180, 0.002496097469702363) (181, 0.002374449511989951) (182, 0.002258440712466836) (183, 0.0021484275348484516) (184, 0.0020438111387193203) (185, 0.0019439324969425797) (186, 0.0018486876506358385) (187, 0.0017577959224581718) (188, 0.0016712404321879148) (189, 0.001588677056133747) (190, 0.0015099109150469303) (191, 0.0014348553959280252) (192, 0.0013634080532938242) (193, 0.0012953367549926043) (194, 0.0012304179836064577) (195, 0.001168722053989768) (196, 0.0011100124102085829) (197, 0.0010541121009737253) (198, 0.001000907621346414) (199, 0.0009502557804808021) (200, 0.0009021104197017848) (201, 0.0008562676375731826) (202, 0.0008126312168315053) (203, 0.0007711296784691513) (204, 0.0007316690753214061) (205, 0.0006941977189853787) (206, 0.000658564327750355) (207, 0.0006246989360079169) (208, 0.0005924883880652487) (209, 0.0005618963623419404) (210, 0.0005328060360625386) (211, 0.0005051691550761461) (212, 0.00047892332077026367) (213, 0.00045398552902042866) (214, 0.00043031509267166257) (215, 0.0004078551137354225) (216, 0.00038650663918815553) (217, 0.0003662478702608496) (218, 0.0003470259252935648) (219, 0.0003287854779046029) (220, 0.0003114747232757509) (221, 0.00029504759004339576) (222, 0.0002794682513922453) (223, 0.00026468021678738296) (224, 0.0002506655000615865) (225, 0.0002373730530962348) (226, 0.00022476489539258182) (227, 0.00021280634973663837) (228, 0.00020147397299297154) (229, 0.00019075145246461034) (230, 0.00018057074339594692) (231, 0.00017092855705413967) (232, 0.0001617853995412588) (233, 0.00015311819151975214) (234, 0.00014489951718132943) (235, 0.0001371085672872141) (236, 0.00012973410775884986) (237, 0.00012275201152078807) (238, 0.00011614339746301994) (239, 0.00010987896530423313) (240, 0.00010394058335805312) (241, 9.832110663410276e-05) (242, 9.308203880209476e-05) (243, 8.818191417958587e-05) (244, 8.354875899385661e-05) (245, 7.916612958069891e-05) (246, 7.501698564738035e-05) (247, 7.109301805030555e-05) (248, 6.738306547049433e-05) (249, 6.387292523868382e-05) (250, 6.0550402849912643e-05) (251, 5.7407876738579944e-05) (252, 5.443486588774249e-05) (253, 5.1619583246065304e-05) (254, 4.895710662822239e-05) (255, 4.643625652533956e-05) (256, 4.404992432682775e-05) (257, 4.1790415707509965e-05) (258, 3.96504947275389e-05) (259, 3.76257739844732e-05) (260, 3.570581611711532e-05) (261, 3.388716504559852e-05) (262, 3.2165069569600746e-05) (263, 3.053358886973001e-05) (264, 2.898678030760493e-05) (265, 2.752110958681442e-05) (266, 2.6130601327167824e-05) (267, 2.4813711206661537e-05) (268, 2.356454569962807e-05) (269, 2.238028173451312e-05) (270, 2.1257856133161113e-05) (271, 2.019203020608984e-05) (272, 1.918136331369169e-05) (273, 1.822264675865881e-05) (274, 1.7313343050773256e-05) (275, 1.6450307157356292e-05) (276, 1.5631119822501205e-05) (277, 1.4854309483780526e-05) (278, 1.4116184502199758e-05) (279, 1.3420431059785187e-05) (280, 1.2760826393787283e-05) (281, 1.2135450560890604e-05) (282, 1.1541494131961372e-05) (283, 1.0977548299706541e-05) (284, 1.0442034181323834e-05) (285, 9.933161891240161e-06) (286, 9.449999197386205e-06) (287, 8.991125469037797e-06) (288, 8.554933629056904e-06) (289, 8.140485078911297e-06) (290, 7.746303708699998e-06) (291, 7.371833817160223e-06) (292, 7.015826213319087e-06) (293, 6.677261808363255e-06) (294, 6.355513960443204e-06) (295, 6.049228431947995e-06) (296, 5.758022780355532e-06) (297, 5.480962045112392e-06) (298, 5.2174100346746854e-06) (299, 4.966630513081327e-06) (300, 4.728053681901656e-06) (301, 4.500909653870622e-06) (302, 4.2848114389926195e-06) (303, 4.079059635841986e-06) (304, 3.883188583131414e-06) (305, 3.696725343615981e-06) (306, 3.5192911127523985e-06) (307, 3.350350425534998e-06) (308, 3.1894519452180248e-06) (309, 3.0362607503775507e-06) (310, 2.8903527891088743e-06) (311, 2.751462716332753e-06) (312, 2.619142378534889e-06) (313, 2.4931789539550664e-06) (314, 2.373164761593216e-06) (315, 2.258883341710316e-06) (316, 2.1500404727703426e-06) (317, 2.0463553482841235e-06) (318, 1.947614236996742e-06) (319, 1.8535372419137275e-06) (320, 1.763929162734712e-06) (321, 1.6786398191470653e-06) (322, 1.5973082554410212e-06) (323, 1.5198972960206447e-06) (324, 1.4461664932241547e-06) (325, 1.3759232615484507e-06) (326, 1.3090059383102925e-06) (327, 1.2453165254555643e-06) (328, 1.1845813787658699e-06) (329, 1.1267944728388102e-06) (330, 1.071732071977749e-06) (331, 1.019369847199414e-06) (332, 9.694083473732462e-07) (333, 9.21849562018906e-07) (334, 8.765552479417238e-07) (335, 8.334499170814524e-07) (336, 7.924181772978045e-07) (337, 7.53332585645694e-07) (338, 7.161202120187227e-07) (339, 6.806727697039605e-07) (340, 6.469285267485247e-07) (341, 6.148216016299557e-07) (342, 5.842518930876395e-07) (343, 5.551452773033816e-07) (344, 5.274819727674185e-07) (345, 5.010953145756503e-07) (346, 4.7603234065718425e-07) (347, 4.521808136814798e-07) (348, 4.294666382520518e-07) (349, 4.0785184296510124e-07) (350, 3.872977742958028e-07) (351, 3.6774352452084713e-07) (352, 3.4915359492515563e-07) (353, 3.314467562631762e-07) (354, 3.1462960237149673e-07) (355, 2.9861658390473167e-07) (356, 2.834055976563832e-07) (357, 2.689356222163042e-07) (358, 2.551726652200159e-07) (359, 2.420917155632196e-07) (360, 2.2965392076912394e-07) (361, 2.1783189652069268e-07) (362, 2.0660498023516993e-07) (363, 1.9593305466969468e-07) (364, 1.8579125082851533e-07) (365, 1.7615684555494227e-07) (366, 1.669901195100465e-07) (367, 1.58297211783065e-07) (368, 1.5004204101387586e-07) (369, 1.421953470526205e-07) (370, 1.3473618309944868e-07) (371, 1.2766743395786762e-07) (372, 1.2094550072561105e-07) (373, 1.1457071025233745e-07) (374, 1.0852708953734691e-07) (375, 1.0277980067030512e-07) (376, 9.732351458069388e-08) (377, 9.214895158038416e-08) (378, 8.724376954205582e-08) (379, 8.257699590785705e-08) (380, 7.816607450195079e-08) (381, 7.397560608524145e-08) (382, 7.000635093845631e-08) (383, 6.623267978511649e-08) (384, 6.265680951855757e-08) (385, 5.927207169520443e-08) (386, 5.6061349340552624e-08) (387, 5.301594541151644e-08) (388, 5.0128829087725535e-08) (389, 4.7395829483320995e-08) (390, 4.4803186938224826e-08) (391, 4.2353285323315504e-08) (392, 4.0024566771990067e-08) (393, 3.7823372878165173e-08) (394, 3.5740665538241956e-08) (395, 3.376369761554088e-08) (396, 3.1895854846197835e-08) (397, 3.0124532202080445e-08) (398, 2.845016311425752e-08) (399, 2.6865688340649285e-08) (400, 2.5362421496311072e-08) (401, 2.394560283391911e-08) (402, 2.2600948668127785e-08) (403, 2.132805398957771e-08) (404, 2.012602351442183e-08) (405, 1.8989609884556558e-08) (406, 1.791919501670236e-08) (407, 1.6899569743600296e-08) (408, 1.5938814712512794e-08) (409, 1.502868407499136e-08) (410, 1.4169384776607785e-08) (411, 1.3360429207409652e-08) (412, 1.2594531639820161e-08) (413, 1.1870530336466345e-08) (414, 1.1188017623453561e-08) (415, 1.0539845440860063e-08) (416, 9.929935096408826e-09) (417, 9.35527122436497e-09) (418, 8.811599450098129e-09) (419, 8.296192177681405e-09) (420, 7.8125976799015e-09) (421, 7.3550809887024116e-09) (422, 6.923850826012767e-09) (423, 6.515894934722155e-09) (424, 6.13362738377532e-09) (425, 5.7696039057475446e-09) (426, 5.429736660289564e-09) (427, 5.106991274317352e-09) (428, 4.80323114615544e-09) (429, 4.517337615084216e-09) (430, 4.246915263905748e-09) (431, 3.99334743050872e-09) (432, 3.754371924458155e-09) (433, 3.5283007626674134e-09) (434, 3.315397067993331e-09) (435, 3.11529890772988e-09) (436, 2.9264335399403762e-09) (437, 2.7491962040215867e-09) (438, 2.582485114643873e-09) (439, 2.4254425134984103e-09) (440, 2.276655752808665e-09) (441, 2.137675370050829e-09) (442, 2.0062038696977424e-09) (443, 1.882483502413379e-09) (444, 1.7678546404553686e-09) (445, 1.657392112441869e-09) (446, 1.5543278886198664e-09) (447, 1.4579614182608225e-09) (448, 1.3675752752462245e-09) (449, 1.2823485606716645e-09) (450, 1.2022631779018411e-09) (451, 1.12726528112006e-09) (452, 1.0564906727239531e-09) (453, 9.897802577540915e-10) (454, 9.272635992374489e-10) (455, 8.690672070876815e-10) (456, 8.1366680060313e-10) (457, 7.620251096795982e-10) (458, 7.135042001671366e-10) (459, 6.682516762168689e-10) (460, 6.256473117360883e-10) (461, 5.855460005754765e-10) (462, 5.47932088590386e-10) (463, 5.123678148422073e-10) (464, 4.79403516884247e-10) (465, 4.486133131642589e-10) (466, 4.1955416918426636e-10) (467, 3.9219907876919535e-10) (468, 3.668534920286959e-10) (469, 3.4258793024655176e-10) (470, 3.204245757615354e-10) (471, 2.9940164236741396e-10) (472, 2.7976837513321584e-10) (473, 2.6117866203101414e-10) (474, 2.4409740895237064e-10) (475, 2.2786179887379632e-10) (476, 2.1272021644147543e-10) (477, 1.985208386567905e-10) (478, 1.8559552505958976e-10) (479, 1.7323617540476732e-10) (480, 1.6175036310350777e-10) (481, 1.5090424743124942e-10) (482, 1.4066703357684673e-10) (483, 1.3133036325108094e-10) (484, 1.2264977922171738e-10) (485, 1.1381937753407456e-10) (486, 1.0641694470070462e-10) (487, 9.904745079669652e-11) (488, 9.260037181491043e-11) (489, 8.619395475140124e-11) (490, 8.019883368515224e-11) (491, 7.474771496207566e-11) (492, 6.964414461796409e-11) (493, 6.509921768316218e-11) (494, 6.053542389583555e-11) (495, 5.633932209647696e-11) (496, 5.24747363617184e-11) (497, 4.876997070080158e-11) (498, 4.545695417301765e-11) (499, 4.2353270196526793e-11)