深度学习笔记

深度学习笔记

pytorch在linux下似乎比在windows下有几倍性能的提高

虽然没有独显(穷),但是仍然希望能尽可能快的训练模型,看看收敛的结果。之前就有听人说在linux跑比在windows下快。于是便简单测试了一下最简单的CNN网络在不同系统下的执行所花的时间。由于没有Linux系统的电脑(穷),加上是win10是家庭版,所以使用virtualbox版的docker来代替,docker使用的是pytorch的官方镜像(系统为ubuntu),Window系统使用Windows10。代码修改自莫烦Python教学里的pytorch的卷积神经网络一集。数据集使用的是Mnist手写数字数据集。两个卷积层加一个输出层。EPOCH是2。BATCH_SIZE是50。代码如下:

1 2""" 3View more, visit my tutorial page: https://morvanzhou.github.io/tutorials/ 4My Youtube Channel: https://www.youtube.com/user/MorvanZhou 5Dependencies: 6torch: 0.4 7torchvision 8matplotlib 9""" 10# library 11# standard library 12import os 13import datetime 14# third-party library 15import torch 16import torch.nn as nn 17import torch.utils.data as Data 18import torchvision 19# torch.manual_seed(1) # reproducible 20# Hyper Parameters 21EPOCH = 2 # train the training data n times, to save time, we just train 1 epoch 22BATCH_SIZE = 50 23LR = 0.001 # learning rate 24DOWNLOAD_MNIST = False 25 26# Mnist digits dataset 27if not(os.path.exists('./mnist/')) or not os.listdir('./mnist/'): 28 # not mnist dir or mnist is empyt dir 29 DOWNLOAD_MNIST = True 30 31train_data = torchvision.datasets.MNIST( 32 root='./mnist/', 33 train=True, # this is training data 34 transform=torchvision.transforms.ToTensor(), # Converts a PIL.Image or numpy.ndarray to 35 # torch.FloatTensor of shape (C x H x W) and normalize in the range [0.0, 1.0] 36 download=DOWNLOAD_MNIST, 37) 38 39print(train_data.train_data.size()) # (60000, 28, 28) 40print(train_data.train_labels.size()) # (60000) 41 42# Data Loader for easy mini-batch return in training, the image batch shape will be (50, 1, 28, 28) 43train_loader = Data.DataLoader(dataset=train_data, batch_size=BATCH_SIZE, shuffle=True) 44 45# pick 2000 samples to speed up testing 46test_data = torchvision.datasets.MNIST(root='./mnist/', train=False) 47test_x = torch.unsqueeze(test_data.test_data, dim=1).type(torch.FloatTensor)[:2000]/255. # shape from (2000, 28, 28) to (2000, 1, 28, 28), value in range(0,1) 48test_y = test_data.test_labels[:2000] 49 50 51class CNN(nn.Module): 52 def __init__(self): 53 super(CNN, self).__init__() 54 self.conv1 = nn.Sequential( # input shape (1, 28, 28) 55 nn.Conv2d( 56 in_channels=1, # input height 57 out_channels=4, # n_filters 58 kernel_size=5, # filter size 59 stride=1, # filter movement/step 60 padding=2, # if want same width and length of this image after Conv2d, padding=(kernel_size-1)/2 if stride=1 61 ), # output shape (16, 28, 28) 62 nn.ReLU(), # activation 63 nn.MaxPool2d(kernel_size=2), # choose max value in 2x2 area, output shape (16, 14, 14) 64 ) 65 self.conv2 = nn.Sequential( # input shape (16, 14, 14) 66 nn.Conv2d(4, 8, 5, 1, 2), # output shape (32, 14, 14) 67 nn.ReLU(), # activation 68 nn.MaxPool2d(2), # output shape (32, 7, 7) 69 ) 70 self.out = nn.Linear(8 * 7 * 7, 10) # fully connected layer, output 10 classes 71 72 def forward(self, x): 73 x = self.conv1(x) 74 x = self.conv2(x) 75 x = x.view(x.size(0), -1) # flatten the output of conv2 to (batch_size, 32 * 7 * 7) 76 output = self.out(x) 77 return output, x # return x for visualization 78 79 80cnn = CNN() 81print(cnn) # net architecture 82 83optimizer = torch.optim.Adam(cnn.parameters(), lr=LR) # optimize all cnn parameters 84loss_func = nn.CrossEntropyLoss() # the target label is not one-hotted 85 86starttime = datetime.datetime.now() 87# training and testing 88for epoch in range(EPOCH): 89 for step, (b_x, b_y) in enumerate(train_loader): # gives batch data, normalize x when iterate train_loader 90 91 output = cnn(b_x)[0] # cnn output 92 loss = loss_func(output, b_y) # cross entropy loss 93 optimizer.zero_grad() # clear gradients for this training step 94 loss.backward() # backpropagation, compute gradients 95 optimizer.step() # apply gradients 96 97 if step % 50 == 0: 98 test_output, last_layer = cnn(test_x) 99 pred_y = torch.max(test_output, 1)[1].data.numpy() 100 accuracy = float((pred_y == test_y.data.numpy()).astype(int).sum()) / float(test_y.size(0)) 101 print('Epoch: ', epoch, 'Step: ', step, '| train loss: %.4f' % loss.data.numpy(), '| test accuracy: %.2f' % accuracy) 102 103endtime = datetime.datetime.now() 104elapsed_time = (endtime-starttime).total_seconds() 105# print 10 predictions from test data 106test_output, _ = cnn(test_x[:10]) 107pred_y = torch.max(test_output, 1)[1].data.numpy() 108print(pred_y, 'prediction number') 109print(test_y[:10].numpy(), 'real number') 110print('time used: ',elapsed_time,'s') 111 112 113

结果如下:
linux下耗时为50s
linux下耗时
windows下耗时为115s
windows下耗时
测试的结果是在Linux下耗时似乎是Windows下的1/2左右,如果增加卷积核个数,网络更加复杂的话,感觉差距还会拉大。同时Linux下的CPU占用也优于Windows(可能是虚拟机的限制)。
所以木有独显,即使是CPU计算,使用linux还是坠吼的,虚拟机里的linux也比windows下的表现好。

附录

docker常用指令:

1docker run -idt -v /c/Users/brett/volume:/opt/sr pytorch/pytorch bash //启动容器并挂载本地目录为volume 2docker exec -it 6a49 bash //进入容器bash 3docker start 6a49 //启动已存在的容器 4 5

代码交流 2021