首页 > 编程入门 文章详情

ResNet笔记 Deep Residual Learning for Image Recognition

编程入门 行业动态 更新时间:2024-10-24 01:53:45

ResNet<a href=https://www.elefans.com/category/jswz/34/1770047.html style=笔记 Deep Residual Learning for Image Recognition"/>

ResNet笔记 Deep Residual Learning for Image Recognition

目录

  • 论文基本信息
  • 解决的问题
  • Abstract
  • Introduction
  • Deep Residual Learning
  • Experiments
  • 新学习的概念
  • 本文心得
  • 附录：代码实践
  • 其他

论文基本信息

作者：Kaiming He Xiangyu Zhang Shaoqing Ren Jian Sun

关键词：CNN、ResNet

领域：CV

链接：.03385.pdf

参考：

  • 【ResNet论文逐段精读【论文精读】】 /?share_source=copy_web&vd_source=0ac66901e8dd624384e51529673d725a

  • 【【精读AI论文】ResNet深度残差网络】 /?share_source=copy_web&vd_source=0ac66901e8dd624384e51529673d725a

  • 【6.2 使用pytorch搭建ResNet并基于迁移学习训练】/?spm_id_from=333.337.search-card.all.click&vd_source=20c37d260c9535497db1af49e1fd2cbd

  • 深度学习经典论文分析（六）-Deep Residual Learning for Image Recognition

解决的问题

深层的网络很难去训练，会有“退化”现象产生。残差块使得很深的网络更加容易训练，甚至可以训练一千层的网络。

残差网络对随后的深层神经网络设计产生了深远的影响，无论是卷积类网络还是全连接网络。

Deep Residual Networks：

  • Easy to train
  • Simply gain accuracy from depth
  • Well transferrable

Abstract

深层的神经网络很难去训练，所以我们提出了一个残差学习框架去应对这个问题。所提出的网络更容易优化、能够通过显着增加的深度获得准确性。并且网络在ILSVRC 2015比赛上获得第一名的成绩，并且该网络具有很好的泛化性，可以应用到各种计算机视觉任务（检测、定位、分割）中。

Introduction

  • 神经网络的深度很重要：深度网络自然地以端到端的多层方式集成低/中/高级特征和分类器，并且特征的“级别”可以通过堆叠层的数量（深度）来丰富。
  • 直接堆叠层数的问题：梯度消失/梯度爆炸
    • 可以通过适当的权重初始化+Batch Normalization实现
  • 退化现象：即使更深的网络开始收敛了，随着深度的增加，准确率也会趋于饱和，之后迅速下降。这个问题不是过拟合引起的，作者的例子证明层数越深，训练误差也会增加
  • 构想出一个解决方案：模型不需要拟合底层的映射，只需要拟合相对于输入的残差（图更容易理解）。而且这种方案还不会增加额外的计算复杂度

Deep Residual Learning

网络结构很容易理解

三种投影方式：(A) 零填充方式用于增加维度，并且是无参数的； (B) 需要增加维度就使用投影，其他为恒等； © 所有shortcuts都是投影。

Experiments

利用不同的数据集验证模型泛化能力，都取得了不错的效果。

新学习的概念

泛函的距离空间（泛函分析）

泛函的最优逼近

网络架构的自动选取

本文心得

  • ResNet解决网络退化的机理：

    • 深层梯度回传顺畅：防止梯度消失
    • 类比其他机器学习模型：
      • 集成学习boosting，每一个弱分类器拟合“前面的模型与GT之差”
      • 长短时记忆神经网络LSTM的遗忘门
      • ReLU激活函数（重要的时候让它输出，不重要的时候为0）

  • 传统线性结构网络难以拟合“恒等映射”

    • skip connection可以让模型自行选择要不要更新
    • 弥补了高度非线性造成的不可逆的信息损失（MobileNet V2）

  • 神经网络的可解释性分析：

    The Shattered Gradients Problem: If resnets are the answer, then what is the question?.08591

    Residual Networks Behave Like Ensembles of Relatively Shallow Networks：.06431

    为什么ResNet有效？

  • BN层

    • 有BN不需要Bias

附录：代码实践

  • 残差结构
import os.pathimport torch.nn as nn
import torch# 定义残差结构（18层和34层）
class BasicBlock(nn.Module):expansion = 1 # 一个残差块中卷积层的卷积核个数的变化def __init__(self, in_channel, out_channel, stride=1, downsample=None):super(BasicBlock, self).__init__()self.conv1 = nn.Conv2d(in_channels=in_channel, out_channels=out_channel,kernel_size=3, stride=stride, padding=1, bias=False)self.bn1 = nn.BatchNorm2d(out_channel)self.relu = nn.ReLU()self.conv2 = nn.Conv2d(in_channels=out_channel, out_channels=out_channel,kernel_size=3, stride=1, padding=1, bias=False)self.bn2 = nn.BatchNorm2d(out_channel)self.downsample = downsampledef forward(self, x):identity = xif self.downsample is not None:identity = self.downsample(x)out = self.conv1(x)out = self.bn1(out)out = self.relu(out)out = self.conv2(out)out = self.bn2(out)out += identityout = self.relu(out)return out# 定义残差结构（50层、101层、152层）
class Bottleneck(nn.Module):expansion = 4def __init__(self, in_channel, out_channel, stride=1, downsample=None):super(Bottleneck, self).__init__()self.conv1 = nn.Conv2d(in_channels=in_channel, out_channels=out_channel,kernel_size=1, stride=1, padding=1, bias=False)self.bn1 = nn.BatchNorm2d(out_channel)self.conv2 = nn.Conv2d(in_channels=out_channel, out_channels=out_channel,kernel_size=3, stride=stride, padding=1, bias=False)self.bn2 = nn.BatchNorm2d(out_channel)self.conv3 = nn.Conv2d(in_channels=out_channel, out_channels=out_channel * self.expansion,kernel_size=1, stride=1, padding=1, bias=False)self.bn3 = nn.BatchNorm2d(out_channel * self.expansion)self.relu = nn.ReLU(inplace=True)self.downsample = downsampledef forward(self, x):identity = x#  不太明白为什么这样设计，当残差块上面identity和最后输出大小不一致时，不应该一定要用卷积操作吗，这里如果是None，不是大小就不合适了吗if self.downsample is not None:identity = self.downsample(x)out = self.conv1(x)out = self.bn1(out)out = self.relu(out)out = self.conv2(out)out = self.bn2(out)out = self.relu(out)out = self.conv3(out)out = self.bn3(out)out += identityout =  self.relu(out)return out
  • ResNet
class ResNet(nn.Module):def __init__(self, block, block_num, num_classes=1000, include_top=True):super(ResNet, self).__init__()self.include_top = include_topself.in_channel = 64self.conv1 = nn.Conv2d(3, out_channels=self.in_channel,kernel_size=7, stride=2, padding=3, bias=False)self.bn1 = nn.BatchNorm2d(self.in_channel)self.relu = nn.ReLU(inplace=True)self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1)self.layer1 = self.make_layer(block, 64, block_num[0])self.layer2 = self.make_layer(block, 128, block_num[1], stride=2)self.layer3 = self.make_layer(block, 256, block_num[2], stride=2)self.layer4 = self.make_layer(block, 512, block_num[3], stride=2)if self.include_top:self.avgpool = nn.AdaptiveAvgPool2d((1,1))self.fc = nn.Linear(512 * block.expansion, num_classes)# 对卷积层进行初始化for m in self.modules():if isinstance(m, nn.Conv2d):nn.init.kaiming_normal_(m.weight, mode='fan_out', nonlinearity='relu')'''block:各种残差块channel:残差结构中卷积层1的卷积核个数block_num:包含了多少个残差结构stride'''def make_layer(self, block, channel, block_num, stride=1):downsample = Noneif stride != 1 or self.in_channel != channel * block.expansion:downsample = nn.Sequential(nn.Conv2d(self.in_channel, channel * block.expansion, kernel_size=1, stride=stride, bias=False),nn.BatchNorm2d(channel * block.expansion))layers = []layers.append(block(self.in_channel, channel, downsample=downsample, stride=stride))self.in_channel = channel * block.expansionfor _ in range(1, block_num):layers.append(block(self.in_channel, channel))return nn.Sequential(*layers)def forward(self, x):out = self.conv1(x)out = self.bn1(out)out = self.relu(out)out = self.maxpool(out)out = self.layer1(out)out = self.layer2(out)out = self.layer3(out)out = self.layer4(out)if self.include_top:out = self.avgpool(out)out = torch.flatten(out, 1)out = self.fc(out)return outdef resnet34(num_classes=1000, include_top=True):# .pthreturn ResNet(BasicBlock, [3, 4, 6, 3], num_classes=num_classes, include_top=include_top)def resnet101(num_classes=1000, include_top=True):return ResNet(Bottleneck, [3, 4, 23, 3], num_classes=num_classes, include_top=include_top)
  • 训练代码
import torchvision.models.resnet
from torchvision import transforms, datasets
import os
import json
import torch.optim as optim
from tqdm import tqdm
import sysdevice = torch.device('cuda:0'if torch.cuda.is_available() else 'cpu')
print("using {} device".format(device))data_transform = {"train": transforms.Compose([transforms.RandomResizedCrop(224),transforms.RandomHorizontalFlip(),transforms.ToTensor(),transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])]),"val": transforms.Compose([transforms.Resize(256),transforms.CenterCrop(224),transforms.ToTensor(),transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])])}data_root = os.path.abspath(os.path.join(os.getcwd(), "../.."))
image_path = os.path.join(data_root, 'dataset', 'flower_photos')
assert os.path.exists(image_path), '{} path does not exist.'.format(image_path)
train_dataset = datasets.ImageFolder(root=os.path.join(image_path, "train"),transform=data_transform['train'])
train_num = len(train_dataset)
# {'daisy': 0, 'dandelion': 1, 'roses': 2, 'sunflowers': 3, 'tulips': 4}
flower_list = train_dataset.class_to_idx
# {0: 'daisy', 1: 'dandelion', 2: 'roses', 3: 'sunflowers', 4: 'tulips'}
cla_dict = dict((val, key) for key, val in flower_list.items())
json_str = json.dumps(cla_dict, indent=4) # indent=4 参数指定了缩进为 4 个空格
with open('class_indices.json', 'w') as json_file:json_file.write(json_str)batch_size = 16
# nw = min([os.cpu_count(), batch_size if batch_size > 1 else 0, 8])
nw = 0
print('Using {} dataloader workers every process'.format(nw))train_loader = torch.utils.data.DataLoader(train_dataset,batch_size=batch_size, shuffle=True,num_workers=nw)validate_dataset = datasets.ImageFolder(root=os.path.join(image_path, 'val'),transform=data_transform['val'])
val_num = len(validate_dataset)
validate_loader = torch.utils.data.DataLoader(validate_dataset, batch_size=batch_size, shuffle=False, num_workers=nw)
print("using {} images for training, {} images for validation.".format(train_num, val_num))net = resnet34()
model_weight_path = './resnet34-pre.pth'
assert os.path.exists(model_weight_path), "file {} does not exist.".format(model_weight_path)
# load pretrain weights
# 载入预训练模型参数的一种方法
net.load_state_dict(torch.load(model_weight_path, map_location='cpu'))
in_channel = net.fc.in_features # 输入特征矩阵的深度
net.fc = nn.Linear(in_channel, 5)
net.to(device)
# 另一种方法可以是可以先载入内存，之后再删掉fcloss_function = nn.CrossEntropyLoss()params = [p for p in net.parameters() if p.requires_grad]
optimizer = optim.Adam(params, lr=0.0001)epochs = 3
best_acc = 0.0
save_path = './resnet34.pth'
train_steps = len(train_loader)for epoch in range(epochs):# trainnet.train()running_loss = 0.0train_bar = tqdm(train_loader, file=sys.stdout)for step, data in enumerate(train_bar):images, labels = dataoptimizer.zero_grad()logits = net(images.to(device))loss = loss_function(logits, labels.to(device))loss.backward()optimizer.step()# print statisticsrunning_loss += loss.item()train_bar.desc = "train epoch[{}/{}] loss:{:.3f}".format(epoch + 1,epochs,loss)# validatenet.eval()acc = 0.0  # accumulate accurate number / epochwith torch.no_grad():val_bar = tqdm(validate_loader, file=sys.stdout)for val_data in val_bar:val_images, val_labels = val_dataoutputs = net(val_images.to(device))# loss = loss_function(outputs, test_labels)predict_y = torch.max(outputs, dim=1)[1]acc += torch.eq(predict_y, val_labels.to(device)).sum().item()val_bar.desc = "valid epoch[{}/{}]".format(epoch + 1,epochs)val_accurate = acc / val_numprint('[epoch %d] train_loss: %.3f  val_accuracy: %.3f' %(epoch + 1, running_loss / train_steps, val_accurate))if val_accurate > best_acc:best_acc = val_accuratetorch.save(net.state_dict(), save_path)print('Finished Training')
  • 推理代码
import os
import jsonimport torch
from PIL import Image
from torchvision import transforms
import matplotlib.pyplot as pltfrom model import resnet34device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
data_transform = transforms.Compose([transforms.Resize(256),transforms.CenterCrop(224),transforms.ToTensor(),transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])])# load image
img_path = r"D:\workplace\dl\cnn\sun_flower.jpg"
assert os.path.exists(img_path), "file: '{}' dose not exist.".format(img_path)
img = Image.open(img_path)
plt.imshow(img)
# [N, C, H, W]
img = data_transform(img)
# expand batch dimension
img = torch.unsqueeze(img, dim=0)# read class_indict
json_path = './class_indices.json'
assert os.path.exists(json_path), "file: '{}' dose not exist.".format(json_path)with open(json_path, "r") as f:class_indict = json.load(f)# create model
model = resnet34(num_classes=5).to(device)# load model weights
weights_path = "./resNet34.pth"
assert os.path.exists(weights_path), "file: '{}' dose not exist.".format(weights_path)
model.load_state_dict(torch.load(weights_path, map_location=device))# prediction
model.eval()
with torch.no_grad():# predict classoutput = torch.squeeze(model(img.to(device))).cpu()predict = torch.softmax(output, dim=0)predict_cla = torch.argmax(predict).numpy()print_res = "class: {}   prob: {:.3}".format(class_indict[str(predict_cla)],predict[predict_cla].numpy())
plt.title(print_res)
for i in range(len(predict)):print("class: {:10}   prob: {:.3}".format(class_indict[str(i)],predict[i].numpy()))
plt.show()

其他

微软亚洲研究院

学术追星

更多推荐

ResNet笔记 Deep Residual Learning for Image Recognition

本文发布于:2024-02-27 08:37:23，感谢您对本站的认可！
本文链接:https://www.elefans.com/category/jswz/34/1705905.html
版权声明:本站内容均来自互联网，仅供演示用，请勿用于商业和其他非法用途。如果侵犯了您的权益请与我们联系，我们将在24小时内删除。
本文标签：笔记   Deep   ResNet   Residual   Recognition

发布评论

评论列表 （有 0 条评论）

最近发表

草根站长

>www.elefans.com

编程频道|电子爱好者 - 技术资讯及电子产品介绍！

热门文章

标签列表