我想要用grad_cam实现对人脸的嘴部区域进行可视化的demo,使其效果如下图所示,
然后我目前的代码运行后可以达到第一行的效果(如下图):
我的问题是我该如何实现第一张图中后面两类图像的效果!!
以下是我的运行代码
import os
import numpy as np
import torch
from PIL import Image
import matplotlib.pyplot as plt
from torchvision import models
from torchvision import transforms
from utils import GradCAM, show_cam_on_image, center_crop_img
# 遍历文件夹
def getFileList(dir, Filelist, ext=None):
"""
获取文件夹及其子文件夹中文件列表
输入 dir:文件夹根目录
输入 ext: 扩展名
返回: 文件路径列表
"""
# newDir = dir
if os.path.isfile(dir):
if ext is None:
Filelist.append(dir)
else:
if ext in dir[-3:]:
Filelist.append(dir)
elif os.path.isdir(dir):
for s in os.listdir(dir):
newDir = os.path.join(dir, s)
getFileList(newDir, Filelist, ext)
return Filelist
if __name__ == '__main__':
model = models.resnet18(pretrained=False)
pth_file = './lrw_resnet18_mstcn.pth.tar'
model.load_state_dict(torch.load(pth_file, map_location='cpu'), strict=False)
target_layers = [model.layer1]
data_transform = transforms.Compose([transforms.ToTensor(),
transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])])
# load image
org_img_folder = '.\\frame'
imglist = getFileList(org_img_folder, [], 'jpg')
for img_path in imglist:
# img_path = "55.jpg"
assert os.path.exists(img_path), "file: '{}' dose not exist.".format(img_path)
img = Image.open(img_path).convert('RGB')
img = np.array(img, dtype=np.uint8)
# img = center_crop_img(img, 224)
# [C, H, W]
img_tensor = data_transform(img)
# expand batch dimension
# [C, H, W] -> [N, C, H, W]
input_tensor = torch.unsqueeze(img_tensor, dim=0)
cam = GradCAM(model=model, target_layers=target_layers, use_cuda=False)
target_category = None # tabby, tabby cat 281
# target_category = 254 # pug, pug-dog
# print(target_category)
grayscale_cam = cam(input_tensor=input_tensor, target_category=target_category)
grayscale_cam = grayscale_cam[0, :]
visualization = show_cam_on_image(img.astype(dtype=np.float32) / 255.,
grayscale_cam,
use_rgb=True)
plt.imshow(visualization)
plt.show()
还有一个代码是https://github.com/WZMIAOMIAO/deep-learning-for-image-processing/tree/master/pytorch_classification/grad_cam中的utils.py文件。
有试过分别提取resnet18中的前面三层的特征计算cam进行输出,但效果也很差,如下图就是提取layer1中的特征计算cam的可视化图(将target_layers那里改成layer1)
所以最后感谢提供帮助的各位,教教我如何达到第一张图片中后面两类图像的的热图效果!