opencv：使用dlib/face++ 人脸关键点检测

关键点检测"/>

opencv：使用dlib/face++ 人脸关键点检测

人脸关键点检测

人脸，是人身份的一种关键信息。从古代来说，会有面相之说。有的说，一个人心里怎么样，就会长得怎么样，相由心生。据说，这一些经验也被应用于警察在路边抽查的时候，比如贼眉鼠眼等等的样子，就容易受到盘问。

这种靠经验总结的传统信息在现在这个时代可能会有些用，但随着科技的发展，那么，人脸到底是如何被应用于现代科技上面的呢？答案是依靠人脸关键点。人脸关键点检测是人脸在工业应用中一个比较重要的基础性研究，可以说，大多数的人脸相关业务都是在此基础上进行的二次开发。比如人脸识别，试妆算法，人脸融合等。人脸的关键点检测的好坏，直接影响后续的业务。

目前很多公司都有关于人脸关键点检测的api的业务，比如腾讯云，百度云，face++等。这里不做过多的介绍，只是讲解一下，利用opencv的包dlib以及调用face++的接口，展示一下人脸关键点检测大致概念。

使用dlib进行人脸关键点检测

dlib封装了一个人脸关键点提取算法，下载好模型后可直接调用。目前是实现的是68点 关键点检测：

（68点）

使用face++ APi接口进行人脸关键点检测

face++ 实现了83点以及106点的关键点检测，直接去官网申请账号，进行调用接口就行。

​ (83点)

​ （106点）

代码

#!/usr/bin/pythonimport cv2
import dlib
import numpy
import numpy as np
import requests
import json
import sysPREDICTOR_PATH = "./shape_predictor_68_face_landmarks.dat"class Face_Detect(object):def __init__(self):self.url = ''self.params = {'api_key': 'your key','api_secret': 'your secret','return_landmark': 2,   # 1：83点  2：106点}def landmarks_face(self, image):img_file = {'image_file': open(image, 'rb')}r = requests.post(url=self.url, files=img_file, data=self.params)if r.status_code == requests.codes.ok:return r.content.decode('utf-8')else:return r.contentdef face_points(self, image):points = self.landmarks_face(image)faces = json.loads(points)['faces']matrix_list = np.matrix(self.matrix_marks(faces[0]['landmark']))point_list = []for p in matrix_list.tolist():point_list.append((int(p[0]), int(p[1])))return point_listdef matrix_marks(self, res):if self.params['return_landmark'] == 1:pointer = [[res['contour_left1']['x'], res['contour_left1']['y']],[res['contour_left2']['x'], res['contour_left2']['y']],[res['contour_left3']['x'], res['contour_left3']['y']],[res['contour_left4']['x'], res['contour_left4']['y']],[res['contour_left5']['x'], res['contour_left5']['y']],[res['contour_left6']['x'], res['contour_left6']['y']],[res['contour_left7']['x'], res['contour_left7']['y']],[res['contour_left8']['x'], res['contour_left8']['y']],[res['contour_left9']['x'], res['contour_left9']['y']],[res['contour_chin']['x'], res['contour_chin']['y']],[res['contour_right9']['x'], res['contour_right9']['y']],[res['contour_right8']['x'], res['contour_right8']['y']],[res['contour_right7']['x'], res['contour_right7']['y']],[res['contour_right6']['x'], res['contour_right6']['y']],[res['contour_right5']['x'], res['contour_right5']['y']],[res['contour_right4']['x'], res['contour_right4']['y']],[res['contour_right3']['x'], res['contour_right3']['y']],[res['contour_right2']['x'], res['contour_right2']['y']],[res['contour_right1']['x'], res['contour_right1']['y']],[res['left_eye_bottom']['x'], res['left_eye_bottom']['y']],[res['left_eye_center']['x'], res['left_eye_center']['y']],[res['left_eye_left_corner']['x'], res['left_eye_left_corner']['y']],[res['left_eye_lower_left_quarter']['x'], res['left_eye_lower_left_quarter']['y']],[res['left_eye_lower_right_quarter']['x'], res['left_eye_lower_right_quarter']['y']],[res['left_eye_pupil']['x'], res['left_eye_pupil']['y']],[res['left_eye_right_corner']['x'], res['left_eye_right_corner']['y']],[res['left_eye_top']['x'], res['left_eye_top']['y']],[res['left_eye_upper_left_quarter']['x'], res['left_eye_upper_left_quarter']['y']],[res['left_eye_upper_right_quarter']['x'], res['left_eye_upper_right_quarter']['y']],[res['left_eyebrow_left_corner']['x'], res['left_eyebrow_left_corner']['y']],[res['left_eyebrow_lower_left_quarter']['x'], res['left_eyebrow_lower_left_quarter']['y']],[res['left_eyebrow_lower_middle']['x'], res['left_eyebrow_lower_middle']['y']],[res['left_eyebrow_lower_right_quarter']['x'], res['left_eyebrow_lower_right_quarter']['y']],[res['left_eyebrow_right_corner']['x'], res['left_eyebrow_right_corner']['y']],[res['left_eyebrow_upper_left_quarter']['x'], res['left_eyebrow_upper_left_quarter']['y']],[res['left_eyebrow_upper_middle']['x'], res['left_eyebrow_upper_middle']['y']],[res['left_eyebrow_upper_right_quarter']['x'], res['left_eyebrow_upper_right_quarter']['y']],[res['mouth_left_corner']['x'], res['mouth_left_corner']['y']],[res['mouth_lower_lip_bottom']['x'], res['mouth_lower_lip_bottom']['y']],[res['mouth_lower_lip_left_contour1']['x'], res['mouth_lower_lip_left_contour1']['y']],[res['mouth_lower_lip_left_contour2']['x'], res['mouth_lower_lip_left_contour2']['y']],[res['mouth_lower_lip_left_contour3']['x'], res['mouth_lower_lip_left_contour3']['y']],[res['mouth_lower_lip_right_contour1']['x'], res['mouth_lower_lip_right_contour1']['y']],[res['mouth_lower_lip_right_contour2']['x'], res['mouth_lower_lip_right_contour2']['y']],[res['mouth_lower_lip_right_contour3']['x'], res['mouth_lower_lip_right_contour3']['y']],[res['mouth_lower_lip_top']['x'], res['mouth_lower_lip_top']['y']],[res['mouth_right_corner']['x'], res['mouth_right_corner']['y']],[res['mouth_upper_lip_bottom']['x'], res['mouth_upper_lip_bottom']['y']],[res['mouth_upper_lip_left_contour1']['x'], res['mouth_upper_lip_left_contour1']['y']],[res['mouth_upper_lip_left_contour2']['x'], res['mouth_upper_lip_left_contour2']['y']],[res['mouth_upper_lip_left_contour3']['x'], res['mouth_upper_lip_left_contour3']['y']],[res['mouth_upper_lip_right_contour1']['x'], res['mouth_upper_lip_right_contour1']['y']],[res['mouth_upper_lip_right_contour2']['x'], res['mouth_upper_lip_right_contour2']['y']],[res['mouth_upper_lip_right_contour3']['x'], res['mouth_upper_lip_right_contour3']['y']],[res['mouth_upper_lip_top']['x'], res['mouth_upper_lip_top']['y']],[res['nose_contour_left1']['x'], res['nose_contour_left1']['y']],[res['nose_contour_left2']['x'], res['nose_contour_left2']['y']],[res['nose_contour_left3']['x'], res['nose_contour_left3']['y']],[res['nose_contour_lower_middle']['x'], res['nose_contour_lower_middle']['y']],[res['nose_contour_right1']['x'], res['nose_contour_right1']['y']],[res['nose_contour_right2']['x'], res['nose_contour_right2']['y']],[res['nose_contour_right3']['x'], res['nose_contour_right3']['y']],[res['nose_left']['x'], res['nose_left']['y']],[res['nose_right']['x'], res['nose_right']['y']],[res['nose_tip']['x'], res['nose_tip']['y']],[res['right_eye_bottom']['x'], res['right_eye_bottom']['y']],[res['right_eye_center']['x'], res['right_eye_center']['y']],[res['right_eye_left_corner']['x'], res['right_eye_left_corner']['y']],[res['right_eye_lower_left_quarter']['x'], res['right_eye_lower_left_quarter']['y']],[res['right_eye_lower_right_quarter']['x'], res['right_eye_lower_right_quarter']['y']],[res['right_eye_pupil']['x'], res['right_eye_pupil']['y']],[res['right_eye_right_corner']['x'], res['right_eye_right_corner']['y']],[res['right_eye_top']['x'], res['right_eye_top']['y']],[res['right_eye_upper_left_quarter']['x'], res['right_eye_upper_left_quarter']['y']],[res['right_eye_upper_right_quarter']['x'], res['right_eye_upper_right_quarter']['y']],[res['right_eyebrow_left_corner']['x'], res['right_eyebrow_left_corner']['y']],[res['right_eyebrow_lower_left_quarter']['x'], res['right_eyebrow_lower_left_quarter']['y']],[res['right_eyebrow_lower_middle']['x'], res['right_eyebrow_lower_middle']['y']],[res['right_eyebrow_lower_right_quarter']['x'], res['right_eyebrow_lower_right_quarter']['y']],[res['right_eyebrow_right_corner']['x'], res['right_eyebrow_right_corner']['y']],[res['right_eyebrow_upper_left_quarter']['x'], res['right_eyebrow_upper_left_quarter']['y']],[res['right_eyebrow_upper_middle']['x'], res['right_eyebrow_upper_middle']['y']],[res['right_eyebrow_upper_right_quarter']['x'], res['right_eyebrow_upper_right_quarter']['y']]]elif self.params['return_landmark'] == 2:pointer = [[res['contour_left1']['x'], res['contour_left1']['y']],[res['contour_left2']['x'], res['contour_left2']['y']],[res['contour_left3']['x'], res['contour_left3']['y']],[res['contour_left4']['x'], res['contour_left4']['y']],[res['contour_left5']['x'], res['contour_left5']['y']],[res['contour_left6']['x'], res['contour_left6']['y']],[res['contour_left7']['x'], res['contour_left7']['y']],[res['contour_left8']['x'], res['contour_left8']['y']],[res['contour_left9']['x'], res['contour_left9']['y']],[res['contour_left10']['x'], res['contour_left10']['y']],[res['contour_left11']['x'], res['contour_left11']['y']],[res['contour_left12']['x'], res['contour_left12']['y']],[res['contour_left13']['x'], res['contour_left13']['y']],[res['contour_left14']['x'], res['contour_left14']['y']],[res['contour_left15']['x'], res['contour_left15']['y']],[res['contour_left16']['x'], res['contour_left16']['y']],[res['contour_chin']['x'], res['contour_chin']['y']],[res['contour_right16']['x'], res['contour_right16']['y']],[res['contour_right15']['x'], res['contour_right15']['y']],[res['contour_right14']['x'], res['contour_right14']['y']],[res['contour_right13']['x'], res['contour_right13']['y']],[res['contour_right12']['x'], res['contour_right12']['y']],[res['contour_right11']['x'], res['contour_right11']['y']],[res['contour_right10']['x'], res['contour_right10']['y']],[res['contour_right9']['x'], res['contour_right9']['y']],[res['contour_right8']['x'], res['contour_right8']['y']],[res['contour_right7']['x'], res['contour_right7']['y']],[res['contour_right6']['x'], res['contour_right6']['y']],[res['contour_right5']['x'], res['contour_right5']['y']],[res['contour_right4']['x'], res['contour_right4']['y']],[res['contour_right3']['x'], res['contour_right3']['y']],[res['contour_right2']['x'], res['contour_right2']['y']],[res['contour_right1']['x'], res['contour_right1']['y']],[res['left_eyebrow_left_corner']['x'], res['left_eyebrow_left_corner']['y']],[res['left_eyebrow_upper_left_quarter']['x'], res['left_eyebrow_upper_left_quarter']['y']],[res['left_eyebrow_upper_middle']['x'], res['left_eyebrow_upper_middle']['y']],[res['left_eyebrow_upper_right_quarter']['x'], res['left_eyebrow_upper_right_quarter']['y']],[res['left_eyebrow_upper_right_corner']['x'], res['left_eyebrow_upper_right_corner']['y']],[res['left_eyebrow_lower_left_quarter']['x'], res['left_eyebrow_lower_left_quarter']['y']],[res['left_eyebrow_lower_middle']['x'], res['left_eyebrow_lower_middle']['y']],[res['left_eyebrow_lower_right_quarter']['x'], res['left_eyebrow_lower_right_quarter']['y']],[res['left_eyebrow_lower_right_corner']['x'], res['left_eyebrow_lower_right_corner']['y']],[res['right_eyebrow_upper_left_corner']['x'], res['right_eyebrow_upper_left_corner']['y']],[res['right_eyebrow_upper_left_quarter']['x'], res['right_eyebrow_upper_left_quarter']['y']],[res['right_eyebrow_upper_middle']['x'], res['right_eyebrow_upper_middle']['y']],[res['right_eyebrow_upper_right_quarter']['x'], res['right_eyebrow_upper_right_quarter']['y']],[res['right_eyebrow_right_corner']['x'], res['right_eyebrow_right_corner']['y']],[res['right_eyebrow_lower_left_corner']['x'], res['right_eyebrow_lower_left_corner']['y']],[res['right_eyebrow_lower_left_quarter']['x'], res['right_eyebrow_lower_left_quarter']['y']],[res['right_eyebrow_lower_middle']['x'], res['right_eyebrow_lower_middle']['y']],[res['right_eyebrow_lower_right_quarter']['x'], res['right_eyebrow_lower_right_quarter']['y']],[res['nose_bridge1']['x'], res['nose_bridge1']['y']],[res['nose_bridge2']['x'], res['nose_bridge2']['y']],[res['nose_bridge3']['x'], res['nose_bridge3']['y']],[res['nose_tip']['x'], res['nose_tip']['y']],[res['nose_left_contour1']['x'], res['nose_left_contour1']['y']],[res['nose_left_contour2']['x'], res['nose_left_contour2']['y']],[res['nose_left_contour3']['x'], res['nose_left_contour3']['y']],[res['nose_left_contour4']['x'], res['nose_left_contour4']['y']],[res['nose_left_contour5']['x'], res['nose_left_contour5']['y']],[res['nose_middle_contour']['x'], res['nose_middle_contour']['y']],[res['nose_right_contour1']['x'], res['nose_right_contour1']['y']],[res['nose_right_contour2']['x'], res['nose_right_contour2']['y']],[res['nose_right_contour3']['x'], res['nose_right_contour3']['y']],[res['nose_right_contour4']['x'], res['nose_right_contour4']['y']],[res['nose_right_contour5']['x'], res['nose_right_contour5']['y']],[res['left_eye_left_corner']['x'], res['left_eye_left_corner']['y']],[res['left_eye_upper_left_quarter']['x'], res['left_eye_upper_left_quarter']['y']],[res['left_eye_top']['x'], res['left_eye_top']['y']],[res['left_eye_upper_right_quarter']['x'], res['left_eye_upper_right_quarter']['y']],[res['left_eye_right_corner']['x'], res['left_eye_right_corner']['y']],[res['left_eye_lower_right_quarter']['x'], res['left_eye_lower_right_quarter']['y']],[res['left_eye_bottom']['x'], res['left_eye_bottom']['y']],[res['left_eye_lower_left_quarter']['x'], res['left_eye_lower_left_quarter']['y']],[res['left_eye_pupil']['x'], res['left_eye_pupil']['y']],[res['left_eye_center']['x'], res['left_eye_center']['y']],[res['right_eye_left_corner']['x'], res['right_eye_left_corner']['y']],[res['right_eye_upper_left_quarter']['x'], res['right_eye_upper_left_quarter']['y']],[res['right_eye_top']['x'], res['right_eye_top']['y']],[res['right_eye_upper_right_quarter']['x'], res['right_eye_upper_right_quarter']['y']],[res['right_eye_right_corner']['x'], res['right_eye_right_corner']['y']],[res['right_eye_lower_right_quarter']['x'], res['right_eye_lower_right_quarter']['y']],[res['right_eye_bottom']['x'], res['right_eye_bottom']['y']],[res['right_eye_lower_left_quarter']['x'], res['right_eye_lower_left_quarter']['y']],[res['right_eye_pupil']['x'], res['right_eye_pupil']['y']],[res['right_eye_center']['x'], res['right_eye_center']['y']],[res['mouth_left_corner']['x'], res['mouth_left_corner']['y']],[res['mouth_upper_lip_left_contour1']['x'], res['mouth_upper_lip_left_contour1']['y']],[res['mouth_upper_lip_left_contour2']['x'], res['mouth_upper_lip_left_contour2']['y']],[res['mouth_upper_lip_left_contour3']['x'], res['mouth_upper_lip_left_contour3']['y']],[res['mouth_upper_lip_left_contour4']['x'], res['mouth_upper_lip_left_contour4']['y']],[res['mouth_right_corner']['x'], res['mouth_right_corner']['y']],[res['mouth_upper_lip_right_contour1']['x'], res['mouth_upper_lip_right_contour1']['y']],[res['mouth_upper_lip_right_contour2']['x'], res['mouth_upper_lip_right_contour2']['y']],[res['mouth_upper_lip_right_contour3']['x'], res['mouth_upper_lip_right_contour3']['y']],[res['mouth_upper_lip_right_contour4']['x'], res['mouth_upper_lip_right_contour4']['y']],[res['mouth_upper_lip_top']['x'], res['mouth_upper_lip_top']['y']],[res['mouth_upper_lip_bottom']['x'], res['mouth_upper_lip_bottom']['y']],[res['mouth_lower_lip_right_contour1']['x'], res['mouth_lower_lip_right_contour1']['y']],[res['mouth_lower_lip_right_contour2']['x'], res['mouth_lower_lip_right_contour2']['y']],[res['mouth_lower_lip_right_contour3']['x'], res['mouth_lower_lip_right_contour3']['y']],[res['mouth_lower_lip_left_contour1']['x'], res['mouth_lower_lip_left_contour1']['y']],[res['mouth_lower_lip_left_contour2']['x'], res['mouth_lower_lip_left_contour2']['y']],[res['mouth_lower_lip_left_contour3']['x'], res['mouth_lower_lip_left_contour3']['y']],[res['mouth_lower_lip_top']['x'], res['mouth_lower_lip_top']['y']],[res['mouth_lower_lip_bottom']['x'], res['mouth_lower_lip_bottom']['y']]]return pointerFD = Face_Detect()FACE_POINTS = list(range(17, 68))
MOUTH_POINTS = list(range(48, 61))
RIGHT_BROW_POINTS = list(range(17, 22))
LEFT_BROW_POINTS = list(range(22, 27))
RIGHT_EYE_POINTS = list(range(36, 42))
LEFT_EYE_POINTS = list(range(42, 48))
NOSE_POINTS = list(range(27, 35))
JAW_POINTS = list(range(0, 17))detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor(PREDICTOR_PATH)class TooManyFaces(Exception):passclass NoFaces(Exception):passdef get_landmarks_dlib(im):rects = detector(im, 1)if len(rects) > 1:raise TooManyFacesif len(rects) == 0:raise NoFacesreturn [[p.x, p.y] for p in predictor(im, rects[0]).parts()]img_file = sys.argv[1]
img = cv2.imread(img_file)# landmarks = get_landmarks_dlib(img)  # get landmarks by opencv dlib
landmarks = FD.face_points(img_file)   # get landmarks by face plus plus api
print(landmarks, type(landmarks), len(landmarks))for landmark in landmarks:pos = tuple(landmark)cv2.circle(img, pos, 5, color=(0, 255, 0))  # 画圈font = cv2.FONT_HERSHEY_SIMPLEX  # 点数字cv2.putText(img, str(landmarks.index(landmark) + 1), pos, font, 0.4, (0, 0, 255), 1, cv2.LINE_AA)cv2.namedWindow("img", 2)cv2.imshow("img", img)cv2.waitKey(0)

参考

• 面相

• 面相是否真的存在

• 人脸关键点检测综述

• face++人脸关键点