大眼和瘦脸"/>
OpenGL 实现大眼和瘦脸
借鉴博客:iOS原生框架Vision实现瘦脸大眼特效、仿抖音特效相机之大眼瘦脸
本文达成效果如下图:
效果图
106个特征点如下图
特征点
原理解析
主要是以下3点,具体请前往参考博客和原理解析。
- 1.圆内放大
- 2.圆内缩小
- 3.向某一点拉伸
用一张gif图来概括上面的内容,也是本文章最终的达成的效果,如开头所展示的效果图
经过前面我们了解了
-
日常开发中OpenGL开发流程
- 1.设置图层
- 2.设置图形上下文
- 3.设置渲染缓冲区(renderBuffer)
- 4.设置帧缓冲区(frameBuffer)
- 5.编译、链接着色器(shader)
- 6.设置VBO (Vertex Buffer Objects)
- 7.设置纹理
- 8.渲染
这些基本步骤大致是不变的。这章是摄像头渲染+"多滤镜"渲染思想的结合提现。内容是感觉是增加了,但是实际的开发流程还是一样的。接下来让我们进入正题。
经过分析我们主要有以下3个工作:
需求图
核心代码:
///绘制面部特征点func renderFacePoint() {//MARK: - 1.绘制摄像头//使用着色器glUseProgram(renderProgram)//绑定frameBufferglBindFramebuffer(GLenum(GL_FRAMEBUFFER), facePointFrameBuffer)//设置清屏颜色glClearColor(0.0, 0.0, 0.0, 1.0)//清除屏幕glClear(GLbitfield(GL_COLOR_BUFFER_BIT))//1.设置视口大小let scale = self.contentScaleFactorglViewport(0, 0, GLsizei(self.frame.size.width * scale), GLsizei(self.frame.size.height * scale))#warning("注意⚠️:想要获取shader里面的变量,这里要记住要在glLinkProgram后面、后面、后面")//----处理顶点数据-------//将顶点数据通过renderProgram中的传递到顶点着色程序的position/*1.glGetAttribLocation,用来获取vertex attribute的入口的.2.告诉OpenGL ES,通过glEnableVertexAttribArray,3.最后数据是通过glVertexAttribPointer传递过去的。*///注意:第二参数字符串必须和shaderv.vsh中的输入变量:position保持一致let position = glGetAttribLocation(renderProgram, "position")//设置合适的格式从buffer里面读取数据glEnableVertexAttribArray(GLuint(position))//设置读取方式//参数1:index,顶点数据的索引//参数2:size,每个顶点属性的组件数量,1,2,3,或者4.默认初始值是4.//参数3:type,数据中的每个组件的类型,常用的有GL_FLOAT,GL_BYTE,GL_SHORT。默认初始值为GL_FLOAT//参数4:normalized,固定点数据值是否应该归一化,或者直接转换为固定值。(GL_FALSE)//参数5:stride,连续顶点属性之间的偏移量,默认为0;//参数6:指定一个指针,指向数组中的第一个顶点属性的第一个组件。默认为0
// glVertexAttribPointer(GLuint(position), 3, GLenum(GL_FLOAT), GLboolean(GL_FALSE), GLsizei(MemoryLayout<GLfloat>.size * 5), UnsafeRawPointer(bitPattern: MemoryLayout<GLfloat>.size * 0))glVertexAttribPointer(GLuint(position), 3, GLenum(GL_FLOAT), GLboolean(GL_FALSE), 0, standardVertex)//----处理纹理数据-------//1.glGetAttribLocation,用来获取vertex attribute的入口的.//注意:第二参数字符串必须和shaderv.vsh中的输入变量:textCoordinate保持一致let textCoord = glGetAttribLocation(renderProgram, "textCoordinate")//设置合适的格式从buffer里面读取数据glEnableVertexAttribArray(GLuint(textCoord))//3.设置读取方式//参数1:index,顶点数据的索引//参数2:size,每个顶点属性的组件数量,1,2,3,或者4.默认初始值是4.//参数3:type,数据中的每个组件的类型,常用的有GL_FLOAT,GL_BYTE,GL_SHORT。默认初始值为GL_FLOAT//参数4:normalized,固定点数据值是否应该归一化,或者直接转换为固定值。(GL_FALSE)//参数5:stride,连续顶点属性之间的偏移量,默认为0;//参数6:指定一个指针,指向数组中的第一个顶点属性的第一个组件。默认为0
// glVertexAttribPointer(GLuint(textCoord), 2, GLenum(GL_FLOAT), GLboolean(GL_FALSE), GLsizei(MemoryLayout<GLfloat>.size * 5), UnsafeRawPointer(bitPattern: MemoryLayout<GLfloat>.size * 3))glVertexAttribPointer(GLuint(textCoord), 2, GLenum(GL_FLOAT), GLboolean(GL_FALSE), 0, standardVerticalInvertFragment)//法一:使用 CVOpenGLESTexture进行加载,打开下面glActiveTexture(GLenum(GL_TEXTURE0))glUniform1i(glGetUniformLocation(self.renderProgram, "colorMap"), 0)//法二:使用 glTexImage2D 方式加载,打开下面
// glActiveTexture(GLenum(GL_TEXTURE1))
// glBindTexture(GLenum(GL_TEXTURE_2D), originalTexture)
// glUniform1i(glGetUniformLocation(self.renderProgram, "colorMap"), 1) //单个纹理可以不用设置glDrawArrays(GLenum(GL_TRIANGLES), 0, 6)//MARK: - 2.绘制面部特征点if drawLandMark {//注意⚠️:不能清屏。否则看不到照相机画面// glClearColor(0.0, 0.0, 0.0, 1.0)//清除屏幕// glClear(GLbitfield(GL_COLOR_BUFFER_BIT))//1.设置视口大小glViewport(0, 0, GLsizei(self.frame.size.width * scale), GLsizei(self.frame.size.height * scale))//使用着色器glUseProgram(faceProgram)for faceInfo in FaceDetector.shareInstance().faceModels {var tempPoint: [GLfloat] = [GLfloat].init(repeating: 0, count: faceInfo.landmarks.count * 3)var indices: [GLubyte] = [GLubyte].init(repeating: 0, count: faceInfo.landmarks.count)for i in 0..<faceInfo.landmarks.count {let point = faceInfo.landmarks[i].cgPointValuetempPoint[i*3+0] = GLfloat(point.x * 2 - 1)tempPoint[i*3+1] = GLfloat(point.y * 2 - 1)tempPoint[i*3+2] = 0.0indices[i] = GLubyte(i)}let position = glGetAttribLocation(faceProgram, "position")glEnableVertexAttribArray(GLuint(position))//这种方式得先把顶点数据提交到GPU// glVertexAttribPointer(GLuint(position), 3, GLenum(GL_FLOAT), GLboolean(GL_FALSE), GLsizei(MemoryLayout<GLfloat>.size * 3), UnsafeRawPointer(bitPattern: MemoryLayout<GLfloat>.size * 0))glVertexAttribPointer(GLuint(position), 3, GLenum(GL_FLOAT), GLboolean(GL_FALSE), 0, tempPoint)let lineWidth = faceInfo.bounds.size.width / CGFloat(self.frame.width * scale)let sizeScaleUniform = glGetUniformLocation(self.faceProgram, "sizeScale")glUniform1f(GLint(sizeScaleUniform), GLfloat(lineWidth * 20))// var scaleMatrix = GLKMatrix4Identity//GLKMatrix4Scale(GLKMatrix4Identity, 1/Float(lineWidth), 1/Float(lineWidth), 0)// let scaleMatrixUniform = shader.uniformIndex("scaleMatrix")!// glUniformMatrix4fv(GLint(scaleMatrixUniform), 1, GLboolean(GL_FALSE), &scaleMatrix.m.0)glDrawElements(GLenum(GL_POINTS), GLsizei(indices.count), GLenum(GL_UNSIGNED_BYTE), indices)}}//MARK: - 3.绘制纹理完毕,开始瘦脸renderThinFace()}//MARK: - 绘制瘦脸///绘制瘦脸func renderThinFace() {//使用着色器glUseProgram(thinFaceProgram)//绑定frameBufferglBindFramebuffer(GLenum(GL_FRAMEBUFFER), thinFaceFrameBuffer)let faceInfo = FaceDetector.shareInstance().oneFaceif faceInfo.landmarks.count == 0 {glUniform1i(hasFaceUniform, 0)//3.绘制纹理完毕,开始渲染到屏幕上displayRenderToScreen(facePointTexture)return}glClearColor(0.0, 0.0, 0.0, 1.0)//清除屏幕glClear(GLbitfield(GL_COLOR_BUFFER_BIT))//1.设置视口大小let scale = self.contentScaleFactorglViewport(0, 0, GLsizei(self.frame.size.width * scale), GLsizei(self.frame.size.height * scale))hasFaceUniform = glGetUniformLocation(self.thinFaceProgram, "hasFace")aspectRatioUniform = glGetUniformLocation(self.thinFaceProgram, "aspectRatio")facePointsUniform = glGetUniformLocation(self.thinFaceProgram, "facePoints")thinFaceDeltaUniform = glGetUniformLocation(self.thinFaceProgram, "thinFaceDelta")bigEyeDeltaUniform = glGetUniformLocation(self.thinFaceProgram, "bigEyeDelta")glUniform1i(hasFaceUniform, 1)let aspect: Float = Float(inputTextureW / inputTextureH)glUniform1f(aspectRatioUniform, aspect)glUniform1f(thinFaceDeltaUniform, thinFaceDelta)glUniform1f(bigEyeDeltaUniform, bigEyeDelta)let size = 106 * 2var tempPoint: [GLfloat] = [GLfloat].init(repeating: 0, count: size)var index = 0for i in 0..<faceInfo.landmarks.count {let point = faceInfo.landmarks[i].cgPointValuetempPoint[i*2+0] = GLfloat(point.x)tempPoint[i*2+1] = GLfloat(point.y)index += 2if (index == size) {break}}glUniform1fv(facePointsUniform, GLsizei(size), tempPoint)//注意:第二参数字符串必须和shaderv.vsh中的输入变量:position保持一致let position = glGetAttribLocation(thinFaceProgram, "position")glEnableVertexAttribArray(GLuint(position))glVertexAttribPointer(GLuint(position), 3, GLenum(GL_FLOAT), GLboolean(GL_FALSE), 0, standardVertex)//----处理纹理数据-------let textCoord = glGetAttribLocation(thinFaceProgram, "inputTextureCoordinate")//设置合适的格式从buffer里面读取数据glEnableVertexAttribArray(GLuint(textCoord))glVertexAttribPointer(GLuint(textCoord), 2, GLenum(GL_FLOAT), GLboolean(GL_FALSE), 0, standardVerticalInvertFragment)glActiveTexture(GLenum(GL_TEXTURE1))glBindTexture(GLenum(GL_TEXTURE_2D), facePointTexture)glUniform1i(glGetUniformLocation(self.thinFaceProgram, "inputImageTexture"), 1) //单个纹理可以不用设置glDrawArrays(GLenum(GL_TRIANGLES), 0, 6)//MARK: - 3.绘制纹理完毕,开始渲染到屏幕上displayRenderToScreen(thinFaceTexture)}//8.渲染到屏幕上private func displayRenderToScreen(_ texture: GLuint) {//注意⚠️:打破之前的纹理绑定关系,使OpenGL的纹理绑定状态恢复到默认状态。glBindTexture(GLenum(GL_TEXTURE_2D), 0) //将2D纹理绑定到默认的纹理,一般用于打破之前的纹理绑定关系,使OpenGL的纹理绑定状态恢复到默认状态。glBindFramebuffer(GLenum(GL_FRAMEBUFFER), 0)//将framebuffer绑定到默认的FBO处,一般用于打破之前的FBO绑定关系,使OpenGL的FBO绑定状态恢复到默认状态。//设置清屏颜色glClearColor(0.0, 0.0, 0.0, 1.0)//清除屏幕glClear(GLbitfield(GL_COLOR_BUFFER_BIT))//1.设置视口大小let scale = self.contentScaleFactorglViewport(0, 0, GLsizei(self.frame.size.width * scale), GLsizei(self.frame.size.height * scale))//使用着色器glUseProgram(displayProgram)//绑定frameBufferglBindFramebuffer(GLenum(GL_FRAMEBUFFER), frameBuffer)#warning("注意⚠️:想要获取shader里面的变量,这里要记住要在glLinkProgram后面、后面、后面")//----处理顶点数据-------//将顶点数据通过renderProgram中的传递到顶点着色程序的position/*1.glGetAttribLocation,用来获取vertex attribute的入口的.2.告诉OpenGL ES,通过glEnableVertexAttribArray,3.最后数据是通过glVertexAttribPointer传递过去的。*///注意:第二参数字符串必须和shaderv.vsh中的输入变量:position保持一致let position = glGetAttribLocation(displayProgram, "position")//设置合适的格式从buffer里面读取数据glEnableVertexAttribArray(GLuint(position))//设置读取方式//参数1:index,顶点数据的索引//参数2:size,每个顶点属性的组件数量,1,2,3,或者4.默认初始值是4.//参数3:type,数据中的每个组件的类型,常用的有GL_FLOAT,GL_BYTE,GL_SHORT。默认初始值为GL_FLOAT//参数4:normalized,固定点数据值是否应该归一化,或者直接转换为固定值。(GL_FALSE)//参数5:stride,连续顶点属性之间的偏移量,默认为0;//参数6:指定一个指针,指向数组中的第一个顶点属性的第一个组件。默认为0
// glVertexAttribPointer(GLuint(position), 3, GLenum(GL_FLOAT), GLboolean(GL_FALSE), GLsizei(MemoryLayout<GLfloat>.size * 5), UnsafeRawPointer(bitPattern: MemoryLayout<GLfloat>.size * 0))glVertexAttribPointer(GLuint(position), 3, GLenum(GL_FLOAT), GLboolean(GL_FALSE), 0, standardVertex)//----处理纹理数据-------//1.glGetAttribLocation,用来获取vertex attribute的入口的.//注意:第二参数字符串必须和shaderv.vsh中的输入变量:textCoordinate保持一致let textCoord = glGetAttribLocation(displayProgram, "textCoordinate")//设置合适的格式从buffer里面读取数据glEnableVertexAttribArray(GLuint(textCoord))//3.设置读取方式//参数1:index,顶点数据的索引//参数2:size,每个顶点属性的组件数量,1,2,3,或者4.默认初始值是4.//参数3:type,数据中的每个组件的类型,常用的有GL_FLOAT,GL_BYTE,GL_SHORT。默认初始值为GL_FLOAT//参数4:normalized,固定点数据值是否应该归一化,或者直接转换为固定值。(GL_FALSE)//参数5:stride,连续顶点属性之间的偏移量,默认为0;//参数6:指定一个指针,指向数组中的第一个顶点属性的第一个组件。默认为0
// glVertexAttribPointer(GLuint(textCoord), 2, GLenum(GL_FLOAT), GLboolean(GL_FALSE), GLsizei(MemoryLayout<GLfloat>.size * 5), UnsafeRawPointer(bitPattern: MemoryLayout<GLfloat>.size * 3))glVertexAttribPointer(GLuint(textCoord), 2, GLenum(GL_FLOAT), GLboolean(GL_FALSE), 0, standardVerticalInvertFragment)glActiveTexture(GLenum(GL_TEXTURE0))glBindTexture(GLenum(GL_TEXTURE_2D), texture)glUniform1i(glGetUniformLocation(self.displayProgram, "inputImageTexture"), 0) //单个纹理可以不用设置glDrawArrays(GLenum(GL_TRIANGLES), 0, 6)if (EAGLContext.current() == myContext) {myContext.presentRenderbuffer(Int(GL_RENDERBUFFER))}}
这里值得注意的是:绘制特征点的时候不能进行Clear清屏操作
,否则会看不摄像头所捕获的内容
大眼片元着色器算法:
//圓內放大vec2 enlargeEye(vec2 textureCoord, vec2 originPosition, float radius, float delta) {float weight = distance(vec2(textureCoord.x, textureCoord.y / aspectRatio), vec2(originPosition.x, originPosition.y / aspectRatio)) / radius;weight = 1.0 - (1.0 - weight * weight) * delta;weight = clamp(weight,0.0,1.0);textureCoord = originPosition + (textureCoord - originPosition) * weight;return textureCoord;
}vec2 bigEye(vec2 currentCoordinate) {vec2 faceIndexs[2];faceIndexs[0] = vec2(74., 72.);//如下图中,以74为圆心,74到72作为半径RfaceIndexs[1] = vec2(77., 75.);for(int i = 0; i < 2; I++){int originIndex = int(faceIndexs[i].x);int targetIndex = int(faceIndexs[i].y);vec2 originPoint = vec2(facePoints[originIndex * 2], facePoints[originIndex * 2 + 1]);vec2 targetPoint = vec2(facePoints[targetIndex * 2], facePoints[targetIndex * 2 + 1]);float radius = distance(vec2(targetPoint.x, targetPoint.y / aspectRatio), vec2(originPoint.x, originPoint.y / aspectRatio));radius = radius * 5.;currentCoordinate = enlargeEye(currentCoordinate, originPoint, radius, bigEyeDelta);}return currentCoordinate;
}
textureCoord
表示当前要修改的坐标,originPosition
表示圆心坐标,radius
表示圆的半径,delta
用来控制变形强度。 和瘦脸的算法类似,根据originPosition
和targetPosition
确定一个圆,圆内的坐标会参与计算,圆外的不变。 圆内的坐标围绕圆心originPosition
在变化,最终的坐标完全是由weight
的值决定,weight
越大,最终的坐标变化越小,当weight
为1,即坐标处于圆边界或圆外时,最终的坐标不变;当weight
小于1时,最终的坐标会落在原坐标和圆点之间,也就是说最终返回的像素点比原像素点距离圆点更近,这样就产生了以圆点为中心的放大效果。
如下图中,以74为圆心,74到72作为半径R
1.png
瘦脸片元着色器算法:
vec2 curveWarp(vec2 textureCoord, vec2 originPosition, vec2 targetPosition, float delta) {vec2 offset = vec2(0.0);vec2 result = vec2(0.0);vec2 direction = (targetPosition - originPosition) ;float radius = distance(vec2(targetPosition.x, targetPosition.y / aspectRatio), vec2(originPosition.x, originPosition.y / aspectRatio));float ratio = distance(vec2(textureCoord.x, textureCoord.y / aspectRatio), vec2(originPosition.x, originPosition.y / aspectRatio)) / radius;ratio = 1.0 - ratio;ratio = clamp(ratio, 0.0, 1.0);offset = direction * ratio * delta;result = textureCoord - offset;return result;
}//指定9对 圆心坐标和目标坐标,如下图
vec2 thinFace(vec2 currentCoordinate) {vec2 faceIndexs[9];faceIndexs[0] = vec2(3., 44.);faceIndexs[1] = vec2(29., 44.);faceIndexs[2] = vec2(7., 45.);faceIndexs[3] = vec2(25., 45.);faceIndexs[4] = vec2(10., 46.);faceIndexs[5] = vec2(22., 46.);faceIndexs[6] = vec2(14., 49.);faceIndexs[7] = vec2(18., 49.);faceIndexs[8] = vec2(16., 49.);for(int i = 0; i < 9; I++){int originIndex = int(faceIndexs[i].x);int targetIndex = int(faceIndexs[i].y);vec2 originPoint = vec2(facePoints[originIndex * 2], facePoints[originIndex * 2 + 1]);vec2 targetPoint = vec2(facePoints[targetIndex * 2], facePoints[targetIndex * 2 + 1]);currentCoordinate = curveWarp(currentCoordinate, originPoint, targetPoint, thinFaceDelta);}return currentCoordinate;
}
textureCoord
表示当前要修改的坐标,originPosition
表示圆心坐标,targetPosition
表示目标坐标,delta
用来控制变形强度。
上述shader方法可以这样理解,首先确定一个以originPosition
为圆心、targetPosition
和 originPosition
之间的距离为半径的圆,然后将圆内的像素朝着同一个方向移动一个偏移值,且偏移值在距离圆心越近时越大,最终将变换后的坐标返回。
如果将方法简化为这样的表达式变换后的坐标 = 原坐标 - (目标坐标 - 圆心坐标) * 变形强度
,也就是说,方法的作用就是要在原坐标的基础上减去一个偏移值,而(targetPosition - originPosition)
决定了移动的方向和最大值。
- 指定9对 圆心坐标和目标坐标,如下图
2.png
刚开始想的是实现像开头动图那样的效果,但是在实现的时候遇到了一些问题。刚开始的想法是这样的,如下图
3.png
后面想到在实现多滤镜的时候,上一个片元着色器的输出,作为下一个片元着色器的输入, 如下图所示:
流程图
具体详情请查看源码。
本文Demo:码云、Github
更多推荐
OpenGL 实现大眼和瘦脸
发布评论