introductory
With the arrival of Christmas, everyone has @official weibo to add a Christmas hat to their avatar. Of course this kind of thing can be done with a lot of P picture software. But as a technical person who learns image processing, still feel we need to write a program to do this thing. And this can be completely as a practice small project, the workload is not big, and it is very interesting.
Tools used
OpenCV (after all, that's what we're mostly about...)
dlib (as just mentioned in a previous post, dlib's face detection works better than OpenCV, and dlib has keypoint detection that OpenCV doesn't have.)
The language used is Python, but it is possible to change it to C++, so I won't write about it because of time constraints. If you are interested, you can use it to practice.
Process one,material preparation
First of all, we need to prepare a Christmas hat material, the format is best for the PNG, because PNG, we can directly use the Alpha channel as a mask. We use the Christmas hat as shown below:
We can get the alpha channel of the Christmas hat image by channel separation. The code is as follows:
r,g,b,a = (hat_img)
rgb_hat = ((r,g,b))
("hat_alpha.jpg",a)
In order to be able to operate with the rgb channel header image, we combine the rgb three channels into one rgb colored hat image. the Alpha channel image is shown below.
II. Face Detection and Face Keypoint Detection
We use the image below as our test image.
In the following, we use dlib's positive face detector for face detection and extract the five key points of the face using the model provided by dlib. The code is as follows:
# dlib face keypoint detector
predictor_path = "shape_predictor_5_face_landmarks.dat"
predictor = dlib.shape_predictor(predictor_path)
# dlib face detector
detector = dlib.get_frontal_face_detector()
# Face detection
dets = detector(img, 1)
# If a face is detected
if len(dets)>0:
for d in dets:
x,y,w,h = (),(), ()-(), ()-()
# x,y,w,h = faceRect
(img,(x,y),(x+w,y+h),(255,0,0),2,8,0)
# Critical point detection, 5 critical points
shape = predictor(img, d)
for point in ():
(img,(,),3,color=(0,255,0))
("image",img)
()
The effect of this part is shown below:
III. Adjusting the size of the hat
We select the points at the corners of the two eyes and seek the center as the reference coordinate in the x-direction for placing the hat, and the coordinate in the y-direction is represented by the y-coordinate on the upper line of the face frame. Then we adjust the size of the hat according to the size of the face obtained from the face detection to make the hat the right size.
# Pick the point at the corner of the left and right eye
point1 = (0)
point2 = (2)
# Find the center of two points
eyes_center = ((+)//2,(+)//2)
# (img,eyes_center,3,color=(0,255,0))
# ("image",img)
# ()
# Adjust hat size to face size
factor = 1.5
resized_hat_h = int(round(rgb_hat.shape[0]*w/rgb_hat.shape[1]*factor))
resized_hat_w = int(round(rgb_hat.shape[1]*w/rgb_hat.shape[1]*factor))
if resized_hat_h > y:
resized_hat_h = y-1
# Adjust hat size to face size
resized_hat = (rgb_hat,(resized_hat_w,resized_hat_h))
IV. Extracting hats and areas where hats need to be added
Go to the Alpha channel as mask as described previously. and invert it. One of these two masks is used to take the hat area out of the hat map, and one is used to empty out the area of the character map that needs to be filled with hats. You will see that later.
# Use the alpha channel as a mask mask = (a,(resized_hat_w,resized_hat_h)) mask_inv = cv2.bitwise_not(mask)
Take the area from the original image where the hat needs to be added, here we are using bitwise operations.
# Offset of the hat relative to the upper line of the face frame
dh = 0
dw = 0
# Original ROI
# bg_roi = img[y+dh-resized_hat_h:y+dh, x+dw:x+dw+resized_hat_w]
bg_roi = img[y+dh-resized_hat_h:y+dh,(eyes_center[0]-resized_hat_w//3):(eyes_center[0]+resized_hat_w//3*2)]
# Extract the area where the hat was placed from the original ROI.
bg_roi = bg_roi.astype(float)
mask_inv = ((mask_inv,mask_inv,mask_inv))
alpha = mask_inv.astype(float)/255
# Ensure that they are the same size before multiplying (may not be consistent due to rounding)
alpha = (alpha,(bg_roi.shape[1],bg_roi.shape[0]))
# print("alpha size: ",)
# print("bg_roi size: ",bg_roi.shape)
bg = (alpha, bg_roi)
bg = ('uint8')
This is the background area (bg) shown below. You can see that just the area that needs to be filled in for the hat is missing.
Then we extract the hat area.
# Extract the hat area hat = cv2.bitwise_and(resized_hat,resized_hat,mask = mask)
The hat region obtained from the extraction is shown below. The hat region is exactly complementary to the previous background region.
V. Adding Christmas hats
Finally we add the two areas together. And then put back to the original picture, you can get the Christmas hat we want. Here you need to pay attention to is that, before adding resize to ensure that the size of the two consistent, because it may be due to rounding reasons are not consistent.
# Ensure that the two are the same size before adding (may not be consistent due to rounding)
hat = (hat,(bg_roi.shape[1],bg_roi.shape[0]))
# Two ROI areas added together
add_hat = (bg,hat)
# ("add_hat",add_hat)
# Put the area where the hat was added back into the original image
img[y+dh-resized_hat_h:y+dh,(eyes_center[0]-resized_hat_w//3):(eyes_center[0]+resized_hat_w//3*2)] = add_hat
Finally we get the effect image shown below.
Download:Full Code。
This is the whole content of this article.