In the course of watching the image processing of Mr. Tang Yudi, which has a small project of the answer card recognition, here combined with their own understanding to do a brief summary.
1. Project analysis
First of all, when you get the project, analyze what is the purpose of the project, what kind of goal you want to achieve, what are the things you need to pay attention to, and at the same time conceptualize the general flow of the experiment.
Figure 1. Answer key test image
For example, in the project of answer card recognition, for the test image in Figure 1, the first function that should be realized is.
Ability to capture each fill-in option in the answer key.
Calculate the percentage of correct answers by comparing the obtained filled-in options with the correct options.
2. Project experiments
In the morphological operations performed on the test image, it is first converted to gray scale image followed by Gaussian filtering operation for noise reduction.
gray = (image, cv2.COLOR_BGR2GRAY)
blurred = (gray, (5, 5), 0)
cv_show('blurred',blurred)
After obtaining the Gaussian filtering result, it is subjected to edge detection as well as contour detection for extracting the boundaries of all the contents of the answer key.
edged = (blurred, 75, 200)
cv_show('edged',edged)
# Profile Detection
cnts, hierarchy = ((), cv2.RETR_EXTERNAL,
cv2.CHAIN_APPROX_SIMPLE)
(contours_img,cnts,-1,(0,0,255),3)
cv_show('contours_img',contours_img)
docCnt = None
Figure 2. Gaussian filter plot
Figure 3. Edge detection map
After obtaining the edge detection image, the outer contour detection is performed as well as the perspective transformation is performed.
# Profile Detection
cnts, hierarchy = ((), cv2.RETR_EXTERNAL,
cv2.CHAIN_APPROX_SIMPLE)
(contours_img,cnts,-1,(0,0,255),3)
cv_show('contours_img',contours_img)
def four_point_transform(image, pts): # Get input coordinate points rect = order_points(pts) (tl, tr, br, bl) = rect # Calculate the input w and h values widthA = (((br[0] - bl[0]) ** 2) + ((br[1] - bl[1]) ** 2)) widthB = (((tr[0] - tl[0]) ** 2) + ((tr[1] - tl[1]) ** 2)) maxWidth = max(int(widthA), int(widthB)) heightA = (((tr[0] - br[0]) ** 2) + ((tr[1] - br[1]) ** 2)) heightB = (((tl[0] - bl[0]) ** 2) + ((tl[1] - bl[1]) ** 2)) maxHeight = max(int(heightA), int(heightB)) # Transformed coordinate positions dst = ([ [0, 0], [maxWidth - 1, 0], [maxWidth - 1, maxHeight - 1], [0, maxHeight - 1]], dtype = "float32") # Calculate the transformation matrix M = (rect, dst) warped = (image, M, (maxWidth, maxHeight)) # Return the transformed result return warped
# Perform perspective transformations
warped = four_point_transform(gray, (4, 2))
cv_show('warped',warped)
After the perspective transformation, it is necessary to carry out binary transformation again, in order to find the ROI circle contour, the secondary contour detection is used to execute the traversal loop as well as the if judgment to find all the circle contours that meet the screening conditions. The reason for not using the Hough transform here is that in the process of filling in the answer card, there will inevitably be filled in more than the circle area, the use of the Hough transform of the straight line detection will affect the accuracy of the experimental results.
# Find every circle outline
cnts, hierarchy = ((), cv2.RETR_EXTERNAL,
cv2.CHAIN_APPROX_SIMPLE)
(thresh_Contours,cnts,-1,(0,0,255),3)
cv_show('thresh_Contours',thresh_Contours)
questionCnts = []
# Traverse for c in cnts: # Calculate ratios and sizes (x, y, w, h) = (c) ar = w / float(h) # Criteria for designation based on the actual situation if w >= 20 and h >= 20 and ar >= 0.9 and ar <= 1.1: (c) # Sorted from top to bottom questionCnts = sort_contours(questionCnts, method="top-to-bottom")[0] correct = 0
Figure 4. Profile detection map
Figure 5. Perspective transformation diagram
Figure 6.
Figure 7: Contour Screening Chart
After obtaining each circle outline, it needs to be sorted in a way that is from left to right and from top to bottom, for example, in Fig. 7, the answer key is distributed in five rows and five columns, and in each column, the value of the horizontal coordinate x of the option A in each row is similar, while in each row, the vertical coordinates y of A, B, C, D, and E are similar, so this property is utilized to sort the obtained circle outlines, and the code is as follows:
def sort_contours(cnts, method="left-to-right"):
reverse = False
i = 0
if method == "right-to-left" or method == "bottom-to-top":
reverse = True
if method == "top-to-bottom" or method == "bottom-to-top":
i = 1
boundingBoxes = [(c) for c in cnts]
(cnts, boundingBoxes) = zip(*sorted(zip(cnts, boundingBoxes),
key=lambda b: b[1][i], reverse=reverse))
return cnts, boundingBoxes
After obtaining each specific outline, it is to determine whether the answer filled in for each question is the correct answer, using the method of traversing each specific circle outline through a two-layer loop, and calculating the number of non-zero points through the mask image to determine whether the answer is correct or not.
# 5 options per row
for (q, i) in enumerate((0, len(questionCnts), 5)):
# Sort
cnts = sort_contours(questionCnts[i:i + 5])[0] # Arrange them from left to right, keeping the order: A B C D E
bubbled = None
# Iterate over each result
for (j, c) in enumerate(cnts):
# Use mask to determine results
mask = (, dtype="uint8")
(mask, [c], -1, 255, -1) #-1 indicates fill
cv_show('mask',mask)
# Calculate whether to choose this answer by counting the number of non-zero points
mask = cv2.bitwise_and(thresh, thresh, mask=mask)
total = (mask)
# Judged by thresholds
if bubbled is None or total > bubbled[0]:
bubbled = (total, j)
# Versus the right answer
color = (0, 0, 255)
k = ANSWER_KEY[q]
# The judgment is right
if k == bubbled[1]:
color = (0, 255, 0)
correct += 1
# Mapping
(warped, [cnts[k]], -1, color, 3)
Figure 8. Circle contour traversal map
3. Project results
At the completion of the experiment, output the results of the experiment
score = (correct / 5.0) * 100
print("[INFO] score: {:.2f}%".format(score))
(warped, "{:.2f}%".format(score), (10, 30),
cv2.FONT_HERSHEY_SIMPLEX, 0.9, (0, 0, 255), 2)
("Exam", warped)
(0)
Connected to pydev debugger (build 201.6668.115) [INFO] score: 100.00% Process finished with exit code 0
Figure 9. Answer key recognition results
summarize
In dealing with the small item of answer key identification, I personally feel that the focus is on the following areas:
- Morphological manipulation of images, each step of processing should be pre-thought and the most appropriate processing should be selected, e.g., secondary contour detection was used without Hough transform.
- Use the mask image to compare whether the answer is correct or incorrect, and make a choice by determining the number of non-zero pixel values.
- Make clever use of double for loops and if statements to traverse all the circle outlines, sort them, and then compare the answers.
The above is based on Opencv image recognition to achieve the answer card recognition example details, more information about Opencv answer card recognition please pay attention to my other related articles!