OpenCV QR code tracking

We can see QR codes everywhere. They are used for payments, marketing, and whatnot. But, tracking these QR codes and analyzing the data in real-time is a complex task. Python Libraries like OpenCV come in handy for these tasks.

As we know OpenCV stands for Open Source Computer Vision Library. It is a powerful tool that provides a range of functionalities for computer vision tasks, including QR code detection and tracking.

In this blog, we will explain to you how to use OpenCV for QR code tracking.

What is QR Code Tracking?

QR code tracking means the ability to detect and track the movement of a QR code in a video or real-time application. Consider situations where QR codes need to be constantly scanned or continuously monitored for which QR code tracking becomes important.

QR codes enable us to encode information like URLs (Uniform resource locator), product details, contact information, text, etc. Tracking them allows businesses and developers to gather information like how many times it was scanned, where it was scanned, and do a specific action when it is scanned.

How OpenCV Tracks QR Codes

OpenCV QR code tracking consists of three main steps.

  • QR Code Detection: In OpenCV, we use cv2.QRCodeDetector() class for identifying the location of a QR code within an image or video frame.
  • Decoding the QR Code: Once we detect the QR code the information needs to be decoded.
  • Tracking Movement: To track a QR code’s movement across video frames, we can use various tracking algorithms in OpenCV like the Meanshift or KLT (Kanade-Lucas-Tomasi) feature tracker. By associating the QR code’s location in one frame with its location in the next, you can continuously track its position in real-time.

Basic Steps to Implement QR Code Tracking with OpenCV

Now, Let’s develop the code for QR tracking using OpenCV. We will be using a video feed in which we will detect and track the movement of the QR code.

Step 1: Install OpenCV

Install OpenCV using the following command.

pip install opencv-python

Step 2: OpenCV QR Code Detection and Tracking in a Video Stream

import cv2
import numpy as np

# Initialize the webcam
cap = cv2.VideoCapture(0)

# Initialize QR Code detector
qr_decoder = cv2.QRCodeDetector()

while True:
    ret, frame = cap.read()
    if not ret:
        print('not getting frames')
        break

    # Detect and decode QR code
    decoded_text, points, _ = qr_decoder.detectAndDecode(frame)

    if points is not None:
        points = points[0].astype(int)

        # Draw the bounding box around the QR code
        cv2.polylines(frame, [points], isClosed=True, color=(0, 255, 0), thickness=5)

        # Display the decoded text
        font = cv2.FONT_HERSHEY_SIMPLEX
        cv2.putText(frame, decoded_text, tuple(points[0]), font, 1, (0, 0, 255), 2)

    # Show the frame with QR code tracking (only if GUI is supported)
    try:
        cv2.imshow('QR Code Tracking', frame)
    except cv2.error as e:

        print("cv2.imshow not available in this environment:", e)
        break

    # Exit on pressing the 'q' key and release the resources
    if cv2.waitKey(1) & 0xFF == ord('q'):
        cap.release()
        cv2.destroyAllWindows()
        break

Output
QR detection

Explanation of algorithm:

  1. We use cv2.VideoCapture(0) function to capture video from the default webcam.
  2. The cv2.QRCodeDetector() function in OpenCV is used to detect QR codes in each frame.
  3. When a QR code is detected, a green bounding box is drawn around it using cv2.polylines function.
  4. Then decoded information from the QR code is displayed on the screen.
  5. The processed video with QR code detection is displayed using cv2.imshow.
  6. The loop runs continuously until the user presses the “q” key.

Tracking QR Codes across video frames

Step 1: QR code detection

Detect the QR code in the first frame and extract its bounding box. We will use QRCodeDetector from OpenCV to detect QR. After detecting to get the four points of the QR code.

Step 2: Track QR using Meanshift algorithm

We initialize the Meanshift tracking algorithm after detecting the QR code for the first time. The algorithm requires a Region of Interest (ROI) i.e. the bounding box we extracted earlier.

We create a histogram of the HSV color space for the region of interest (ROI), which is used to track the QR code in the subsequent frames.

In each frame, we calculate the backprojection of the histogram on the new frame and apply the Meanshift algorithm to find the best match of the region.

Step 3 : Tracking Update

We update the QR code position in every upcoming frame and draw a rectangle around it to indicate the tracked region.

Step 4: Exit

We can exit the tracking loop by pressing the Esc key.

Code for QR Code Tracking using Meanshift algorithm with comments

import cv2
import numpy as np

# Initialize the QRCodeDetector object
qr_detector = cv2.QRCodeDetector()

# Open the webcam or use a video file
cap = cv2.VideoCapture(0)  # Change 0 to a video file path to use a video instead of webcam

# Variable to store the initial position of the QR code (ROI - Region of Interest)
roi = None
tracking = False  # Flag to indicate whether tracking has started

while True:
    # Read frame from the camera or video file
    ret, frame = cap.read()
    
    if not ret:
        print("Failed to capture image")
        break

    # If QR code is detected in the current frame, initialize tracking
    if not tracking:
        value, points, _ = qr_detector.detectAndDecode(frame)
        if value:
            points = points[0]  # The points form a quadrilateral
            x, y, w, h = cv2.boundingRect(np.array(points))  # Get the bounding box

            # Set up the ROI for tracking
            roi = (x, y, w, h)
            tracking = True

            # Initialize the tracking window (Meanshift)
            track_window = roi
            # Create a mask for the tracking region (helps with stability)
            mask = np.zeros(frame.shape[:2], dtype=np.uint8)  # Mask should be 2D and uint8
            mask[y:y+h, x:x+w] = 255  # Set the region of interest to 255

            # Convert frame to HSV
            hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)

            # Calculate the histogram for the ROI
            roi_hist = cv2.calcHist([hsv], [0], mask, [16], [0, 256])
            roi_hist = cv2.normalize(roi_hist, roi_hist, 0, 255, cv2.NORM_MINMAX)

        else:
            # No QR code detected, keep searching
            continue

    # If tracking is enabled, perform tracking in subsequent frames using Meanshift
    if tracking:
        hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)

        # Calculate the back projection of the current frame
        back_proj = cv2.calcBackProject([hsv], [0], roi_hist, [0, 256], 1)

        # Apply the Meanshift algorithm
        ret, track_window = cv2.meanShift(back_proj, track_window, (cv2.TERM_CRITERIA_EPS | cv2.TERM_CRITERIA_COUNT, 10, 1))

        # Draw the tracking window (rectangle) on the frame
        x, y, w, h = track_window
        cv2.rectangle(frame, (x, y), (x+w, y+h), (0, 255, 0), 3)

        # Optionally, display the current tracking status (value of QR code)
        cv2.putText(frame, "Tracking QR Code", (10, 50), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 255, 0), 2, cv2.LINE_AA)

    # Show the frame with the tracking
    cv2.imshow('QR Code Tracker', frame)

    # Exit on pressing the 'Esc' key
    if cv2.waitKey(1) & 0xFF == 27:  # 27 is the key code for the 'Esc' key
        break

# Release resources and close the window
cap.release()
cv2.destroyAllWindows()

Challenges and Considerations in tracking QR code using OpenCV.

  • Lighting Conditions: If we have low light conditions, it is difficult to detect QR code. We can perform image processing or adjust brightness to overcome the problem.
  • Camera Quality: When we use cameras with low resolution accuracy of QR code detection and tracking is affected. Use of high quality camera is always preferred.
  • Fast Movement: We should use cameras with high frame rate for QR code tracking.

Conclusion

We have understood how to track QR codes using OpenCV in the blog.

You can also read about our other blog on Layer normalization vs Batch normalization and ChatGPT API Python : A Complete Guide.

Subscribe
Subscribe