Real-Time Object Detection (w/Yolo-Python)

Real-Time Object Detection (w/Yolo-Python)

Hello everyone,

In this article, where we will seek answers to questions such as "What is Computer Vision?",

"How is real-time object tracking done?",

"What is the CNN algorithm?",

I will also try to share my experiences with real-time object detection stages using YOLO and Python.

Computer Vision?

A human can perceive and interpret images and moving objects. Computer vision research began so that a machine could perform the same operations. Thanks to the algorithms developed as a result of this research, computers can learn, recognize, and interpret digital images or video images just like a human. In short, the sub-discipline that enables computers to understand what they see is called "Computer Vision".

Before moving on to the real-time object tracking stages, I should mention some general terms that will be used.

What is an Image?

An image is a visual representation of something. From an image processing perspective, it is a picture that is electronically created, copied, processed, and stored.

Images are made up of pixels. As the number of pixels increases, the quality of the image also increases.

A digital representation of an image (illustration)
Computers are deterministic machines. Therefore, for computers, an image is simply a matrix.

An image is a visual representation of something. In terms of image processing, it is a picture that is electronically created, copied, processed, and stored.

What is Video?

It is the creation of a visual image that appears to be moving, formed by a series of images arranged in sequence. The FPS (Frames per second) value is very important for videos.

Furthermore, the compression of the image during transmission provides great convenience in today's technologies. For example:

1 Pixel = 8 bits * 3 (RGB) => 24 bits

• A 1024x1024 pixel image occupies approximately 24 Mbit of space (1K * 1K * 24).
  
  
• If we were transmitting this image without compression/encoding, we would need 24 Mbit of bandwidth for 1 frame per second (1 FPS).
  
  
• This is an unacceptable FPS value for moving images.

Now that we have learned about images and videos, we can move on to the algorithmic side of object detection.

CNN (Convolitional Neural Network)

• The CNN (ConvNet) algorithm is an algorithm used to detect objects in images and videos.
  
  
• CNN algorithms use unique features of objects in an image to distinguish (learn) from one another.

The CNN algorithm could be discussed in a separate article. Since our topic isn't solely CNN, let's talk about some basic information you need to know about the algorithm.

Structure:

Convolution Layer: Detects unique features on the image/frame.

• Non-Linearity Layer: Normalizes the image matrix by passing it through an activation function. The chosen activation function affects the speed of the neural network training that will take place later.
  
  
• Pooling Layer: Reduces the number of weights in the matrix. Some users increase the size of the filter matrix used in the convolution layer instead of using the pooling layer.

Max Pooling

• Fully-Connected Layer: The events up to this layer represent the stage where the image is prepared for the artificial neural network. This layer is where the system is trained, meaning the artificial neural network operates. It can be said that the output of the CNN algorithm is this layer.

Now we can move on to YOLO, the main character of our article 🙂

What is YOLO?

YOLO is an algorithm that uses convolutional neural networks to detect objects. Its full name is "You Only Look Once".

• The reason is that the algorithm can detect objects very quickly and in a single pass.
  
  
• The reason the YOLO algorithm is faster than other algorithms is that it passes the entire image through a neural network in one go.
  
  
• The YOLO algorithm surrounds the objects it detects in images with a bounding box.
  
  
• YOLO divides the image given as input into NxN grids. These grids can be 5x5, 9x9, 17x17, etc.
  
  
• Each grid checks whether there is an object within it, and if it thinks there is an object, it considers whether the object's center point is within its area.
  
  
• The grid that decides that the object has a center point must find the object's class, height, and width and draw a bounding box around that object.

YOLO Grid System (Representative)

• Multiple grids may assume an object is contained within them. This creates unnecessary bounding boxes on the screen.
• All bounding boxes have a confidence score.
  
  
• To prevent this, the Non-Maximum Suppression algorithm is used.
  
  
• In short, the Non-Maximum Suppression algorithm draws the bounding box with the highest confidence score on the screen for objects detected on the image.

Non-Max Suppression

The following graphs show the object detection performance of YOLO and some other algorithms for the MS COCO dataset.

https://arxiv.org/abs/2004.10934

As seen in the graphs, if we consider a case where the number of classifiers is equal, YOLOv4 is almost three times ahead of its competitors.

Now let's briefly discuss the object recognition stages with YOLO. As an example, let's examine a project that detects masked and unmasked faces.

1) Data Collection/Labeling

You can perform data labeling via https://www.makesense.ai/ Labeling is important so that we can provide the algorithm with training data that allows it to distinguish between masked and unmasked people and train itself. Besides Make Sense, you can find many other platforms for image labeling. I prefer Make Sense because it provides the labeled data in the format required by YOLO. Below, we see example images for the labeling stage.

For illustrative purposes, I labeled two images and when I exported the result in YOLO format, I had two different matrices for the two images, as shown below.

After tagging the images, I trained my own YOLO model via Google Colab through the DarkNet. I'm sharing some screenshots of the training stages with you.

• To run the YOLO algorithm, the Darknet requires certain files from us. One of these is the .cfg file.
  
  
• The config file requests properties that will affect the success, speed, etc., of the neural network within YOLO.
  
  
• Another file that YOLO requires is the .names file. Names holds the names of the classes. There are two classes for mask detection: Mask and No-Mask.
  
  
• It also requests a .data file that specifies the location of all files within the DarkNet folder.

A screenshot taken during training with Google Colab

The success of training can be understood from the loss graph generated after training. If we train too much data for our YOLO model and increase the number of iterations of our artificial neural network too much,

the system may experience Overtraining. This is a situation we don't want to happen in our model.

We assume that we have completed the training phase and have our .weights file. Now our model is ready.

Dealing with data labeling, training, etc., can be tedious. Alternatively, you can use a model trained by someone else. Since the mask is a simple example with 2 classes, we might be able to use a model we don't know, but for special cases and a unique topic, using a model whose training method we don't know might not be a good choice.

Up to this point, I assume that we all have our weights and config files for Mask Recognition.

You can access the program that allows me to perform mask recognition in real-time with a local camera below.

Github: https://github.com/alperenyildiz/mask-detection

import cv2

import numpy as np

 

 

whT = 320

cap = cv2.VideoCapture(0)

confThreshold=0.5

nmsThreshold=0.3

 

classFile="maske.names"

classNames=[]

 

with open(classFile,"rt") as f:

classNames=f.read().rstrip("\n").split("\n")

modelConfiguration="maske.cfg"

modelWeights="maske_final.weights"

model=cv2.dnn.readNetFromDarknet(modelConfiguration,modelWeights)

 

model.setPreferableBackend(cv2.dnn.DNN_TARGET_CPU)

model.setPreferableTarget(cv2.dnn.DNN_TARGET_CPU)

 

def findObject(detectionLayers,img):

hT,wT,cT=img.shape

bbox=[] # List holding our delimiter boxes

classIds=[] # List of IDs holding our classes

confs=[] # Our list holding the confidence value of the found objects

for detectionLayer in detectionLayers:

for objectDetection in detectionLayer:

scores=objectDetection[5:]

classId=np.argmax(scores)

confidence=scores[classId]

if confidence>confThreshold:

w,h=int(objectDetection[2]*wT),int(objectDetection[3]*hT)

x,y=int((objectDetection[0]*wT)-w/2),int((objectDetection[1]*hT)-h/2)

bbox.append([x,y,w,h])

classIds.append(classId)

confs.append(float(confidence))

 

 

#Non maximum suppression ;

indices=cv2.dnn.NMSBoxes(bbox,confs,confThreshold,nmsThreshold)

 

for i in indices:

i=i[0]

box=bbox[i]

x,y,w,h=box[0],box[1],box[2],box[3]

 

if classNames[classIds[i]].upper() == "NO-MASK":

g,b,r = 0.0.255

else:

g,b,r = 0.255.0

 

cv2.rectangle(img,(x,y),(x+w,y+w),(g,b,r),3)

cv2.putText(img, f'{classNames[classIds[i]].upper()} {int(confs[i] * 100)}%',

(x, y - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.6, (g,b,r), 2)

 

while True:

success,img=cap.read()

blob=cv2.dnn.blobFromImage(img,1/255,(whT,whT),[0,0,0],1,crop=False) #[img,scale factor,size,]

model.setInput(blob)

 

layerNames=model.getLayerNames()

outputLayers=[layerNames[i[0]-1] for i in model.getUnconnectedOutLayers()]

detectionLayers=model.forward(outputLayers)

findObject(detectionLayers,img)

#cv2.namedWindow("Mask Detection", cv2.WINDOW_NORMAL)

#cv2.flip(img,1)

cv2.imshow("Mask Detection",img)

k = cv2.waitKey(20) & 0xFF

if k == 27:

break

 

cv2.waitKey(50)