Despite technological advances, factory injuries and fatalities remain a persistent concern across many industries worldwide. The aim of this project was to develop an off-the-shelf, lightweight upstream module that could serve as a foundation for future safety systems, such as human–machine proximity analysis, hazardous-zone entry detection, and near-miss identification. All of these applications rely on accurately determining a worker’s location within each frame.

In addition, many factories require operational insights, such as where workers are positioned on the floor and how long they spend on breaks—information that our model is designed to provide. A further objective was to ensure the system is truly off-the-shelf and lightweight: it requires minimal fine-tuning and can run effectively on relatively outdated hardware, which is still common in many industrial environments.

Methodology (we wrote 2 versions of this, one is less technical than the other)

V1) We built a system that takes CCTV video and automatically finds people in the footage. First, we pull out individual frames from the videos and “clean” them so they’re all in the same format and size. To make the system work better in real conditions (like dark areas, glare, blur, or low-quality camera footage), we also create extra versions of the same images by making small changes—like adjusting brightness, adding a bit of noise, or simulating motion blur (dataset augmentation). Then we label the training images by drawing tight boxes around every person, so the model can learn what a person looks like and where they appear in the frame.

The model works like this: one part scans the image and learns useful visual patterns (Backbone), and a second part decides, for different regions of the image, whether a person is there and exactly where the box should go (Detection Head). During training, the model repeatedly compares its guesses to the correct answers and updates itself to make fewer mistakes (Loss function). We also upgraded the training method so the model is better at placing boxes accurately, not just roughly detecting that someone is present.

Finally, we tested many training settings to find what worked best, trained the best version for longer, and used safeguards to avoid “overfitting” (doing well only on training images but poorly on new footage). We judged performance using common detection measures that reflect how often it finds people correctly, how often it misses them, and how well the boxes match the true locations on new, unseen video frames.

V2) Our method turns raw CCTV video into reliable “person detected” results through a structured pipeline. We first extract frames from surveillance videos, standardize them (format conversion, resizing, normalization), and apply data augmentation to help the model handle real-world variation like lighting changes, blur, noise, and compression. Each person in an image is labeled with a tight bounding box, and those labels are converted into a grid-based target format so the model learns not only if a person is present in a grid cell, but also where they are and how big the box should be.

The model has two parts: a convolutional “backbone” that learns visual features from the image (from simple edges to higher-level human shapes), and a “detection head” that outputs five values per grid cell: four numbers describing the bounding box and one probability that a person is present. During training, the model’s predictions are compared to the targets cell-by-cell using a combined loss that teaches both accurate localization and correct detection. We improved bounding-box accuracy by switching from a basic coordinate loss to Complete IoU (CIoU), which better accounts for overlap, alignment, and shape, leading to stronger precision, recall, and average precision.

Finally, we tuned hyperparameters using a two-stage process: a small grid search to shortlist good settings, followed by longer training runs with the best candidates. Training used AdamW optimization, learning-rate reduction when progress stalled, and early stopping to reduce overfitting. Performance was reported with standard detection metrics such as precision, recall, F1, IoU, and Average Precision on held-out validation and test sets.

Results

Our results show that MLHD (our custom model) beats the YOLOv5s baseline on real factory CCTV footage. YOLOv5 is a widely used, fast, general-purpose object detector that finds objects (like people) by drawing bounding boxes around them in an image. It often works well “out of the box,” but it can underperform in a specific environment—like factory cameras—if it isn’t trained or fine-tuned on that kind of footage.

We trained the best MLHD version on a heavily varied (augmented) dataset and used early stopping so it wouldn’t just memorize the training videos. Training took about 3 hours 43 minutes on a Mac M4 Pro. Under the standard detection setting     (Intersection over Union (IoU) = 0.50, meaning “reasonable overlap” between predicted and true boxes), MLHD performed strongly: Precision = 0.965 (few false alarms), Recall = 0.916 (few misses), F1 = 0.940 (overall balance), and AP = 0.884 (overall detection quality). When we required extremely tight, almost perfect box alignment (IoU = 0.85), performance dropped, which is expected and usually not necessary for real surveillance tasks that mainly need to flag that a person is present in a zone.

Overall, the takeaway is simple: a smaller model designed specifically for factory CCTV can outperform a general-purpose detector that hasn’t been adapted to the same environment.

Conclusions

This work shows that human detection in factory CCTV can be more reliable when the model is designed specifically for industrial environments, instead of relying on large general-purpose detectors. MLHD achieves strong performance at IoU 0.50, with high precision and recall, while using fewer parameters than YOLOv5s (A state of the art detection model used as a reference point in this project). This supports the idea that domain-specific design choices can outperform pretrained models when the footage looks different from typical benchmark datasets. The drop in performance at stricter IoU thresholds reflects a deliberate tradeoff: MLHD focuses on reliably detecting that a person is present rather than drawing perfectly tight boxes. For most factory safety use cases, that tradeoff is acceptable because the main goal is dependable awareness of worker presence. As a result, MLHD can serve as a fast “first step” in a larger safety system, enabling features like restricted-zone alerts, worker–machine interaction monitoring, and near-miss prediction. Its lightweight design and low latency also make it practical to deploy across many cameras in real factory settings.

Teamwork played a huge role in the success of our project, even though collaborating wasn’t always seamless. Early on, we had to rewrite several parts of the code to make sure our training loops and inference scripts ran reliably on both Windows and Mac. We also needed to standardize variable and function names across the entire codebase. If one person used a different name for the same concept, it could break the program—and tracking down those inconsistencies sometimes took hours.

Beyond the technical work, teamwork mattered just as much in the writing. To produce a clear report, we had to communicate consistently and agree on the intent of every paragraph, so the final message didn’t feel disjointed or require major rewrites later. Overall, these challenges made coordination demanding at times, but they ultimately led to stronger, more reliable code and a more polished, coherent report.

Since our project focuses on identifying people within CCTV footage, it can serve as a foundational component for broader safety systems. Accurately locating individuals in each frame creates opportunities to detect entry into restricted or dangerous zones, identify unusual behaviour patterns, and support both active and proactive monitoring solutions. A key next step in the project is the integration of a temporal component that enables tracking of individuals across consecutive frames. Rather than treating each frame independently, the system would learn temporal continuity - understanding that a person detected in one frame is likely to persist in subsequent frames. This would allow us to progressively expand the project toward more advanced safety applications tailored to specific operational needs.

From a technical perspective, an important improvement involves addressing the current limitation of one person per grid cell. At the moment, the image is divided into regions where each area can only report a single detection, which can reduce accuracy in crowded scenes. To overcome this, we would adapt the detection approach so that each region can propose multiple possible people and then apply filtering methods to keep the most reliable results. We would also refine the system to analyse the image at different levels of detail, allowing it to better distinguish individuals who are close together. These adjustments would improve robustness and scalability while keeping the overall design of the project consistent.