HawkWatch
AI-Powered Proctoring & Computer Vision Surveillance
01 / Project Overview
A browser-based AI-powered proctoring and real-time student monitoring platform. Integrating light-weight computer vision models directly in the client browser, HawkWatch tracks eye movements, head orientation, multi-face presence, and audio triggers.
02 / The Challenge & Problem
Real-World Problem Statement
Online testing is plagued by academic integrity concerns. Traditional proctoring tools require invasive desktop applications that collect massive system data and suffer from poor cross-platform compatibility.
03 / The Engineering Solution
Implementation & Architectural Approach
Designed a browser-only proctoring solution using MediaPipe and OpenCV. All computer vision inference (such as facial landmark tracking and gaze estimation) is executed locally or streamed via WebSockets.
04 / Technical Architecture Flow
Browser WebAssembly & MediaPipe
Processes student camera feed on local threads, outputting facial mesh coordinates.
Python FastAPI & OpenCV
Runs head pose estimation (solvePnP), identity verification (ArcFace), and deepfake detection algorithms.
React & Socket.IO Dashboard
Feeds incident logs, live statistics, and alert highlights to examiners via real-time WebSockets.
05 / Key Project Features
Local Gaze Tracking
Uses face landmark coordinates to detect if the student shifts sight away from the screen.
Multi-Face Watchdog
Flags warning signals immediately if secondary individuals enter the camera's viewport.
Deepfake Detection Engine
EfficientNet binary classifier analyzing video feeds for synthetic manipulation.
06 / Engineering Challenges & Mitigations
Running computer vision models concurrently with client exam questions caused lag on entry-level computers.
Delegated heavy model execution to Web Workers using OffscreenCanvas, ensuring the main thread stays locked at 60fps.
Students shifting positions slightly were incorrectly flagged with cheating violations.
Created calibration routines before exams, adjusting baseline head coordinates and gaze boundaries.
07 / Technical & Personal Learnings
Acquired key insights in browser-based model quantization, WebAssembly bindings, and off-thread model inference.
Gained deep experience in asynchronous audio-visual stream processing and real-time Socket communication.