SteelVision: Pixel-Level Steel Defect Segmentation

Problem

Image-level pass/fail does not tell an inspector where a defect is. Small, low-contrast marks on textured steel are easy for a classifier to ignore and hard to show in a UI.

Solution

A segmentation pipeline with mask training, IoU/Dice evaluation, FastAPI inference, and a browser overlay so quality teams can inspect predicted defect regions.

System Architecture

Primary data and decision flow

1. Images and masks Aligned training samples preserve small defect regions during augmentation.
2. Training pipeline Augmentation and train/validation splits account for small foreground regions.
3. UNet++ model UNet++ with an EfficientNet backbone predicts a mask, not only a class.
4. Model selection IoU and Dice guide checkpoint selection under class imbalance.
5. FastAPI inference The service loads the checkpoint and returns a predicted defect mask.
6. Inspection overlay The browser places the mask over the uploaded steel image for review.

1. Aligned masks, augmentation, and train/validation splits for small defect regions.
2. UNet++ with EfficientNet backbone, segmentation losses, and checkpoint selection on IoU/Dice.
3. FastAPI accepts an image, runs no-grad inference, and returns a mask for overlay rendering.
4. Docker separates model artifacts from the service layer for a repeatable run.

Role and Ownership

Sole computer vision and product engineering owner. Dmitri designed the mask workflow, trained and evaluated the model, built the inference API and inspection interface, and published the deployable code.

Technical Decisions

• Chose semantic segmentation over image classification because inspectors need defect location, not only a pass or fail label.
• Selected checkpoints with IoU and Dice metrics that reflect mask quality under severe foreground and background imbalance.
• Used a UNet++ decoder with an EfficientNet backbone to combine multi-scale localization with efficient feature extraction.
• Separated model artifacts from the service layer so inference can be packaged and updated independently.

Technologies

• PyTorch
• UNet++
• EfficientNet
• FastAPI
• OpenCV
• Docker

Result and Impact

A public segmentation demo and GitHub repo that show pixel-level defect localization with a simple upload-and-overlay workflow.

Limitations

• Trained for steel surface textures in the project dataset, not every alloy, coating, or camera setup.
• Class imbalance (tiny defect vs background) can still suppress faint marks.
• This is a technical demo for inspection experiments, not a certified factory QA gate.

Live demo and source

Live demo: https://ai-vision.dmitriderevjanko.com

GitHub: https://github.com/DmitriDerevjanko/ai-vision

Portfolio case study: https://dmitriderevjanko.com/projects/steelvision/

Other independent projects by Dmitri Derevjanko

• Law Agent
• Marketplace Intelligence
• OSINT Radar
• OncoView AI
• Global Risk Forecast
• SmartCam
• FinanceLens Analyst
• Estonian Company Intelligence Hub
• Opportunity Intelligence Studio