Problem

The client faced a difficult problem of optimizing computer vision models running on battery-powered hardware with low memory and processing power. To train computer vision that can run on such hardware, the model’s input needs to be small (below 100x100px), the model can’t have many layers, while inference speed and accuracy shouldn’t suffer greatly.

Solution

Two Brainstorm Computer Vision and AI developers worked on algorithm development for this project. They were engaged in helping with model training, and optimization, as well as with dataset analysis and augmentation to help the model learn from edge cases. The experiments were conducted in Python using the TensorFlow framework. The experiments were conducted on AWS EC2 instances that were managed by the Ray cloud computer framework. Dataset augmentation was performed by training a PyTorch segmentation model based on the PointRend architecture. The model was used to generate realistic new sample images with overlapping objects, considering the trained models often failed when two persons were overlapping.
Other tasks included:

• Dataset conversion and validation
• Model training with scalable experiments using Ray Tune
• Researching and experimenting with different hyperparameter optimization methods

Results

Discoveries and development made by the Brainstorm team played a great role in improving the quality of smart cameras. Clients were happier with how the models performed in crowded areas, and the number of orders increased. The collaboration was a success and we continued working together in a different direction.