Problem

The client wanted the build the MVP in the form of a user-oriented web platform on top of existing machine learning code. The platform would allow its users to upload datasets, register their microchips, and run scalable end-to-end training experiments in the cloud. Before the start of the project, model training was semi-manual, and there was no end-to-end connection between the platform and the machine learning experiments. The goal of our collaboration was to connect all pieces of the platform, add new features as needed, and integrate authentication and authorization features.

Solution

Brainstorm developers (a team of 4 engineers) worked on the frontend (ReactJS), backend (NodeJS), bridge code between the backend and Ray Cluster (Python), and infrastructure (AWS). Some of the features accomplished so far include:

• Redesign of the platform.
• User authorization through Google OAuth 2.0 and MS Oauth.
• Connecting user experiment submissions to the model training code and vice versa.
• Developing features including dataset analysis and verification, live experiment updates, frontend visualizations, etc.
• Managing AWS services, automation, and security.
• Communicating the platform to users, and technical support.

The entire platform is hosted on AWS, using Amazon Aurora MySQL as the database, EC2 instances for model training using Ray for machine learning experiments, S3 for storage, and DataSync for automated data transfers.

Results

Brainstorm developers have a crucial role in product design, decision-making, and planning. The development is executed through the Scrum agile project management framework using Slack, Jira, and GitHub. The collaboration is ongoing as of writing this text, and the future is looking bright.