• TensorFlow.js with React-Native
• Machine Learning Pose Analysis Application
• written by HyeonMin Shin

1. Purpose

• Building TensorFlow machine learning models in JSON format for an application created using React-Native.

2. Limitations

• Data received from users needs to be converted into a format that can be processed by the TensorFlow ML model.
• Predicted values generated by the model need to be converted into a data format supported by React-Native.
• Large amounts of computational cost are incurred when processing data from users with large data sizes for model training and prediction data extraction.
• Due to the above reasons, the time required to generate predicted values and the available range of the model varies depending on the user's device specifications.

3. User Perspective

• When practicing exercises or specific movements, users can intuitively observe the difference between the desired and actual movements.
• Real-time analysis of moving objects is possible through various characteristics, such as the position of specific joints, relative angles of segmented body parts, and ratios.
• When describing or analyzing specific objects other than the human body, a single metric can be presented by analyzing and classifying the movement of the object.
• Unlike traditional analysis, which requires knowledge of the specific characteristics of the object and the model, the user can simply select the desired analysis model and prepare only the required analysis data.

4. Developer Perspective

1. Various characteristics of the object to be analyzed and predicted can be extracted from the analyzed model.
2. Fitting the machine learning model to the user's body type requires server-based model training without incurring additional costs. If fitting is required for the user-specific model already stored, training can be performed on the user's device to avoid generating computational costs.
3. The expected advantages from above include:
   1. Drastic reduction in server costs
   2. Only the required data can be selectively extracted by comparing the user-trained model and the existing model, without collecting all the data required for training.
   3. Additional resources for security are not required since customer data is not handled directly on the server.
   4. After the final tuning, the ML model possessed by the user is trained to match the user's characteristics, resulting in cost savings in maintenance and support for common models that fit all users.
   5. If the pose model is successfully compatible with React Native, additional models built using Tensorflow.js can also be operated on the React-Native platform, significantly increasing scalability.
   6. However, the biggest problem with the project is the limitation based on device performance. The performance of the device has a significant impact on model training and predicted value extraction, which can cause problems with app usage on older Android and iOS devices.