Architecture - KaidiLyu/InfiniteLoopers GitHub Wiki

Description:

AI Nutritional Search is a mobile application designed to provide users with detailed nutritional information about food items through AI-based search and image recognition. Users can either input food items via text or upload images, and the AI will return comprehensive nutritional data, including calories, proteins, carbohydrates, and other nutrients.

Technologies:

Platform:

  • iOS / Android: The app will be developed for both mobile operating systems using a cross-platform framework for efficient development.

React Native (Framework):

  • React Native was selected as the base framework due to its JavaScript foundation, making it accessible for developers and ensuring compatibility across iOS and Android.

Expo (React Native Framework):

  • Expo facilitates faster and easier development by providing file-based routing and a suite of standard React Native modules.

Google Gemini (LLM):

  • Gemini, a free-to-use language model, will be integrated for text-based searches due to its ease of integration, speed, and the ability to fine-tune it for our specific use case of food nutrition.

Firebase (Backend):

  • Firebase will serve as the backend, providing real-time data synchronization, secure user authentication, and an intuitive database solution, fully compatible with React Native.

Custom Image Recognition Model:

  • We will develop a custom image recognition model using publicly available food image datasets with human-verified labels. This model will detect food items in images and return nutritional data. Research on optimizing cost and accuracy for training the model is ongoing.

Expo Go (Deployment Hosting):

  • Expo Go is under consideration for deployment, allowing internal beta testing among the team. The app will be accessible on personal devices through this platform, with potential migration to other services for wider releases.

Node Package Manager (NPM):

  • NPM will be used for managing project dependencies and libraries, ensuring streamlined package management.

Package Management Tool:

  • We rely on NPM to efficiently manage dependencies, handle project builds, and maintain a clean development environment. This tool simplifies updates, installs, and versioning.

Teamwork:

Logan Munn:

  • Responsible for integrating the language model for food text-based search and ensuring accurate nutritional data retrieval.

Kiran:

  • In charge of developing the custom image recognition model and optimizing it with food image datasets.

yingdongfeng:

  • Backend lead, responsible for implementing Firebase integration and managing the database schema for users and food items.

ADhill01:

  • Frontend development lead, focusing on building the user interface and ensuring smooth interaction with the AI features.

KaidiLyu:

  • Oversees deployment processes and manages the Expo Go setup, ensuring team members can easily access the app during testing phases.

Deployment and Release:

We aim to deploy the app internally via Expo Go, giving the team and stakeholders access to test the app on their personal devices. For the final release, APK (Android) and IPA (iOS) files will be generated, making the app downloadable on both platforms.

Development Environment:

The development environment will vary depending on the developer's preference, with support for IDEs such as Visual Studio Code or WebStorm. Expo will simplify the setup, allowing easy access to tools and simulators for debugging.

URL Structure:

URL Structure:

  • /food-item: Allows users to enter food names and retrieve their nutritional data.

    • Parameters: name (string)
    • Response: Nutritional information (object)

  • /upload-image: Accepts image uploads, identifies food items using the AI model, and returns corresponding nutritional data.

    • Parameters: image (file)
    • Response: Detected food (string), nutritional information (object)

  • /save-food-item: Allows users to save a specific food item to their profile.

    • Parameters: userId (string), foodItemId (string)
    • Response: Success message

  • /get-saved-food-items: Retrieves all the food items saved by the user.

    • Parameters: userId (string)
    • Response: Array of saved food items (array of objects)

  • /delete-saved-food-item: Deletes a specific food item from the user's saved list.

    • Parameters: userId (string), foodItemId (string)
    • Response: Success message

  • /get-nutritional-summary: Retrieves a nutritional summary for a list of food items (e.g., total calories, protein).

    • Parameters: userId (string), foodItems (array of strings)
    • Response: Nutritional summary (object)

Application Models:

FoodItem (Food Item Model):

  • Properties:

    • name (String)
    • nutritionalInfo (Object)

  • Description: This model represents each food item and stores its name along with nutritional details like calories, proteins, and fats.

User (User Model):

  • Properties:

    • username (String)
    • email (String)
    • savedFoodItems (Array of FoodItem)

  • Description: Stores basic user data such as username, email, and saved food items for quick retrieval.

ImageRecognitionResult (Image Recognition Model):

  • Properties:

    • image (File)
    • detectedFood (String)
    • confidenceScore (Number)

  • Description: Holds the image uploaded by the user, the detected food item, and the confidence score returned by the AI model.

State Management:

  • Memory: For temporary data, such as unsaved user actions.
  • Firebase Database: For persistent data like saved food items and user profiles.

Database Schema:

Users Collection:

  • username (String)
  • email (String)
  • savedFoodItems (Array of FoodItem IDs)

FoodItems Collection:

  • name (String)
  • nutritionalInfo (Object)

ImageRecognitionResults Collection:

  • image (File)
  • detectedFood (String)
  • confidenceScore (Number)

Common Queries:

Get the user's saved food items:

Retrieve a user’s list of saved food items and display their nutritional data.

Get food recognition results based on images:

Once an image is uploaded, the AI model analyzes it, returning the recognized food item and confidence score.

Get a nutritional summary:

Fetch the nutritional summary of a user's selected food items, providing total calories, protein, and other macronutrients.

Views:

Login Page:

The user interface for registration and login.

Home Page:

Allows users to enter food names or upload food pictures, and displays the associated nutritional data.

History Page:

Displays previously saved food entries for users to review or delete.

Personal Settings Page:

Users can modify personal information or app preferences.