AI-based Food Recognition and Nutrient Prediction

International Journal of Emerging Research in Science, Engineering, and Management Vol. 1, Issue 6, pp.44-54, December 2025. www.ijersem.com eISSN - 3107-9075 IJERSEM@2025 https://doi.org/10.58482/ijersem.v1i6.6 44 AI-based Food Recognition and Nutrient Prediction 1A. Surekha, 2P Aswini, 3M Shavan Kumar, 4V Manisha, 5V Nagendra Reddy, 6B Nithin Kumar Reddy Department of CSE, Siddartha Institute of Science and Technology, Puttur, India. [email protected], [email protected], 3shavank[email protected], [email protected], [email protected], [email protected] Abstract: Accurate food recognition and nutrient estimation are critical for practical dietary assessment, health monitoring, and personalized nutrition management. Traditional calorie tracking methods rely heavily on manual input and self-reporting, which are often inaccurate, time-consuming, and inconsistent. Existing AI-based food recognition systems primarily rely on basic convolutional neural networks and two-dimensional image analysis, limiting their ability to identify complex, mixed, and regional dishes and failing to estimate portion sizes accurately. These limitations significantly reduce their practical applicability, particularly for diverse cuisines such as Indian food. To address these challenges, this project presents Nutri Vision, an AI-driven framework for food recognition, portion size estimation, nutrient prediction, and personalized dietary guidance. The proposed system integrates advanced deep learning models including YOLOv8, Vision Transformers, and Region-Based Convolutional Neural Networks to accurately detect and classify multiple food items from a single image. Portion size estimation is achieved using pixel-to-gram conversion and depth-aware analysis, enabling reliable calorie and nutrient computation through integrated food databases. Furthermore, machine-learning-based decision models are employed to generate personalized diet recommendations and healthier food alternatives based on users' goals and health conditions. The system delivers real-time, culturally adaptive, and scalable nutrition insights with high accuracy, making it suitable for applications in healthcare, fitness management, and nutrition research. Keywords: Artificial Intelligence, Calorie Estimation, Food Recognition, Nutrient Prediction, Vision Transformers 1 INTRODUCTION Recent advances in artificial intelligence (AI) and deep learning have significantly transformed food analysis by enabling automated food recognition and nutrient estimation from images. Conventional dietary assessment methods, such as manual food logging and self-reported intake, are often inaccurate and inconsistent due to reliance on users and estimation errors. To overcome these limitations, AI-based food recognition systems have been increasingly adopted to provide objective and scalable nutritional assessment solutions [1]. Deep learning models, particularly convolutional neural networks and vision-based architectures, have demonstrated strong performance in food category recognition and nutrient prediction. Studies show that these models can effectively learn visual features from food images and map them to nutritional information using integrated food databases. However, most existing systems are trained on limited datasets and primarily focus on western or straightforward food categories, making them less effective for complex and regional cuisines [2]. Accurate estimation of portion sizes remains a critical challenge in image-based nutrition analysis. Many current approaches rely solely on two-dimensional image analysis, which fails to capture food volume and depth, leading to significant errors in calorie estimation [3]. Recent research emphasizes integrating advanced computer vision techniques and intelligent models to improve the accuracy and overall reliability of nutrient prediction systems. In addition to food recognition and calorie estimation, personalized nutrition has gained increasing attention in recent years. AI-driven decision models and intelligent recommendation systems enable personalized dietary guidance tailored to individual health conditions, dietary habits, and nutritional goals. Such systems have shown strong potential in healthcare and fitness applications by supporting personalized and data-driven dietary decision-making [4]. Motivated by these research advances, this project proposes Nutri Vision, an AI-based framework for food recognition, portion-size estimation, nutrient prediction, and personalized dietary guidance. The system integrates advanced object detection models, vision transformers, and machine learning-based recommendation techniques to deliver accurate, culturally adaptive, and real-time nutrition insights. International Journal of Emerging Research in Science, Engineering, and Management Vol. 1, Issue 6, pp.44-54, December 2025. www.ijersem.com eISSN - 3107-9075 IJERSEM@2025 https://doi.org/10.58482/ijersem.v1i6.6 45 2 LITERATURE SURVEY The integration of artificial intelligence with advanced analytical techniques has significantly improved food component analysis and nutritional assessment. Cao et al. proposed a synergistic approach that combines AI with Nuclear Magnetic Resonance (NMR) spectroscopy to enhance the identification of food components. Their study demonstrated that AI-driven pattern recognition improves the accuracy and efficiency of nutrient profiling, enabling reliable analysis of complex food matrices [1]. As intelligent food systems evolve, Yakoubi introduced an Industry 5.0 paradigm emphasizing human-centered and AIenabled food manufacturing. The study focused on AI-based digitization of nano-scale smart nutrient carriers, highlighting the role of intelligent systems in personalized nutrition and sustainable food production. This work underlines the importance of adaptive and human-centric AI models in next-generation nutrition technologies [2]. Zheng et al. conducted a comprehensive scoping review on artificial intelligence applications for measuring food and nutrient intake. Their findings show that AI-based dietary assessment tools reduce human bias and manual errors compared to traditional self-reporting methods. However, the study also identified challenges related to dataset diversity, mixed-meal recognition, and deployment in real-world conditions [3]. In the context of sustainable agri-food systems, Nath et al. reviewed recent advances in AI technologies applied to food quality assessment, nutrient prediction, and supply chain optimization [4]. Their work emphasized that machine learning and deep learning models contribute to both nutritional accuracy and environmental sustainability, reinforcing the relevance of AI in modern food systems [4]. Extending nutrient monitoring to agricultural applications, Bharti et al. explored nano-biosensors for real-time tracking of plant nutrient levels [5]. Their research demonstrated that intelligent sensing technologies combined with AI enable precise nutrient monitoring. Although focused on agriculture, the study suggests that similar AI-assisted techniques can enhance nutrient estimation in consumer-level food analysis systems. Explainability has become a critical requirement in AI-driven nutrition systems, particularly in healthcare applications. Di Martino et al. proposed explainable AI models for malnutrition risk prediction using mobile health and clinical data [6]. Their work highlighted the importance of transparency and interpretability in AI systems to improve trust, usability, and clinical decision-making. Deep learning approaches have shown strong performance in food image recognition tasks. Zhang et al. investigated deep learning models for food category recognition and reported significant improvements in classification accuracy over traditional image processing methods. Their findings confirm that deep neural architectures are effective for recognizing visually complex food items [7]. The real-world applicability of AI-based food recognition systems has also been evaluated. Lozano et al. validated an AI-based mobile application that identifies food items and estimates energy intake [8]. While promising results were achieved under controlled conditions, the study reported reduced performance in real-world scenarios involving poor lighting, occlusion, and complex food presentations [9]. Ingredient-level nutrient estimation has been explored to improve nutritional accuracy [10]. Ma et al. demonstrated that deep learning models can accurately predict food categories and nutrient values from ingredient lists [11]. Their work suggests that integrating ingredient-level analysis with visual recognition can significantly improve the accuracy of nutrient prediction. Early studies on mobile-based food recognition systems were conducted by Knez and Šajn, who evaluated food object recognition using mobile devices [12]. Their study identified limitations related to computational efficiency, dataset constraints, and reduced recognition accuracy in real-world environments. These findings highlight the need for robust, scalable, and adaptive AI-based food recognition frameworks 3 PROBLEM STATEMENT Accurate dietary assessment is essential for effective nutrition monitoring, disease prevention, and personalized health management. However, existing food recognition and calorie estimation systems predominantly rely on manual food logging or fundamental image-based analysis, both of which are prone to inaccuracies and user dependence. Conventional methods fail to provide reliable nutrient estimates due to subjective portion-size reporting, inconsistent food descriptions, and limited user compliance. Most current AI-based food recognition systems utilize standard convolutional neural networks trained on generic datasets. These systems struggle to accurately recognize complex, mixed, and culturally diverse dishes, particularly regional foods with similar visual characteristics and hidden ingredients such as oils, spices, and condiments. Additionally, many approaches rely solely on two-dimensional image analysis, lacking depth and volume information, leading to significant errors in portion size and calorie estimation. Furthermore, existing solutions provide limited or no personalized dietary guidance. While some applications estimate calorie intake, they do not consider individual user profiles, health conditions, dietary preferences, or nutritional goals. The absence of adaptive learning and explainable recommendation mechanisms reduces their effectiveness in healthcare and fitness applications. Real-time performance, scalability, and robustness under varying lighting and environmental conditions also remain significant challenges. Therefore, there is a critical need for an intelligent, automated, and scalable nutrition analysis system that accurately recognizes complex food items, estimates portion sizes and nutrient content, and delivers personalized dietary recommendations. Such a system must integrate advanced deep learning techniques, reliable portion-estimation methods, and machine-learning-based decision models to support accurate, real-time, and culturally adaptive dietary assessment. International Journal of Emerging Research in Science, Engineering, and Management Vol. 1, Issue 6, pp.44-54, December 2025. www.ijersem.com eISSN - 3107-9075 IJERSEM@2025 https://doi.org/10.58482/ijersem.v1i6.6 46 4 PROPOSED SYSTEM The proposed Nutri Vision system is an intelligent AI-based framework for automated food recognition, portion-size estimation, nutrient prediction, and personalized dietary recommendations. The methodology integrates advanced deep learning models for visual analysis, computer vision techniques for portion estimation, and machine learning algorithms for nutrition-based decision making. The system aims to provide accurate, real-time, and culturally adaptive nutrition insights from food images. 4.1 System Overview The proposed system follows a modular architecture comprising image acquisition, preprocessing, food recognition, portionsize estimation, nutrient prediction, and personalized diet recommendation modules. Food images captured using a mobile or webbased interface are processed through deep learning models to identify multiple food items on a single plate. Detected food items are then analyzed for portion size, calorie content, and nutritional composition using integrated food databases. The overall methodology focuses on: • Automated multi-food recognition from a single image • Accurate portion size estimation using vision-based techniques • Reliable calorie and nutrient prediction • Personalized dietary recommendations based on user profiles The complete system workflow is illustrated in the block diagram shown in Fig. 1. Fig. 1. block diagram 4.2 Image Acquisition and Preprocessing Food images are captured using a smartphone or camera under normal lighting conditions. The acquired images are resized and normalized to match the input dimensions required by the deep learning models. Image preprocessing techniques such as noise reduction, contrast enhancement, and color normalization are applied to improve visual quality and robustness under varying environmental conditions. 4.3 Food Recognition Using Deep Learning Models Food recognition is performed using advanced object detection and classification models such as YOLOv8, Vision Transformers (ViT), and Region-Based Convolutional Neural Networks (R-CNN). These models enable accurate detection and classification of multiple food items from a single image, including complex and mixed dishes. For each detected food item, bounding boxes are generated, and classification confidence scores are computed. YOLOv8 enables real-time detection with high precision, while Vision Transformers improve contextual understanding by modeling global dependencies across image patches. 4.4 Portion Size Estimation Accurate portion-size estimation is achieved through pixel-to-gram conversion and depth-aware analysis. The detected food region area in pixels is converted into real-world measurements using calibration factors. The volume of the food item is estimated as: 𝑉=𝐴×𝐻 International Journal of Emerging Research in Science, Engineering, and Management Vol. 1, Issue 6, pp.44-54, December 2025. www.ijersem.com eISSN - 3107-9075 IJERSEM@2025 https://doi.org/10.58482/ijersem.v1i6.6 47 where: 𝑉= Estimated food volume 𝐴= Area of detected food region 𝐻= Estimated height or depth of the food The food weight is then calculated as: 𝑊=𝑉×𝜌 where: 𝑊= Estimated food weight 𝜌= Density of the food item This approach improves the accuracy of calorie estimation compared to traditional 2D-only methods. 4.5 Calorie and Nutrient Prediction Once the food type and portion size are determined, calorie and nutrient values are computed using standardized food composition databases. The total calorie content is calculated as: 𝐶𝑎𝑙𝑜𝑟𝑖𝑒𝑠=𝑊×𝐶𝑔 where: 𝑊= Food weight in grams 𝐶𝑔= Calories per gram of the food Macronutrients (carbohydrates, proteins, fats) and micronutrients are similarly estimated using predefined nutrient coefficients. 4.6 Personalized Diet Recommendation Personalized dietary recommendations are generated using a Decision Tree-based machine learning model. User-specific parameters such as age, weight, dietary preferences, health conditions, and fitness goals are used as input features. Based on these parameters and predicted nutrient intake, the system recommends balanced meals, healthier alternatives, and portion guidance. The decision-making process is explainable and rule-based, allowing transparent interpretation of dietary suggestions. 5 SYSTEM ARCHITECTURE The system architecture of the proposed Nutri Vision: AI-Based Food Recognition and Nutrient Prediction System is designed as a modular and layered framework that integrates computer vision, deep learning, nutritional databases, and personalized recommendation mechanisms. The architecture ensures scalability, real-time performance, and accurate dietary analysis from food images. The Final architecture consists of six major modules: Image Acquisition, Image Preprocessing, Food Recognition, Portion Size Estimation, Nutrient Analysis, and Personalized Recommendation, all interacting through a centralized processing pipeline. The system architecture is shown in Fig. 2. It has the following modules. • Image Acquisition Module: The system begins by acquiring food images via a mobile or web-based interface. Users upload food images through the Nutri Vision application interface. The acquired image serves as the primary input for the analysis pipeline. • Image Preprocessing Module: The captured image undergoes preprocessing operations such as resizing, noise removal, contrast enhancement, and normalization. These operations improve image quality and ensure uniform input dimensions for deep learning models, thereby enhancing recognition accuracy. • Food Recognition Module: The preprocessed image is passed to the food recognition module, which employs advanced deep learning models including YOLOv8, Vision Transformers (ViT), and R-CNN. YOLOv8 performs real-time object detection and classification of food items. Vision Transformers capture global contextual features for complex and mixed dishes. R-CNN improves localization accuracy in multi-food scenarios. This module outputs detected food labels along with bounding boxes and confidence scores. • Portion Size Estimation Module: Detected food regions are analyzed to estimate portion size using pixel-to-gram conversion and volume estimation techniques. The module calculates approximate food weight based on spatial dimensions and food-density parameters, which is critical for accurate calorie and nutrient calculations. • Nutrient Analysis Module: Using the estimated food weight and recognized food type, the nutrient analysis module computes calorie content and macroand micro-nutrients by referencing standardized food composition databases. This module generates detailed nutritional information, including calories, proteins, carbohydrates, fats, and health indicators. • Personalized Recommendation Module: The personalized recommendation module uses a Decision tree–based machine learning model to generate customized dietary suggestions. User-specific data, such as age, weight, height, activity level, health conditions, and nutritional preferences, are processed to create personalized health ratings, meal plans, and food recommendations. International Journal of Emerging Research in Science, Engineering, and Management Vol. 1, Issue 6, pp.44-54, December 2025. www.ijersem.com eISSN - 3107-9075 IJERSEM@2025 https://doi.org/10.58482/ijersem.v1i6.6 48 User Interface Module: The final results are displayed through an intuitive user interface that presents food recognition results, nutritional breakdown, health ratings, recipe suggestions, and personalized meal plans. This module ensures user-friendly interaction and seamless navigation. Fig. 2. System Architecture 6 ALGORITHMS USED The proposed Nutri Vision system employs a combination of deep learning and machine learning algorithms to achieve accurate food recognition, portion size estimation, nutrient prediction, and personalized dietary recommendations. The major algorithms used in the system are YOLOv8, Vision Transformers, Region-Based Convolutional Neural Networks (R-CNN), Pixelto-Gram Conversion, and Decision Tree algorithms. 6.1 YOLOv8 Algorithm for Food Recognition YOLOv8 (You Only Look Once version 8) is used for real-time detection and classification of multiple food items present in a single image. It treats object detection as a regression problem and predicts bounding boxes and class probabilities in a single forward pass, enabling high-speed and high-accuracy performance. For each detected object, YOLOv8 predicts: • Bounding box coordinates (𝑥,𝑦,𝑤,ℎ) • Object confidence score • Food class probability The loss function combines classification loss, localization loss, and confidence loss: 𝐿=𝐿𝑐𝑙𝑠+𝐿𝑏𝑜𝑥+𝐿𝑐𝑜𝑛𝑓 YOLOv8 enables efficient recognition of complex and overlapping food items, making it suitable for real-time nutrition analysis. 6.2 Vision Transformer (ViT) Algorithm Vision Transformers are used to improve recognition accuracy by capturing global contextual information from food images. Unlike CNNs, ViTs divide the input image into fixed-size patches, which are then converted into embeddings and processed using self-attention mechanisms. International Journal of Emerging Research in Science, Engineering, and Management Vol. 1, Issue 6, pp.44-54, December 2025. www.ijersem.com eISSN - 3107-9075 IJERSEM@2025 https://doi.org/10.58482/ijersem.v1i6.6 49 The self-attention operation is defined as: 𝐴𝑡𝑡𝑒𝑛𝑡𝑖𝑜𝑛(𝑄,𝐾,𝑉)=𝑠𝑜𝑓𝑡𝑚𝑎𝑥(𝑄𝐾𝑇 √𝑑𝑘)𝑉 where: • 𝑄, 𝐾, and 𝑉are query, key, and value matrices • 𝑑𝑘is the dimensionality of the key vector 6.3 Region-Based Convolutional Neural Network (R-CNN) R-CNN is employed to improve precise localization of multiple food items, especially in cluttered or overlapping plate images. The algorithm works by: 1. Generating region proposals 2. Extracting features from each region using CNNs 3. Classifying each region into food categories R-CNN enhances detection accuracy in scenarios where multiple food items are closely packed. 6.4 Pixel-to-Gram Conversion Algorithm for Portion Size Estimation Portion size estimation is performed using pixel-to-gram conversion combined with volume estimation. The area of the detected food region is calculated in pixel units and mapped to real-world dimensions. Food volume is estimated as: 𝑉=𝐴×𝐻 where: • 𝐴= projected food area • 𝐻= estimated height or depth The food weight is computed as: 𝑊=𝑉×𝜌 where: • 𝑊= food weight • 𝜌= density of the food item This algorithm significantly improves calorie estimation accuracy compared to 2D-only methods. 6.5 Calorie and Nutrient Estimation Algorithm Once the food weight is estimated, calorie content is calculated using standardized food databases: 𝐶𝑎𝑙𝑜𝑟𝑖𝑒𝑠=𝑊×𝐶𝑔 where: • 𝑊= food weight in grams • 𝐶𝑔= calories per gram Macronutrients and micronutrients are computed using predefined nutrient coefficients associated with each food item. 6.6 Decision Tree Algorithm for Personalized Diet Recommendation A Decision Tree algorithm is used to generate personalized dietary recommendations. The model uses user-specific parameters such as age, weight, health conditions, dietary preferences, and fitness goals as input features. The splitting criterion is based on Information Gain: 𝐸𝑛𝑡𝑟𝑜𝑝𝑦(𝑆)=−∑𝑝𝑖 𝑛 𝑖=1 log⁡2𝑝𝑖 𝐼𝐺(𝑆,𝐴)=𝐸𝑛𝑡𝑟𝑜𝑝𝑦(𝑆)−∑∣𝑆𝑣∣ ∣𝑆∣ 𝑣∈𝐴 𝐸𝑛𝑡𝑟𝑜𝑝𝑦(𝑆𝑣) Decision Trees provide transparent and explainable dietary recommendations, making them suitable for healthcare-oriented nutrition systems. International Journal of Emerging Research in Science, Engineering, and Management Vol. 1, Issue 6, pp.44-54, December 2025. www.ijersem.com eISSN - 3107-9075 IJERSEM@2025 https://doi.org/10.58482/ijersem.v1i6.6 50 7 EXPERIMENTAL SETUP The experimental setup of the proposed Nutri Vision system is designed to evaluate the performance of food recognition, portion-size estimation, nutrient prediction, and personalized dietary recommendations under real-world conditions. The experiments were conducted using a combination of hardware resources, software frameworks, datasets, and evaluation protocols. 7.1 Hardware Requirements The system experiments were performed on a computing platform with the following specifications: • Processor: Intel Core i7 or equivalent • RAM: 16 GB • GPU: NVIDIA GPU with CUDA support • Storage: Minimum 256 GB • Camera: Smartphone or external camera for food image capture 7.2 Software Environment The software environment used for system development and experimentation includes: • Operating System: Windows / Linux • Programming Language: Python • Deep Learning Frameworks: TensorFlow and PyTorch • Computer Vision Library: OpenCV • Machine Learning Library: Scikit-learn • Development Tools: Jupyter Notebook and Visual Studio Code 7.3 Dataset Description The system was trained and evaluated using publicly available and curated food image datasets containing multiple food categories. The dataset includes: • Single and multi-food images • Diverse cuisines and regional dishes • Variations in lighting, angle, and background Food composition data for calorie and nutrient estimation was obtained from standardized nutritional databases. The dataset was divided into training, validation, and testing subsets to ensure unbiased performance evaluation. 7.4 Model Training and Configuration YOLOv8, Vision Transformer, and R-CNN models were trained using labeled food images. Transfer learning was applied to accelerate convergence and improve generalization. Hyperparameters such as learning rate, batch size, and number of epochs were optimized through validation experiments. The Decision Tree model was trained using user profile attributes and nutritional intake data to generate personalized dietary recommendations. 7.5 Experimental Procedure The experimental workflow consisted of the following steps: 1. Capture food images using a camera or mobile device 2. Apply image preprocessing techniques 3. Perform food detection and classification 4. Estimate portion size and food weight 5. Calculate calorie and nutrient values 6. Generate personalized dietary recommendations 7. Record system outputs and performance metrics International Journal of Emerging Research in Science, Engineering, and Management Vol. 1, Issue 6, pp.44-54, December 2025. www.ijersem.com eISSN - 3107-9075 IJERSEM@2025 https://doi.org/10.58482/ijersem.v1i6.6 51 7.6 Performance Evaluation The system performance was evaluated based on recognition accuracy, portion estimation error, calorie prediction accuracy, and recommendation effectiveness. Multiple test cases were conducted to validate robustness across different food types and environmental conditions. 8 RESULTS AND DISCUSSION This section presents the experimental results obtained with the proposed Nutri Vision system and discusses its performance in food recognition, nutrient estimation, health analysis, and personalized meal planning. The results demonstrate the system's effectiveness in real-world use cases. 8.1 Food Recognition and Nutrient Estimation Results The primary functionality of the proposed system is to accurately recognize food items in images and estimate their nutritional values. Fig. 3 illustrates the output generated after analyzing a food image. The system successfully detects the food item (Kheer) and estimates its serving size, calorie content, and macronutrient composition, including protein, carbohydrates, and fat. Fig. 3. Food recognition and nutrient estimation output of the proposed system Fig. 3 shows that the deep learning–based recognition module accurately identifies food items and computes nutritional values in real time. The visual highlighting of detected items further improves interpretability and user confidence. 8.2 Health Rating and Nutritional Assessment In addition to nutrient estimation, the system provides an overall health rating for the analyzed food item. Fig. 3 presents the health score generated by the system along with associated health benefits and cautionary information. This feature helps users make informed dietary decisions. Fig. 4. Health rating and nutritional assessment generated by the system International Journal of Emerging Research in Science, Engineering, and Management Vol. 1, Issue 6, pp.44-54, December 2025. www.ijersem.com eISSN - 3107-9075 IJERSEM@2025 https://doi.org/10.58482/ijersem.v1i6.6 52 Fig. 4 shows the health rating mechanism, which effectively summarizes nutritional quality in an easily understandable format, making the system suitable for general users without technical knowledge of nutrition. 8.3 Recipe and Preparation Guidance Output The system also provides intelligent recipe assistance for the recognized food item. Fig. 5 illustrates the automatically generated ingredient list and step-by-step preparation instructions. This feature enhances usability by providing cooking guidance alongside nutritional insights. Fig. 5. Recipe generation and step-by-step cooking instructions This integrated approach bridges the gap between food analysis and practical meal preparation, improving user engagement. 8.4 User Profile and Goal-Based Recommendation Interface The system incorporates user-specific parameters such as age, weight, height, activity level, dietary preferences, and health conditions to generate personalized recommendations. Fig. 6 shows the user profile input interface used for goal-based meal plan generation. Fig. 6. User profile and goal configuration interface