Using LLMs to Extract Food Entities from Cooking Recipes

Using LLMs to Extract Food Entities from Cooking Recipes Vasiliki Pitsilou Information Management Systems Institute “Athena” Research Center Athens, Greece [email protected] George Papadakis Department of Informatics and Telecommunications National and Kapodistrian University of Athens Athens, Greece [email protected] Dimitrios Skoutas Information Management Systems Institute “Athena” Research Center Athens, Greece [email protected] Abstract—Automatically extracting food entities from cooking recipes has recently gained significant attention as a means of facilitating dataand AI-driven solutions for healthy and sustainable diets. The state-of-the-art approaches in the literature involve a high overhead, as their supervised functionality requires a large number of labelled instances. To facilitate food entity extraction applications, we explore the use of Large Language Model (LLMs) under zero-shot settings, where no labelled instance is required. Instead, we propose a generic methodology that exclusively focuses on prompt and response engineering. We apply it to small LLMs with just 7b parameters so as to increase their timeefficiency, while minimizing the necessary resources. Our experimental analysis yields promising results, but there is room for significant improvements. Index Terms—food entity recognition, food information extraction, LLMs I. INTRODUCTION Over the last years, we are witnessing a growing availability of food-related data from a multitude of diverse sources, ranging from authoritative ones, such as scientific articles and national dietary policies and recommendations, to crowd-sourced ones generated from the Web, social networks, and mobile devices, such as recipe databases [1]. This has led to the rise of food computing [2], which aims at acquiring and analysing large-scale food data to tackle various foodrelated tasks and applications. However, a major issue is that a large portion of this data, including cooking recipes, is in the form of unstructured text, thus not directly machine-processable, which hinders its use in downstream applications [3]. Notably, employing data and AI-driven tools to support a behavioral shift towards healthier and more sustainable diets requires large-scale collections of digital recipes annotated with nutrition and sustainability information. Today, large recipe datasets exist, such as Recipe1M+ [4], RecipeDB [5], FoodBase [6], etc., as well as efforts to extract nutritional and sustainability profiles from recipes [7], [8], [9], [10]. To this end, a fundamental task is to identify mentions of food entities in cooking recipes. Several methods for food entity recognition have been proposed in the literature, including terminology-driven, rule-based, corpus-based, methods based on active learning, and methods based on deep neural networks [3]. The current state of the art is SciFoodNER [11], which has been evaluated both on cooking recipes and scientific articles, achieving high recall and precision. SciFoodNER is a transformer model fine-tuned on a corpus of scientific abstracts annotated with food entities. However, creating such a corpus requires human annotations from food domain experts, hence an investment in manual time and effort. Moreover, the derived model is tailored to the task at hand, and therefore may not generalize well to different types of entities or styles of text compared to the ones included in the used training dataset. In this work, our goal is to investigate whether it is possible to lift these limitations by leveraging the generalization capabilities of Large Language Models (LLMs), which allow them to be successfully applied to new tasks via in-context learning instead of fine-tuning. We present our experiments with three open-source LLMs, specifically Llama2, Mistral and Openhermes. We focus on open-source models and model sizes of 7B parameters, since we are interested in solutions that do not impose high costs in terms of hardware resources or API usage. We discuss our approach with respect to prompt and answer engineering, and present our preliminary results. Our experimental results demonstrate that there is still room for improving the performance of LLMs to reach that of SciFoodNER, yet they offer a promising starting point, especially when taking into consider- ation their zero-shot operation. Overall, we make the following contributions: 1) We propose two granularities of various zeroshot prompts for food entity extraction using small, open-source LLMs. 2) We compare the performance of our LLM-based approach to the state-of-the-art in the field. 3) We identify limitations and point to directions for further improvements. The rest of the paper is structured as follows: We discuss the main relevant works in the field in Section II. We formally define the problem we tackle in Section III. We introduce the LLMs we consider and our methodology in Section IV. We present our experimental analysis in Section Vand conclude with a discussion of the main points in Section VI. II. RELATED WORK The extraction of food entities from recipes is a specialization of the more generic task of named entity recognition (NER) [12],[13]. There is a plethora of works on NER, both in a generic, domain-agnostic context [14],[15] and in domain-specific applications [16], [3], [17]. Most relevant to our work are NER methods that are crafted for extracting food entities from free text like recipes. The main methods of this type are surveyed in [3], which discusses FoodIE [18] and various instantiations of NCBO, namely NCBO (SNOMED CT), NCBO (OntoFood), and NCBO (FoodON). The former is a rule-based system, while the latter is an annotator for biomedical entities based on several ontologies (SNOMED CT, OntoFood, FoodON) that is available as a web service1. Their relative performance is evaluated on the FoodBase corpus (see Section V-B for more details). More advanced are the food NER methods that leverage language models. InstaFoodRoBERTa-NER2 is a BERT model that is fine-tuned on classifying text tokens as food or non-food entities. More specifically, it classifies each token as the beginning of a food entity (“B-FOOD”), inside of a food entity (“I-FOOD”) or no food entity (“O”). InstaFoodRoBERTa-NER is fine-tuned using InstaFoodSet3, a dataset containing free texts related to foods and drinks, with a golden standard annotating each text token as “B-FOOD”,“I-FOOD” or “O”. By fine-tuning InstaFoodRoBERTa on this labelled dataset, the method is able to identify food entities in any other dataset, without the need for additional training or fine-tuning. We compare our approach to InstaFoodRoBERTa-NER in Section V-D. 1https://bioportal.bioontology.org/annotator 2https://huggingface.co/Dizex/InstaFoodRoBERTa-NER 3https://huggingface.co/datasets/Dizex/InstaFoodSet Figure 1: An example with a recipe involving food entities that consists of one,two or three tokens. Another similar approach is the “Bidirectional Encoder Representations from Transformers for Biomedical Text Mining” (BioBERT) [19]. It constitutes a domain-specific language representation model, pre-trained on large-scale biomedical corpora. It is used for biomedical text mining tasks and it is capable of understanding complex biomedical texts. BioBERT is thus useful for biomedical NER, outperforming BERT on that task. Finally, SciFoodNER [11] is a supervised NER approach that requires a labelled dataset. In fact, it splits the input labelled dataset into train, validation and test set to fine-tune a language model on the train set. The following language models are supported: BERT, RoBERTa, SciBERT or BioBERT. In our experiments, we used BioBERT, since this was shown to be the best configuration for SciFoodNER in [11]. The use of LLMs has not been considered in any of these works. In fact, LLMs have only been applied to NER in a generic, context-agnostic context. For example, different types of prompts are examined in [20] in order to optimize the performance of LLMs. To the best of our knowledge, this is the first work that uses LLMs for food NER and compares them to previous techniques that rely on the currently dominant pre-trained, fine-tuned paradigm [11]. III. PROBLEM DEFINITION The task tackled in this work can be formally defined as follows: Given as input a recipe Rin the form of a free text, extract all mentions to food entities that consist of a single or multiple tokens so as to maximize recall and precision, while minimizing the run-time and the human effort required for building the extraction model. As an example, consider the recipe in Figure 1, which describes the first step of preparing hummus. We observe that it comprises 6 mentions to food entities. Some of them are straightforward, consisting of a single token, like “garlic”, while others are more complex ones, formed by 2 or even 3 tokens (e.g., “lemon juice” and “crushed red pepper” respectively). An extraction model is evaluated with respect to the accuracy of identifying every mention. This requires determining exactly those tokens that comprise a specific food entity. That is, adding more tokens or excluding some others yields an error. For example, marking “red pepper” as food entity in the recipe of Figure 1is wrong in this case, even though it is probably correct in other contexts where there is no adjective like crushed. IV. APPROACH A. LLMs In this work, we focus on applying LLMs for food entity extraction from recipes. We consider opensource LLMs that can be run without any direct costs (e.g., subscriptions) – unlike the commercial LLMs like ChatGPT that charge per API call, depending on the complexity and size of the request. We also take special care to ensure that the selected LLMs run on low-end GPU servers. To this end, we consider the smallest versions of recent LLMs with 7 billion parameters, minimizing the memory requirements. We actually experimented with the following LLMs: •Llama2-7b [21], released by Meta AI, was trained on 2 trillion tokens with a context length of 4,096 through an optimized auto-regressive transformer. The training corpus combines data from publicly available sources, up-sampling the most factual sources to increase knowledge and dampen hallucinations. Grouped-query attention (GQA) was also used to improve inference scalability and to reduce memory requirements. •Mistral-7b [22] goes beyond Llama2 through optimization techniques like sliding window attention (SWA), rolling buffer cache, pre-fill and chunking. For example, SWA allows for handling longer sequences more effectively and efficiently. Its context length is also double than that of Llama2. As a result, 7b version outperforms most other LLMs of the same parameter scale, matching the performance of much larger models like LLaMa-34B on several benchmarks. •OpenHermes 2.5 - Mistral 7B4is an LLM crafted for code generation. Yet, it achieves state-of-the-art performance in a series of noncode benchmarks, like TruthfulQA. This should be attributed to its large training corpus, which comprises 1,000,000 entries of primarily GPT-4 generated data, as well as other pre-processed and filtered data from open data sources. •Orca2-7b [23] is built on top of Llama2, finetuning on synthetic data that involve zeroand few-shot prompts. Its training leverages several optimization techniques like progressive learning on subsets of data, Llama byte pair encoding with padding for tokenization and packing, i.e., the concatenation of multiple examples into a single sequence, for higher time efficiency. •Zephyr-7b [24] is an LLM built on top of Mistral-7b with the goal of acting as a helpful assistant. It has been trained on a mixture of publicly available, synthetic datasets through direct preference optimization. User feedback suggests that its performance on chat tasks is comparable to more complex LLMs with 70B parameters. 4https://huggingface.co/teknium/OpenHermes-2.5-Mistral-7B We carried out preliminary experiments to identify the ones combining higher effectiveness with easier adaptation to the task at hand. Zephyr-7b was less effective than all other models, while Orca2-7b was more accurate, but quite verbose, requiring complex parsing for its wordy output. Hence, we exclusively consider Llama2-7b, Mistral-7b and Openhermes-7b5 in the following. Below, we explain how they were applied to the task at hand. B. Methodology In general, the methodology for applying an LLM to food NER consists of two steps [25]: •Prompt engineering formulates the query that is posed to the LLM so as to make the most of its knowledge and capabilities. •Response engineering parses the output of the LLM to identify the entities that were mentioned in the given recipe (i.e., separating the food entities from the whole answer returned). We elaborate on each step below. 1) Prompt Engineering: Our approach exclusively considers zero-shot prompts, where every request to the LLM provides no entity extraction examples that could facilitate its understanding of the task at hand. There are two reasons for this choice: •zero-shot prompts reduce the run-time, as the larger the prompt is, the more time consuming is the response of an LLM. •zero-shot prompts simplify the use of LLMs, saving the cost of carefully selecting multiple representative cases that cover the main types of food entities in the data at hand. In this setting, the LLM prompts can be configured by two parameters: •the form of the query, and •the granularity of the query. With respect to the former parameter, we consider the following forms of zero-shot prompts: •Prompt 1: “Print only one comma-separated list of the foods, drinks or edible ingredients mentioned in the previous text. Do write a very short answer, with no details, just the list. If there are no foods, drinks or edible ingredients mentioned, print no.” •Prompt 2: “You are a food allergy specialist and your task is to find anything edible, i.e. food, drink or ingredient, mentioned in the previous text. If you lose any edible item mentioned, there is a risk of someone getting allergy and you will be penalized. Print the edible items you found in a comma-separated list, each edible item printed separately and without further information. If 5Among the available versions for Openhermes-7b, we consider the latest one, i.e., v.2.5. there are no edible items mentioned in the text, print no.” •Prompt 3: “Find any foods, drinks or edible ingredients mentioned in the previous text. Print them in a comma-separated list. If there are none, print no. Write a short answer.” In formulating these prompts, we followed the instructions in [26]. For example, Prompt 2 assigns a role to the LLM, while all prompts state clearly the requirements of the task that will be addressed by the LLM. Note that Prompts 1 and 3 have slight differences, but we consider both of them as a means of testing the robustness of LLMs, exploring to what extent do minor changes in prompt formulation affect an LLM’s response and subsequently performance. Also crucial is the second configuration parameter, the prompt granularity. We consider two options: •Entire recipe, where all sentences comprising a given recipe are fed to the LLM as a single input, regardless of its overall size. •Individual sentences, where period is used as a delimiter for splitting a recipe into a series of shorter texts, and each sentence is fed to the LLM individually. The first approach involves coarse-grained prompts that minimize the requests to the LLM, which can be time-consuming. These prompts also ensure that the LLM has processed the output so that it comprises a set of the detected food entities, excluding possible repetitions (e.g., due to grammar variations). On the downside, this approach might suffer from lower accuracy, due to the limited attention span of the small LLMs we are considering. This issue is effectively tackled by the finer prompts of the second type. 2) Response engineering: All three prompts considered above request a list of detected entities from the LLM. This is rarely the case, as the LLM responses are typically verbose, entailing irrelevant text that has to be removed to arrive at the desired output. As an example, consider the following LLM response: “The list is: pears, apples, bananas”. It is more wordy than requested, thus calling for simple text manipulation techniques to get the desired output. In this case, we consider the text after the colon and split it by commas, transforming the LLM response into: [pears, apples, bananas]. It is worth noting that the form of responses varies among different LLMs, but for a particular LLM, each response invariably exhibits the same structure. As a result, we developed a separate response parser for each of the three considered LLMs. V. EXPERIMENTAL EVALUATION In this section, we first describe the experimental setup, the datasets used, and the applied evaluation measures. Then, we present and discuss the results of the evaluation. A. Setup All experiments were executed on a server running Ubuntu 22.04.2 LTS with an AMD Ryzen 3960X, 256 GB RAM and a GeForce RTX 2080 Ti with 11 GB RAM. To run the selected LLMs locally, on this server, we used Ollama v0.1.206, an open-source library that facilitates the setup and application of multiple open-source LLMs, on a CPU or GPU. B. Datasets In our experiments, we use the FoodBase dataset [6]7, an established recipe dataset annotated with food entities. In total, it comprises 1,000 recipes with 7,936 sentences and 12,844 mentions to food entities. On average, there are 12.84 and 1.62 food entities per recipe and sentence, respectively. Finally, the recipes constitute relatively large texts, with almost 8 sentences, 527 characters and 106 tokens, on average. C. Evaluation Measures We assess the performance of all considered techniques with respect to time efficiency and effectiveness. For the former, we consider the overall run-time, i.e., the total time required for extracting food entities from all recipes in the FoodBase dataset. For effectiveness, we consider recall re =tp/(tp+ fn)and precision pr =tp/(tp +fp), where tp stands for true positives (i.e., tokens comprising food entities that are correctly identified as such), fp for false positives (i.e., tokens not related to food entities, but incorrectly identified as such) and fn for false negatives (i.e., tokens comprising food entities but not identified as such). We also consider the F-measure (f1), which amounts to the harmonic mean of recall and precision, i.e., f1 = 2 ×re ×pr/(re +pr). Note that for every effectiveness evaluation measure, we report an extended definition, denoted by re∗,pr∗and f1∗, where we consider as true positives the food entities that are partially detected, i.e., they comprise multiple tokens, but one of them has been mistakenly marked as not belonging to the corresponding mention. As an example, consider the phrase “spread with almond mixture”, where the LLM extracts “almond” as food entity, whereas the ground truth contains the entity “almond mixture”. D. Experimental Results The effectiveness of the considered approaches is reported in Figures 2-5, where the variations of the Llama2-7b, Mistral-7b and OpenHermes 2.5 - Mistral 7B models start with L2, MI and OH, respectively, while P1, P2 and P3 indicate Prompts 1, 2 and 3, respectively. The sentence-level prompts are indicated by the suffix S. Rn denotes the repetitions per prompt 6https://github.com/jmorganca/ollama 7http://cs.ijs.si/repository/FoodBase/foodbase.zip Figure 2: The effectiveness results per recipe-level model with respect to recall, precision and f1. Figure 3: The effectiveness results per sentence-level model with respect to recall, precision and f1. – R1 means that each prompt is submitted only once, while R3 indicates three submissions per prompt, with the final result comprising the union of the three responses, i.e., all mentions identified as food entities at least once. In this way, we alleviate the stochastic nature of LLMs. IF stands for InstaFoodRoBERTaner and SF for SciFoodNER-BioBERT. Finally, the time efficiency is reported in Figure 6with respect to the overall run-time (in minutes), which includes LLM querying and response parsing – the latter consistently takes less than 1 minute, with the former accounting for 90-95% of the total time in each case. We observe the following patterns: •The effectiveness of recipe-level prompts is relatively stable across all LLMs and prompt types, with f1(f1∗) fluctuating between 75.1% (82.6%) for L2P1R1 and 78.6% (86.1%) for MIP3R3 and OHP3R3. This might be attributed to the large size of the recipes, which dominate the prompts used in each case, increasing the robustness of LLM responses. •The time efficiency of recipe-level prompts fluctuates significantly among the LLMs and prompt types, but is consistently lower than the sentence-level prompts by 77%, on average, due to the much lower number of queries they involve. •The sentence-level performance of Llama2-7b favors precision over recall, except for Prompt 3. The three repetitions significantly improve the performance of Prompt 1. In fact, the best performance is achieved by unioning the results of the three repetitions of Prompt 1. For this reason, we exclusively combine the other two prompts with three repetitions. •Different behavior is exhibited by the sentencelevel performance of Mistral-7b. The three repetitions Figure 4: The effectiveness results per recipe-level model with respect to re∗,pr∗and f1∗. Figure 5: The effectiveness results per sentence-level model with respect to re∗,pr∗and f1∗. have no significant impact on the performance of Prompt 1, as the increase in recall is counterbalanced by the drop in precision. The same applies to the performance of Prompt 2, which achieves the highest f1for this model, regardless of the number of repetitions. As expected, though, using a single repetition with Prompt 2 is three times faster than the three repetitions. However, this performance is lower than that of Llama2-7b. Prompt 3 underperforms all other prompts, albeit to a limited extent – as in Llama2-7b. •OpenHermes consistently favors recall over precision in sentence-level prompting, unlike Llama2-7b. The best sentence-level performance with respect to effectiveness and the second best with respect to runtime corresponds to Prompt 1 with a single repetition, with all other configurations yielding practically the same F1, despite the small variations in recall and precision. Still, its best performance is much lower than Llama2-7b. •The highest effectiveness is achieved by sentence-level prompts with respect to both f1 and f1∗. The former reaches is maximized when combining Llama2-7b with L2P1R3S (f1=79.2%), and the latter when combining Mistral-7b with MIP2R1S (f1∗=86.4%). Compared to recipe-level prompts, the sentence-level ones trade higher sensitivity (i.e., lower robustness) for higher effectiveness in some cases. •InstaFoodRoBERTa-ner is slightly more effective than the best configuration of sentence-level Llama27b, whereas SciFoodNER-BioBERT achieves significantly higher F1 score, due to its supervised functionality that leverages a large training dataset. Note that InstaFoodRoBERTa-ner and SciFoodNER-BioBERT Figure 6: Run-time (in minutes) per model and prompt granularity. Note the logarithmic scale of the vertical axis. process the input at the level of individual sentences. Therefore, their performance in Figures 2and 3as well as Figures 4and 5is the same. •Figures 4and 5report re∗,pr∗and f1∗, which consider the partially detected entity mentions as correct ones. We observe that performance rises significantly for all LLMs and prompts, while there is no change in the run-time, hence it is not reported separately in Figure 6. The performance of the baseline methods also increases to a significant extent, but their advantage over the LLMs is reduced. VI. DISCUSSION Our experiments are summarized in Figure 7, which associates the F1 score per approach with its overall run-time. Each LLM is represented by two configurations: the one exhibiting the lowest run-time and the one achieving the highest f1. The closer an approach is to the upper left corner, the better balance it exhibits between effectiveness and time efficiency, combining high f1with low run-times. We observe that LLMs underperform the two baseline methods with respect to effectiveness, but typically offer lower run-times. Their advantages in terms of time efficiency is even greater when considering the effort required for preparing the labelled dataset required by the most accurate approach, SciFoodNER-BioBERT. Therefore, our goal is to improve the effectiveness of LLMs without degrading their run-times. To this end, we looked into their responses in more detail. We observed some interesting cases when prompting LLMs to extract food entities. If the given text indeed contains food entities, the correct ones are usually returned. However, in texts that lack any food entities, even though the LLM is explicitly asked to print “no” in such cases, it usually returns nonfood tokens as food entities (e.g., oven is frequently identified as a food entity) or even food products that do not exist in the text. A relevant example appears in Figure 9. Moreover, there are some common food entity misses among all LLMs, such as batter,dough, filling and liquid. Figure 7: Best configurations with respect to runtime (triangle) and F1 score (circle) per LLM, compared to baseline methods (square). When using an LLM to extract food entities in a text, the finer granularity of prompts invariably trades higher recall for lower precision, as seen when comparing the performance of recipe-level prompts in Figures 2and 3with the sentence-level prompts in Figures 4and 5. This is expected, since as mentioned in section IV.B, searching bigger texts results in less successful extraction of food entities, missing some mentions. On the other hand, splitting recipes in sentences results in many texts not containing food entities. That way, many false positives occur, as previously discussed. In Figure 8, we can see the food entities returned by Mistral-7b for a particular recipe, when prompting it for the whole recipe (Figure 8a), and when prompting it for each one of its 18 sentences (Figure 8b). VII. CONCLUSIONS On the whole, LLMs offer an F1-score of up to 79.21% for the Foodbase dataset. This is quite encouraging, considering that no labelled dataset is required for the task of food NER. This particular performance comes at the cost of high runtime, and perhaps high computational resources, but most other LLM-based approaches, especially the recipelevel ones, offer significantly higher time efficiency for negligibly lower effectiveness. The two baseline methods, InstaFoodRoBERTa-ner and SciFoodNERBioBERT, achieve an F1-score of 83.45% and 96.14%, respectively, but rely on a time-consuming training procedure that requires labelled instances. In other words, LLMs offer an effective way of performing food NER, without the need of labelled data. This is indeed a significant step in the field of NER and especially in domain-specific NER, since labelled datasets required for training traditional models are extremely limited, and creating such datasets is a time-consuming and laborious process. At the same time, LLMs can be used to create labelled datasets, which can in turn be used as a gold standard for (a) (b) Figure 8: False negatives, true positives, false positives when prompting Mistral-7B (a) by recipe, (b) by sentence. (a) (b) Figure 9: Llama2’s response for text that (a) contains food entities, and (b) lacks any food entities. training or fine-tuning traditional language models. We plan to explore this direction in our future work. More extensive prompt and response engineering can lead to much higher F1-scores for food NER using LLMs. The same is true for using more sophisticated LLMs, i.e. LLMs with more parameters (e.g. 13 billion and 70 billion). In the future, we will deploy such methods to improve F1-scores in the field of food NER using LLMs. We also intend to investigate the effectiveness of our approaches to NER tasks in other domains. We expect the food domain to be more challenging than others like news, which mostly involve locations, persons and companies that correspond to popular (a.k.a. head) entities, comprising a single or at most two tokens. In contrast, the food entities may involve multiple tokens, as shown in Figure 1, while corresponding to torso or tail entities, as defined in [27]. ACKNOWLEDGEMENT This work was partially funded by the EU Horizon Europe project STELAR (101070122). REFERENCES [1] W. Min, P. Zhou, L. Xu, T. Liu, T. Li, M. Huang, Y. Jin, Y. Yi, M. Wen, S. Jiang et al., “From plate to production: Artificial intelligence in modern consumer-driven food systems,” arXiv preprint arXiv:2311.02400, 2023. [2] W. Min, S. Jiang, L. Liu, Y. Rui, and R. C. Jain, “A survey on food computing,” ACM Comput. Surv., vol. 52, no. 5, pp. 92:1–92:36, 2019. [3] G. Popovski, B. Korousic-Seljak, and T. Eftimov, “A survey of named-entity recognition methods for food information extraction,” IEEE Access, vol. 8, pp. 31 586–31 594, 2020. [4] J. Mar´ ın, A. Biswas, F. Ofli, N. Hynes, A. Salvador, Y. Aytar, I. Weber, and A. Torralba, “Recipe1M+: A dataset for learning cross-modal embeddings for cooking recipes and food images,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 43, no. 1, pp. 187–203, 2021. [5] D. Batra, N. Diwan, U. Upadhyay, J. S. Kalra, T. Sharma, A. K. Sharma, D. Khanna, J. S. Marwah, S. Kalathil, N. Singh, R. Tuwani, and G. Bagler, “RecipeDB: a resource for exploring recipes,” Database J. Biol. Databases Curation, vol. 2020, 2020. [6] G. Popovski, B. Korousic-Seljak, and T. Eftimov, “FoodBase corpus: a new resource of annotated food entities,” Database J. Biol. Databases Curation, vol. 2019, p. baz121, 2019. [7] M. van Erp, C. Reynolds, D. Maynard, A. Starke, R. I. Mart´ ın, F. Andr` es, M. C. A. Leite, D. A. de Toledo, X. S. Rivera, C. Trattner, S. Brewer, C. A. Martins, A. Kluczkovski, A. Frankowska, S. Bridle, R. B. Levy, F. Rauber, J. T. da Silva, and U. Bosma, “Using natural language processing and artificial intelligence to explore the nutrition and sustainability of recipes and food,” Frontiers Artif. Intell., vol. 3, p. 621577, 2020. [8] G. Ispirova, T. Eftimov, S. Dzeroski, and B. Korousic-Seljak, “MsGEN: Measuring generalization of nutrient value prediction across different recipe datasets,” Expert Syst. Appl., vol. 237, no. Part B, p. 121507, 2024. [9] N. M. Abdelrhim, F. Andr` es, and A. Pester, “Food carbon footprint: Challenges and opportunities,” in REV, M. E. Auer, S. A. El-Seoud, and O. H. Karam, Eds., vol. 524, 2022, pp. 664–670. [10] P. Piplani, P. Gulati, S. Malik, S. Goyal, M. Gurbaxani, and G. Bagler, “FoodPrint: Computing carbon footprint of recipes,” in ICDE Workshops. IEEE, 2022, pp. 95–100. [11] G. Cenikj, G. Petelin, B. Korousic-Seljak, and T. Eftimov, “SciFoodNER: Food named entity recognition for scientific text,” in IEEE Big Data, 2022, pp. 4065–4073. [12] V. Yadav and S. Bethard, “A survey on recent advances in named entity recognition from deep learning models,” 2019. [13] B. Jehangir, S. Radhakrishnan, and R. Agarwal, “A survey on named entity recognition — datasets, tools, and methodologies,” Natural Language Processing Journal, vol. 3, p. 100017, 2023. [14] A. Akbik, D. Blythe, and R. Vollgraf, “Contextual string embeddings for sequence labeling,” in Proceedings of the 27th International Conference on Computational Linguistics, E. M. Bender, L. Derczynski, and P. Isabelle, Eds. Santa Fe, New Mexico, USA: Association for Computational Linguistics, Aug. 2018, pp. 1638–1649. [15] P. Qi, Y. Zhang, Y. Zhang, J. Bolton, and C. D. Manning, “Stanza: A python natural language processing toolkit for many human languages,” 2020. [16] H. Keshavarz, Z. Vagena, P. Kouki, I. Fountalis, M. Mabrouki, A. Belaweid, and N. Vasiloglou, “Named entity recognition in long documents: An end-to-end case study in the legal domain,” in 2022 IEEE International Conference on Big Data (Big Data), 2022, pp. 2024–2033. [17] Q. H. Ngo, M. T. Kechadi, and N. Le-Khac, “Domain specific entity recognition with semantic-based deep learning approach,” IEEE Access, vol. 9, pp. 152 892–152 902, 2021. [18] G. Popovski, S. Kochev, B. Seljak, and T. Eftimov, “FoodIE: A rule-based named-entity recognition method for food information extraction,” 02 2019. [19] J. Lee, W. Yoon, S. Kim, D. Kim, S. Kim, C. H. So, and J. Kang, “BioBERT: a pre-trained biomedical language representation model for biomedical text mining,” Bioinform., vol. 36, no. 4, pp. 1234–1240, 2020. [20] D. Ashok and Z. C. Lipton, “PromptNER: Prompting for named entity recognition,” 2023. [21] H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, D. Bikel, L. Blecher, C. C. Ferrer, M. Chen, G. Cucurull, D. Esiobu, J. Fernandes, J. Fu, W. Fu, B. Fuller, C. Gao, V. Goswami, N. Goyal, A. Hartshorn, S. Hosseini, R. Hou, H. Inan, M. Kardas, V. Kerkez, M. Khabsa, I. Kloumann, A. Korenev, P. S. Koura, M.-A. Lachaux, T. Lavril, J. Lee, D. Liskovich, Y. Lu, Y. Mao, X. Martinet, T. Mihaylov, P. Mishra, I. Molybog, Y. Nie, A. Poulton, J. Reizenstein, R. Rungta, K. Saladi, A. Schelten, R. Silva, E. M. Smith, R. Subramanian, X. E. Tan, B. Tang, R. Taylor, A. Williams, J. X. Kuan, P. Xu, Z. Yan, I. Zarov, Y. Zhang, A. Fan, M. Kambadur, S. Narang, A. Rodriguez, R. Stojnic, S. Edunov, and T. Scialom, “Llama 2: Open foundation and fine-tuned chat models,” 2023. [22] A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. de las Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnier, L. R. Lavaud, M.-A. Lachaux, P. Stock, T. L. Scao, T. Lavril, T. Wang, T. Lacroix, and W. E. Sayed, “Mistral 7b,” 2023. [23] A. Mitra, L. D. Corro, S. Mahajan, A. Codas, C. Simoes, S. Agarwal, X. Chen, A. Razdaibiedina, E. Jones, K. Aggarwal, H. Palangi, G. Zheng, C. Rosset, H. Khanpour, and A. Awadallah, “Orca 2: Teaching small language models how to reason,” 2023. [24] L. Tunstall, E. Beeching, N. Lambert, N. Rajani, K. Rasul, Y. Belkada, S. Huang, L. von Werra, C. Fourrier, N. Habib, N. Sarrazin, O. Sanseviero, A. M. Rush, and T. Wolf, “Zephyr: Direct distillation of lm alignment,” 2023. [25] P. Liu, W. Yuan, J. Fu, Z. Jiang, H. Hayashi, and G. Neubig, “Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing,” ACM Comput. Surv., vol. 55, no. 9, pp. 195:1–195:35, 2023. [26] S. M. Bsharat, A. Myrzakhan, and Z. Shen, “Principled instructions are all you need for questioning llama-1/2, gpt3.5/4,” 2024. [27] K. Sun, Y. E. Xu, H. Zha, Y. Liu, and X. L. Dong, “Headto-tail: How knowledgeable are large language models (llm)? A.K.A. will llms replace knowledge graphs?” CoRR, vol. abs/2308.10168, 2023.