Polymer Data Extraction and KG Population Using LLMs

Polymer Data Extraction and KG Population Using LLMs Anmol Saini,1Jeffrey Ethier,2Cogan Shimizu,1 1Wright State University, 2US Air Force Research Laboratory {saini.25, cogan.shimizu}@wright.edu, jeffrey.ethier[email protected] Abstract Computer-aided research techniques for accelerating scientific discovery in polymer (and materials) science has continued to grow in both utilization and access. There remain limitations, however, especially in the curation of data. Currently, data is primarily extracted and compiled from publications and other forms of text manually. This process can be time-consuming and many existing forms of representation are rigid, unable to account for the evolution of data. Knowledge graphs – and ontology – provide a representation that allows for the complex nature of polymer data, but still needs to be populated with data from literature. Given the recent successes of large language models in interpreting massive natural language corpora, we propose a pipeline for populating a modular knowledge graph that captures state-of-the-art polymer characterizations in combination with experimental metadata and methodology. Introduction The expansion of computer-aided research techniques, such as artificial intelligence (AI), in various domains has played a significant role in their advancement. However, their incorporation into some fields remains limited, impeding the rate of discovery. This is especially true in the materials and polymer science domains, in which manual experimentation and data curation remain prominent. These processes can be time-consuming, preventing subject matter experts from investing their efforts in tasks better aligned to their skill set. Furthermore, this manual data curation results in the development of disparate collections of data that make it more difficult for experts to integrate information from a variety of sources into their own work. Automation in these fields serves to expedite research by reducing the mundane aspects of scientists’ workload. By also minimizing human error in experiments, it ensures better reproducibility of results. Such technologies also hold much potential in the process of collecting and maintaining data. For example, computer-aided research techniques can be used to directly incorporate data from experiments into a repository or to draw from already collected data to provide context for another experiment. Even before analyzing the process of integrating and using data from experCopyright © 2024, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. iments in real time, however, it becomes necessary to determine how to incorporate the vast amounts of information that have been created and collected over several generations into one store and how to unify this data. Thus, widespread autonomous data curation acts as a significant milestone in materials and polymer research. Knowledge graphs (KGs), in particular, present a reasonable solution to the problem of data storage by acting as a semantically rich mechanism of providing contextually relevant information and are used across domains and sectors (Hitzler 2021; Noy et al. 2019). By representing data as the relationships between elements in the form of triples, KGs precisely model data and limit ambiguity. Furthermore, their flexible and extensible nature allows them to be updated easily, accounting for the possibility of altering their representation as more information is incorporated (Hogan et al. 2022). KG and ontology development can be a slow and timeconsuming process, especially when the data to be integrated is scattered across disparate sources and when these sources are not organized in a tabular format. For example, publications provide a prominent source of our knowledge on materials and polymers; but extracting and compiling this knowledge presents a significant challenge. Thus, we propose the use of large language models (LLMs) as a means by which to populate KGs. LLMs have drastically grown in popularity in recent years and have borne fruitful results in activities such as the completion of tasks and provision of information. Related Work There are numerous polymer datasets, but they contain varying kinds of data organized and structured in unique ways. For example, PoLyInfo (Otsuka et al. 2011), Polymer Property Predictor and Database (Ellis and Smith 2008), and CHEMnetBASE (chemnetbase) all have collections of polymers that record different properties. Since many of these resources obtain their data from publications and handbooks (Zhao et al. 2018), the format of the primary sources remains the same. Thus, it becomes imperative to develop a feasible and consistent framework to extract data directly from text without having to first convert it into a tabular format. Some graph-based approaches have also been developed in an attempt to create a living resource that can represent concepts with richer semantics. For example, the Com- munity Resource for Innovation in Polymer Technology (CRIPT) (Walsh et al. 2023) utilizes a unique data model to represent various kinds of information, including processes and computations, some of which other representations fail to account for. NanoMine (Zhao et al. 2016; Brinson et al. 2020; Zhao et al. 2018) is another tool that uses a KG and ontology to represent concepts. While possessing more dynamic representations of data, these methods still suffer from the time-consuming nature of manual curation of literature with the latter actually acknowledging challenges associated with it, even posing natural language processing (NLP) techniques as a potential solution to automatically extract targeted data (Zhao et al. 2018). Due to their advanced natural language capabilities and user-friendly nature over distinct NLP techniques, LLMs hold much promise in assisting with the automation of this task. Proposed Methodology The proposition of LLMs for Knowledge Graph and Ontology Engineering (KGOE) has been studied previously and shown promise for various tasks subsumed under it (Shimizu and Hitzler 2024). KG population, in particular, has been observed in relation to the Enslaved.Org Hub Ontology and yielded good results (Norouzi et al. 2025). Thus, our foundational approach is simply to extend that work to the realm of polymer science to determine its potency as a domain agnostic technique. Specifically, we intend to use NanoMine initially since it already has a KG-based representation with data being derived from a number of publications, thus enabling more direct comparison between the LLM-generated triples and the manually created ones. As shown in Figure 1, the process begins with inputting a prompt, polymer publication, and ontology module to an LLM. The ontology module simply delineates triples (in the form of classes and relationships) to be detected in the polymer publication. The prompt provides this information to the LLM along with the task of parsing the publication and populating these triples with instance-level data. The LLM then outputs the relevant data it found in the publication as triples. In an attempt to determine their accuracy, we intend to validate these automatically generated triples against the manually curated NanoMine KG. By analyzing the similarity of triples across both, we can assess the extent to which an LLM can populate a KG relative to a human’s ability. Minimal discrepancies would be indicative of a feasible solution to this problem. Sample Usage In our initial cursory testing of the proposed methodology, we have selected a publication that was used in the NanoMine KG (Roy et al. 2007). By then creating a module file that contains classes and object properties as triples in the format predicate(subject, object), we provided templates for some of the simpler relationships, primarily related to metadata on the publications themselves and some higher-level aspects of the materials discussed within them. The prompt, shown in Figure 2, contains the query and was passed to the LLM along with the two files. We used both Llama 3.21and Claude 3.5 Sonnet2with oneshot prompting for our tests. Multiple attempts were made with the Llama model with slightly varying prompts each time. The results were consistently lackluster with the model populating the triples in the module file with incorrect information, not finding any relevant information, or just outputting the triples as they appear in the module file. The Claude model, on the other hand, performed much better at completing the assigned task, successfully extracting all data related to the triples found in the module except those that were not explicitly mentioned in the publication and required data to be inferred. Additionally, there were some instances of Claude generating extra triples that are not necessarily included in the original KG. Thus, ensuring that incorrect triples are not generated becomes just as important as ensuring that correct triples are generated. Conclusion In this paper, we propose the extension of a technique to populate KGs with LLMs to the realm of polymer science. Such a discovery, if successful, has the potential to expedite advancements by creating a single store of information to which new data can be added quickly. By being able to then draw needed data from one source, researchers can easily obtain the relevant context for an experiment, whether that entail various processes, experimental conditions, or polymer properties. Of course, as this is a fairly nascent approach to populating KGs, it becomes necessary to evaluate the accuracy of triples generated by an LLM as different kinds of data are incorporated. Future Work To determine the efficacy of our technique, we intend to execute it on more sample publications and evaluate the quality of the generated triples. It is possible that certain modifications to the technique may yield better results, however, especially when a focus is placed on more intricate and niche data. For example, since the publications can be long, an LLM may not be able to store the entire paper in its context window, limiting the effective generation of triples. In this case, it would be wise to segment the publications and pass these portions to the LLM one at a time, generating triples for each division and ultimately taking their union. While this process would be slower, it would still be faster than the traditional manual approach while potentially yielding better results. Furthermore, we will extend our approach beyond NanoMine, determining its effectiveness for similar tools. With resources like CRIPT focusing on modeling unique kinds of data like process and computational information, it becomes necessary to analyze the capability of LLMs to extract and populate KGs with it. Thus, in this work, we proposed the foundations of our overarching goal of developing a framework for this task that can be extended to a variety of polymer data. 1https://www.llama.com 2https://www.anthropic.com/claude Figures Figure 1: Graphical Representation of Proposed Methodology Given the attached polymer science-related publication and ontology module, extract data from the publication that satisfies the triples in the module file. Each entry in the module file has the format predicate(subject, object). For example, if the author of the publication is John Doe, the output triple should be hasAuthor(Article, John Doe). In these triples, the subject is primarily used as an identifier and does not need to be replaced with instance-level data. The object does, however, need to be replaced with instance-level data. Some of the provided triples may need to be used multiple times while others may not need to be used at all. Do NOT just output the triples as they appear in the module file. The module file just contains templates for the triples that need to be populated with actual data. Figure 2: Prompt Passed to LLM References Brinson, L. C.; Deagen, M.; Chen, W.; McCusker, J.; McGuinness, D. L.; Schadler, L. S.; Palmeri, M.; Ghumman, U.; Lin, A.; and Hu, B. 2020. Polymer Nanocomposite Data: Curation, Frameworks, Access, and Potential for Discovery and Design. ACS Macro Letters. Publisher: American Chemical Society. chemnetbase. 2024. CHEMnetBASE. https://poly.chemnetbase.com/polymers/PolymerSearch.xhtml? Ellis, B.; and Smith, R., eds. 2008. Polymers: A Property Database, Second Edition. Boca Raton: CRC Press, 2 edition. ISBN 978-0-429-12768-7. Hitzler, P. 2021. A review of the semantic web field. Commun. ACM, 64(2): 76–83. Hogan, A.; Blomqvist, E.; Cochez, M.; d’Amato, C.; de Melo, G.; Gutierrez, C.; Kirrane, S.; Gayo, J. E. L.; Navigli, R.; Neumaier, S.; Ngomo, A. N.; Polleres, A.; Rashid, S. M.; Rula, A.; Schmelzeisen, L.; Sequeda, J. F.; Staab, S.; and Zimmermann, A. 2022. Knowledge Graphs. ACM Comput. Surv., 54(4): 71:1–71:37. Norouzi, S. S.; Barua, A.; Christou, A.; Gautam, N.; Eells, A.; Hitzler, P.; and Shimizu, C. 2025. Ontology Population using LLMs, chapter To Appear. IOS Press. Available on arxiv https://arxiv.org/abs/2411.01612. Noy, N. F.; Gao, Y.; Jain, A.; Narayanan, A.; Patterson, A.; and Taylor, J. 2019. Industry-scale knowledge graphs: lessons and challenges. Commun. ACM, 62(8): 36–43. Otsuka, S.; Kuwajima, I.; Hosoya, J.; Xu, Y.; and Yamazaki, M. 2011. PoLyInfo: Polymer Database for Polymeric Materials Design. In 2011 International Conference on Emerging Intelligent Data and Web Technologies, 22–29. Roy, M.; Nelson, J. K.; MacCrone, R. K.; and Schadler, L. S. 2007. Candidate mechanisms controlling the electrical characteristics of silica/XLPE nanodielectrics. Journal of Materials Science, 42(11): 3789–3799. Shimizu, C.; and Hitzler, P. 2024. Accelerating Knowledge Graph and Ontology Engineering with Large Language Models. ArXiv:2411.09601 [cs]. Walsh, D. J.; Zou, W.; Schneider, L.; Mello, R.; Deagen, M. E.; Mysona, J.; Lin, T.-S.; de Pablo, J. J.; Jensen, K. F.; Audus, D. J.; and Olsen, B. D. 2023. Community Resource for Innovation in Polymer Technology (CRIPT): A Scalable Polymer Material Data Structure. ACS Central Science, 9(3): 330–338. Publisher: American Chemical Society. Zhao, H.; Li, X.; Zhang, Y.; Schadler, L. S.; Chen, W.; and Brinson, L. C. 2016. Perspective: NanoMine: A material genome approach for polymer nanocomposites analysis and design. APL Materials, 4(5): 053204. Zhao, H.; Wang, Y.; Lin, A.; Hu, B.; Yan, R.; McCusker, J.; Chen, W.; McGuinness, D. L.; Schadler, L.; and Brinson, L. C. 2018. NanoMine schema: An extensible data representation for polymer nanocomposites. APL Materials, 6(11): 111108.