User-oriented exploration of semi-structured datasets

User-oriented exploration of semi-structured datasets Nelly Barret 4th year PhD student Supervised by Ioana Manolescu Inria Saclay and Institut Polytechnique de Paris January 25, 2024 Nelly Barret (Inria) Semi-structured Data Exploration January 25, 2024 1 / 40 Context: data is the new gold (1/3) Nelly Barret (Inria) Semi-structured Data Exploration January 25, 2024 2 / 40 Context: data is the new gold (1/3) Nelly Barret (Inria) Semi-structured Data Exploration January 25, 2024 3 / 40 Context: data is the new gold (2/3) Our digital world comes: In various contexts: science, health, political life At various scales: home, city, country, world By different actors: scientists, businesses, policy makers With different needs, constraints, abilities We are overwhelmed by (raw) data, we need: Data-driven applications Data journalism Knowledge graphs Artificial “intelligence” ... Nelly Barret (Inria) Semi-structured Data Exploration January 25, 2024 4 / 40 Context: data is the new gold (3/3) Very heterogeneous data: Mainly RDF (1K datasets in the LODC) Also: XML, JSON, relational, Property Graph... Detection of entities of interest: People, Place, email, ... With heterogeneous data, users need: 1Auniform integration, view over the data 2Efficient algorithms and applications 3A global understanding, description 4Interesting entity connections Nelly Barret (Inria) Semi-structured Data Exploration January 25, 2024 5 / 40 Create a unique data graph “A uniform integration, view” Angelos Anadiotis Oana Balalau Ioana Manolescu et al... IPP, EPFL Inria, IPP Inria, IPP INSEC, ... Heterogeneous input Loading Collection graph Abstra PathWays New features Studio Nelly Barret (Inria) Semi-structured Data Exploration January 25, 2024 6 / 40 Graph construction Ingest any dataset into a directed graph (•,→) Extract named entities, NEs, from the graph values (◦,99K): Temporal: date , time reference Web: URI, email address , hashtag, Twitter citation Complex entities: People , Place , Organization Nelly Barret (Inria) Semi-structured Data Exploration January 25, 2024 7 / 40 Create a compact representation of the data graph “Efficient algorithms and applications” Heterogeneous input Loading Collection graph Abstra PathWays New features Studio Nelly Barret (Inria) Semi-structured Data Exploration January 25, 2024 8 / 40 A uniform view of data formats Each data format has its own specificities: RDF (URIs) JSON (labels) XML (ID-IDREF) PG (edge attrs) But, we encode the same logic: Record: piece of data, an object Value: record with no children Same-kind records: schema or “intuitive” order Relationship: how records relate Nelly Barret (Inria) Semi-structured Data Exploration January 25, 2024 9 / 40 Collections weights, boundaries and graph updates Collection weight wdesck wleafk wDAG wPageRank wdwPageRank Nelly Barret (Inria) Semi-structured Data Exploration January 25, 2024 15 / 40 Collections weights, boundaries and graph updates Collection weight wdesck wleafk wDAG wPageRank wdwPageRank Boundary bounddesc boundleaf boundDAG boundflood boundacyclic−flood Nelly Barret (Inria) Semi-structured Data Exploration January 25, 2024 16 / 40 Collections weights, boundaries and graph updates Collection weight wdesck wleafk wDAG wPageRank wdwPageRank Boundary bounddesc boundleaf boundDAG boundflood boundacyclic−flood Graph update updateboolean updateexact Nelly Barret (Inria) Semi-structured Data Exploration January 25, 2024 17 / 40 Find relationships between main collections Possible relationships The set of relationships connecting a pair of collections is the set of their paths. paper →wB →author paper →pIn →conf author →hW →paper conf →inv →author Nelly Barret (Inria) Semi-structured Data Exploration January 25, 2024 18 / 40 The final output in Abstra https://team.inria.fr/cedar/projects/abstra/ Nelly Barret (Inria) Semi-structured Data Exploration January 25, 2024 19 / 40 Enumerate entity paths “Interesting entity connections” Nelly Barret Antoine Gauquier Jia Jean Law Ioana Manolescu Inria, IPP IMT IPP Inria, IPP Heterogeneous input Loading Collection graph Abstra PathWays New features Studio Nelly Barret (Inria) Semi-structured Data Exploration January 25, 2024 20 / 40 PathWays: find interesting connections in the data Problem statement How to interactively explore entity connections in heterogeneous datasets? 1No query writing, nor prior knowledge 2Atabular, high-level output, easy to grasp for NTUs 3Do it efficiently even if the data graph is large =⇒Connect named entities (People, Places, ...) in and across datasets. Keyword Graph Reachability PathWays search query query No query writing X× × X Tabular output ∼ ∼ ×X Efficient ×X X X Nelly Barret (Inria) Semi-structured Data Exploration January 25, 2024 21 / 40 PathWays: find interesting connections in the data Problem statement How to interactively explore entity connections in heterogeneous datasets? 1No query writing, nor prior knowledge 2Atabular, high-level output, easy to grasp for NTUs 3Do it efficiently even if the data graph is large =⇒Connect named entities (People, Places, ...) in and across datasets. Keyword Graph Reachability PathWays search query query No query writing X× × X Tabular output ∼ ∼ ×X Efficient ×X X X Nelly Barret (Inria) Semi-structured Data Exploration January 25, 2024 21 / 40 Scenario and terminology Adata (entity) path is a path in the data graph Acollection (entity) path is a path in the collection graph The evaluation of a collection path leads to a set of data paths Nelly Barret (Inria) Semi-structured Data Exploration January 25, 2024 22 / 40 Collection (entity) path enumeration (τ1, τ2); max path length; non-specific connections Nelly Barret (Inria) Semi-structured Data Exploration January 25, 2024 23 / 40 Data path results in PathWays https://team.inria.fr/cedar/projects/pathways/ Nelly Barret (Inria) Semi-structured Data Exploration January 25, 2024 30 / 40 Quick overview of experiments On widely-used open data formats: JSON, RDF, XML and PG. Abstra PathWays •User study •# paths: 0 to very high •Comparison to schemas •Filter spurious paths (path interestingness) Nelly Barret (Inria) Semi-structured Data Exploration January 25, 2024 31 / 40 Future work, takeaways and open questions Heterogeneous input Loading Collection graph Abstra PathWays New features Studio Nelly Barret (Inria) Semi-structured Data Exploration January 25, 2024 32 / 40 Future work: Studio Studio: a data lake for ingesting, querying, cleaning and understanding heterogeneous data French media are interested (DataJournos, CFI) Nelly Barret (Inria) Semi-structured Data Exploration January 25, 2024 33 / 40 Future work: Studio Studio: a data lake for ingesting, querying, cleaning and understanding heterogeneous data Nelly Barret (Inria) Semi-structured Data Exploration January 25, 2024 34 / 40 Future work: Studio Studio: a data lake for ingesting, querying, cleaning and understanding heterogeneous data Nelly Barret (Inria) Semi-structured Data Exploration January 25, 2024 35 / 40 Future work: Studio Studio: a data lake for ingesting, querying, cleaning and understanding heterogeneous data Nelly Barret (Inria) Semi-structured Data Exploration January 25, 2024 36 / 40 Future work: Studio Studio: a data lake for ingesting, querying, cleaning and understanding heterogeneous data Nelly Barret (Inria) Semi-structured Data Exploration January 25, 2024 37 / 40 Future work: Studio Studio: a data lake for ingesting, querying, cleaning and understanding heterogeneous data Nelly Barret (Inria) Semi-structured Data Exploration January 25, 2024 38 / 40 Takeaways and open questions Abstra: a dataset abstraction system for heterogeneous data PathWays: an entity-focused exploration system Studio: a user-oriented data lake for data exploration Abstra PathWays Studio EDBT 2024 ADBIS 2023 CoopIS 2023 Nelly Barret (Inria) Semi-structured Data Exploration January 25, 2024 39 / 40