Review of porosity uncertainty estimation methods in computed tomography dataset

Measurement Science and Technology Meas. Sci. Technol. 32 (2021) 122001 (17pp) https://doi.org/10.1088/1361-6501/ac1b40 Topical Review Review of porosity uncertainty estimation methods in computed tomography dataset Victory A J Jaques1, Anton Du Plessis2, Marek Zemek1, Jakub Šalplachta1, Zuzana Stubianov´ a1, Tom´ aš Zikmund1,∗and Jozef Kaiser1 1CEITEC—Central European Institute of Technology, Brno University of Technology, Purkyˇ nova 123, Brno 612 00, Czech Republic 2Research Group 3D Innovation, Stellenbosch University, Stellenbosch 7602, South Africa E-mail: [email protected].cz Received 15 June 2021, revised 28 July 2021 Accepted for publication 6 August 2021 Published 23 August 2021 Abstract X-ray computed tomography is a common tool for non-destructive testing and analysis. One major application of this imaging technique is 3D porosity identification and quantification, which involves image segmentation of the analysed dataset. This segmentation step, which is most commonly performed using a global thresholding algorithm, has a major impact on the results of the analysis. Therefore, a thorough description of the workflow and a general uncertainty estimation should be provided alongside the results of porosity analysis to ensure a certain level of confidence and reproducibility. A review of current literature in the field shows that a sufficient workflow description and an uncertainty estimation of the result are often missing. This work provides recommendations on how to report the processing steps for porosity evaluation in computed tomography data using global thresholding, and reviews the methods for the estimation of the general uncertainty in porosity measurements. Keywords: computed tomography, porosity evaluation, uncertainty estimation, results comparison, segmentation , global thresholding (Some figures may appear in colour only in the online journal) 1. Introduction Porous materials and samples are common in a wide range of scientific fields. For instance, permeability and reservoir characteristics of porous rocks are useful parameters in the oil and ∗Author to whom any correspondence should be addressed. Original Content from this work may be used under the terms of the Creative Commons Attribution 4.0 licence. Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI. gas industry and related fluid research [1], whereas in paleontology, the shape and volume of pores are used to identify fossils [2]. Porosity analysis is also widely used in engineering [3], manufacturing [4,5], or material development [6] as an indicator of a material’s strength [7]. Pores in metals often indicate crack initiation locations in cyclic loading applications, and they even influence static strength and ductility of materials, making non-destructive porosity testing valuable for quality control purposes [8,9]. Generally, porosity refers to a measurement of the presence of voids within a sample. Allaby [10] defines pores as voids that can be empty or filled with trapped gas and/or fluids, and surrounded by any type of material. Using this definition, 1361-6501/21/122001+17$33.00 1 © 2021 The Author(s). Published by IOP Publishing Ltd Printed in the UK Meas. Sci. Technol. 32 (2021) 122001 Topical Review porosity is the volume of all pores present in a sample [10]. It is commonly expressed as a percentage of empty space within the total volume of an object [11]: Porosity [%] = Volume of pores Volume of solid (incl.pores)×100.(1) The shape and volume of pores is connected to the material’s formation, temperature, environment and type [12]. Pores can be cylindrical, slits, conical, spherical, ink bottle-like, and interstitial [13], but they can also feature more complex shapes. Pores can form networks that may or may not be accessible from the outside of the object (open porosity), or they can be isolated (closed porosity) [14]. Pore characteristics influence bulk density, mechanical strength, and thermal conductivity of objects [8]. Various methods can be used to analyze the porosity of a sample. The choice of a method depends on the type of porosity present in the particular sample, as well as on other parameters. X-ray computed tomography (CT) is among the most widespread and broadly applicable methods. CT is an imaging modality which is based on the absorption of x-rays in materials [15], and makes non-destructive three-dimensional analysis of samples and their internal structures possible [16]. The output of a typical CT measurement is a set of cross-sectional slices stacked in a 3D volume (figure 1Steps 1 and 2). This forms a grid of voxels, which are volumetric elements with a specific gray value determined by the density and atomic number of materials contained within, and the x-ray energy used [17]. The edge length of a voxel influences the best possible resolution of a measurement. The scanning, tomographic reconstruction, and subsequent analysis of a CT dataset are all potential sources of uncertainty and variation between measurements (figure 1). The various error sources in figure 1influence the quality of the resulting images, which in turn has a major impact on pore segmentation and the subsequent porosity measurement [18]. In this case, quality refers to the combination of noise level, contrast between material and pores, and sharpness of edges between the two regions [19]. As an example of the complex influences of the image quality on porosity assessment, image denoising may decrease the amount of noise falsely detected as pores, but it may also cause some smaller pores to be blurred and therefore missed, skewing the results [20,21]. Due to this, it is important to report on the various sample, measurement, and processing parameters used in a porosity study, and to take uncertainty into account [18]. Data used in studies can be shared through data repositories such as the GigaScience Database, see Goodman [22], as in the case of Du Plessis et al [23]. The workflows and protocols can also be shared on services like Protocols.io [24]. Characterization of porosity in a CT dataset is directly related to the segmentation procedure, the partitioning of a volume into two or more separate sections (e.g. material and voids). Segmentation is based on the intrinsic characteristics of voxels or regions of the volume, such as gray values, edges, or texture [28]. Figure 1. The creation of a 3D dataset from a given object using x-ray CT has three steps: (1) the scan or measurement, (2) reconstruction, and (3) segmentation. Then, analysis (4) of the volume can be done. Errors that occur during step 1 can lead to tomographic artifacts (discrepancies between an object and its image). In steps 2–4, other types of errors can lead to uncertainty in the final analysed results. The red lines show the focus of this work. Figure inspired by Villarraga-Gómez et al [25], Smet et al [26] and Hiller and Reindl [27]. One of the most widely used segmentation methods is thresholding [29], where voxels are separated into distinct categories based on a threshold set for one or more of the characteristics mentioned above. Thresholding can be global or locally adaptive [30,31]. The latter is mainly used for complex objects, where the optimal threshold value may change throughout the dataset [32]. On the other hand, global thresholding defines a single threshold value for the entire dataset, influencing all further analysis and interpretations [29]. For its simplicity, ease of use, and ease of access, global thresholding is the go-to segmentation method in many CT data analyses. Porosity analysis may yield different results based on the chosen method of segmentation and researcher input [33]. The conclusions drawn from a study can be ambiguous if the methodology used for segmentation is not clearly described. This has caused some researchers to call for standardisation [18,34–37]. It is commonplace to describe measurement parameters used for data acquisition in porosity studies, such as the tube voltage and current, and the voxel size. However, to ensure reproducibility, the applied segmentation approach should be described and the uncertainty of the results should be estimated too. Otherwise, the 2 Meas. Sci. Technol. 32 (2021) 122001 Topical Review Figure 2. The relationship between accuracy and precision shown using multiple measurements (small red dots) and a reference value (big green dot). Precision estimation can be determined with several measurements. A standalone CT measurement (small blue dot) has no reference value (the green dot is unknown) and different a different approach must be adopted for precision and uncertainty estimation. This figure was inspired by Pospíšil and Ludvík [48], and Taylor [49]. reliability of these studies runs the risk of being disputed [26,29,38,39]. Measurements are typically expressed with an error ratio, a confidence interval, or a standard deviation presented as a±value. This value is based on the cumulative effect of device, measurement and processing errors [27,40]. Knigge [41] states that a measurement does not need to be accurate (close to the reference value), but its precision should be known (figure 2). The accuracy refers to the closeness between a measured value and a reference value [42,43]. A precise measurement is not necessarily close to the reference value, but it has little variability when repeated. Thus, precision quantifies the reproducibility and level of uncertainty of a measurement. International metrological standards for estimating the measurement uncertainty exist [44–46], but they cannot be directly applied for the purposes discussed here, as they do not provide specific guidelines for the uncertainty of 3D CT data segmentation [47]. Two causes of errors [42] can affect the precision and accuracy of a result, namely: systematic error (affects all measurements in a similar manner, observable through repeated measurements [50,51]), and random error (mainly caused by operator errors, and affects individual measurements and thus the measurement precision [52]). Acquisition, hardware, and reconstruction discrepancies all significantly influence the final porosity evaluation. A complex analysis of uncertainties in CT measurements is the domain of metrology and of some industrial fields, where calibrated devices or calibration methodologies are used [51,53,54]. Without a ground truth, which is a measurement that is considered to have the exact true value, accuracy of a measurement cannot be assessed [55]. In terms of precision, the influence of uncertainties stemming from the measurement process is a complex issue, and it is already the subject of thorough research [40]. In contrast, uncertainties associated with segmentation are seldom referenced or explained thoroughly within current literature. This work offers an overview of CT data segmentation methods that use global thresholding and are commonly used for porosity analysis. Crucial aspects of the segmentation process, which should be disclosed in studies, are identified in the section ‘Thresholding and reproducibility’. The need for a thorough description of approaches used in studies is supported by a systematic review of recent relevant literature. Methods for the uncertainty estimation of porosity analysis in CT data are discussed in the section ‘Uncertainties of CT data segmentation’. Due to financial and time constraints, researchers may often only have access to a single CT dataset for analysis [9], so particular attention is paid to those methods that can be used in these situations. Uncertainty estimation is still needed in such cases, but the approaches to perform it may be less obvious. We hope to provide a practical overview of the possibilities available to researchers to ensure the reproducibility of their results. 2. Thresholding and reproducibility Global thresholding methods can be divided into manual, semi-automatic, and automatic [56], depending on the extent of operator involvement in the selection of the threshold value. Automatic algorithms calculate a threshold objectively based on the characteristics of the input dataset, such as voxel grayscale values and features of the image histogram (figure 3). Common automatic algorithms include minimum error thresholding [57], Otsu’s method [58], valleyemphasis [59], optimal thresholding [60], histogram concavity analysis [61], iterative thresholding (isodata method) [62], entropy-based thresholding [63], Bayesian thresholding [64], and others. The results of these may be used directly or further fine-tuned manually. Manual threshold selection is subjective and observer-dependent, and it is usually based on visualizing the segmentation result on a slice of the CT dataset and tuning it until it is satisfactory [35]. The potential human bias inherent in manual thresholding may lead to larger differences between data segmented by different operators. Despite this, manual thresholding is still very common due to its simplicity. One of the simplest global thresholding methods is ISO50, which sets a threshold at the mean of two extreme peak values in the grayscale histogram of a dataset [65]. Results of this method tend to be satisfactory when the analysed histogram is bior multi-modal (figure 3) [66]. However, Horner et al [67] showed that ISO50 might be influenced by local variations in the image, in which case the threshold should be modified accordingly. It is also challenging to use it with low porosity values because the histogram of 3 Meas. Sci. Technol. 32 (2021) 122001 Topical Review Figure 3. Tomographic images with a uni- (a), (d), bi- (b), (e), and multimodal (c), (f) grayscale histogram. The histograms show thresholds set using Otsu’s method (dark dashed red line) and ISO50 (light green line). The multimodal histogram in (f) shows multi-level thresholding, which divides the dataset into three parts. Boundaries of segmented areas are shown in (a)–(c) using an outline with colors corresponding to the two thresholds. The slices show datasets measured in our laboratory: (a) and (b) are slices of a chalk sample, while (c) is an image of a seed. such data may lack a clear peak corresponding to pore values. Otsu’s thresholding [58] is another simple method, and along with Kittler’s thresholding [57], it is one of the most used algorithms for porosity segmentation (table 2). Similar to ISO50, the results of Otsu’s method are affected by the modality of the histogram [29]. Algorithms such as Otsu’s method can also be used for multilevel thresholding, which may be used to classify datasets into pores, grains and high-density inclusions (figures 3(e) and (f)) [68,69]. In cases where a dataset’s histogram is approximately unimodal, the algorithms mentioned above are likely to perform poorly, and a threshold can be set using probability-based algorithms [59]. There is no consensus in the scientific community about which thresholding method is ideal for porosity analysis in CT data. In fact, the wide selection of published specialized algorithms in various fields suggests that the effectiveness of an algorithm changes with the dataset type and application [29,37,70,71]. There is, however, an agreement that the reproducibility of manual segmentation is lower than that of an automated or trainable procedure, as remarked by Kalasov´ a et al [72]. A round robin test, which is porosity in a specific scan evaluated by multiple operators, was reported by Du Plessis et al [18]. This study found a good agreement in the qualitative pore distribution assessment of ten operators, but quantitative results varied significantly, partly due to the low porosity content in the test sample used. Works of Zikmund et al [11,73] and Baveye et al [37] also feature a comparison of various manual segmentation strategies in addition to algorithm-based ones. Significant discrepancies were found both within and between these groups. Therefore, automated methods do not ensure an accurate or reliable result either, as the choice of algorithm significantly impacts the threshold value and the obtained results (figure 3) [26,29,37,38,74,75]. Regardless of the segmentation method used, porosity analysis needs to be reproducible for a reliable inter-study comparison of results. This means that parameters of the segmentation process should be described and explained thoroughly, as remarked in multiple works [18,34–36,73]. 2.1. Analysis of published porosity methodologies To assess the trends regarding reproducibility in the current literature, we chose 53 articles from geosciences and material sciences (industry, engineering, metrology, agriculture, and cultural heritage) that deal with porosity analysis in CT data, and analysed their segmentation methodologies (table 2). The articles were selected through Google Scholar using the keywords CT, Porosity, Segmentation, Global Thresholding, Quantitative analysis, and Uncertainty evaluation. Our selection was narrowed down to articles that were cited at least once. Twenty of the 53 articles (57%) featured either no description of the segmentation procedure, or their description was not sufficient for their results to be reliably reproducible. Ten of these articles (19%) had a description but was not accompanied by any visualization of the histogram and 4 Meas. Sci. Technol. 32 (2021) 122001 Topical Review threshold value. Out of the remaining 23 articles (43%), ten (19%) featured a sufficient description and provided an example CT slice showing segmentation results, along with either a grayscale histogram of the slice, or an estimation of the uncertainty of the results. These results can be considered reproducible, but not optimally so. Only 13 (25%) of the surveyed articles disclosed all parameters needed to ensure measurement reproducibility, including a description, an example slice along with its histogram, an uncertainty estimation or the threshold value, and a mention of the software used. Over the observed period (1992–2020), the overall thoroughness of thresholding methodology descriptions seems to not have changed. There are no clear distinctions between methodology descriptions in the various fields of study, except that works dealing with soil porosity (21% of the studied articles) are more prone to inter-study comparison, and therefore they generally include a more thorough definition of the parameters used for thresholding. The examined studies are mostly based on a single segmentation method (43% of the examined articles), followed by comparison (28%) and combination (19%) of segmentation methods. Otsu’s method is the most commonly used (36%), both on its own and in comparison to, or in combination with, other techniques. It is closely followed by manual global thresholding segmentation (34%). This is fairly consistent across the fields, which shows that the choice of thresholding method is probably mainly dependent on the operator experience and sample type. A mention of the software used, which is present in 68% of the selected articles, can aid in the reproducibility of results. The most commonly mentioned software in table 2includes VG Studio (15%; Volume Graphics GmbH, Heidelberg/D), ImageJ/Fiji (15%; National Institutes of Health, Bethesda, Maryland/US and LOCI, University of Wisconsin-Madison, Madison, Wisconsin/US), and Avizo (13%; ThermoFisher Scientific, Waltham, Massachusetts/US). We have observed that industrial and engineering fields tend to utilize VG Studio and Avizo, whereas the selection of software used in geosciences and geology is broader. This may be because the complex samples in geosciences are difficult to process in general purpose image processing software, forcing researchers to use more specialized solutions. 2.2. Reproducibility Analysis of table 2and previous work done by Taina et al [76], Lievers and Pilkey [77], and Iassonov et al [74] has led to the identification of six major parameters of the thresholding process, which should be included in a study to ensure result reproducibility. These parameters include: histogram shape, an image of a slice from the dataset showing pore determination, the chosen threshold value, description of the thresholding procedure, and the name and version of the software used. The parameters are also listed in table 1[74,76,77]. The most common thresholding algorithms operate on the grayscale histograms of images, and manually selected thresholding is often partly determined by the histogram shape, too. Therefore, including an example histogram in a Table 1. Thresholding parameters that ensure methodology reproducibility, listed in the order of their importance. For the sake of clarity, abbreviations used in table 2are included here, as well. Thresholding parameters 1 [h] Histogram 2 [s] Slice showing pore determination 3 [t] Threshold grey-level value 4 [d] Manual: Reason for the threshold choice (visual or material-dependent, or with reference) Semi-automated: Algorithm choice with reference and modifications Automated: Implemented software algorithm title and values chosen 5 [u] Uncertainty estimation (±) 6 Software +version study can aid its reproducibility. Likewise, including a slice illustrating the final segmentation and the threshold value provides a valuable visual and numeric reference. Documenting the software used, along with its version, may also be relevant for reproducibility. In some types of software, the internal workings of the algorithms may not be accessible to the end user (these algorithms are called black boxes). Due to this, analyses carried out in different software may be hard to compare, and the name and version of the software used for a given study becomes relevant. An important but often omitted piece of information (table 2) in terms of reproducibility is the estimation of the porosity result uncertainty. Porosity measurement in CT is directly related to the segmentation process, which affects the size, shape, distribution, and total volume of pores [71]. Different segmentation methods may lead to an overor underestimation and increased uncertainty of porosity (figure 3) [33]. An uncertainty range will therefore help other researchers assess whether their reproduced results differ significantly from the outcomes of the original study. However, uncertainty of porosity in CT data may not always be straightforward to estimate. The next section goes over a selection of possible approaches to perform this estimation in a variety of scenarios. 3. Uncertainties of CT data segmentation Several studies were conducted concerning the uncertainty estimation of both manual and automatic threshold selection in CT datasets [51,53]. These works describe various procedures for estimating the uncertainty of porosity analysis (table 3), which can be separated into empirical, analytical, and sensitivity approaches. All these procedures mitigate different kinds of errors in the final uncertainty estimation. Empirical procedures are based on the comparison of a CT dataset with a reference. This reference may take the form of a calibrated workpiece or a measurement conducted using another calibrated method. Such procedures are demanding, costly, and impractical for very complex or non-homogeneous samples. Since these methods require a reference measurement, they will increase the error sources of step 1 in figure 1 (mostly systematic errors) but will reduce operator errors. 5 Meas. Sci. Technol. 32 (2021) 122001 Topical Review Table 2. A list of articles that use CT for porosity analysis in different fields, highlighting the amount of information provided concerning the threshold selection. Author Year Threshold selection based on Software Threshold parameters Field of research Material Lin et al [78] 2016 Cp—Otsu [58], Liao et al [69], Watershed Avizo d, s, h, u Geosciences Limestone Fishman et al [79] 2010 Otsu [58] Fiji d, s, h, u Industry Fibers Freire-Gormaly et al [80] 2014 Cb—Otsu [58], Ji et al [81] Fiji d, s, h, u Geology Limestone, dolomite Abera et al [29] 2017 Cp—Kittler and Illingworth [82], Otsu [58], Kapur et al [63], Johannsen and Bille [83], Pun [84] Matlab +Image-Pro +Standalone segmentation software d, s, h, u Engineering Glass beads, concrete, sand Sleutel et al [36] 2008 Manual Morpho+tool Vlassenbroeck et al [85] d, s, h, u Geosciences Soil Baveye et al [37] 2010 Cp—Otsu [58], Iterative, entropy, IK, manual, Sezgin and Sankur [31] OTIMEC d, s, h, u Geosciences Soil Rozenbaum et al [86] 2012 Otsu [58] VG Studio d, s, h, u Geosciences Soil Borges de Oliveira et al [87] 2016 Cp—ISO-50, advanced local-mode, region growing, greyscale, ROI VG Studio MAX 2.2.6 d, s, h, u Metrology Polymer Zikmund et al [73] 2019 Cp—Otsu [58], K-means RIdler et al [62], manual VG Studio MAX 3.0 d, s, h, u Industry Additive manufacturing Hermanek et al [66] 2019 Cb—ISO-50, ROI Iterative optimization VG Studio MAX 3.0 (Defect analysis module) d, s, h, u Metrology Aluminium Smet et al [26] 2018 Cp—Otsu [58], IK, gradient mask, porosity-based Beckers et al [88] MATLAB R2015a, AWIK (custom) d, s, h, u, t Geosciences Soil Wang et al [89] 2011 Cp—Sahoo et al [90], Otsu [58], Ridler et al [62], IK, entropy, iterative R, 3DMA-Rock d, s, h, u, t Geosciences Rock Gantzer and Anderson [91] 2002 Cb—‘Threshold feature’, ‘Measure’ tool ImageJ 1.20 d, s, h, t Agriculture Soil Luo et al [92]2010 Maximum entropy Jassogne et al [93] ImageJ 1.39 d, s, u Geosciences Soil Thompson et al [34] 1992 Manual No mention d, s, u Geosciences Soil Sander et al [94] 2008 Manual SlicerDicer d, s, u Agriculture Soil Ishutov et al [1] 2015 Manual 3Dvis, paraview d, s, h Geosciences Rock Ji et al [81] 2012 Otsu [58] ImageJ d, s, h Geology Limestone Coker et al [95] 1996 Edge-based Cocker et al [96] No mention d, s, h Geology Sandstone Taud et al [11] 2005 Cp—Grey-level, manual, k-means Riedler et al [62] No mention d, s, h Engineering Rock Schlüter et al [97] 2010 Cb—Yanowitz and Bruckstein [98], Vogel and Kretzschmar [99] No mention d, s, h Geosciences Soil Andrä et al [75] 2013 Cp—VSG, Kongju, Stanford No mention d, s, h Geosciences Sandstone, carbonate, sphere pack Zhang et al [100] 2017 Otsu [58] ImageJ d, s, t Geosciences Shale (Continued.) 6 Meas. Sci. Technol. 32 (2021) 122001 Topical Review Table 2. (Continued.) Author Year Threshold selection based on Software Threshold parameters Field of research Material Heinzl [101] 2007 Cp—Otsu [58], Manual, Watershed, Calypso Carl Zeiss Calypso d, u Industry Metal Moroni and Petr` o [33] 2016 Cp—ISO-50, manual No mention d, u Metrology Calibrated balls Hermanek and Carmignato [102] 2017 ISO-50, local adaptative VG Studio MAX 2.2.6 (Defect analysis module ‘Only threshold’), Volume Player, iVolume, d, u Metrology Multi-material Salarian and Toyserkani [103] 2018 Cb—Greyscale, morphological operation, manual Dragonfly Pro v3.1 d, s Industry Aluminium Manahilo [104] 2013 Cp +Cb—Otsu [58], Region-based, iterative minimization, manual, watershed Image-Pro Plus d, s Geosciences Soil, sand, glass beads Iassonov and Tuller [105] 2010 Otsu [58] No mention d, s Geosciences Bentonite, glass beads Alyafei et al [68] 2015 Otsu [58] No mention d, s Geology Sandstone, limestone Rezaei et al [38] 2019 Cp—Kittler and Illingworth [57], Otsu [58], Ridler et al [62], three locally adaptative (Niblack 1986, Sauvola 2000, Bernsen 1986) No mention d, s Engineering Limestones, dolomite Kumar et al [106] 2012 Cb—Automatic histogram-based threshold (10% sensitivity manual), ‘Defect detection’ module VG Studio MAX 2.0 (Defect detection module) d, s Engineering Foam Pavan et al [107]. 2018 Otsu [58] VG Studio MAX 2.2 (Defect analysis module ‘Only threshold’) d, s Industry Laser sintered PA12 Rogasik et al [108]. 2003 Manual No mention d Geosciences Soil Andrä et al [75]2013 Cp—Otsu [58], VGL (manual + watershed), Kongju (single threshold), stanford No mention d Geosciences Sandstone, Carbonate Madra et al [109] 2014 Cp—Manual, thresholding, learning algorithm (trainable WEKA) Fiji p, s, u Engineering Plastic +glass fibers Fusi and Martinez-Martinez [110] 2013 Manual Avizo p, s, t Engineering Dolostone, limestone, marble Spierings et al [111] 2011 Manual VG Studio MAX 2.0 p, u Industry Metal Skorpa et al [112] 2017 ‘Interactive thresholding’ based on greyscale Avizo p, s Geosciences Chalk, cement, sandstone Blunt et al [113] 2013 Otsu [58] No mention p, s Geology Limestone, carbonate, sandstone Vanderesse et al [114] 2011 Manual Avizo p Industry Cast Al-alloy Mahanta et al [115] 2020 Cb—Greyscale Andrä et al [75], Watershed Avizo 9.0 p Geosciences Sandstones (Continued.) 7 Meas. Sci. Technol. 32 (2021) 122001 Topical Review Table 2. (Continued.) Author Year Threshold selection based on Software Threshold parameters Field of research Material Wevers et al [116] 2012 Cb—Otsu [58] , Manual ImageJ, CTAn, Avizo p Engineering Multi-material Arns et al [117] 2019 Cb—Manual, Active contour from Sheppard et al [118] No mention p Geosciences Limestone Pearce et al [119] 2016 Greyscale Golab et al [120] QEMSCAN p Geosciences Rock Bugani et al [121] 2007 ROI CTAn package n, s, u Cultural heritage Limestone Müller et al [122] 2012 Cb—Automatic ‘optimal’, local adaptative No mention n, s Metrology Aluminium, polymer Jiang et al [123] 2013 No mention Pore Analysis Tools (PAT, custom) n, s Geosciences Rock Vrålstad and Skorpa [124] 2020 No mention Avizo n Engineering Cement Maire et al [125] 2007 Cp—Greyscale, region growing No mention n Industry Ceramic Nicoletto et al [126] 2010 No mention No mention n Engineering Al-alloy Pak et al [127] 2016 Jiang et al [123] No mention n Geosciences Carbonate Fintov´ aet al [128] 2010 No mention Original software n Engineering Al–Si alloys Cp: comparison; Cb: combination; Threshold choice method d: described, p: partially described, n: non described; h: histogram; s: slice; t: threshold value; u: uncertainty; ROI: region of interest; IK: indicator kriging [129]. 8 Meas. Sci. Technol. 32 (2021) 122001 Topical Review Table 3. An overview of uncertainty estimation procedures for porosity segmentation. The number of thresholds, objects, operators, and CT datasets required for each method are listed, but the listed values should serve only as a general guideline. The arrangement of the table follows that of the text below, with multi-dataset methods listed first, and single-dataset methods second. This table can be used to select an appropriate method for a particular situation by matching it with the parameters in columns ‘Sample’ to ‘ground truth’. Uncertainty estimation procedure Sample Object Dataset Operator Thresholding process Type Ground truth Reference EMP. Methods correlation Open porosity, destructible 1≥2 1 2 Comparison Yes [11,130] Calibrated object (CAD) Known shapes, distribution 2 2 1 2 Comparison Yes [51,53, 66,102, 131] Reference object Known reference’s porosity 2111Comparison Yes [131] ANAL. Inter-studies All ≥2≥2≥2≥1 Comparison No [132] Multiple scans All 1 ≥2 1 ≥2 Averaging No [133] Multiple location Homogeneous 1 1 1 1 Averaging No [134] Ground truth creation All 1 1 1 ≥3 Comparison Averaging Yes [135, 136] SENS. Multiple manual All 1 1 ≥2≥2 Averaging No [37] Manual ±n%All 1 1 1 1 ±n%Averaging No [73] Erosion/Dilation All 1 1 1 1 ±npixels Averaging No [137] Analytical approaches use various statistical concepts to enumerate uncertainty. They usually utilize multiple porosity measurements in some way, so they can be time-consuming and potentially expensive. This makes them unsuitable for large amounts of data. Despite this, analytical approaches are applicable to a large variety of samples. When this approach is used, data processing errors are increased, and statistical and operator errors are reduced. Sensitivity approaches are based on the operator’s behavior, knowledge of the dataset, and expectations. Here, an uncertainty of the operator’s measurement is estimated using some heuristics. Methods in this category can be unreliable if certain rules are not followed, but they are usually quick, cheap, and easy to apply on any dataset and in a wide range of software (table 3). Since sensitivity approaches are based on the experience of the operator and visualization of the data, they are expected to reduce systematic and analytical errors, but increase random errors [73]. It should be emphasized that the uncertainty estimated using any of these methods is relative, not absolute. Various methods in the three categories described above are suitable for different scenarios, depending on the number of samples, operators, time, and other resources available. The following text discusses the methods in table 3, and offers recommendations concerning their proper application. As the number of available datasets is likely to be a major factor when choosing an appropriate method for uncertainty estimation, the text is primarily divided into approaches for multiple datasets and for a single dataset. Despite their importance, approaches that require multiple datasets are described briefly, as they have already been exhaustively described in the literature. Single-dataset methods are often easier to apply and fit a wide range of datasets, but they are rarely explained in sufficient detail. For this reason, we describe those approaches more thoroughly. 3.1. Several datasets 3.1.1. Empirical approach. Correlation of results of various measurement methods for the same sample (figure 4(c); table 3) is a common approach. For example, Taud et al [11] and Robin et al [130] estimated the uncertainty of their porosity measurement by comparing the results of CT and helium injection measurements. Taud et al [11] found an uncertainty of about ±2%. This approach is straightforward and reliable, but the methods that are compared must be selected carefully, and the measurement must be clearly planned out beforehand. Additional demands are placed on the researchers who choose this approach, as they need both access to, and the knowhow for, multiple measurement methods. Different methods are suited for evaluating different types of porosity, making the comparison complex. The voxel size of a CT scan strongly influences measurement results, particularly in samples with a wide range of pore sizes (e.g. concrete). For more precise results, multiple CT scans might be required [138]. Similarly, other quantification methods are limited to specific ranges of pore sizes, making a direct comparison between methods challenging. Additionally, if a chosen method is destructive, the non-destructive nature of CT may no longer be an advantage. 9 Meas. Sci. Technol. 32 (2021) 122001 Topical Review [82] Kittler J and Illingworth J 1986 Minimum error thresholding Pattern Recognit. 19 41–7 [83] Johannsen G and Bille, J 1982 A threshold selection method using information measures Int. Conf. on Pattern Recognition, Proc. 6th (Munich) pp 140–3 [84] Pun T 1980 A new method for grey-level picture thresholding using the entropy of the histogram Signal Process. 2223–37 [85] Vlassenbroeck J, Dierick M, Masschaele B, Cnudde V, Hoorebeke L V and Jacobs P 2007 Software tools for quantification of x-ray microtomography at the UGCT Nucl. Inst. Methods Phys. Res. A580 442–5 (Proceedings of the 10th Int. Symp. on Radiation Physics) [86] Rozenbaum O, Bruand A and Le Trong E 2012 Soil porosity resulting from the assemblage of silt grains with a clay phase: new perspectives related to utilization of x-ray synchrotron computed microtomography Comptes Rendus Geosc. 344 516–25 [87] Borges de Oliveira F, Stolfi A, Bartscher M, Chiffre L and Neuschaefer-Rube U 2016 Experimental investigation of surface determination process on multi-material components for dimensional computed tomography Case stud. Nondestruct. Test. Eval. 693–103 [88] Beckers E, Plougonven E, Roisin C, Hapca S, Léonard A and Degré A 2014 X-ray microtomography: a porosity-based thresholding method to improve soil pore network characterization Geoderma 219–220 145–54 [89] Wang W, Kravchenko A, Smucker A and Rivers M 2011 Comparison of image segmentation methods in simulated 2D and 3D microtomographic images of soil aggregates Geoderma 162 231–41 [90] Sahoo P, Wilkins C and Yeager J 1997 Threshold selection using Renyi’s entropy Pattern Recognit. 30 71–84 [91] Gantzer C J and Anderson S H 2002 Computed tomographic measurement of macroporosity in chisel-disk and no-tillage seedbeds Soil Tillage Res. 64 101–11 [92] Luo L, Lin H and Li S 2010 Quantification of 3-D soil macropore networks in different soil types and land uses using computed tomography J. Hydrol. 393 53–64 (Soil Architecture and Preferential Flow across Scales) [93] Jassogne L, McNeill A and Chittleborough D 2007 3D-visualization and analysis of macro-and meso-porosity of the upper horizons of a sodic, texture-contrast soil Eur. J. Soil Sci. 58 589–98 [94] Sander T, Gerke H H and Rogasik H 2008 Assessment of Chinese paddy-soil structure using x-ray computed tomography Geoderma 145 303–14 [95] Coker D A, Torquato S and Dunsmuir J H 1996 Morphology and physical properties of Fontainebleau sandstone via a tomographic analysis J. Geophys. Res.: Solid Earth 101 17497–506 [96] Coker D, Lindquist W 1994 Edge-based algorithm to filter tomographic data sets, preprint SUNYIT-MS-1-1994 [97] Schlüter S, Weller U and Vogel H-J 2010 Segmentation of x-ray microtomography images of soil using gradient masks Comput. Geosci. 36 1246–51 [98] Yanowitz S D and Bruckstein A M 1989 A new method for image segmentation Comput. Vis. Graph. Image Process. 46 82–95 [99] Vogel H J and Kretzschmar A 1996 Topological characterization of pore space in soil—sample preparation and digital image-processing Geoderma 73 23–38 [100] Zhang P, Lu S, Li J, Zhang P, Xie L, Xue H and Zhang J 2017 Multi-component segmentation of x-ray computed tomography (CT) image using multi-Otsu thresholding algorithm and scanning electron microscopy Energy Explor. Exploit. 35 281–94 [101] Heinzl C, Kastner J, Georgi B and Lettenbauer H 2007 Comparison of surface detection methods to evaluate cone beam computed tomography data for three dimensional metrology Int. Symp. on Digital Industrial Radiology and Computed Tomography - DIR 2007 (Villeurbanne, France, 25–27 June 2007) (Lion, France) pp 21–9 [102] Hermanek P and Carmignato S 2017 Porosity measurements by x-ray computed tomography: accuracy evaluation using a calibrated object Precis. Eng. 49 377–87 [103] Salarian M and Toyserkani E 2018 The use of nano-computed tomography (nano-CT) in non-destructive testing of metallic parts made by laser powder-bed fusion additive manufacturing Int. J. Adv. Manuf. Technol. 98 3147–53 [104] Manahiloh K N 2013 Microstructural analysis of unsaturated granular soils using x-ray computed tomography PhD Thesis Department of Civil and Environmental Engineering, Washington State University [105] Iassonov P and Tuller M 2010 Application of segmentation for correction of intensity bias in x-ray computed tomography images Vadose Zone J. 9187–91 [106] Kumar J, Abulrub A H G, Attridge A and Williams M A 2012 Effect of x-ray computed tomography scanning parameters on the estimated porosity of foam specimens Mechanical and Aerospace Engineering, ICMAE2011 vol 110 ed Fan W Applied Mechanics and Materials (Zurich: Trans Tech Publications Ltd) pp 808–15 [107] Pavan M, Craeghs T, Kruth J-P and Dewulf W 2018 Investigating the influence of x-ray CT parameters on porosity measurement of laser sintered PA12 parts using a design-of-experiment approach Polym. Test. 66 203–12 [108] Rogasik H, Onasch I, Brunotte J, Jegou D and Wendroth O 2003 Assessment of soil structure using x-ray computed tomography Geol. Soc. Spec. Publ. 215 151–65 [109] Madra A, Hajj N E and Benzeggagh M 2014 X-ray microtomography applications for quantitative and qualitative analysis of porosity in woven glass fiber reinforced thermoplastic Compos. Sci. Technol. 95 50–8 [110] Fusi N and Martinez-Martinez J 2013 Mercury porosimetry as a tool for improving quality of micro-CT images in low porosity carbonate rocks Eng. Geol. 166 272–82 [111] Spierings A B, Schneider M and Eggenberger R 2011 Comparison of density measurement techniques for additive manufactured metallic parts Rapid Prototyp. J. 17 380–6 [112] Skorpa R, Todorovic J and Torsæter M 2017 Porosity changes in mud-affected rock and cement upon reaction with CO2Energy Procedia 114 5266–74 [113] Blunt M J, Bijeljic B, Dong H, Gharbi O, Iglauer S, Mostaghimi P, Paluszny A and Pentland C 2013 Pore-scale imaging and modelling Adv. Water Resour. 51 197–216 [114] Vanderesse N, Buffiere J-Y, Maire E and Chabod A 2011 Effect of porosity on the fatigue life of a cast Al alloy Optical Measurements, Modeling and Metrology ed Proulx T (Conf. Proc. of the Society for Experimental Mechanics, vol 5) (New York: Springer) pp 55–61 [115] Mahanta B, Vishal V, Ranjith P and Singh T 2020 An insight into pore-network models of high-temperature heat-treated sandstones using computed tomography J. Nat. Gas Sci. Eng. 77 103227 [116] Wevers M, Kerckhofs G, Pyka G, Herremans E, Van Ende A, Hendrickx R and Wilderjans E 2012 X-ray computed tomography for nondestructive testing Proc. ICT 2012, Int. Conf. on Industrial Computed Tomography (Wels, Austria) pp 13–29 [117] Arns C H, Jiang H, Dai H, Shikhov I, Sayedakram N and Arns J-Y 2019 A digital rock physics approach to effective and total porosity for complex carbonates: pore-typing and applications to electrical conductivity The 2018 Int. Symp. 16 Meas. Sci. Technol. 32 (2021) 122001 Topical Review of the Society of Core Analysts (SCA 2018) E3S Web of Conf. (Trondheim, Nowray,27–31 August 2018) vol 89, ed Nicot B (EDP Sciences) p 05002 [118] Sheppard A P, Sok R M and Averdunk H 2004 Techniques for image enhancement and segmentation of tomographic images of porous materials Phys. A339 145–51 [119] Pearce J K, Golab A, Dawson G K, Knuefing L, Goodwin C and Golding S D 2016 Mineralogical controls on porosity and water chemistry during O2-SO2-CO2reaction of CO2 storage reservoir and cap-rock core Appl. Geochem. 75 152–68 [120] Golab A N, Knackstedt M A, Averdunk H, Senden T, Butcher A R and Jaime P 2010 3D porosity and mineralogy characterization in tight gas sandstones Leading Edge 29 1476–83 [121] Bugani S, Camaiti M, Morselli L, Van de Casteele E and Janssens K 2007 Investigation on porosity changes of Lecce stone due to conservation treatments by means of x-ray nanoand improved micro-computed tomography: preliminary results X-Ray Spectrom. 36 316–20 [122] Müller P, Hiller J, Cantatore A and De Chiffre L 2012 A study on evaluation strategies in dimensional x-ray computed tomography by estimation of measurement uncertainties Int. J. Metrol. Quality Eng. 3107–15 [123] Jiang Z, Van Dijke M, Sorbie K S and Couples G D 2013 Representation of multiscale heterogeneity via multiscale pore networks Water Resour. Res. 49 5437–49 [124] Vrålstad T and Skorpa R 2020 Digital cement integrity: a methodology for 3D visualization of cracks and microannuli in well cement Sustainability 12 41128 [125] Maire E, Colombo P, Adrien J, Babout L and Biasetto L 2007 Characterization of the morphology of cellular ceramics by 3D image processing of x-ray tomography J. Eur. Ceram. Soc. 27 1973–81 [126] Nicoletto G, Anzelotti G and Koneˇ cn´ a R 2010 X-ray computed tomography vs. metallography for pore sizing and fatigue of cast Al-alloys Procedia Engineering—Fatigue 2010 vol 2, ed Luk´ aš L (Amsterdam: Elsevier) p 7 [127] Pak T, Butler I B, Geiger S, van Dijke M I, Jiang Z and Surmas R 2016 Multiscale pore-network representation of heterogeneous carbonate rocks Water Resour. Res. 52 5433–41 [128] Fintov´ a S, Anzelotti G, Koneˇ cn´ a R and Nicoletto G 2010 Casting pore characterization by x-ray computed tomography and metallography Arch. Mech. Eng. 57 263–73 [129] Oh W and Lindquist B 1999 Image thresholding by indicator kriging IEEE Trans. Pattern Anal. Mach. Intell. 21 590–602 [130] Robin V, Sardini P, Mazurier A, Regnault O and Descostes M 2016 Effective porosity measurements of poorly consolidated materials using non-destructive methods Eng. Geol. 205 24–9 [131] Du Plessis A, le Roux S G, Booysen G and Els J 2016 Quality control of a laser additive manufactured medical implant by x-ray tomography 3D Print. Addit. Manuf. 3175–82 [132] Kerckhofs G, Schrooten J, Van Cleynenbreugel T, Lomov S V and Wevers M 2008 Validation of x-ray microfocus computed tomography as an imaging tool for porous structures Rev. Sci. Instrum. 79 013711 [133] Huo D, Pini R and Benson S M 2016 A calibration-free approach for measuring fracture aperture distributions using x-ray computed tomography Geosphere 12 558–71 [134] Cooper D, Matyas J, Katzenberg M and Hallgrimsson B 2004 Comparison of microcomputed tomographic and microradiographic measurements of cortical bone porosity Calcif. Tissue Int. 74 437–47 [135] Vieira P and Paciornik S 2001 Uncertainty evaluation of metallographic measurements by image analysis and thermodynamic modeling Mater. Charact. 47 219–26 [136] Panigrahi S K and Gupta S 2018 Automatic ranking of image thresholding techniques using consensus of ground truth Trait. Signal 35 121–36 [137] Kalasov´ a D, Zikmund T, Spurný P, Haloda J, Boroviˇ cka J and Kaiser J 2020 Chemical and physical properties of Žd’´ ar nad s´ azavou l chondrite and porosity differentiation using computed tomography Meteorit. Planet. Sci. 55 1073–81 [138] Du Plessis A, Olawuyi B J, Boshoff W P and Le Roux S G 2016 Simple and fast porosity analysis of concrete using x-ray computed tomography Mater. Struct. 49 553–62 [139] Bredemann J and Schmitt R H 2018 Task-specific uncertainty estimation for medical CT measurements J. Sens. Sens. Syst. 7627–35 17