Performance of US and European exchange traded funds: A base point-slack-based measure approach

Henriques, Carla Oliveira; Neves, Elisabete Duarte; Conceição, Jeremias A.; Vieira, Elisabete F. Simões Article Performance of US and European exchange traded funds: A base point-slack-based measure approach Journal of Risk and Financial Management Provided in Cooperation with: MDPI – Multidisciplinary Digital Publishing Institute, Basel Suggested Citation: Henriques, Carla Oliveira; Neves, Elisabete Duarte; Conceição, Jeremias A.; Vieira, Elisabete F. Simões (2023) : Performance of US and European exchange traded funds: A base pointslack-based measure approach, Journal of Risk and Financial Management, ISSN 1911-8074, MDPI, Basel, Vol. 16, Iss. 2, pp. 1-34, https://doi.org/10.3390/jrfm16020130 This Version is available at: https://hdl.handle.net/10419/275198 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. https://creativecommons.org/licenses/by/4.0/ Citation: Henriques, Carla O., Maria E. Neves, Jeremias A. Conceição, and Elisabete S. Vieira. 2023. Performance of US and European Exchange Traded Funds: A Base Point-Slack-Based Measure Approach. Journal of Risk and Financial Management 16: 130. https://doi.org/10.3390/ jrfm16020130 Academic Editor: Thanasis Stengos Received: 21 September 2022 Revised: 6 February 2023 Accepted: 14 February 2023 Published: 16 February 2023 Copyright: © 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/). Journal of Risk and Financial Management Article Performance of US and European Exchange Traded Funds: A Base Point-Slack-Based Measure Approach Carla O. Henriques 1,2,3 , Maria E. Neves 1,4,* , Jeremias A. Conceição1and Elisabete S. Vieira 5 1Polytechnic Institute of Coimbra, Coimbra Business School|ISCAC, Bencanta, 3040-316 Coimbra, Portugal 2INESCC, Departamento de Engenharia Electrotécnica e de Computadores, University of Coimbra, Polo 2, 3030-290 Coimbra, Portugal 3CeBER, Faculty of Economics, University of Coimbra, Av Dias da Silva 165, 3004-512 Coimbra, Portugal 4 Centre for Transdisciplinary Development Studies (CETRAD), University of Trás-os-Montes and Alto Douro, 5000-801 Vila Real, Portugal 5 GOVCOPP Unit Research, ISCA-UA|University of Aveiro, Campus Universitário, 3810-902 Aveiro, Portugal *Correspondence: [email protected] Abstract: This study evaluates the performance of United States (US) and European Exchange Traded Funds (ETFs) using the non-oriented version of the base point-slack-based measure (BP-SBM) Data Envelopment Analysis (DEA) model, which allows for handling negative data that can arise in some of the metrics traditionally used in this type of analysis. Our findings show that US efficient ETFs are considered benchmarks more often than European efficient ETFs. Nonetheless, it was possible to conclude that European inefficient ETFs were generally less inefficient than US ETFs. Our findings also show that ETFs’ efficiency (particularly for US ETFs) in the short run is more related to risk than to profitability factors. This implies that as the time horizon lengthens, the importance of profitability factors for the ETFs’ financial performance grows. Keywords: ETFs; performance; base point slack-based measure; US; Europe 1. Introduction ETFs are open-end investment funds that aim to replicate the return and risk of their reference index (Gastineau 2001;Neves et al. 2019;Tsolas 2022). Since the early 1990s, the popularity of ETFs has been continuously growing. ETFs offer diversification benefits and indirect access to overseas equities that are otherwise inaccessible or hard to acquire due to their low fees, transparency, liquidity, and tax efficiency (Dragomirescu-Gaina et al. 2021;Bowes and Ausloos 2021;Tsolas 2022;Henriques et al. 2022). Most of the research concerning ETFs has been devoted to examining their performance (e.g., Roll 1978;Golany and Roll 1989;Gouveia et al. 2018;Poterba and Shoven 2002;Kostovetsky 2003;Zopounidis et al. 2010;Hill et al. 2015;Neves et al. 2019;Navratil et al. 2021;Henriques et al. 2022). The performance assessment of these funds is generally carried out according to risk-adjusted measures (Admati and Ross 1985;Blitz and Huij 2012;Osterhoff and Kaserer 2016;Lettau and Pelger 2020;Lobato et al. 2021). The Sharpe ratio (Sharpe 1966) and Jensen’s Alpha (Jensen 1968) are the indicators mostly used to measure the performance of ETFs (Tsolas 2022;Henriques et al. 2022). However, besides the criticism inherent to their isolated use, i.e., the need to find benchmarks and the significance of market timing (Tsolas 2022; Henriques et al. 2022), there is also the challenge of combining them into a single metric (Murthi et al. 1997;Henriques et al. 2022;Tsolas 2022). This article aims to explore the different risk-adjusted performance measures used in assessing ETFs. To this end, the BP-SBM DEA model (Tone et al. 2019) is used to address the difficulties associated with the consideration of negative values that usually arise in this type of analysis and to combine into a single score the multiplicity of metrics available in the assessment of ETFs’ performance. J. Risk Financial Manag. 2023,16, 130. https://doi.org/10.3390/jrfm16020130 https://www.mdpi.com/journal/jrfm J. Risk Financial Manag. 2023,16, 130 2 of 34 In this context, the DEA approach originally proposed by Charnes et al. (1978) can be particularly useful (Basso and Funari 2001,2007,2014,2016;Proença et al. 2023). DEA is a mathematical programming methodology that allows assessing the efficiency of a set of units called Decision-Making Units (DMUs), in our case the ETFs under evaluation, whose performance is characterized by multiple inputs (risk indicators) and outputs (profitability indicators). With this methodology, DMUs are classified as efficient or inefficient according to a single efficiency score. Additionally, DEA enables finding the benchmarks of inefficient DMUs, providing managers with valuable information regarding the best practices to be followed. Furthermore, the DEA methodology has been accepted and used to measure the performance of ETFs because it allows for the elimination of limitations associated with traditional performance measures (Murthi et al. 1997;Galagedera and Silvapulle 2002;Choi and Min 2017;Kiymaz 2019). In fact, Murthi et al. (1997) highlighted the advantages offered by this tool in the assessment of portfolio performance. First, this methodology does not require any theoretical reference model (e.g., CAPM or APT). The efficiency of each fund (DMU) is measured against a set of efficient funds within the same category. Second, it allows simultaneously considering both risk (inputs) and profit measures (outputs), resulting in a single performance assessment score. In this framework, the literature devoted to the use of DEA in the performance assessment of ETFs can be grouped into three main branches. The first considers the single use of DEA models. In this vein, Chu et al. (2010) used a range directional measure (RDM) (Portela et al. 2004) to evaluate the performance of IShare World exchange-traded funds (ETFs), including major countries in Asia Pacific, Europe, North America, and some emerging markets, from 2006 to 2009. The inputs considered in their study were downside risk, and expense ratio, and the outputs were monthly return and upside deviation. Prasanna (2012) employed the Charnes, Cooper, and Rhodes (CCR) DEA model (Charnes et al. 1978) to evaluate the performance of 82 ETFs floated and traded on the Indian stock market. They considered as inputs the standard deviation, maximum drawdown, and monthly downside deviation. The profitable month’s percentage and the compounded monthly return were used as outputs. Acharya et al. (2015) assessed five gold ETFs versus four index ETFs during 2009–2013 with the CCR and Banker, Charnes, and Cooper (BCC) DEA model (Banker et al. 1984). The standard deviation and maximum drawdown have been used as inputs, whereas market value, annual return, and the Sharpe ratio have been used as outputs. Choi and Min (2017) analyzed the efficiency of the KOSPI 200 index, the eight ETFs that follow it, and the 200 stocks that make up the index. The RDM was used to measure the efficiency of the ETFs. The standard deviation and systematic risk were used as inputs, and the return of these funds was considered an output. Isakov (2019) assessed the performance of ETFs on the Xetra platform (Germany) through the CCR model. The expense ratio and downside risk-return were used as inputs, and the upper deviation was used as an output. The second uses two-stage DEA assessments intending to identify performance-related factors. After obtaining the efficiency scores from the first stage, regression models are used to find the reasons behind inefficiency. In this context, Tsolas and Charles (2015) employed a two-step technique to evaluate natural resources ETFs, integrating the generalized proportional distance function (Kerstens and Van de Woestyne 2011) with a censored Tobit model. Tsolas and Charles (2015) used a two-stage process for evaluating the performance of “green ETFs” using the range-adjusted measure (RAM-BCC) DEA model, Tobit censored, ordinary least squares (OLS), and boot-strapped-truncated regression. The performance was measured by considering as inputs the price cash flow (P/CF) and price book (P/B) and as outputs the Sharpe ratio and Jensen’s Alpha. The third combines DEA with other techniques. In this context, Tsolas (2019) assessed the efficiency of ten “utility ETFs” through grey relational analysis, the BCC model, and the additive model. The factors considered were the P/E ratio and expense ratio as inputs and the Sharpe ratio as output. Henriques et al. (2022) evaluated the performance of 60 ETFs in J. Risk Financial Manag. 2023,16, 130 3 of 34 the energy sector from 2014 to 2018. The Weighted Russell Directional Distance DEA model was combined with a multiobjective interval portfolio model to measure the performance of these ETFs. Beta, standard deviation, and Jensen’s Alpha were used to measure risk (inputs), while the Sharpe index, mean annual return, and trailing total return were used to measure profitability (outputs). Later, Tsolas (2022) also used the superefficient RAM DEA model to assess the performance of utility ETFs. Although the advantages associated with the use of the DEA methodology to assess ETFs’ performance have been well documented, it can be established that the available scientific literature is not very prolific (Gregoriou 2006;Gregoriou and Henry 2015). Overall, only a few studies (e.g., Tsolas and Charles 2015;Choi and Min 2017;Tsolas 2022) have dealt with negative data. Additionally, from the literature review conducted, it can be concluded that most of the studies used DEA models that are radial (unrealistically considering proportionate changes of either the inputs or the outputs) or oriented (either fixing the outputs or the inputs) models. As a result, unlike the CCR and BCC models, we employ the SBM model (Tone 2001), which offers a more comprehensive analysis of efficiency. The SBM model is non-radial (i.e., inputs and outputs are allowed to have non-proportional changes) and non-oriented in its analysis of efficiency (i.e., inputs and outputs can change simultaneously). In the SBM model, all possible improvements in inputs or outputs are fully considered in the objective function. This is a key feature of the SBM model, as it ensures that all opportunities for improving efficiency are thoroughly analyzed and considered. The main feature of the SBM model is that it leaves no input or output slack unaccounted for, i.e., all possible improvements are exhausted and properly considered in the objective function. All in all, this work aims at contributing to the existing literature by further exploring the use of DEA in the evaluation of ETFs’ performance. Another novelty introduced by this study is the selection of ETFs in pairs, following the same reference index, with one member of the pair domiciled in the US and the other in Europe (Graham et al. 2020). This work is structured as follows: in the next section, a description of the DEA methodology used is given. In Section 2, the main premises regarding the data collected for assessing the performance of the funds under scrutiny are given. Section 3delivers the main results obtained and their corresponding analyses. Finally, the main conclusions of this work and future developments are established. 2. Methodology In this work, we use the non-oriented version of the DEA model proposed by Tone (2001), called Slack-Based Measure (SBM), which is a non-radial DEA model. This model provides a more comprehensive efficiency assessment tool because, since it is non-radial and non-oriented, it considers that the inefficiency linked to the use of a specific input by a DMU is not necessarily related to the inefficiency regarding the use of another input by the same DMU; also, it contemplates the fact that a DMU may produce distinct outputs at the same time, but with a different production capacity, and hence the production efficiency for different outputs may also be distinct. Consider the set of n DMUs (DMU1,DMU2, . . . , DMUn) , where X= [x ij ,i= 1, 2, . . . , m,j= 1, 2, . . . ,n] is the (m × n) matrix of inputs, Y= [y rj , r = 1, 2, . . . ,s,j= 1, 2, . . . ,n] is the matrix of outputs (s × n), and the rows of these matrices corresponding to the inputs and outputs of DMU o are given by xT o and yT o , respectively, with T denoting the transpose of a vector. The SBM model is given by (Tone 2001): Min λ,s−,s+ρ=1−1 m∑m i=1s− i/xio 1+1 s∑s r=1s+ r/yro s.t. xo=Xλ+s−, yo=Yλ−s+, λ≥0,s−≥0,s+≥0. (1) J. Risk Financial Manag. 2023,16, 130 4 of 34 In problem (1) it is assumed that the matrices of inputs and outputs have non-negative elements, i.e., X ≥ 0 and Y ≥ 0. Furthermore, it is also possible to conclude that an increase in each element of s− or s+ ,ceteris paribus, will decrease the value of the objective function of the problem (1). Therefore, it can be concluded that 0 < ρ< 1. The value of ρin (1) can also be written as follows: ρ= 1 m∑m i=1 xio −s− i xio !1 s∑s r=1 yro +s+ r yro −1 (2) In terms of interpretation, the ratio xio−s− i xio assesses the rate of reduction of input i and therefore, 1 m∑m i=1 xio−s− i xio assesses the average percentage reduction of inputs. Similarly, the ratio yro+s+ r yro evaluates the rate of increase of output rand 1 s∑s r=1 yro+s+ r yro is the average percentage increase in outputs. Therefore, ρ can be seen as the ratio of average inefficiencies of inputs and outputs. To consider variable returns to scale, it is only necessary to add the constraint eTλ= 1 to model (1). So, problem (1) can be converted into problem (3), by using a positive scalar variable t: Min t,λ,s−,s+τ=t−1 m m ∑ i=1 ts− i/xio s.t. t+1 s∑s r=1ts+ r/yro =1, xo=Xλ+s−, yo=Yλ−s+, λ≥0,s−≥0,s+≥0,t>0. (3) Then, problem (3) becomes: Min t,λ,s−,s+τ=t−1 m∑m i=1S− i/xio s.t. t+1 s∑s r=1S+ r/yro =1, txo=XΛ+S−, tyo=YΛ−S+, Λ≥0,S−≥0,S+≥0,t>0 (4) The optimal solution corresponds to: ρ∗=τ∗,λ∗=Λ∗/t∗,s−∗=S−/t∗,s+∗=S+/t∗ Definition 1. A DMU o is SBM-Efficient if ρ∗= 1. This condition is equivalent to s−∗ = 0 and s+∗= 0. Definition 2. The set of efficient reference units for the SBM-inefficient DMU o is obtained by considering the indices of the DMUs associated with λ∗ j>0. Consider the reference set of the SBM-inefficient DMUoas follows: Eo=nj:λ∗ j>0, j=1 . . . , no The point of the efficient frontier that can be seen as a reference DMU for the SBMinefficient DMUois: (ˆxo, ˆyo) = (xo−s−∗,yO+s+∗) = ∑jeEoλ∗ jxj,∑jeEoλ∗ jyj(5) One of the limitations of the original model proposed by Tone (2001) is that it does not allow the treatment of negative factors. To overcome this problem, Tone et al. (2019) have J. Risk Financial Manag. 2023,16, 130 5 of 34 recently proposed the BP-SBM DEA model, which transforms negative factors into positive factors by considering “reference points”. In this context, Sharp et al. (2007) proposed an SBM DEA model similar to the one suggested by Tone et al. (2019). The first model uses the scheme suggested by Portela et al. (2004) to accept negative data. Nevertheless, although the Sharp et al. (2007) model, unlike the BP-SBM DEA model, has the property of translation invariance, it can only be run under the VRS assumption in order to obtain an efficiency score within [0, 1]. Furthermore, the efficiency scores obtained by this model are very sensitive to the maximum output values and the minimum input values, whereas these extreme values have only small effects on the efficiency scores computed with the BP-SBM DEA model because only inputs and outputs with negative values are translated into positive values. Finally, the first DEA model cannot prevent division by zero in the objective function. Definition 3. The reference points are the minimum values of inputs and outputs. The minimum value of the input i,xmin i(i=1, 2, . . . , m)is: δi=min{xi1,xi2, . . . , xin}(i=1, 2, . . . , m)(6) The proposed transformation is carried out as follows: If δi>0, then xmin i=0 If δi=0, then xmin i=−σi If δi<0, then xmin i=δi(1+τi) (7) Therefore, if δi> 0, it is not necessary to change input isince it is positive for all DMUs. If δi= 0, i.e., if one or more DMUs do not use this input, then a very small perturbation, σi> 0, should be considered instead. If δi< 0, i.e., if one or more DMUs have negative input i , this input is converted into an amount large enough to make it strictly positive for all DMUs. To ensure that the use of this input is strictly positive, another term for disturbance, δiτi , is considered. It should be noted that both σ and τi are positive numbers. Similarly, the minimum value of output r , ymin r(r=1, 2, . . . , s) , is obtained as follows: ωr=min{yr1,yr2, . . . , yrn}(r=1, 2, . . . , s) and the proposed transformation is then: If ωr>0, then ymin r=0 If ωr=0, then ymin r=−ρi If ωr<0, then ymin r=ωr(1+γi) (8) Finally, the transformed factors correspond to: xij =xij −xmin i>0(∀i,j)and yrj =yrj −ymin r>0(∀r,j)(9) 3. Data and Assumptions The sample of this study consists of 38 ETFs 1 ordered in pairs, each pair following the same index, domiciled in the US (19 funds), and Europe (14 funds from Ireland and five funds from Luxembourg). The indicators collected for each ETF were Jensen’s Alpha, the Sharpe ratio, the trailing total return for evaluating their performance, and Beta and standard deviation 2 to account for their risk. The time horizon of the study is between 2014 and 2018 and refers to one year (2018), three years (2016–2018), and five years (2014–2018). Table A1 in the Appendix Apresents the data for the DMUs evaluated, specifically their acronyms, names, countries of origin, benchmarks, release dates, and legal structure. J. Risk Financial Manag. 2023,16, 130 6 of 34 The regions chosen for this study were the US and Europe (Ireland and Luxembourg), because these represent the largest ETF world markets. According to the Irish fund 3 , around 968 fund managers from more than 53 countries have assets managed in Ireland. Ireland offers managers access to the European Union (EU) market passport, the Undertaking for Collective Investment in Transferable Securities (UCITS), and Alternative Investment Funds (AIFs). Ireland leads in this fast-growing sector and is the center of excellence for supporting ETFs all over Europe. The ETFs domiciled in Ireland correspond to more than half of the European ETF market. In addition, as claimed by the Association of the Luxembourg Funds Industry (ALFI) 4 , Luxembourg is the second-largest center of ETFs in Europe, with funds distributed across 70 countries. Finally, the three largest US fund providers (which are BlackRock, Vanguard, and State Street) represent more than 70% of the global ETF market. Inputs and outputs were selected according to the factors generally considered in the literature. Thus, we have selected as inputs the Beta and standard deviation, and as outputs, we have considered Jensen’s Alpha, the Sharpe ratio, and the total trailing return. Beta is a market risk indicator that measures the sensitivity of an ETF to a portfolio of assets representing the market (benchmark) and is calculated as the covariance between the return of the ETF and the benchmark index, divided by the variance of the return of the benchmark index. A Beta greater than one means that the ETF is high-risk, as its profitability shows higher volatility than the market. In turn, the standard deviation represents the total risk of a fund and measures the volatility of a fund over a period. In this case, a high standard deviation means that the fund is very volatile. The Sharpe ratio measures the excess return of an ETF compared to a risk-free asset per unit of total risk. The higher the value of this indicator, the better the risk/return ratio of the ETF, i.e., the better the historical performance adjusted to the risk of the ETF. Regarding Jensen’s Alpha, this indicator is a risk-adjusted measure that measures the excess return of an ETF above its expected return determined by the CAPM. The higher the value of this indicator, the higher the performance of the ETF. Finally, the trailing total return indicates the return that could be obtained from an ETF in a specific period. Table 1presents the four models that will be used to measure the performance of investments in ETFs after adjusting for their risk. These models differ only in the factors to be considered as outputs. Table 1. DEA models. Models Input Output DEA 1 (M1) Beta and standard deviation Jensen’s Alpha and Sharpe ratio DEA 2 (M2) Beta and standard deviation Jensen’s Alpha and trailing total return DEA 3 (M3) Beta and standard deviation Sharpe ratio and total trailing return DEA 4 (M4) Beta and standard deviation Jensen’s Alpha, Sharpe ratio, and trailing total return Tables A2 and A3 in the Appendix Aprovide information on the risk and performance indicators, respectively, for each ETF considered in the analysis. 4. Results In this section, we present and discuss the results obtained for the four models considered in the periods of one, three, and five years, respectively. J. Risk Financial Manag. 2023,16, 130 7 of 34 4.1. Results for the Complete Set of ETFs The results for DEA Model 1 (M1) show that, in general, domiciled US ETFs are used more often as benchmarks (see Table A4 in the Appendix A). For one year, five efficient ETFs were obtained, three of which were domiciled in Ireland (CPXJ, CUKS, and IEMS) and two in the US (EWA, EWM) (see Table A4 in the Appendix A). The ETFs based in the US were more often used as a benchmark (EWA, 32 times; EWM, twice)—see Table A4 in the Appendix A. Among the efficient ETFs based in Ireland, the CPXJ served as a reference only once (see Table A4 in the Appendix A). No combination of ETFs that track the same index was efficient. For the period of three years, six efficient ETFs were obtained, four of which are based in the US (IVV, URTH, EWA, and EWM) and two of which are based in Ireland (CSPX and SSAC)—see Table A4 in the Appendix A. The ETFs URTH, SSAC, and EWA are used as references 31 times, six times, and three times, respectively (see Table A4 in the Appendix A). In this case, there are two ETFs (IVV and CSPX) that track the same index but do not serve as a benchmark. Finally, the 5-year period includes three efficient ETFs, two of which are domiciled in Ireland (CSPX and SSAC) and one in the US (SCJ) (see Table A4 in the Appendix A). The ETF SCJ was used as a benchmark 34 times (having a lower sensitivity to the market movement and a higher Jensen’s Alpha), followed by the ETFs CSPX (five times) and SSAC (two times)—see Table A4 in the Appendix A. Nevertheless, it should be noted that inefficient ETFs domiciled in the US show, in general, greater variability in terms of inefficiency than the corresponding pair in Europe (the scores of efficiency of EU ETFs ranged between 0.31 and 0.91, 0.16 and 0.77, and 0.36 and 0.85, for one-, threeand five-year periods, respectively, whereas those for US ETFs ranged between 0.19 and 0.73, 0.04 and 0.92, and 0 and 0.81, for one year, three years, and five years, respectively). Table A5 in the Appendix Apresents the analysis of the descriptive statistics related to the efficient and inefficient ETFs for M1 for the three periods under analysis. It is possible to verify that the average values of the risk factors associated with efficient ETFs are maintained as the time horizon increases, always being below the unit (the opposite situation occurs for inefficient ETFs). On the other hand, in a three-year time horizon, there is a very significant reduction in the average values of the outputs associated with these same funds. From the analysis of Figure 1a–l, it can be seen that in M1, the efficiency scores of efficient ETFs do not seem to be related to their volatility with the market, regardless of the period of analysis considered. In the case of inefficient ETFs, there seems to be an evident correlation between efficiency levels and Beta. This result is particularly corroborated for one year (see the projections associated with this factor in Figure 1d). Regarding the standard deviation, efficient ETFs present, in general, average values that are more or less uniform, varying approximately between 7 and 13. For inefficient ETFs, these values vary between 10 and 24, showing higher volatility (see Table A5 in the Appendix A). Then again, this situation has been particularly evident for one year (see projections associated with this factor in Figure 1j). Concerning the model’s outputs depicted in Figure 2a–l (Jensen’s Alpha and trailing total return), efficiency is much more correlated to the profitability measures over five years (see Table A6 in the Appendix A), meaning that the investor exceeds his or her profitability expectations in relation to the value he or she previously estimated. J. Risk Financial Manag. 2023,16, 130 8 of 34 J. Risk Financial Manag. 2023, 16, x FOR PEER REVIEW 9 of 33 (a) 1 year (b) 3 years (c) 5 years (d) 1 year (e) 3 years (f) 5 years (g) 1 year (h) 3 years (i) 5 years (j) 1 Year (k) 3 Years (l) 5 Years 0.00 0.20 0.40 0.60 0.80 1.00 1.20 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 Efficiency Beta 0.00 0.20 0.40 0.60 0.80 1.00 1.20 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 Efficiency Beta 0.00 0.20 0.40 0.60 0.80 1.00 1.20 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 Efficiency Beta -0.4 0.1 0.6 1.1 1.6 SAUS EPP XMJD ISFR EEMS CEU SSAC ISJP CEMA SCJ EWJ CSUK EWQ CSPX URTH ACWI CCAU IVV AAXJ EEMA XMWO SEMA DX2J DXS5 EZU EEM EWC XCS6 EWU EWUS MCHI IEUS Beta Beta Projection -0.4 0.1 0.6 1.1 1.6 ACWI CUKS ISFR EWJ XMJD EPP EWQ XMWO CEU CPXJ SCJ DX2J CSUK EWUS SAUS IEUS ISJP EWU EZU EWC CCAU CEMA DXS5 SEMA EEMA AAXJ XCS6 MCHI EEM IEMS EEMS Beta Beta Projection -0.4 0.1 0.6 1.1 1.6 ISFR ISJP URTH CUKS ACWI CSUK EWJ IVV XMJD XMWO IEUS EWQ DX2J CPXJ CEU CCAU SAUS CEMA IEMS EZU EWUS EPP EEMA AAXJ EWA DXS5 SEMA XCS6 MCHI EWC EEM EWU EEMS EWM Beta Beta Projection 0.00 0.20 0.40 0.60 0.80 1.00 1.20 6 11162126 Efficiency Standard Deviation 0.00 0.20 0.40 0.60 0.80 1.00 1.20 6 11162126 Efficiency Standard Deviation 0.00 0.20 0.40 0.60 0.80 1.00 1.20 6 11162126 Efficiency Standard Deviation 0 5 10 15 20 25 SAUS EPP XMJD ISFR EEMS CEU SSAC ISJP CEMA SCJ EWJ CSUK EWQ CSPX URTH ACWI CCAU IVV AAXJ EEMA XMWO SEMA DX2J DXS5 EZU EEM EWC XCS6 EWU EWUS MCHI IEUS Standard Deviation Standard Deviation Projection 0 5 10 15 20 25 ACWI CUKS ISFR EWJ XMJD EPP EWQ XMWO CEU CPXJ SCJ DX2J CSUK EWUS SAUS IEUS ISJP EWU EZU EWC CCAU CEMA DXS5 SEMA EEMA AAXJ XCS6 MCHI EEM IEMS EEMS Standard Deviation Standard Deviation Projection 0 5 10 15 20 25 ISFR ISJP URTH CUKS ACWI CSUK EWJ IVV XMJD XMWO IEUS EWQ DX2J CPXJ CEU CCAU SAUS CEMA IEMS EZU EWUS EPP EEMA AAXJ EWA DXS5 SEMA XCS6 MCHI EWC EEM EWU EEMS EWM Standard Deviation Standard Deviation Projection Figure 1. Efficiency scores vs. inputs and projections (ETFs are ordered in decreasing order of efficiency) for the different time horizons–M1. J. Risk Financial Manag. 2023,16, 130 15 of 34 When contrasting the results obtained for DEA M3 with the previous models, it was observed that the European ETFs obtained higher levels of performance with the indicators used. For the period of five years, this latter model also differed from the others, with an ETF domiciled in Ireland becoming a benchmark. In regards to inefficient ETFs, the volatility of inefficiency is, generally, lower in European ETFs (the scores of efficiency of EU ETFs ranged between 0.4 and 0.75, 0.14 and 0.86, and 0.33 and 0.90, for one-, threeand five-year periods, respectively, whereas those for US ETFs ranged between 0.33 and 0.77, 0.19 and 0.91, and 0 and 0.88, for one year, three years, and five years, respectively). According to Table A10 in the Appendix Aand Figures 5a–l and 6a–l, the results suggest that the risk factors of inefficient ETFs practically show no need for adjustments at this level for the threeand five-year periods, in contrast to what happens for one year. In the case of using the Sharpe ratio as a risk-adjusted performance indicator, the opposite situation is verified. This result highlights the trade-off traditionally studied in the literature between risk and return (Henriques and Neves 2019). Finally, it is important to note that the choice of indicators can influence the efficiency levels of ETFs, suggesting that the investor or manager must take a leading role in the choice of the factors to be used. Table A11, in the Appendix A, shows the efficiency scores obtained from Model 4 (M4) over one, three, and five years, with all factors considered simultaneously. According to M4, there are four efficient ETFs based in Ireland (CPXJ, SAUS, CUKS, and IEMS) and two in the US (EWA and EWM) for the one-year period—see Table A11 in the Appendix A. However, only the US-based ETF EWA has been used as a reference 31 times (see Table A11 in the Appendix A), corroborating the results of Chu et al. (2010). During the three-year period, four efficient ETFs from the US (IVV, URTH, EWA, and EWM) and two from Ireland (CSPX and SSAC) were obtained, with the existence of a pair of efficient ETFs (IVV and CSPX) never being considered as a reference (see Table A11 in the Appendix A). The ETF URTH stood out, serving as a reference 31 times. SSAC ranked first among European ETFs, serving as a reference six times. Over a five-year period, there were two efficient ETFs based in the US (EEMA and SCJ), two in Ireland (CSPX and SSAC), and one in Luxembourg (KXMJD)—see Table A11 in the Appendix A. The SSAC and XMJD served as references twice, the CSPX served as references three times, and the SCJ served as references 32 times, demonstrating once more the best positioning of efficient US-based ETFs. After analyzing Table A12 in the Appendix Aand Figures 7a–l and 8a–r, it is possible to conclude that, as in the previous model, the risk factors of ETFs are less susceptible to undergo adjustments to make these funds efficient for the periods of three and five years, registering the opposite situation for one year. Regarding the risk-adjusted performance factors, the trade-off between risk and return is again evident. From the analysis of the different models for one year, the influence of inputs on the efficiency measure obtained far exceeds that of outputs. We discovered a more balanced situation over the three-year period, with inputs having a slightly greater influence than outputs on M1 and M2, except for the Sharpe ratio on M1, and outputs having a greater influence on M3 and M4, except for Jensen’s Alpha on M4. With the exception of Jensen’s Alpha, which has a lower influence than the standard deviation over the five years, the influence of outputs on the obtained measure of efficiency is much greater than that of inputs. In summary, short-term efficiency is more related to risk factors, and with the expansion of the time horizon, the efficiency of funds becomes increasingly related to return factors. J. Risk Financial Manag. 2023,16, 130 16 of 34 J. Risk Financial Manag. 2023, 16, x FOR PEER REVIEW 17 of 33 (a) 1 year (b) 3 years (c) 5 years (d) 1 year (e) 3 years (f) 5 years (g) 1 year (h) 3 years (i) 5 years (j) 1 year (k) 3 years (l) 5 years Figure 7. Efficiency scores vs. inputs and projections (ETFs are ordered in decreasing order of efficiency) for the different time horizons–M4. 0.00 0.20 0.40 0.60 0.80 1.00 1.20 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 Efficiency Beta 0.00 0.20 0.40 0.60 0.80 1.00 1.20 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 Efficiency Beta 0.00 0.20 0.40 0.60 0.80 1.00 1.20 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 Efficiency Beta 0 0.5 1 1.5 EPP ISFR XMJD EEMS CEU SSAC CEMA ISJP CSPX CCAU EWQ EWJ URTH SCJ ACWI IVV AAXJ CSUK XMWO DXS5 SEMA EZU EEM DX2J EWC XCS6 EEMA EWU EWUS MCHI IEUS Beta Beta Projection 0 0.5 1 1.5 ACWI ISFR XMWO EWJ EWQ CUKS XMJD EPP CPXJ CEU DX2J SCJ EWUS SAUS IEUS ISJP EWU CSUK CEMA EZU EWC CCAU DXS5 SEMA XCS6 EEMA AAXJ MCHI EEM IEMS EEMS Beta Beta Projection 0 0.5 1 1.5 ISFR IVV ISJP CUKS URTH ACWI CSUK EWJ XMWO DX2J IEUS EWQ CEU CEMA CCAU CPXJ SAUS EZU EWUS EPP AAXJ DXS5 EWA IEMS XCS6 SEMA MCHI EWC EEM EWU EEMS EWM Beta Beta Projection 0.00 0.20 0.40 0.60 0.80 1.00 1.20 6 11162126 Efficiency Standard Deviation 0.00 0.20 0.40 0.60 0.80 1.00 1.20 6 11162126 Efficiency Standard Deviation 0.00 0.20 0.40 0.60 0.80 1.00 1.20 6 11162126 Efficiency Standard Deviation 0 5 10 15 20 25 EPP ISFR XMJD EEMS CEU SSAC CEMA ISJP CSPX CCAU EWQ EWJ URTH SCJ ACWI IVV AAXJ CSUK XMWO DXS5 SEMA EZU EEM DX2J EWC XCS6 EEMA EWU EWUS MCHI IEUS Standard Deviation Standard Deviation Projection 0 5 10 15 20 25 ACWI ISFR XMWO EWJ EWQ CUKS XMJD EPP CPXJ CEU DX2J SCJ EWUS SAUS IEUS ISJP EWU CSUK CEMA EZU EWC CCAU DXS5 SEMA XCS6 EEMA AAXJ MCHI EEM IEMS EEMS Standard Deviation Standard Deviation Projection 0 5 10 15 20 25 ISFR IVV ISJP CUKS URTH ACWI CSUK EWJ XMWO DX2J IEUS EWQ CEU CEMA CCAU CPXJ SAUS EZU EWUS EPP AAXJ DXS5 EWA IEMS XCS6 SEMA MCHI EWC EEM EWU EEMS EWM Standard Deviation Standard Deviation Projection Figure 7. Efficiency scores vs. inputs and projections (ETFs are ordered in decreasing order of efficiency) for the different time horizons–M4. J. Risk Financial Manag. 2023,16, 130 17 of 34 J. Risk Financial Manag. 2023, 16, x FOR PEER REVIEW 18 of 33 (a) 1 year (b) 3 years (c) 5 years (d) 1 year (e) 3 years (f) 5 years (g) 1 year (h) 3 years (i) 5 years (j) 1 year (k) 3 years (l) 5 years 0.00 0.20 0.40 0.60 0.80 1.00 1.20 0 5 10 15 Efficiency Jensen's Alpha 0.00 0.20 0.40 0.60 0.80 1.00 1.20 0 5 10 15 Efficiency Jensen's Alpha 0.00 0.20 0.40 0.60 0.80 1.00 1.20 0 5 10 15 Efficiency Jensen's Alpha 0 5 10 15 20 EPP ISFR XMJD EEMS CEU SSAC CEMA ISJP CSPX CCAU EWQ EWJ URTH SCJ ACWI IVV AAXJ CSUK XMWO DXS5 SEMA EZU EEM DX2J EWC XCS6 EEMA EWU EWUS MCHI IEUS Jensen's Alpha Jensen's Alpha Projection 0 5 10 15 20 ACWI ISFR XMWO EWJ EWQ CUKS XMJD EPP CPXJ CEU DX2J SCJ EWUS SAUS IEUS ISJP EWU CSUK CEMA EZU EWC CCAU DXS5 SEMA XCS6 EEMA AAXJ MCHI EEM IEMS EEMS Jensen's Alpha Jensen's Alpha Projection 0 5 10 15 20 ISFR IVV ISJP CUKS URTH ACWI CSUK EWJ XMWO DX2J IEUS EWQ CEU CEMA CCAU CPXJ SAUS EZU EWUS EPP AAXJ DXS5 EWA IEMS XCS6 SEMA MCHI EWC EEM EWU EEMS EWM Jensen'sAlpha Jensen's Alpha Projection 0.00 0.20 0.40 0.60 0.80 1.00 1.20 0.00 0.50 1.00 1.50 2.00 2.50 Efficiency Sharpe 0.00 0.20 0.40 0.60 0.80 1.00 1.20 0.00 0.50 1.00 1.50 2.00 2.50 Efficiency Sharpe 0.00 0.20 0.40 0.60 0.80 1.00 1.20 0.00 0.50 1.00 1.50 2.00 2.50 Efficiency Sharpe 0.00 0.50 1.00 1.50 2.00 2.50 EPP ISFR XMJD EEMS CEU SSAC CEMA ISJP CSPX CCAU EWQ EWJ URTH SCJ ACWI IVV AAXJ CSUK XMWO DXS5 SEMA EZU EEM DX2J EWC XCS6 EEMA EWU EWUS MCHI IEUS Sharpe ratio Sharpe Projection 0 0.2 0.4 0.6 0.8 1 ACWI ISFR XMWO EWJ EWQ CUKS XMJD EPP CPXJ CEU DX2J SCJ EWUS SAUS IEUS ISJP EWU CSUK CEMA EZU EWC CCAU DXS5 SEMA XCS6 EEMA AAXJ MCHI EEM IEMS EEMS Sharpe ratio Sharpe Projection 0 0.5 1 1.5 2 ISFR IVV ISJP CUKS URTH ACWI CSUK EWJ XMWO DX2J IEUS EWQ CEU CEMA CCAU CPXJ SAUS EZU EWUS EPP AAXJ DXS5 EWA IEMS XCS6 SEMA MCHI EWC EEM EWU EEMS EWM Sharpe ratio Sharpe Projection Figure 8. Cont. J. Risk Financial Manag. 2023,16, 130 18 of 34 J. Risk Financial Manag. 2023, 16, x FOR PEER REVIEW 19 of 33 (m) 1 year (n) 3 years (o) 5 years (p) 1 year (q) 3 years (r) 5 years Figure 8. Efficiency scores vs. outputs and projections (ETFs are ordered in decreasing order of efficiency) for the different time horizons–M4. 0.00 0.20 0.40 0.60 0.80 1.00 1.20 0 5 10 15 20 Efficiency Trailing total return 0.00 0.20 0.40 0.60 0.80 1.00 1.20 0 5 10 15 20 Efficiency Trailing total return 0.00 0.20 0.40 0.60 0.80 1.00 1.20 0 5 10 15 20 Efficiency Trailing total return 0 5 10 15 20 25 EPP ISFR XMJD EEMS CEU SSAC CEMA ISJP CSPX CCAU EWQ EWJ URTH SCJ ACWI IVV AAXJ CSUK XMWO DXS5 SEMA EZU EEM DX2J EWC XCS6 EEMA EWU EWUS MCHI IEUS Trailing total return Trailing Total Return(%) Projection 0 5 10 15 20 25 ACWI ISFR XMWO EWJ EWQ CUKS XMJD EPP CPXJ CEU DX2J SCJ EWUS SAUS IEUS ISJP EWU CSUK CEMA EZU EWC CCAU DXS5 SEMA XCS6 EEMA AAXJ MCHI EEM IEMS EEMS Trailing total return Trailing Total Return(%) Projection 0 5 10 15 20 25 ISFR IVV ISJP CUKS URTH ACWI CSUK EWJ XMWO DX2J IEUS EWQ CEU CEMA CCAU CPXJ SAUS EZU EWUS EPP AAXJ DXS5 EWA IEMS XCS6 SEMA MCHI EWC EEM EWU EEMS EWM Trailing total return Trailing Total Return(%) Projection Figure 8. Efficiency scores vs. outputs and projections (ETFs are ordered in decreasing order of efficiency) for the different time horizons–M4. J. Risk Financial Manag. 2023,16, 130 19 of 34 4.2. Results by Region In addition to the study carried out for the different models regarding the periods considered, an analogous empirical study by region was also performed, with the sample being divided into US (19) and European (18) ETFs (Table 2). Table 2. Summary table of efficient ETFs from the global sample and by region. Models Period Input Output Efficient ETFs Global US Efficient ETFs EUR Efficient ETFs M1 1 Year Beta Standard deviation Jensen’s Alpha Sharpe ratio EWA, EWM, CPXJ, CUKS, IEMS EWA, EWM CPXJ, CUKS, IEMS, ISFR 3 Years IVV, URTH, EWA, EWM, CSPX, SSAC IVV, URTH, EWA, EWM, EWJ CSPX, CUKS, ISFR 5 Years SCJ, CSPX, SSAC SCJ,IVV CSPX, SSAC CUKS, ISFR M2 1 Year Beta Standard deviation Jensen’s Alpha trailing total return EWA, EWM, CPXJ, CUKS, and IEMS EWA, EWM CPXJ, CUKS, SAUS, ISFR, IEMS 3 Years IVV, URTH, EWA, EWM, CSPX, SSAC IVV, URTH, EWA, EWM, EWJ CSPX, SSAC CUKS, ISFR 5 Years EEMA, SCJ, CSPX, SSAC, XMJD EEMA, SCJ,IVV CSPX, SSAC, CUKS, ISFR, XMJD M3 1 Year Beta Standard deviation Sharpe ratio, trailing total return EWA, EWM, CPXJ, CUKS, IEMS EWA, EWM CPXJ, IEMS, SAUS, ISFR 3 Years IVV, URTH, EWA, EWM, CSPX, SSAC IVV, URTH, EWA, EWM, EWJ CSPX, SSAC CUKS, ISFR 5 Years EEMA, SCJ, CSPX, SSAC XMJD EEMA, SCJ,IVV CSPX, SSAC, XMJD, ISFR M4 1 Year Beta Standard deviation Jensen’s Alpha Sharpe ratio Trailing total return EWA, EWM, CPXJ, EWA, EWM CPXJ, SAUS, IEMS, CUKS, ISFR 3 Years IVV, URTH, EWA, EWM, CSPX, SSAC IVV, URTH, EWA, EWM, EWJ CSPX, SSAC CUKS, ISFR 5 Years EEMA, SCJ, CSPX, SSAC, XMJD EEMA, SCJ,IVV CSPX, SSAC, XMJD, ISFR, CUKS Note: the efficient ETFs more often selected as benchmarks are highlighted in bold. The results obtained corroborate the conclusions previously established regarding US ETFs. Tables A13 and A14 in the Appendix Ashow the descriptive statistics of the efficient and inefficient ETFs for each region for the periods of one, three, and five years for M1. Regardless of the period considered, European inefficient ETFs always presented a lower magnitude of average inefficiency compared to US ETFs (the scores of efficiency of EU ETFs ranged between 0.52 and 0.98, 0.19 and 0.89, and 0.40 and 0.92, for one-, threeand five-year periods, respectively, whereas those for US ETFs ranged between 0.19 and 0.73, 0.04 and 0.92, and 0 and 0.81, for one year, three years, and five years, respectively)—see Tables A13 and A14. On the other hand, efficient US ETFs presented, on average, less risk, while efficient European ETFs showed, on average, greater performance. For the one-year period in the US, the risk indicators (Beta and standard deviation) had a significant correlation with efficiency in the four models (see Table A15 in the Appendix A). Furthermore, these results suggest that efficiency is more related to short-term risk factors. On the European side, the results obtained indicate a situation of greater balance concerning J. Risk Financial Manag. 2023,16, 130 20 of 34 the factors that are correlated with the efficiency of this type of fund (see Table A15 in the Appendix A). In turn, for the three years in the US sample, the results are similar to those obtained in the previous period. Concerning Europe, risk factors are less correlated with efficiency when compared to profitability factors (see Table A15 in the Appendix A). Finally, for the five years in the US, risk indicators had a lower correlation with efficiency when compared to profitability indicators. In Europe, the standard deviation and the Sharpe ratio showed a higher correlation with efficiency for these funds (see Table A15 in the Appendix A). These results suggest that the US ETFs follow the trend observed in the global sample, i.e., they show a greater correlation of efficiency with risk in the short term and a greater correlation with profitability as the time horizon expands. However, this trend has not been confirmed for European ETFs. 5. Conclusions ETFs are the future of the fund industry, and several characteristics contribute to their popularity: transparency, ease of access, tax efficiency, and low cost, among others. Traditionally, their performance is assessed by risk-adjusted profitability indicators, namely the Sharpe ratio and Jensen’s Alpha. However, these measures have limitations, as they do not incorporate some of the factors seen as essential for assessing the performance of portfolios of assets: Jensen’s Alpha is sensitive to the choice of benchmark and market timing, and the Sharpe ratio can be manipulated by individuals to present their best side (i.e., in theory, if the three-year Sharpe ratio of a portfolio is not appealing, the fund manager could calculate the ratio for a 5-year period considering that the portfolio had a good performance in the past). On the other hand, the isolated consideration of these indicators does not allow the investor to take into account the multiple factors underlying the decision-making process. To overcome this issue, in this study, a non-parametric technique that considers multiple assessment factors was used. Traditionally, the literature on ETFs has fundamentally referred to the performance of the ETFs in comparison with their benchmark index or with the performance of other types of funds, mainly mutual funds (Ippolito 1989). In this study, some originality was sought, and the DEA methodology was used to evaluate the efficiency of ETFs domiciled in the US and Europe (Ireland and Luxembourg). These ETFs were selected in pairs, following the same reference index, with one member of the pair domiciled in the US and the other in Europe. The sample included 38 ETFs (19 in the US, 14 in Ireland, and five in Luxembourg), and the time horizon considered was from 2014 to 2018, with data for one year (2018), three years (2018–2016), and five years (2018–2014), respectively. Four models were used, whose inputs are the Beta and standard deviation and whose outputs are the Sharpe ratio, Jensen’s Alpha, and the total trailing return. First, the models for the complete sample were obtained, and then the sample was separated by regions. Efficient US ETFs were viewed more often as a reference for best practices than efficient European ETFs. For inefficient ETFs, those from Europe showed, in general, less variability in terms of inefficiency than those from the US. Overall, for one year, efficiency depended more on risk factors than on profitability, and as the time horizon increased, efficiency became more related to profitability factors. This trend was also seen in the sample of ETFs from the US. However, these findings were not corroborated in the case of Europe. The results also showed that efficiency levels are related to the risk-adjusted performance factors used, thus highlighting the need to reflect the preferences of investors/managers in the choice of the factors to be used in this assessment. Similarly to other empirical studies (e.g., Tsolas and Charles 2015;Choi 1995;Choi and Min 2017), it was found that the DEA methodology complements traditional performance measures, providing investors with a broader perspective of ETFs’ performance since it allows the contemplation of several risk-adjusted-performance assessment factors simultaneously. J. Risk Financial Manag. 2023,16, 130 21 of 34 This work presents relevant theoretical and practical implications, which it is important to highlight. On the one hand, academics and researchers may find this article a good foundation for further work as we identified a gap in the literature when studying ETFS on a comparative basis between the US and Europe using an innovative methodology. On the other hand, both fund managers, potential investors, and civil society can understand what contributes to the performance and risk levels of this type of diversified funds, in different periods. At a time when interest rates are rising given the high inflation rates and weak returns on bank investments, this investment alternative can translate into an increasingly attractive reality for long term investors. While in the US this type of product is already truly used to diversify portfolios, in Europe it is a reality that is still recent but that is here to stay given the increasingly adverse macroeconomic environment in terms of banking dependence in these countries and the increasing weaknesses of long-term investments in banking. As a future study, it would be interesting to analyze the efficiency of ETFs from different regions and see if their domiciliation has any impact on efficiency, using, for example, portfolio models and considering new periods, including bear markets such as the war in Ukraine. Author Contributions: Methodology, C.O.H. and J.A.C.; Software, C.O.H.; Formal analysis, M.E.N., J.A.C. and E.S.V.; Investigation, C.O.H. and J.A.C.; Resources, M.E.N.; Data curation, C.O.H., M.E.N. and J.A.C.; Writing—original draft, J.A.C.; Writing—review & editing, C.O.H. and E.S.V.; Visualization, M.E.N.; Supervision, E.S.V. All authors have read and agreed to the published version of the manuscript. Funding: This research was funded by national funds, through the FCT—Portuguese Foundation for Science and Technology under the project UIDB/04011/2020. This work has also been funded by the Portuguese Foundation for Science and Technology funds through Projects UID/MULTI/00308/2020 and UIDB/05037/2020. Data Availability Statement: Data is contained within the article. Conflicts of Interest: The authors declare no conflict of interest. Appendix A Table A1. Data on DMUs. DMU ETF Home Index Followed Launch Legal Structure IVV iShares core S&P 500 ETF US S&P 500 15/05/2000 ETF CSPX iShares Core S&P 500 UCITS ETF Ireland S&P 500 19/05/2010 UCITS ETF EWU iShares MSCI United Kingdom ETF US MSCI United Kingdom 12/03/1996 ETF CSUK iShares MSCI United Kingdom UCITS ETF Ireland MSCI United Kingdom 12/01/2010 UCITS ETF EWQ iShares MSCI France ETF US MSCI France 12/03/1996 ETF ISFR iShares MSCI France UCITS ETF Ireland MSCI France 05/09/2014 UCITS ETF URTH iShares MSCI World EFT US MSCI World 10/01/2012 ETF XMWO XTRACKERS MSCI World UCITS ETF 1C Ireland MSCI World 22/07/2014 UCITS ETF EWJ iShares MSCI Japan ETF US MSCI Japan 12/03/1996 ETF XMJD Xtrackers MSCI Japan UCITS ETF 1C Luxembourg MSCI Japan 09/01/2007 UCITS ETF EEMA iShares MSCI Emergent Market Asia ETF US MSCI Emergent Market Asia 08/02/2012 ETF J. Risk Financial Manag. 2023,16, 130 22 of 34 Table A1. Cont. DMU ETF Home Index Followed Launch Legal Structure CEMA iShares MSCI Emergent Asia UCITS ETF Ireland MSCI Emergent Market Asia 06/08/2010 UCITS ETF MCHI iShares MSCI China ETF US MSCI China 29/03/2011 ETF XCS6 Xtrackers MSCI China UCITS ETF 1C Luxembourg MSCI China 24/06/2010 UCITS ETF EWC iShares MSCI Canada ETF US MSCI Custom Capped Canada 12/03/1996 ETF CCAU iShares MSCI Canada UCITS ETF Ireland MSCI Canada 12/01/2010 UCITS ETF EEM iShares MSCI Emerging Markets ETF US MSCI Emerging Markets Index (SM) 07/04/2003 ETF SEMA iShares MSCI EM UCITS ETF USD (Acc) Ireland MSCI Emerging Markets Index (SM) 25/09/2009 UCITS ETF EZU iShares MSCI EMU ETF US MSCI MEU 25/07/2000 ETF CEU iShares MSCI EMU UCITS ETF Ireland MSCI MEU 12/01/2010 UCITS ETF EPP iShares MSCI Pacific ex Japan ETF US MSCI Pacific ex-Japan Index 25/10/2001 ETF CPXJ iShares Core MSCI Pacific ex-Japan UCITS ETF Ireland MSCI Pacific ex-Japan Index 12/01/2010 UCITS ETF EWA iShares MSCI Australia ETF US MSCI Australia Index 12/03/1996 ETF SAUS iShares MSCI Australia UCITS ETF Ireland MSCI Australia Index 22/01/2010 UCITS ETF ACWI iShares MSCI ACWI ETF US MSCI ACWI Index 26/03/2008 ETF SSAC iShares MSCI ACWI UCITS ETF Ireland MSCI ACWI Index 21/10/2011 UCITS ETF SCJ iShares MSCI Japan Small-Cap ETF US MSCI Japan Small Cap Index 20/12/2007 ETF ISJP iShares MSCI Japan Small Cap UCITS ETF Ireland MSCI Japan Small Cap Index 09/05/2008 UCITS ETF EWUS iShares MSCI United Kingdom Small-Cap ETF US MSCI United Kingdom Small Cap Index 25/01/2012 ETF CUKS iShares MSCI UK Small Cap UCITS ETF Ireland MSCI United Kingdom Small Cap Index 01/07/2009 UCITS ETF EWM iShares MSCI Malaysia ETF US MSCI Malaysia Index 12/03/1996 ETF LG6 Xtrackers MSCI Malaysia UCITS ETF 1C Luxembourg MSCI Malaysia TRN INDEX 24/06/2010 UCITS ETF EEMS iShares MSCI Emerging Markets Small-Cap ETF US MSCI Emerging Markets Small Cap Index 16/08/2011 ETF IEMS iShares MSCI EM Small Cap UCITS ETF Ireland MSCI Emerging Markets Small Cap Index 06/03/2009 UCITS ETF J. Risk Financial Manag. 2023,16, 130 23 of 34 Table A1. Cont. DMU ETF Home Index Followed Launch Legal Structure IEUS iShares MSCI Europe Small-Cap ETF US MSCI Europe Small Cap Index 12/11/2007 ETF DX2J Xtrackers MSCI Europe Small Cap UCITS ETF 1C Luxembourg MSCI Europe Small Cap Index 17/01/2008 UCITS ETF AAXJ iShares MSCI All Country Asia ex Japan ETF US MSCI AC Asia ex Japan Index 13/08/2008 ETF DXS5 Xtrackers MSCI AC Asia ex Japan Swap UCITS ETF 1C Luxembourg MSCI AC Asia ex Japan Index 20/01/2009 UCITS ETF Table A2. Input-related data. Inputs Beta Standard Deviation (%) DMU 1 Year 3 Years 5 Years 1 Year 3 Years 5 Years IVV 1 1 1 17 12.07 11.92 CSPX 0.98 0.99 0.99 15.17 11.4 11.29 EWU 1.01 0.93 0.95 14.46 12.01 13.14 CSUK 1.02 1.04 1.05 10.66 10.48 10.47 EWQ 1.04 1.1 1.04 15.38 14 14.37 ISFR 0.96 0.95 0.96 11.53 11.76 11.91 URTH 1.09 0.89 0.86 15.54 11.21 11.65 XMWO 1.06 1.02 1 16.28 11.54 11.81 EWJ 0.84 0.82 0.87 12.72 10.7 12.3 XMJD 1.02 1.01 1.02 10.44 9.94 11.53 EEMA 1.09 1.17 1.09 16.13 14.76 15.46 CEMA 0.99 1.01 1 12.22 12.72 14.49 MCHI 1.52 1.45 1.39 23.04 19.44 21.12 XCS6 1.02 1.01 1 23.57 19.66 21.24 EWC 1.17 0.9 0.92 18.04 13.28 14.68 CCAU 0.98 0.99 1 15.38 12.19 13.1 EEM 1.09 1.16 1.13 15.91 14.18 15.65 SEMA 1 1 1 15.9 14.21 15.65 EZU 1.08 1.15 1.09 15.42 14.2 14.74 CEU 1 1 1 14.04 12.04 13.98 EPP 0.85 0.86 0.99 12.88 11.47 14.02 CPXJ 1 1 1 10.34 10.88 13.17 EWA 0.67 0.7 0.92 10.97 10.98 14.5 SAUS 1 1 1 10.52 11.82 14.08 ACWI 1.09 0.92 0.89 15.37 11.28 11.74 SSAC 1 1 1 12.7 9.92 10.43 SCJ 0.83 0.86 0.74 13.69 11.65 11.53 ISJP 0.99 1 1 11.42 11.14 11.72 J. Risk Financial Manag. 2023,16, 130 24 of 34 Table A2. Cont. Inputs Beta Standard Deviation (%) DMU 1 Year 3 Years 5 Years 1 Year 3 Years 5 Years EWUS 1.21 1.06 0.99 18.86 14.67 16.46 CUKS 1.25 0.96 0.99 13.17 11.1 11.32 EWM 0.24 0.62 0.76 7.07 11.84 15.01 EEMS 0.76 0.97 0.97 12.48 13.09 14.4 IEMS 0.93 0.98 0.99 9.42 11.86 13.93 IEUS 1.16 1.14 1.03 16.81 14 14.58 DX2J 106 1.03 1.01 16.56 12.4 13.61 AAXJ 1.1 1.16 1.09 16.35 14.67 15.4 DXS5 1.02 1.01 1 16.7 14.89 15.48 Average 1.00 1.00 0.99 14.44 12.69 13.83 Minimum 0.24 0.62 0.74 7.07 9.92 10.43 Maximum 1.52 1.45 1.39 23.57 19.66 21.24 Source: Financial Times. Table A3. Output-related data. Outputs Jensen’s Alpha Sharpe Ratio Trailing Total Return (%) DMU 1 Year 3 Years 5 Years 1 Years 3 Years 5 Years 1 Year 3 Years 5 Years IVV −0.03 −0.03 −0.04 0.74 1.08 0.83 14.29 14.87 10.73 CSPX −0.05 −0.07 −0.09 0.8 1.03 1.27 13.92 14.5 10.37 EWU −4.39 −0.54 −2.52 0.34 0.48 0.06 6.48 6.93 0.97 CSUK −1.44 −1.13 −1.32 0.47 0.47 0.52 5.35 5.06 5.35 EWQ 1.03 3.1 3 0.7 0.75 0.46 12.62 11.88 6.85 ISFR 0.52 1.49 1.54 0.97 0.86 0.99 15.16 11.93 10.1 URTH 1.06 4.47 4.2 0.72 0.94 0.62 13.07 12.16 5.85 XMWO −0.34 0.42 0.52 0.68 0.89 0.59 12.75 11.95 7.61 EWJ −0.96 0.21 3.02 0.54 0.54 0.49 8.76 7.05 6.54 XMJD 0.37 −0.49 −1.2 0.67 0.5 0.93 10.08 7.75 11.18 EEMA 0.34 −0.88 0.58 0.65 0.48 0.28 4.32 7.93 12.16 CEMA −1.02 −0.75 −0.69 0.82 0.45 0.6 12.79 9.22 9.18 MCHI −2.39 −0.74 1.63 0.51 0.47 0.31 12.21 9.27 5.44 XCS6 −0.8 −0.67 −0.6 0.5 0.46 0.3 12.19 9.19 5.36 EWC −0.78 −0.5 −1.78 0.56 0.42 0.1 11.53 6.61 1.42 CCAU −0.08 −0.22 −0.22 0.64 0.37 0.45 15.28 7.21 5.9 EEM −0.7 −1.97 −1.47 0.6 0.41 0.16 11.12 6.74 2.34 SEMA −0.73 −0.64 −0.58 0.59 0.41 0.16 11.06 6.69 2.34 EZU −0.83 −0.12 0.43 0.6 0.54 0.29 10.94 8.69 4.29 CEU 0.65 0.4 0.37 1 0.78 0.59 13.43 8.58 7.25 J. Risk Financial Manag. 2023,16, 130 31 of 34 Table A11. Cont. DMU 1 Year 3 Years 5 Years Number of Times as Benchmark (1 Year) Number of Times as Benchmark (3 Years) Number of Times as Benchmark (5 Years) Home ACWI 0.46 0.92 0.76 US SSAC 0.52 1.00 1.00 6 2 Ireland SCJ 0.47 0.56 1.00 32 US ISJP 0.49 0.49 0.86 Ireland EWUS 0.26 0.53 0.44 US CUKS 1.00 0.70 0.82 Ireland EWM 1.00 1.00 0.00 2 US EEMS 0.56 0.05 0.31 US IEMS 1.00 0.18 0.41 Ireland IEUS 0.23 0.50 0.59 US DX2J 0.37 0.57 0.60 Luxembourg AAXJ 0.44 0.37 0.43 US DXS5 0.42 0.42 0.42 Luxembourg Table A12. Descriptive statistics of efficient and inefficient DMUs in model M4. Model DEA 4 1 Year 3 Years 5 Years Mean Stand. Deviation Minimum Maximum Mean Stand. Deviation Minimum Maximum Mean Stand. Deviation Minimum Maximum Efficient DMUs Beta 0.85 0.35 0.24 1.25 0.87 0.17 0.62 1.00 0.97 0.13 0.74 1.09 Standard deviation 10.25 2.00 7.07 13.17 11.24 0.76 9.92 12.07 12.05 1.96 10.43 15.46 Alfa 9.05 5.90 0.01 16.03 4.77 2.99 0.00 8.81 8.96 2.88 6.73 13.99 TTR 1.55 0.82 0.00 2.13 0.77 0.41 0.00 1.09 1.20 0.37 0.63 1.62 Efficiency 13.97 7.87 0.00 20.00 10.39 5.18 0.85 14.87 15.24 2.03 12.10 17.41 Inefficient DMUs 1.00 0.00 1.00 1.00 1.00 0.00 1.00 1.00 1.00 0.00 1.00 1.00 Beta Standard deviation 1.03 0.13 0.76 1.52 1.02 0.12 0.82 1.45 1.00 0.10 0.76 1.39 Alfa 15.25 3.04 10.44 23.57 12.90 2.27 9.94 19.66 14.11 2.38 10.47 21.24 TTR 7.58 1.76 3.31 13.30 4.36 1.57 0.19 8.12 8.01 2.25 0.01 12.13 Efficiency 1.47 0.17 1.16 1.85 0.55 0.18 0.11 0.90 0.78 0.30 0.00 1.49 DMU efficient 14.76 2.92 7.97 19.43 8.14 2.17 3.00 11.95 10.18 3.32 0.01 15.98 Beta 0.44 0.11 0.23 0.75 0.51 0.18 0.05 0.92 0.53 0.21 0.00 0.92 Table A13. Descriptive statistics of efficient and inefficient DMUs (US. M1). Model DEA 1US 1 Year 3 Years 5 Years Mean Stand. Deviation Minimum Maximum Mean Stand. Deviation Minimum Maximum Mean Stand. Deviation Minimum Maximum Efficient DMUs Beta 0.46 0.30 0.24 0.67 0.81 0.15 0.62 1.00 0.87 0.18 0.74 1.00 Standard deviation 9.02 2.76 7.07 10.97 11.36 0.58 10.70 12.07 11.73 0.28 11.53 11.92 Alfa 7.61 10.76 0.01 15.22 4.94 3.32 0.00 8.81 10.94 4.31 7.89 13.99 TTR 1.03 1.45 0.00 2.05 0.65 0.42 0.00 1.09 1.14 0.06 1.09 1.18 Efficiency 1.00 0.00 1.00 1.00 1.00 0.00 1.00 1.00 1.00 0.00 1.00 1.00 Inefficient DMUs Beta 1.05 0.18 0.76 1.52 1.06 0.16 0.86 1.45 1.00 0.14 0.76 1.39 Standard deviation 15.89 2.57 12.48 23.04 13.76 2.06 11.28 19.44 14.66 2.14 11.65 21.12 Alfa 7.48 2.35 3.31 13.30 4.36 1.99 0.19 8.12 8.16 2.99 0.01 12.13 TTR 1.43 0.16 1.16 1.85 0.54 0.16 0.19 0.90 0.61 0.23 0.00 0.97 Efficiency 0.39 0.13 0.19 0.73 0.45 0.21 0.04 0.92 0.45 0.19 0.00 0.81 J. Risk Financial Manag. 2023,16, 130 32 of 34 Table A14. Descriptive statistics of efficient and inefficient DMUs (EUR. M1). Model DEA 1EUR 1 Year 3 Years 5 Years Mean Stand. Deviation Minimum Maximum Mean Stand. Deviation Minimum Maximum Mean Stand. Deviation Minimum Maximum Efficient DMUs Beta 1.04 0.15 0.93 1.25 0.98 0.02 0.95 1.00 0.99 0.02 0.96 1.00 Standard deviation 11.12 1.62 9.42 13.17 11.05 0.80 9.92 11.76 11.24 0.61 10.43 11.91 Alfa 9.99 4.05 7.50 16.03 5.57 1.77 4.20 7.96 8.86 1.31 7.72 10.43 TTR 1.73 0.32 1.38 2.13 0.86 0.16 0.65 1.04 1.37 0.21 1.10 1.62 Efficiency 1.00 0.00 1.00 1.00 1.00 0.00 1.00 1.00 1.00 0.00 1.00 1.00 Inefficient DMUS Beta 1.01 0.03 0.98 1.06 1.01 0.02 0.98 1.04 1.01 0.01 0.99 1.05 Standard deviation 14.40 3.54 10.44 23.57 12.56 2.44 9.94 19.66 13.88 2.60 10.47 21.24 Alfa 7.64 0.54 6.59 8.68 3.93 0.47 3.21 4.76 7.51 0.51 6.61 8.45 TTR 1.55 0.22 1.29 2.12 0.53 0.20 0.11 0.90 0.91 0.26 0.51 1.49 Efficiency 0.73 0.12 0.52 0.98 0.58 0.17 0.19 0.89 0.64 0.14 0.40 0.92 Table A15. Correlation of the efficiency scores with inputs and outputs for each model by region (US and Europe). Correlation with Efficiency US (M1) Correlation with Efficiency Europe (M1) Input/output 1 year 3 years 5 years Bigger Input/output 1 year 3 years 5 years Bigger Beta −0.86 −0.66 −0.25 1 year Beta 0.02 −0.31 −0.25 3 years Standard deviation −0.82 −0.67 −0.62 1 year Standard deviation −0.77 −0.46 −0.77 5 years Jensen’s Alfa 0.34 0.50 0.80 5 years Jensen’s Alfa 0.45 0.69 0.53 3 years Sharpe ratio −0.09 0.48 0.97 5 years Sharpe ratio 0.66 0.91 0.92 5 years Correlation with efficiency US (M2) Correlation with efficiency Europe (M2) Input/output 1 year 3 years 5 years Bigger Input/output 1 year 3 years 5 years Bigger Beta −0.83 −0.63 −0.11 1 year Beta −0.02 −0.40 −0.06 3 years Standard deviation −0.80 −0.64 −0.48 1 year Standard deviation −0.61 −0.45 −0.73 5 years Jensen’s Alfa 0.35 0.52 0.75 5 years Jensen’s Alfa 0.44 0.69 0.40 3 years Trailing total return −0.03 0.36 0.95 5 years Trailing total return 0.47 0.77 0.86 5 years Correlation with efficiency US (M3) Correlation with efficiency Europe (M3) Input/output 1 year 3 years 5 years Bigger Input/output 1 year 3 years 5 years Bigger Beta −0.86 −0.67 −0.13 1 year Beta −0.60 −0.36 −0.09 1 year Standard deviation −0.82 −0.69 −0.50 1 year Standard deviation −0.59 −0.40 −0.72 5 years Jensen’s Alfa −0.10 0.47 0.87 5 years Jensen’s Alfa 0.83 0.89 0.94 5 years Trailing total return −0.03 0.31 0.94 5 years Trailing total return 0.66 0.77 0.84 5 years Correlation with efficiency US (M4) Correlation with efficiency Europe (M4) Input/output 1 year 3 years 5 years Bigger Input/output 1 year 3 years 5 years Bigger Beta −0.85 −0.64 −0.13 1 year Beta −0.01 −0.31 −0.12 3 years Standard deviation −0.82 −0.65 −0.50 1 year Standard deviation −0.67 −0.42 −0.75 5 years Jensen’s Alfa 0.34 0.51 0.74 5 years Jensen’s Alfa 0.45 0.69 0.42 3 years Sharpe ratio −0.08 0.49 0.87 5 years Sharpe ratio 0.70 0.91 0.91 3 and 5 years Trailing total return −0.05 0.35 0.94 5 years Trailing total return 0.39 0.78 0.84 5 years Notes 1ETF LG6, which is domiciled in Luxembourg, is excluded from the analysis due to a lack of data for the period under analysis. 2These measures will be further explained in Section 3. 3 Irish funds are the abbreviation for Irish Funds Industry Association, which is an association that aims to support, complement, and develop the funds industry in Ireland. 4ALFI represents the face and voice of asset management in Luxembourg and the investment fund community. J. Risk Financial Manag. 2023,16, 130 33 of 34 References Acharya, Satya Ranjan, Amit Kumar Dwivedi, and Bhumika Darshak Panchal. 2015. Application od Data Envelopment analysis on Indian Gold ETFs. International Journal of Business Continuity and Risk Management 6: 147–61. [CrossRef] Admati, Anat R., and Stephen A. Ross. 1985. Measuring Investment Performance in a Rational Expectations Equilibrium Model. The Journal of Business 53: 1–26. [CrossRef] Banker, Rajiv D., Abraham Charnes, and William Wager Cooper. 1984. Some models for estimating technical and scale inefficiencies in data envelopment analysis. Management Science 30: 1078–92. [CrossRef] Basso, Antonella, and Stefania Funari. 2001. A data envelopment analysis approach to measure mutual fund performance. European Journal of Operational Research 135: 477–92. [CrossRef] Basso, Antonella, and Stefania Funari. 2007. DEA models for ethical and non ethical mutual funds with negative data. Mathematics and Methods in Economics Finance 2: 21–40. Basso, Antonella, and Stefania Funari. 2014. Constant and variable returns to scale DEA models for socially responsible investment funds. European Journal of Operational Research 235: 775–83. [CrossRef] Basso, Antonella, and Stefania Funari. 2016. DEA performance assessment of mutual funds. In Data Envelopment Analysis. Boston: Springer, pp. 229–87. [CrossRef] Blitz, David, and Joop Huij. 2012. Evaluating the performance of global emerging markets equity exchange-traded funds. Emerging Markets Review 13: 149–58. [CrossRef] Bowes, Jordan, and Marcel Ausloos. 2021. Financial Risk and Better Returns through smart beta exchange-traded funds? Journal of Risk and Financial Management 14: 283. [CrossRef] Charnes, Abraham, William W. Cooper, and Edwardo Rhodes. 1978. Measuring the efficiency of decision making units. European Journal of Operational Research 2: 429–44. [CrossRef] Choi, Hyung-Suk, and Daiki Min. 2017. Efficiency of well-diversified portfolios: Evidence from data envelopment analysis. Omega 73: 104–13. [CrossRef] Choi, Yoon K. 1995. The sensitivity in tests of the efficiency of a portfolio and portfolio performance measurement. Quarterly Review of Economics and Finance 35: 187–206. [CrossRef] Chu, Jacky, Frank Chen, and Philip Leung. 2010. ETF performance measurement-Data envelopment analysis. Paper present at 2010 7th International Conference on Service Systems and Service Management, Proceedings of ICSSSM’, Tokyo, Japan, June 28–30; vol. 10, pp. 173–78. [CrossRef] Dragomirescu-Gaina, Catalin, Emilios Galariotis, and Dionisis Philippas. 2021. Chasing the ‘green bandwagon’ in times of uncertainty. Energy Policy 151: 112190. [CrossRef] Galagedera, Don UA, and Param Silvapulle. 2002. Australian mutual fund performance appraisal using data envelopment analysis. Managerial Finance 28: 60–73. [CrossRef] Gastineau, Gary L. 2001. Exchange-Traded Funds. The Journal of Portfolio Management 27: 88–96. [CrossRef] Golany, Boaz, and Yaakov Roll. 1989. An application procedure for DEA. Omega 17: 237–50. [CrossRef] Gouveia, Maria, Elisabete Duarte Neves, Luís Cândido Dias, and Carlos Henggeler Antunes. 2018. Performance evaluation of Portuguese mutual fund portfolios using the value-based DEA method. Journal of the Operational Research Society 69: 1628–39. [CrossRef] Graham, J. Edward, Carlos Lassala, and Belén Ribeiro Navarrete. 2020. Influences on mutual fund performance: Comparing US and Europe using qualitative comparative analysis. Economic research-Ekonomska istraživanja 33: 3049–70. [CrossRef] Gregoriou, Greg N. 2006. Optimisation of the largest US mutual funds using data envelopment analysis. Journal of Asset Management 6: 445–55. [CrossRef] Gregoriou, Greg N., and Stephen C. Henry. 2015. Undesirable outputs in commodities trading advisers: A data envelopment analysis approach. Journal of Wealth Management 17: 85–92. [CrossRef] Henriques, Carla Oliveira, and Maria Elisabete Duarte Neves. 2019. A multiobjective interval portfolio framework for supporting investor’s preferences under different risk assumptions. Journal of the Operational Research Society 70: 1639–61. [CrossRef] Henriques, Carla Oliveira, Maria Elisabete Neves, Licínio Castelão, and Duc Khuong Nguyen. 2022. Assessing the performance of exchange traded funds in the energy sector: A hybrid DEA multiobjective linear programming approach. Annals of Operations Research 313: 341–66. [CrossRef] [PubMed] Hill, Joanne M., Dave Nadig, and Matt Hougan. 2015. A Comprehensive Guide to Exchange-Traded Funds (ETFs). Charlottesville: CFA Institute Research Foundation. [CrossRef] Ippolito, Richard A. 1989. Efficiency with costly information: A study of mutual fund performance, 1965–1984. The Quarterly Journal of Economics 104: 1–23. [CrossRef] Isakov, Vsevolod. 2019. Performance Appraisal of Exchange-Traded Funds using Clustering and Data Envelopment analysis (Xetra, Germany). Master’s thesis, LUT School of Business and Management, Lappeenranta, Finland. Jensen, Michael C. 1968. The Performance of Mutual Funds in the Period 1945–1964. The Journal of Finance 23: 389–416. [CrossRef] Kerstens, Kristiaan, and Ignace Van de Woestyne. 2011. Negative data in DEA: A simple proportional distance function approach. Journal of the Operational Research Society 62: 1413–19. [CrossRef] Kiymaz, Halil. 2019. Factors influencing SRI fund performance. Journal of Capital Markets Studies 3: 68–81. [CrossRef] Kostovetsky, Leonard. 2003. Index mutual funds and exchange-traded funds. Journal of Portfolio Management 29: 80–92. [CrossRef] J. Risk Financial Manag. 2023,16, 130 34 of 34 Lettau, Martin, and Markus Pelger. 2020. Factors that fit the time series and cross-section of stock returns. The Review of Financial Studies 33: 2274–325. [CrossRef] Lobato, Manuel, Javier Rodríguez, and Herminio Romero. 2021. A volatility-match approach to measure performance: The case of socially responsible exchange-traded funds (ETFs). The Journal of Risk Finance 22: 34–43. [CrossRef] Murthi, B. P. S., Yoon K. Choi, and Preyas Desai. 1997. Efficiency of mutual funds and portfolio performance measurement: A non-parametric approach. European Journal of Operational Research 98: 408–18. [CrossRef] Navratil, Robert, Stephen Taylor, and Jan Vecer. 2021. On equity market inefficiency during the COVID-19 pandemic. International Review of Financial Analysis 77: 101820. [CrossRef] Neves, Maria Elisabete Duarte, Carla Manuela Fernandes, and Pedro Coimbra Martins. 2019. Are ETFs good vehicles for diversification? New evidence for critical investment periods. Borsa Istanbul Review 19: 149–57. [CrossRef] Osterhoff, Friedrich, and Christoph Kaserer. 2016. Determinants of tracking error in German ETFs–the role of market liquidity. Managerial Finance 42: 417–37. [CrossRef] Portela, Silva, Emmanuel Thanassoulis, and Gary Simpson. 2004. Negative data in DEA: A directional distance approach applied to bank branches. Journal of the Operational Research Society 55: 1111–21. [CrossRef] Poterba, James M., and John B. Shoven. 2002. Exchange-traded Funds: A New Investment Option for Taxable Investors. American Economic Review 92: 422–27. [CrossRef] Prasanna, P. Krishna. 2012. Performance of Exchange-Traded Funds in India. International Journal of Business and Management 7: 122. [CrossRef] Proença, C., E. Neves, M. Gouveia, and M. Madaleno. 2023. Technological, healthcare and consumer: Influence of COVID-19. Operational Research,unpublished. Roll, Richard. 1978. Ambiguity when performance is measured by the securities market line. The Journal of Finance 33: 1051–69. [CrossRef] Sharp, John A., Wang Meng, and Wei Liu. 2007. A modified slacks-based measure model for data envelopment analysis with “natural” negative outputs and inputs. Journal of the Operational Research Society 58: 1672–77. [CrossRef] Sharpe, William F. 1966. Mutual Fund Performance, Part 2: Supplement on Security Prices. The Journal of Business 39: 119–38. [CrossRef] Tone, Kaoru. 2001. Slacks-Based measure of efficiency. European Journal of Operational Research 130: 498–509. [CrossRef] Tone, Kaoru, Tsung-Sheng Chang, and Chen-Hui Wu. 2019. Handling negative data in slacks-based measure data envelopment analysis models. European Journal of Operational Research 282: 926–35. [CrossRef] Tsolas, Ioannis E. 2019. Utility exchange traded fund performance evaluation. A comparative approach using grey relational analysis and data envelopment analysis Modelling. International Journal of Financial Studies 7: 67. [CrossRef] Tsolas, Ioannis E. 2022. Performance Evaluation of Utility Exchange-Traded Funds: A Super-Efficiency Approach. Journal of Risk and Financial Management 15: 318. [CrossRef] Tsolas, Ioannis E., and Vincent Charles. 2015. Green exchange-traded fund performance appraisal using slacks-based DEA models. Operational Research 15: 51–77. [CrossRef] Zopounidis, Constantin, Michael Doumpos, and Panos M. Pardalos, eds. 2010. Handbook of Financial Engineering. Berlin: Springer Science & Business Media, vol. 18. Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.