Limits to the information gain from lidar measurements

Limits to the information gain from lidar measurements Aniceto Belmonte Technical University of Catalonia, BarcelonaTech, Dept. of Signal Theory and Communications, 08034 Barcelona, Spain [email protected] Received December 3, 2014 Measurements over the return signal are an integral part of lidar remote sensing by which we gather information about the characteristics of specific targets. But how much information is gained by performing a given lidar measurement? By defining Shannon’s mutual information of a lidar observation, here we consider the bits of information content on the measurement and describe mathematically the capacity of lidar estimates to represent a corresponding property in the target. For heterodyne Doppler lidars in particular, we have found simple analytical formulas that considers the information gain in mean-frequency estimates. © 2014 Optical Society of America OCIS Codes: (280.4788) Optical sensing, (280.3640) Lidar, (110.3055) Information theoretical analysis, (010.0010) Atmospheric and oceanic optics, (170.3880) Medical and biological imaging. The problem of lidar remote sensing is to gain information from the received optical signals about the characteristics of the observed target. In the context of statistical parameter estimation, where a sample of observations depends on a set of target parameters that needs to be estimated, the precision of the signal estimates is commonly investigated as a measure of how well the parameters can be anticipated from the lidar observations. However, the precise amount of information gained by performing a lidar measurement is unknown. Here we use the framework of information theory to provide the limits to the information capacity of a lidar system, or how many bits of information can be obtained from the received signal about the identity of the target that is being observed. Unwanted noise and target speckle effects, those producing great signal variability, limit information gain. Without consideration of the properties of the target to be observed, limits to the information content of lidar estimates in atmospheric, oceanic, or biological media can be quantified. In this paper we shall reflect on the role of Shannon mutual information in the evaluation of lidar performance and use the analysis of information content of lidar estimates to complement classical studies on precision of signal parameter estimation. Although the work presented in this letter is particular to the lidar remote sensing problem, we note that the subject of information theory with applications to radar has been considered in the past. Early work used information-theoretic ideas to formulate a posteriori radar receiver, which in the case of additive white Gaussian noise results in the correlation receiver [1]. Later, the emphasis changed to the problem of radar waveform design for the purpose of extracting information about a target [2]. Radar signature analysis techniques using information theory are also of interest to radar sensor optimization [3]. We consider a lidar receiver that responds to the targetscattering signals of a sequence of 𝑁 transmitted laser pulses. We denote the lidar responses by a vector 𝑧!=𝑧!,𝑧!,…,𝑧! describing the 𝑀 stochastic signal samples obtained from the 𝑛th lidar pulse. As 𝑁 of these response vectors are accumulated, then the total lidar response is 𝑧=𝑧! ! !!!. The accumulated lidar observation 𝑧 depends on a set of target parameters –such as position, speed, or scattering mechanism– which are usually continuous and need to be estimated. The target parameters are denoted by the scalar variable 𝜃, which have probability 𝑃𝜃. The lidar observation vector 𝑧 is also conditional on the transmitted pulse characteristics – such as power, temporal and spectral width, or rate – and other measurement system parameters, which are denoted by 𝜎 and are usually known at the receiver. The conditional probability of 𝜃 given 𝑧 and 𝜎, written as 𝑃𝜃|𝑧,𝜎, defines the probability that a specific value of parameter 𝜃 will occur given transmitted pulse parameters 𝜎 and the knowledge that observations 𝑧 have already occurred. If we consider the case of statistically independent lidar pulses, the parameter probability is given by 𝑃𝜃|𝑧,𝜎=𝑃!𝜃|𝑧!,𝜎 ! !!!, where 𝑃!𝜃|𝑧!,𝜎 is the probability of a target parameter 𝜃 conditional on the 𝑖th lidar sample and system parameters 𝜎. In this paper we focus on the amount of information 𝐼𝜃;𝑧|𝜎 gained about the target parameter 𝜃 by performing the measurement 𝑧 with system parameters 𝜎 and consider the Shannon mutual information [4] of the lidar observation. In our case, it is the average amount of information on the target parameter 𝜃 that is added by observing the lidar response 𝑧. For a specific parameter 𝜃, the Shannon’s mutual information needs to be defined as 𝐼𝜃;𝑧|𝜎=ℎ𝜃|𝜎−ℎ𝜃|𝑧,𝜎!. (1) Here, the term ℎ𝜃|𝜎 is the entropy of the prior distribution 𝑃𝜃|𝜎=𝑑𝑧!𝑃𝜃|𝑧,𝜎 for the target parameter 𝜃 and describes the initial ignorance about the parameter: ℎ𝜃|𝜎=−𝑑𝜃 !𝑃𝜃|𝜎!𝑙𝑜𝑔!𝑃𝜃|𝜎!. (2) The second term ℎ𝜃|𝑧,𝜎 in Eq. (1) is the entropy of the posterior distribution 𝑃𝜃|𝑧,𝜎, ℎ𝜃|𝑧,𝜎=𝑑𝜃!𝑃𝜃!𝑑𝜃!𝑃𝜃|𝑧,𝜎!𝑙𝑜𝑔!𝑃𝜃|𝑧,𝜎!,! (3) and describes the final ignorance about the target parameter 𝜃 after the measurements 𝑧. Because the target parameter 𝜃 is itself a random quantity, Eq. (3) considers the average over the parameter distribution 𝑃𝜃. In the expression Eq. (1) for the gain of information 𝐼𝜃;!𝑧!|𝜎 from lidar signals – for the purposes of this analysis, we use logarithms to the base two so 𝐼𝜃;!𝑧!|𝜎 is measured in bits, – the entropy of the prior distribution 𝑝𝜃|𝜎!for the target parameter 𝜃 is considered along with the corresponding entropy of the posterior distribution 𝑃𝜃|𝑧,𝜎 after measurement 𝑧. Entropy is a measure of ignorance, and the gain of information 𝐼𝜃;𝑧!|𝜎 is a difference between the initial ignorance and the final ignorance. A lidar system observes a target to decrease the a priori lack of knowledge about that target and, in this sense, the mutual information 𝐼𝜃;𝑧!|𝜎 tell us how many bits of information observation 𝑧 provides about the target parameter 𝜃. Lidar measurements provide sufficient information gain to classify parameter 𝜃 into a maximum of 𝜅=2!!;!!!|! equally likely partitions or choices —by definition, the number of information bits is equal to the logarithm of the number of choices taken to the base two. At this point we need to consider the characteristics of the posterior entropy ℎ𝜃|𝑧,𝜎 for lidar observations. As the dimension of the data samples from lidar measurements –given by 𝑁𝑀– is usually large, 𝑃𝜃|𝑧,𝜎 will appear to be peaked around its most probable value given observations, i.e., the maximum likelihood estimation of parameter 𝜃. The large sample notion allows the use of a saddle-point method to solve the integral over 𝑃𝜃|𝑧,𝜎!𝑙𝑜𝑔!𝑃𝜃|𝑧,𝜎 in Eq. (3), and approach the final ignorance ℎ𝜃|𝑧,𝜎 in terms of Fisher’s information metric 𝒥!𝜃 of the continuous parameter 𝜃 as [5, 6] ℎ𝜃|𝑧,𝜎≈12𝑑𝜃!𝑃𝜃!𝑙𝑜𝑔!2𝜋𝑒 𝒥!𝜃!. (4) Error terms 𝑂1𝑁𝑀 in Eq. (4) can be neglected for large 𝑁𝑀. The well-known Fisher’s metric 𝒥!𝜃 –usually considered on the problem of quantifying the efficiency of measuring 𝜃 in the lidar responses 𝑧 – relates to the derivative with respect to the target parameter of the lidar response probability. When the lidar measurement of 𝜃 considers an efficient maximum likelihood estimator, 𝒥!𝜃 is just the reciprocal of the estimate variance, i.e., 𝒥!𝜃=1𝜎! ! [7]. Finally, putting Eqs. (2) and (4) in Eq. (1) leads to the following expression for the information gain from lidar measurements: 𝐼𝜃;𝑧!|𝜎=−𝑑𝜃 !𝑃𝜃|𝜎!𝑙𝑜𝑔!𝑃𝜃|𝜎 −12𝑑𝜃!𝑃𝜃!𝑙𝑜𝑔!2𝜋𝑒 𝒥!𝜃!. !! (5) To research and clarify the implications for lidar remote sensing of Eq. (5), we now turn to a specific case and discuss the information gain from the measurement of heterodyne Doppler lidar return signals [8, 9]. Optical Doppler techniques are of interest in systems operating in the atmosphere, ocean, or biological media. Heterodyne lidars sample the output signals of a complex receiver and vector 𝑧 describes the 𝑀 complex data samples obtained from the transmitted pulses. Many Doppler lidar estimations are based on the spectrum 𝑢 of the complex data vector 𝑧 and therefore require a sufficiently long data sequence 𝑀 for the spectrum to be well defined. The vector 𝑢 describe the power levels obtained in the 𝑀 spectral channels of the spectral sample. Because the spectral vector 𝑢 is the linear discrete Fourier transform of the data vector 𝑧, using the transform vector 𝑢 produces the same information gain than using 𝑧. Note that the lidar return signal spectrum is a convolution of the transmitted pulse spectrum and the target Doppler spectrum. With short pulses, target motion broad barely the signal spectrum and the spectral width of the signal 𝜔 will be determined only by the pulse profile and therefore known a priori. For a laser pulse with Gaussian temporal profile, the spectral width 𝜔 of the signal is inversely proportional to the pulse temporal width. The estimations of the mean power – or, equivalently, the signal-to-noise ratio 𝛾– and the Doppler mean frequency shift 𝑓 are important for lidar remote sensing. Although both of these parameters have unbiased, efficient maximum likelihood estimators, here we will be concerned only with 𝑓 Doppler estimates providing information about the radial component of the target velocity. If we assume that SNR 𝛾 and signal spectral width 𝜔 are known, the performance of the maximum likelihood estimator of 𝑓 is well known [10-13]. Earlier work on the processing of radar and lidar signals found analytical formulas describing asymptotic bounds to the performance 𝒥!𝑓 of the mean frequency estimator [1011]. Exact, numerically efficient representations of the Fisher’s metric 𝒥!𝑓 have been presented in detail [12]. Approximate 𝒥!𝑓 expressions for the estimation of the mean frequency, which can be easily computed in most practical situations, are also relevant on bounding Doppler lidar performance [13]. Without lost of generality, in our analysis we will consider these simplified expressions for 𝒥!𝑓. In order to solve the information gain Eq. (5) for the mean frequency 𝑓, we need to express the prior and posterior entropies, ℎ𝑓|𝜎 and ℎ𝑓|𝑧,𝜎, respectively. For the posterior probability, as the Fisher’s metric 𝒥!𝑓 is independent of the mean frequency 𝑓, i.e., 𝒥!𝑓=𝒥! [13], the entropy of the posterior distribution Eq. (4) simplifies to ℎ𝑓|𝑧=12!𝑙𝑜𝑔!!2𝜋𝑒!!𝒥!. For the prior probability, as the pulse spectral width 𝜔 determines unambiguously the signal spectral width, the probability of 𝑓 is conditional on the uncertainty 𝜔 of the received signal frequency. Also, as the lidar measurement uses a frequency estimator, the uncertainty of the estimator 𝜎 ! needs to be added to the prior probability. Under the usual assumptions of Gaussian pulse profile and efficient, normally distributed maximum likelihood frequency estimator, the best prior distribution 𝑃𝑓|𝜎 for the mean frequency 𝑓 is a Gaussian with variance 𝜔!+1𝒥!. On these terms, the prior entropy Eq. (2) results ℎ𝑓|!𝜎= 12𝑙𝑜𝑔!!2𝜋𝑒!𝜔!+1𝒥!!. Now, using the prior ℎ𝑓|𝜎 and posterior ℎ𝑓|𝑧,𝜎 entropies above, the information gain 𝐼𝑓;𝑧!|𝜎 for the mean frequency 𝑓 can be stated using Eq. (5) as 𝐼𝑓;𝑧!|𝜎=12!!𝑙𝑜𝑔!!1+!𝜔!!𝒥!!!.!! (6) This result offers an intuitively appealing explanation. If the estimation problem has small variance, which for an efficient estimator is given by the reciprocal of the Fisher information 𝜎 ! !=1𝒥! [7], then the occurrence of this measurement brings much information. If 𝜔!𝒥!= 𝜔!𝜎 ! !≫1, then 𝐼𝑓;𝑧!|𝜎≈𝑙𝑜𝑔!!𝜔𝜎 !! the information gain is logarithmic in the ratio of pulse bandwidth 𝜔 to estimation uncertainty 𝜎 !. This ratio 𝜔𝜎 ! defines the maximum number 𝜅 of equiprobable partitions of 𝑓. When the ratio 𝜔𝜎 ! is small, the information gain 𝐼𝑓;𝑧!|𝜎≈12𝜔!𝜎 ! !!𝑙𝑜𝑔!𝑒 is also small and the number of likely partitions of 𝑓 tends asymptotically to 𝜅=1. Note that the information gain 𝐼𝑓;𝑧!|𝜎 from the parameter measurement cannot be less than zero as the ignorance about the parameter after the measurement cannot be larger than the initial ignorance. We need to consider briefly the information capacity of the mean-frequency measurement. In essence, information capacity can be used to establish fundamental limits to the problem of lidar information measurement that is independent of any particular approach to solving it. It provides an upper bound on lidar information gain that can be approached but not exceeded. The capacity is defined as the maximized mutual information over all the priors, i.e., 𝐶=max!!𝐼𝑓;𝑧!|𝜎 [4]. For a given mean and variance, Gaussian distributions always reach this maximum. Consequently, by considering a Gaussian laser pulse profile and an efficient Gaussian estimator, the result presented in Eq. (6) achieves capacity and defines an upper bound to Doppler information gain. This is an important result as the assumptions of a Gaussian pulse profile or an efficient Gaussian estimator may not be well founded in some practical situations. Even in these cases where 𝐼𝑓;𝑧!|𝜎 cannot achieve capacity, Eq. (6) is useful describing the upper bound to the information gain of mean frequency 𝑓 estimates. The information gain in mean-frequency estimates, calculated from Eq. (6), is shown in Fig. 1 as a function of different system parameters. Typical parameters for a heterodyne lidar are chosen to be signal bandwidth normalized to the sampling frequency 𝜔=0.1, and number of complex samples per estimate 𝑀=128. The plots consider different values of lidar pulse return accumulation (spectral accumulation) 𝑁. The signal energy per mean-frequency estimate is measured with the effective number of coherently detected photons 𝛾 for the accumulated return data sample. For a photon noise limited heterodyne receiver, 𝛾 corresponds to the ratio over the entire spectral bandwidth of the signal power to noise power, or wideband SNR per observation time. Information gain 𝐼𝑓;𝑧!|𝜎 as a function of SNR 𝛾 is shown in Fig. 1 (a). The figure considers when a discretely sampled spectral is measured from only a lidar pulse return (𝑁=1), and when several of these sample spectra are accumulated to define the total sample spectrum (𝑁 of 10, 100, and 1000). For small SNR, when the accumulated signal is too poor to produce good estimates, by gradually increasing the accumulated energy guarantees a steady gain of information. But, at large SNR, when performing the lidar measurement has eliminated most ignorance about the mean frequency 𝑓, the amount of information gain reaches a threshold. Information content in further received signals from the same target mostly echo already known information. These two different behaviors of the information gain 𝐼𝑓,𝑧!|𝜎 can be easily quantified by considering the analytical formulas describing asymptotic bounds to Fisher’s metric 𝒥! of the mean frequency estimator [11]. On the one hand, for small SNR 𝛾, Fisher’s metric 𝒥!≈𝑁𝑀𝛾!4𝜋!!𝜔! and the mutual information Eq. (6) simplifies to 𝐼𝑓;𝑧!|𝜎=12!𝑙𝑜𝑔!!1+!𝑁𝑀𝛾!4𝜋!!𝜔!!.!! (7) This result explains the logarithmic increases in information with SNR 𝛾 shown in Fig. 1 (a). On the other hand, for large SNR, 𝒥!≈!𝑁𝑀 12𝜔! and information gain reach a threshold independent of SNR 𝛾: 𝐼𝑓;𝑧!|𝜎=12!!𝑙𝑜𝑔!!1+!𝑁𝑀 12𝜔!!!.!! (8) Equation (8) describes how the information threshold for high SNR grows logarithmically with accumulation order 𝑁. To illustrate these information asymptotic bounds, Fig. 1 (a) considers information gain for a discretely sampled spectral measured from a set of 𝑁=1000 lidar pulse returns, an otherwise usual accumulation order in practical heterodyne lidars. The dashed, broken curve shows asymptotic bounds for this accumulation level. In this case, information gain shows values as high as 9 bits, corresponding to a maximum 𝜅=512 number of equiprobable partitions for 𝑓. Note that for this accumulation number, information gain increases rapidly with SNR but only when the amount of received energy reach a minimum level. The minimum SNR required to reach a mere 2-bit gain of information is a considerable high 10-dB per accumulated sample spectrum. As the accumulation order 𝑁 increases, and the signal spectrum become less noisy, it is easy to understand how spectral accumulation produces a higher information gain from lidar estimates. To assess the effects of accumulation from a different perspective, Fig. 1 (b) shows the information gain at several fixed levels of 𝛾 (SNR of 0, 10, 20, and 40 dB) as the accumulation order 𝑁 increases. After reaching a maximum at a certain accumulation order, the information gain falls and ultimately goes to zero. This is not surprising, as the estimation of the mean frequency involves obtaining the location of the spectral peak at the signal spectra in the presence of random noise, and positioning the precise peak is easier when the spectral feature of the signal is not buried in the noise. Usually, when the accumulated SNR 𝛾 is large –and the energy into one single pulse is enough to produce optimal signals for make good estimates, – increasing the accumulation 𝑁 of independent samples result in a decrease of the statistical uncertainty and, consequently, an increase of the information gain. However, when the accumulated SNR 𝛾 is small, and the energy received from a single pulse averages less than 1 photon per sample, spectral signals are too noisy to produce good estimates. In this case, using more energetic pulses, to concentrate all the energy into a few sampled spectra, is the best strategy to overcome noise. Information gain increases with spectral accumulation –and eventually reaches a maximum, – when every sampled spectrum collects enough energy to prevail upon noise. As expected, the maximum value of information gain increases logarithmically with the SNR 𝛾 per observation time. In conclusion, the problem of finding the information content of a lidar observation has been confronted. We identify Shannon’s mutual information of a lidar measurement as the bits of information gained by performing it, independent of the target state on which it is performed. The current work depicts in Eq. (5) a simple dependence of the mutual information on the precision of signal estimates that can be used to explain the basic information capacity of lidar measurements. We study new logarithmic measures of information in lidar sensing. For the use of the heterodyne receiver scheme in particular, we have analyzed with Eq. (6) the information gain in mean-frequency estimates and found how many bits of information can be extracted from heterodyne signals as a function of different measurement parameters (see Fig. 1). We have shown that Eq. (6) achieves capacity and describes an upper bound to the information content of Doppler estimates. In this letter we have considered how much information we can infer from a particular set of lidar data about the observed target in terms of mutual information. As a necessary complement to conventional estimation theory, and to capture all important features of signal information content, the subject of this work has been the study of novel logarithmic measures of information in lidar observations. The rigorous framework of the information theoretic approach may offer new perspectives on the field of lidar remote sensing. This research was partially funded by the Spanish Department of Science and Innovation MICINN Grant No. TEC 2012-34799. REFERENCES 1. P. M. Woodward, I. L. Davies, Proc. IEE 99, 37-44 (1952). 2. M. R. Bell, IEEE Trans. Inform. Theory 39, 15781597 (1993). 3. J. A. Malas, K. M. Pasala, Proc. IEEE Radar Conf. 2008, 26-30 (2008). 4. T. M. Cover, J. A. Thomas, Elements of Information Theory (Wiley-Interscience, 2006). 5. J. Rissanen, IEEE Trans. Inform. Theory 42, 40-47 (1996). 6. N. Brunel and J.P. Nadal, MIT Press Neur. Comp. 10, 1731-1757 (1998). 7. L. L. Scharf, Statistical Signal Processing: Detection, Estimation, and Time Series Analysis (Prentice Hall, 1991). 8. T. Fujii and T. Fukuchi (Eds.), Laser Remote Sensing (CRC Press, 2005). 9. Claus Weitkamp (Ed.), Lidar: Range-Resolved Optical Remote Sensing of the Atmosphere (Springer, 2005). 10. M. J. Levin, IEEE Trans. Inform. Theory 11, 100107 (1965). 11. D. S. Zrnic, IEEE Trans. Geosci. Electron. 17, 113128 (1979). 12. R. Frehlich, IEEE Trans. Geosci. Remote Sensing 31, 1123-1131 (1993). 13. B. J. Rye and R. M. Hardesty, IEEE Trans. Geosci. Remote Sensing 31, 16-27 (1993). ! Fig. 1. For heterodyne Doppler lidars, the curves represent bits of information gain 𝐼!𝑓 !;𝑧!!|𝜎! obtained by mean-frequency estimators 𝑓 ! over the signal returns 𝑧! about the Doppler shift of the observed target. The curves consider a signal bandwidth, normalized to the sampling frequency, of 𝜔=0.1 and use 𝑀=128 spectral channels. (a) The results are a function of the coherent SNR (number of photons 𝛾! coherently detected) per accumulated sample spectrum and ponder different values of lidar spectral accumulation 𝑁 . (b) Now the curves are a function of the spectral accumulation 𝑁 and consider different coherent SNR levels per accumulated spectrum (𝛾! of 0, 10, 20, and 40 dB).