Digital communication receivers using Gaussian processes for machine learning

Digital Communication Receivers Using Gaussian Processes for Machine Learning Fernando P´ erez-Cruz and Juan Jos´ e Murillo-Fuentes ∗ May 27, 2008 Abstract We propose Gaussian processes (GPs) as a novel nonlinear receiver for digital communication systems. The GPs framework can be used to solve both classification (GPC) and regression (GPR) problems. The optimal minimum mean squared error solution is the expectation of the transmitted symbol given the information at the receiver, which is a nonlinear function of the received symbols for discrete inputs. GPR can be presented as a nonlinear MMSE estimator and thus capable of achieving optimal performance from MMSE viewpoint. Also, the design of digital communication receivers can be viewed as a detection problem, in which GPC is specially suited as it assigns posterior probabilities to each possible transmitted symbol. In this paper, we explore the suitability of GPC and GPR as nonlinear digital communication receivers. The major advantage of GPs is that they are Bayesian ∗This work was partially funded by the Spanish government (Ministerio de Educaci´ on y Ciencia TEC2006-13514-C02-01/TCM and TEC2006-13514-C02-02/TCM), the European Union (FEDER) and the Comunidad de Madrid (project ‘PRO-MULTIDIS-CM’, id. S0505/TIC/0223). Fernando P´ erez-Cruz is supported by Marie Curie Fellowship 040883-AI-COM. F. P´ erez-Cruz is in the Electrical Engineering Department at Princeton University. Princeton, NJ 08544. He is also an Associate Professor in Universidad Carlos III de Madrid. E-mail: [email protected] J.J. Murillo- Fuentes is with the Dept. Teor´ ıa de la Se˜ nal y Comunicaciones, Escuela T´ ecnica Superior de Ingenieros, Universidad de Sevilla, Paseo de los Descubrimientos s/n, 41092 Sevilla, Spain. E-mail: [email protected] 1 machine learning tools that allow a formulation of a likelihood function for its hyperparameters, which can then be set optimally. Thereby, it uses an optimal hyperparameter setting to achieve the best nonlinear receiver for each training sequence. GPs outperform state-of-the-art nonlinear machine learning approaches that prespecify their hyperparameters or rely on crossvalidation, whose computational complexity is high and unpredictable. We illustrate the advantages of GPs as digital communication receivers for linear and nonlinear channel models for short training sequences and compare them to other state-of-the-art nonlinear machine learning tools such as support vector machines. 1 Introduction Gaussian Processes are typically used to characterize the noise component in digital communication systems, as it is mainly caused by thermal noise fluctuations [28]. In this paper, we propose the Gaussian processes (GPs) framework to design nonlinear receivers in digital communication systems. GPs were initially presented as a nonlinear estimation technique in 1978 [22] and were rapidly forgotten due to its computation complexity. In the mid-nineties, they were independently rediscovered [39]. Since then they have been shown to fit many different applications [31] and nowadays their computational complexity is no longer a limiting issue [29]. There is a vast literature on machine learning techniques for designing digital communication systems. The channel equalization problem has been addressed with different machine learning tools, such as: multi-layered perceptrons (MLPs) [10], radial basis function networks (RBFNs) [5], recurrent RBFNs [7], self-organizing feature maps (SOFMs) [16], wavelet neural networks [4], GCMAC [13], kernel adaline (KA) [20] or support vector machines (SVMs) [27], among many others. Other digital communication systems that have also benefited from nonlinear detection and estimation algorithms are multi-user detection [9, 35], multiple-input multiple-output systems [32], beam forming [19], pre-distortion [12] and plant identification [1], to name a few. For these machine learning approaches, it is necessary to prespecify the hy- 2 perparameters (structure), since standard methods for searching the optimal hyperparameters (i.e. cross-validation [15, 2]) require immense computational resources, which are not available in most communication receivers, and also their training time is highly variable. As a result, they use a suboptimal structure that requires longer training sequences for ensuring optimal receiver performance. Also, it makes the length of the training sequence hard to predict, as it depends on how well the chosen structure or hypeparameters fits the current problem. For example, SVM with a Gaussian kernel needs to fit its width, which is proportional to the noise level [20, 27, 6]. If the width is too large, the SVM can be optimized with short training sequences, but its performance is poor. If it is too small, it requires a significantly longer training sequence to avoid overfitting. For each instantiation of the problem there is an optimal width. This kernel width depends not only on the channel values and noise level, as we would expect, but also on the actual values of the noise themselves. Ideally, we would like to choose the kernel width every time we receive a new training sequence. But this would involve training a different SVM for each possible width and then choosing the optimal receiver (validation). In addition, this width is not the only SVM’s hyperparameter. We must also validate the soft-margin that trades off the minimization of the training errors and the maximization of the margin. Therefore, we would have to train a set of receivers with different width and soft-margin hyperparameters to find the optimal setting in each problem. However, typically, we can only solve a single optimization problem in the receiver. We thus prespecify the SVM hyperparameters, as it is the case with other nonlinear tools referenced earlier. In previous work, we introduced Gaussian processes for machine learning as a novel nonlinear tool for designing digital communication receivers. Gaussian processes can be applied to regression and classification problems [31] and in this paper we use both settings for tuning digital communication receivers with short training sequences. We compare Gaussian processes for regression (GPR) and Gaussian processes for classification (GPC) to state-of-the-art linear and nonlinear receivers to show their strength in solving this relevant problem. We have presented 3 some preliminaries results for multi-user detection in CDMA systems [21, 26] and channel equalization in [3]. In this paper we extend these results and include GPC in our comparisons. Gaussian processes for machine learning are rooted in Bayesian statistics [31] and, consequently, build a likelihood function for its hyperparameters given the training examples. This likelihood can be optimized to set the hyperparameters. This property makes GPs an attractive tool for designing nonlinear digital communication receivers, compared to other nonlinear machine learning tools, because the hyperparameters can be optimally set for each instantiation of our problem with a single optimization procedure. For short training sequences hyperparameter mismatch significantly affects the performance of digital communication receivers, while for longer training sequences this performance is not sensitive to variations in the hyperparameters. Most papers applying nonlinear machine learning for designing digital communication receivers propose fixed hyperparameters and sufficiently long training sequences. We focus on short training sequences and show that fixed hyperparameters underperform compared to GPR receivers with optimally trained hyperparameters. Gaussian processes can be extended for solving classification problems. In this case the posterior is no longer tractable and we need to use approximations to compute the prediction for each class label [31]. A Gaussian distribution is typically used to approximate the GPC’s posterior, either using Laplace [38] or expectation propagation methods [17]. However, GPC computational complexity is significantly higher than that of GPR and hence they might not be as suited for designing digital communication receivers as GPR are. Moreover, their performance is not as good as that of GPR receivers as we show and explain in the experimental section. The rest of the paper is organized as follows. We present the design of digital communication receivers as an optimization problem in Section 2 and show how different nonlinear machine learning tools can be fitted in this framework. Section 3 is devoted to Gaussian processes for regression and how it can be understood as 4 a nonlinear MMSE estimation. The optimization of the GPR hyperparameters is proposed in Section 4. GPC are introduced briefly in Section 5. We present some computer simulations in Section 6 to illustrate the benefits of GPR for channel equalization and multi-user detection compared to other state-of-the-art nonlinear tools. We conclude with some final remarks and proposed further work in Section 7. 2 Nonlinear optimization for communication receivers 2.1 Channel model and MMSE We consider throughout the paper the following deterministic channel model: x=Hs +z(1) where sis a random variable column-vector representing the transmitted symbols, Hcorresponds to the deterministic channel gains, unknown to both the transmitter and receiver, zis zero-mean Gaussian noise, and xrepresents the received symbols. This model is general enough to capture most standard communication systems. For example: •Inter Symbol Interference: Each element in sis a symbol transmitted at a different time instant. His a Toeplitz matrix, in which each row represents the channel impulsive response. •Multiple-Input Multiple-Output:(H)ij represents the gain from the ith receiving antenna to the jth transmitting antenna and srepresents the symbols transmitted by the antenna array. •Fading:His a diagonal matrix with the fading coefficients and srepresents the symbols transmitted at each time instant. •CDMA: The columns of Hcollect each user’s spreading code and each element of srepresents the symbol transmitted by the users. 5 We can also combine different Hmatrices to accommodate other communication systems. For example H=H1H2H3, where H1is a Toeplitz matrix representing an inter-symbol interference channel model, H2contains the spreading codes of a CDMA system, and H3is a diagonal matrix assigning different power to each user. This Hmatrix represents the downlink channel in a mobile communication network. The source sthat achieves capacity (maximum information transmission rate) [8] is a zero-mean Gaussian distribution with a covariance matrix given by the right eigenvectors of the channel matrix [30]. sbeing a continuous random variable, we can estimate in the receiver the transmitted vector using a minimum mean squared error (MMSE) detector: fmmse(x) = argmin f(·)Ehks−f(x)k2i(2) The function fmmse(x)is the mean value of sgiven the received vector x, E[s|x], which is a linear function of xif sis Gaussianly distributed. Practical structural constraints dictate the use of discrete constellations, such as PSK and QAM, which depart from the optimal Gaussian distributions. Although linear detectors cannot achieve E[s|x]if sis a discrete random variable1, and thus the MMSE is only a proxy for minimizing the probability of misclassification, still digital communication receivers use linear MMSE detectors for estimating the transmitted vector, because they can be easily implemented and hopefully their performance is not severely degraded. The linear MMSE solution is given by: wmmse = argmin w Es−w>x2=Ehxx>i−1E[xs].(3) If His unknown, we can replace the expectations by sample averages using a training sequence. 1Even for a simple example, if s∈ {±1}and equally likely and H= 1 then E[s|x] = tanh(x/σ2 z). 6 2.2 Machine learning for digital communication receivers The design of digital communication receivers can be readily understood as a supervised classification problem [10, 23], in which the receiver constructs a classifier for deciding over the incoming symbols. Machine learning tools optimize the risk of misclassification: fopt(x) = argmin f(·)E[L(s, f(x))] = argmin f(·)ZL(s, f(x)) p(s, x)dsdx,(4) where L(·)is a loss-function that measures the penalty for wrongly classifying a pattern and f(x)is the nonlinear model to predict s. The joint density, p(s, x), is typically unknown and thus we use a training sequence {xi, si}n i=1 and the empirical risk minimization (ERM) inductive principle [36] to obtain the optimal solution: b fopt(x) = argmin f(·)(n X i=1 L(si, f(xi)) + λΩ(||f||)),(5) where we have included a regularization term, λΩ(||f||), to avoid overfitting and to ensure that the minimum of the empirical risk converges to the minimum risk [36] as the number of training samples increases. The number of training patterns ndetermines the symbols in the preamble of each transmission needed to adjust the receiver. This number should be small to maximize the number of bits used to transmit information, as we need to retransmit the preamble in each burst of data. The nonlinear machine learning approaches mentioned in the introduction can be cast as the optimization in (5) using an appropriate nonlinear model, lossfunction and regularizer. For example: f(x) = w>φ(x), where φ(x)is a nonlinear transformation to a higher dimensional space; L(si, f(xi)) = (1 −siw>xi)+, hinge-loss2; and Ω(||f||)) = ||w||2weight decay [2], gives an SVM for a binary antipodal constellation, which constructs the nonlinear classifier using the ‘kernel trick’ for φ(·)[34]. The convexity of the optimization in (5) depends on f(·),L(·,·)and Ω(·). In some cases, as in SVM or KA, it leads to a convex functional and in others, as in 2(y)+= max(y, 0) 7 MLP or RBFN, it does not. But in any case, these machine learning approaches rely on an iterative optimization tool [2, 34] for solving (5). If we choose f(x) = w>φ(x),L(s, f(x)) = (s−w>φ(x))2and Ω(f) = ||w||2we get a convex functional: wnl mmse = argmin w(n X i=1 (si−w>φ(xi))2+λ||w||2)(6) that can be analytically optimized: wnl mmse =Φ>Φ+λI−1Φ>s.(7) where Φ= [φ(x1),...,φ(xn)]>and s= [s1, . . . , sn]>. We denote this solution as nonlinear MMSE, since it is a nonlinear extension of (3), in which we have substituted xby φ(x)and we have replaced the expectations by sample averages. In the next section we show (7) is equivalent to the mean solution provided by Gaussian processes for regression with a Gaussian likelihood function and that it can be solved using kernels [24]. Moreover, interpreting (7) as GPR allows optimizing its hyperparameters by maximum likelihood (Section 4). This optimization improves the performance of (7) with respect to other nonlinear machine learning procedures when the number of training samples is low, because for reduced training datasets the performance of nonlinear machine learning methods significantly depends on its hyperparameters. 3 Gaussian Processes for Regression In the past few years a new Bayesian machine learning tool based on Gaussian processes (GPs) has been developed for nonlinear regression estimation [39, 31, 37]. In a nutshell, Gaussian processes for regression (GPR) assume that a GP prior governs the set of possible regressors. Consequently, the joint distribution of training and test data is given by a multidimensional Gaussian density function and the predicted distribution for each test point is estimated by conditioning on the training data. 8 We present GPR from the Bayesian generalized linear regression viewpoint. Although from this opening we lose the GPs interpretation and we can only work with Gaussian likelihood models, we believe it is a simpler way to understand GPR. This approach mimics how most machine learning textbooks introduce nonlinear regression [2, 34, 14] and it helps understanding GPR as a nonlinear MMSE estimation. Therefore, practitioners in signal processing for digital communications can readily relate to this new tool for estimation and detection. Both interpretations are described in [37], where they are shown to be identical for Gaussian likelihood models. There is more to GPs than what we introduce in this summary, for interested readers GPs extensions can be found in [31]. A generalized linear regressor expresses the input-output relation as s=w>φ(x) + ν, (8) where φ(·)is a nonlinear transformation to a higher dimensional feature space and νis a random variable that measures the deviation between sand its estimate. Given a labeled training sequence (D={xi, si}n i=1, where the input xi∈Rdand the output si∈R) and a statistical model for ν, we can compute the regressor w by maximum likelihood (ML), wML = argmax w n Y i=1 p(νi) = argmax w n Y i=1 p(si−w>φ(xi)).(9) We use these ML weights to predict the outputs for future test points x∗: s∗=w> MLφ(x∗).(10) In Bayesian machine learning wis considered to be a random variable and, to predict the outcome of x∗, we use its conditional density given the training dataset, p(w|D). This conditional density, known as the posterior of w, can be computed through Bayes rule, p(w|D) = p(w|s,X) = p(s|X,w)p(w) p(s|X)=p(w) p(s|X) n Y i=1 p(si|xi,w),(11) 9 tractable density models. A generalized linear binary classifier predicts for an input xthe class label as follow: p(s= +1|w,x) = p(s= +1|f) = σ(f),(29) where f=w>φ(x)is an underlying continuous function, σ(·)is a sigmoid3that squashes fbetween 0 and 1, and p(s=−1|f)=1−p(s= +1|f). Given a labeled training sequence (D={xi, si}n i=1, where the input xi∈Rd and the output si∈ {±1}), we can compute the posterior over the underlying function f= [f1, . . . , fn]>using Bayes rule, as we did in Section 3 for GPR with w, and we can integrate out fto predict the class label for any new test point x∗. We can compute the class label for the test samples as follows: p(s∗= +1|x∗,D) = Zσ(f∗)p(f∗|x∗,D)df∗(30) where p(f∗|x∗,D) = Zp(f∗|x∗,X,f)p(f|D)df(31) and p(f|D) = p(f|X,s) = Qip(si|fi)p(f|X) p(s|X).(32) In (31) we compute the distribution for the underlying function in the test point and in (30) we integrate out the underlying function to predict the probability that the class label of that point is +1. Both integrals are intractable due to the likelihood model employed for fin (29). GPC typically relies on a Gaussian approximation4for the posterior density p(f|D), to analytically solve (31), and (30) is a one-dimensional integral that can be easily solved numerically. Further details on how to approximate the posterior and train the covariance function hyperparameters can be found in [31]. 3This function is typically the logistic or the cumulative density function of a Gaussian [31]. 4The standard approximations to the posterior are Laplace or expectation propagation, as explained in [17]. 16 6 Experimental Results We carry out two sets of experiments. First, we design a receiver for a CDMA system with strong near-far requirements and inter-symbol interference. In the second experiment, we deal with a channel equalization problem with a nonlinear amplifier in the receiver. The results in these experiments allow drawing some general conclusions about the advantages of GPs for designing digital communication receivers. For both experiments the channel model is given by: h(z)=0.3763 + 0.8466z−1+ 0.3763z−2.(33) For all these systems we train a linear MMSE receiver (denoted by ‘MMSE’ and a dashed line), a GPR (‘GPR’ and a solid line) and a GPC with an EP approximation to its posterior (‘GPC’ and a dash-dotted line). We approximate the GPC posterior using the EP algorithm, because it provides superior performances than the Laplace approximation as suggested in [17]. For the GPs receivers we work with the covariance matrix in (28). We also report a linear SVM receiver (‘SVMl’ and a dotted line with circles) and a nonlinear SVM (‘SVMnl’ and a dotted line with bullets) with an RBF kernel [34]. For the SVMs we train a set of receivers with different hyperparameters and we report the best result. We use C= 0.5,1,2,5 and 10 and σ=kσzwith k= 1,2,5and 10. Thereby, the comparison is biased in favor of the SVM when compared to the GPR and GPC solutions. All the figures are obtained for 100 independently trained trials with 105test symbols. 6.1 Linear multi-user detection In our first experiment we employ Gold spreading codes with 31 chips per user, because they have favorable cross-correlation properties that limit the interferences by other users and their delayed replicas [11]. We report results for systems operating with 3 and 16 users and we assume the user of interest is 50dB bellow the other users. This is a fairly standard scenario when one of the users is close to the base station and it is assigned little power. We use the received 31 chips to detect each transmitted symbol. 17 0 2 4 6 8 10 12 14 10−6 10−5 10−4 10−3 10−2 10−1 100 snr BER n=512 MMSE GPR GPC SVMl SVMnl 2 4 6 8 10 12 14 16 18 10−6 10−5 10−4 10−3 10−2 10−1 100 snr BER n=512 MMSE GPR GPC SVMl SVMnl (a) (b) Figure 1: We report the BER versus the snr for a multi-user detector with 3 users in (a) and 16 users in (b). The dashed line represents the linear MMSE receiver, the solid line the GPR, the dash-dotted line the GPC, the dotted line with circles the linear SVM and the dotted line with bullets the nonlinear SVM. We show the bit error rate (BER) versus the signal to noise ratio (snr) for 3 users in Figure 1(a) and 16 users in Figure 1(b) with 512 training symbols. The solution is almost linear and all the receivers perform similarly well except for the nonlinear SVM for 16 users. The training sequence for the nonlinear SVM with 16 users is not long enough, and hence the nonlinear SVM is unable to detect the transmitted bits and reports chance level performances. The GPR solution is quite similar to the MMSE solution, because it almost shuts down its nonlinear part in (28). As we show in Section 3, the GPR with a linear kernel and the linear MMSE provide equivalent solutions in this case. This result is quite relevant, as we do not tell the GPR receiver that the solution is linear. It finds out on its own, when it maximizes the hyperparameters’ likelihood. The GPC also cancels its nonlinear part and it is able to avoid overfitting. The linear SVM detector presents the worse performance among the proposed methods that converge in both cases, although it is barely noticeable in the figures. The optimal solution is almost linear and all the proposed procedures perform 18 3456789 10−6 10−5 10−4 10−3 10−2 10−1 100 log2 n BER snr=14dB MMSE GPR GPC SVMl SVMnl 3456789 10−7 10−6 10−5 10−4 10−3 10−2 10−1 100 log2 n BER snr=18dB MMSE GPR GPC SVMl SVMnl (a) (b) Figure 2: We report the BER versus the length of the training sequence for a multiuser detector with 3 users and snr=14dB in (a) and 16 users and snr=18dB in (b). The dashed line represents the linear MMSE receiver, the solid line the GPR, the dash-dotted line the GPC, the dotted line with circles the linear SVM and the dotted line with bullets the nonlinear SVM. equally well, once the training sequence is long enough. The training sequence of 512 symbols is not long enough for the nonlinear SVM with 16 users and it is unable to correctly tune its multi-user detector. If we had increased the training sequence to several thousands samples, the nonlinear SVM would converge and it would provide a solution close to the other algorithms. The differences in BER are not significant to decide which method is best, but the differences in training time might lead us to choose one over the others, as we discuss in short. We report the BER as a function of the training examples for 3 users in Figure 2(a) and 16 users in Figure 2(b). For this experiment, these results are more meaningful than the BER versus snr reported in Figure 1, because there is a significant disparity between the performances of the different methods. For 3 users (Figure 2(a)) the GPR and linear SVM are able to reduce the BER for very short training sequences while GPC, MMSE and nonlinear SVM need substantially longer training sequences before they provide non-chance level performances. For 32 training 19 symbols, there are 3 orders of magnitude difference in BER between the former and latter methods. From these 2 plots, we can easily understand why the nonlinear SVM is unable to converge for 16 users with 512 training symbols. For 3 users the nonlinear SVM needs longer training sequences than the other methods, before it can significantly reduce the BER. For 16 users, the learning problem is harder and it needs several thousands samples to achieve convergence. The GPR, MMSE and linear SVM learn the solution as the number of training examples increases and they behave almost equally well for 16 users. The GPC needs the training sequence to be long enough before it can produce a meaningful solution. It needs at least 64 symbols for 3 users and 256 for 16 to be able to produce non-chance level performances. But once the training sequence is long enough, it converges to the optimal solution. It does not provide intermediate solutions as the other methods do. For 16 users, the GPR receiver presents the fastest learning curve closely followed by the linear MMSE and linear SVM solutions. We conjecture this is due to the GPR optimal training of its hyperparameter, because it is able to adjust them for each training sequence, while the linear SVM uses a constant setting, which might be good for a long training sequence, but not as good for shorter ones. In this example we can readily understand the advantages of using GPR for solving multi-user detection problems, as for very short training sequences we are able to obtain the best possible solution, and if it is linear, it even improves the linear MMSE solution. The GPR and linear MMSE detectors provide the same solution as the number of samples increases, but for short training sequence the GPR detector is able to optimally set its hyperparameters to provide better performance than the linear MMSE. Also, as we see in the next example, if the solution is nonlinear, it is able to achieve nonlinear multi-user detectors, significantly improving the linear MMSE solution. 20 6.2 Nonlinear multi-user detection We repeat the Experiment 2 in [6], in which 3 users transmit with an orthogonal 8-dimension spreading code. The solution for user 2 is highly nonlinear and we report the BER versus the snr in Figure 3. The linear SVM and MMSE clearly underperform compared to the nonlinear methods. The GPR and nonlinear SVM achieve almost identical results. The GPC for low snr mimics the results of the nonlinear methods (snr < 14dB) and for high snr it reports the same results as the linear receivers (snr > 16dB). This behavior is explained by the length and diversity of the training sequence. If the training sequence is long enough, the GPC receiver provides the best nonlinear decision function, otherwise it reports the best linear decision function to avoid overfitting. For low snr, 512 symbols is long enough for the GPC to achieve the best nonlinear decision function and the GPC receiver trains its hyperparameters to obtain this nonlinear detector. For high snr there is not enough diversity in a training sequence with 512 symbols and it is only able to report the best linear detector, as it shuts down its nonlinear part to avoid overfitting. In the first experiment, we already saw that GPC receivers need longer training sequences than GPR, even to achieve the best linear detector. It is clear in this experiment that for nonlinear decision function, GPC receivers even need longer training sequences. In these two experiments, we are able to show that the GPR with the covariance function in (28) is able to obtain the best results in both scenarios. If the solution is linear, it performs as the linear MMSE, needing shorter training sequences. If the solution is nonlinear the GPC receiver builds a nonlinear detector that significantly improves the linear MMSE and reports the same solution as a nonlinear SVM. The nonlinear SVM is not as good as the GPR with the covariance matrix in (28), because for (almost) linear solutions, it needs significantly longer training sequences, which is a waste of resources in wireless communication systems, as the preamble must be as short as possible. Also a SVM cannot use a kernel as in (28), because it would need to cross validate (or hand pick) too many hyperparameters. 21 2 4 6 8 10 12 14 16 18 10−7 10−6 10−5 10−4 10−3 10−2 10−1 100 snr BER n=512 MMSE GPR GPC SVMl SVMnl Figure 3: We report the BER versus snr for a multi-user detector with 3 users and a training sequence of 512 symbols. The dashed line represents the linear MMSE receiver, the solid line the GPR, the dash-dotted line the GPC, the dotted line with circles the linear SVM and the dotted line with bullets the nonlinear SVM. The linear SVM is on top of the linear MMSE line. 6.3 Nonlinear channel equalization Now we turn to the channel equalization problem, in which the channel is represented by (33), and we add a memoryless nonlinearity to the receiver that transforms each received signal as follows: xi= ˆxi+ 0.2ˆx2 i−0.1ˆx3 i+zi(34) where ˆxi= (Hs)i. This channel model is typically used to described nonlinear amplifiers in wireless communication receivers as explained in [20]. To construct the equalizers, we use 6 received samples to predict each transmitted symbol with a delay of 2 samples. 22 In Figure 4, we show the BER versus the snr for all equalizers and n= 512. For snr less than 22dB the nonlinear GPR equalizer achieves the minimum BER with a gain larger than 3dB for BER around 10−3. For larger snr the performance of this nonlinear equalizer degrades and the linear equalizers perform significantly better. The nonlinear SVM equalizer performs as the GPR equalizer for snr lower than 17dB, but for larger snr the training sequence is not long enough and its solution degrades (overfiting). For snr larger than 20dB, the nonlinear SVM equalizer is not able to reduce the achieved BER. The nonlinear SVM and the GPR as the snr increases are not able to get optimal equalizers, because there is not enough diversity in the training sequence and they overfit to it. The GPR performance is better than the SVM for large snr, because it uses a covariance function in (28) that incorporates a linear term. Although it overfits the nonlinear part, the linear component allows the GPR to reduce the BER for large snr. If we had increased the training sequence, the SVM and GPR would perform better than the linear methods for larger values of the snr. The GPC shuts down the nonlinear part and performs as the linear SVM. This is the same effect that we saw for large snr in Figure 3, the training set is not long enough to ensure it can train the nonlinear part of its covariance function and it consequently sets it to zero. In Figure 4 for snr less than 10dB, although we can barely notice it, the GPC equalizer follows the nonlinear solutions, as the training sequence is long enough to train its nonlinear component in this case. The linear SVM and GPC are able to perform significantly better than the linear MMSE, because the channel model is nonlinear. For a nonlinear channel the received constellation is no longer symmetric and penalizing the squared error is suboptimal, as it forces that all the detected symbols to be equally far from its optimal value. The SVM and GPC equalizers only care if the points are correctly classified and they only focus on those that might not be, which explains the BER gap between the linear MMSE equalizer and the GPC and linear SVM ones. In any case, for the snr of interests between 10 and 20dB, the GPR receivers (and nonlinear SVM) are significantly better than the linear methods and the GPC. 23 0 5 10 15 20 25 10−5 10−4 10−3 10−2 10−1 100 snr BER n=512 MMSE GPR GPC SVMl SVMnl Figure 4: We report the BER versus snr for a channel equalization problem with a nonlinear channel model. The dashed line represents the linear MMSE receiver, the solid line the GPR, the dash-dotted line the GPC, the dotted line with circles the linear SVM and the dotted line with bullets the nonlinear SVM. For this range of snr the BER is not low enough for most digital communication applications, but we can significantly reduce the BER using channel coding strategies [18] with high data rates, instead of increasing the snr. 6.4 Discussion In the experiments we show the behavior of GPR for designing digital communication receivers and we show it has many favorable properties for solving such task when we use it with the covariance function in (28): •If the solution is linear, the GPR receiver shuts down the nonlinear part of the covariance function and performs as the linear MMSE detector for long 24 training sequences. It converges faster than the MMSE detector to the optimal solution. It does not degrade its performance when canceling the nonlinear part of the kernel. •If the solution is nonlinear, the GPR receiver is able to achieve very good performances, comparable to a nonlinear SVM receiver with optimal hyperparameters, and it needs shorter training sequences to achieve such solutions. The GPR receiver performs significantly better than the linear detectors. •The GPR receiver performs a single optimization procedure. This is a highly desirable quality as in one step we get the optimal hyperparameters without needing to try several solutions and check which one is best. The GPR decides if it needs a linear or a nonlinear solution in that single optimization without relying on a ‘genie’ or another procedure to check if the optimal solution is linear. •The GPR can overfit if the training sequence is not sufficiently long, as we can see in Figure 4. But in this case the overfitting does not degrade the solution as much as it does for the nonlinear SVM. It only happens for very large snr in which we do not typically transmit. •The GPR receiver uses a least square lost function, which is not ideal for solving classification problems when we are interested in minimizing the misclassification error. But for digital communication problems in which the noise is Gaussian, the use of this loss-function is not critical and the GPR-receiver performs as well as the receivers based on classification lossfunctions (GPC and SVM). The GPC would initially seems like a better choice for designing digital communication receivers, because it minimizes the misclassification error and it can optimize the hyperparameters, just as the GPR does. But in our experiments we show that GPC receivers usually need longer training sequences before it can tune its nonlinear part and it decides to train a linear detector in cases where a nonlinear 25