publications

2021

1. Biologically-Inspired Pulse Signal Processing for Intelligence at the Edge

   Kan Li and José C. Príncipe,

   Frontiers in Artificial Intelligence 2021

   Abs HTML PDF

   There is an ever-growing mismatch between the proliferation of data-intensive, power-hungry deep learning solutions in the machine learning (ML) community and the need for agile, portable solutions in resource-constrained devices, particularly for intelligence at the edge. In this paper, we present a fundamentally novel approach that leverages data-driven intelligence with biologically-inspired efficiency. The proposed Sparse Embodiment Neural-Statistical Architecture (SENSA) decomposes the learning task into two distinct phases: a training phase and a hardware embedment phase where prototypes are extracted from the trained network and used to construct fast, sparse embodiment for hardware deployment at the edge. Specifically, we propose the Sparse Pulse Automata via Reproducing Kernel (SPARK) method, which first constructs a learning machine in the form of a dynamical system using energy-efficient spike or pulse trains, commonly used in neuroscience and neuromorphic engineering, then extracts a rule-based solution in the form of automata or lookup tables for rapid deployment in edge computing platforms. We propose to use the theoretically-grounded unifying framework of the Reproducing Kernel Hilbert Space (RKHS) to provide interpretable, nonlinear, and nonparametric solutions, compared to the typical neural network approach. In kernel methods, the explicit representation of the data is of secondary nature, allowing the same algorithm to be used for different data types without altering the learning rules. To showcase SPARK’s capabilities, we carried out the first proof-of-concept demonstration on the task of isolated-word automatic speech recognition (ASR) or keyword spotting, benchmarked on the TI-46 digit corpus. Together, these energy-efficient and resource-conscious techniques will bring advanced machine learning solutions closer to the edge.

2. Functional Bayesian Filter

   Kan Li and José C. Príncipe,

   IEEE Transactions on Signal Processing 2021

   Abs HTML PDF

   We present a general nonlinear Bayesian filter for high-dimensional state estimation using the theory of reproducing kernel Hilbert space (RKHS). By applying the kernel method and the representer theorem to perform linear quadratic estimation in a functional space, we derive a Bayesian recursive state estimator for a general nonlinear dynamical system in the original input space. Unlike existing nonlinear extensions of the Kalman filter where the system dynamics are assumed known, the state-space representation for the Functional Bayesian Filter (FBF) is completely learned online from measurement data in the form of an infinite impulse response (IIR) filter or recurrent network in the RKHS, with universal approximation property. Using a positive definite kernel function satisfying Mercer’s conditions to compute and evolve information quantities, the FBF exploits both the statistical and time-domain information about the signal, extracts higher-order moments, and preserves the properties of covariances without the ill effects due to conventional arithmetic operations. We apply this novel kernel adaptive filtering (KAF) to recurrent network training, chaotic time-series estimation and cooperative filtering using Gaussian and non-Gaussian noises, and inverse kinematics modeling. Simulation results show FBF outperforms existing Kalman-based algorithms.

3. Non-invasive Assessment of Elevated LVEDP Using a Small Portable Device: Design for Telemonitoring

   Kan Li, Diego Pava, Arash Andalib, and Kaustubh Kale,

   In Heart Failure Society of America (HFSA) Annual Scientific Meeting Sep 2021

   Abs HTML PDF AUDIO

   Left ventricular end-diastolic pressure (LVEDP) provides crucial information on the LV operating compliance and is a key marker for congestive heart failure and pulmonary hypertension. The diagnosis and monitoring of the left-sided filling pressure often requiring invasive procedures. HEMOTAG, is a small portable device that uses micro-sensors to capture cardiac vibrations and electrocardiogram (ECG), transduced via thoracic electrodes. It is a viable option to measure cardiac time intervals (CTIs), surrogate markers of hemodynamics and intracardiac pressure. In this study, diastolic-to-systolic time ratio (DSR) and systolic time ratio (STR) along with the patient’s age, height, and weight were assessed as valid markers for elevated LVEDP, compared against left heart catheterization LVEDP measurement.

4. A Novel Non-Invasive Device for Screening and Optimized Management to Improve Heart Failure Outcomes in Patients with Diabetes Mellitus

   Diego Pava, Arash Andalib, Kan Li, and Kaustubh Kale,

   In Heart Failure Society of America (HFSA) Annual Scientific Meeting Sep 2021

   Abs HTML PDF AUDIO

   Evidence shows diabetes mellitus (DM) is major independent risk factor for several cardiovascular disorders including heart failure (HF). Large, randomized clinical trials for screening and revascularization of stable macrovascular disease in diabetics, have failed to demonstrate a significant reduction in cardiac events and HF episodes. Studies have demonstrated that early detection of left ventricular dysfunction and prevention of microvascular complications through glycemic control in diabetes patients is a critical mechanism for reducing the incidence and severity of left ventricular dysfunction and HF. There is a need for a non-invasive, affordable, accurate, absolute and actionable method to facilitate optimized management of diabetes patients across the continuity of care. HEMOTAG, is a small portable device that uses micro-sensors to capture cardiac vibrations and electrocardiogram, transduced via thoracic electrodes. It is a viable option to measure cardiac time intervals (CTIs), surrogate markers for measurements of left ventricular dysfunction.

2020

1. Fast Estimation of Information Theoretic Learning Descriptors using Explicit Inner Product Spaces

   Kan Li and José C. Príncipe,

   2020

   Abs HTML

   Kernel methods form a theoretically-grounded, powerful and versatile framework to solve nonlinear problems in signal processing and machine learning. The standard approach relies on the \emphkernel trick to perform pairwise evaluations of a kernel function, leading to scalability issues for large datasets due to its linear and superlinear growth with respect to the training data. Recently, we proposed \emphno-trick (NT) kernel adaptive filtering (KAF) that leverages explicit feature space mappings using data-independent basis with constant complexity. The inner product defined by the feature mapping corresponds to a positive-definite finite-rank kernel that induces a finite-dimensional reproducing kernel Hilbert space (RKHS). Information theoretic learning (ITL) is a framework where information theory descriptors based on non-parametric estimator of Renyi entropy replace conventional second-order statistics for the design of adaptive systems. An RKHS for ITL defined on a space of probability density functions simplifies statistical inference for supervised or unsupervised learning. ITL criteria take into account the higher-order statistical behavior of the systems and signals as desired. However, this comes at a cost of increased computational complexity. In this paper, we extend the NT kernel concept to ITL for improved information extraction from the signal without compromising scalability. Specifically, we focus on a family of fast, scalable, and accurate estimators for ITL using explicit inner product space (EIPS) kernels. We demonstrate the superior performance of EIPS-ITL estimators and combined NT-KAF using EIPS-ITL cost functions through experiments.

2019

1. No-Trick (Treat) Kernel Adaptive Filtering using Deterministic Features

   Kan Li and José C. Príncipe,

   2019

   Abs HTML

   Kernel methods form a powerful, versatile, and theoretically-grounded unifying framework to solve nonlinear problems in signal processing and machine learning. The standard approach relies on the kernel trick to perform pairwise evaluations of a kernel function, which leads to scalability issues for large datasets due to its linear and superlinear growth with respect to the training data. A popular approach to tackle this problem, known as random Fourier features (RFFs), samples from a distribution to obtain the data-independent basis of a higher finite-dimensional feature space, where its dot product approximates the kernel function. Recently, deterministic, rather than random construction has been shown to outperform RFFs, by approximating the kernel in the frequency domain using Gaussian quadrature. In this paper, we view the dot product of these explicit mappings not as an approximation, but as an equivalent positive-definite kernel that induces a new finite-dimensional reproducing kernel Hilbert space (RKHS). This opens the door to no-trick (NT) online kernel adaptive filtering (KAF) that is scalable and robust. Random features are prone to large variances in performance, especially for smaller dimensions. Here, we focus on deterministic feature-map construction based on polynomial-exact solutions and show their superiority over random constructions. Without loss of generality, we apply this approach to classical adaptive filtering algorithms and validate the methodology to show that deterministic features are faster to generate and outperform state-of-the-art kernel methods based on random Fourier features.

2018

1. Surprise-Novelty Information Processing for Gaussian Online Active Learning (SNIP-GOAL)

   Kan Li and José C. Príncipe,

   In 2018 International Joint Conference on Neural Networks (IJCNN) 2018

   Abs HTML

   In this paper, we propose a novel, combined surprise-novelty approach to online active learning for kernel adaptive filters. Surprise and novelty criteria have always been used individually in designing sparse kernel machines. While closely related, they are in fact two distinct concepts. In this paper, we highlight the key differences between surprise and novelty, define quantitative measures, and propose a unifying framework that leverages the complementary properties of the two concepts combined. We test the information theoretic approach of designing sparse kernel adaptive filters using the surprise-novelty information processing for Gaussian online active learning (SNIP-GOAL) on the task of nonlinear chaotic time series prediction. The proposed method outperforms existing algorithms using the measures individually. Results show that combining surprise and novelty can be advantageous in terms of efficiency and performance. Leveraging both measures allows the system to be not only sparse but also generalizes better.

2. Biologically-Inspired Spike-Based Automatic Speech Recognition of Isolated Digits Over a Reproducing Kernel Hilbert Space

   Li, Kan, and Príncipe, José C.

   Frontiers in Neuroscience 2018

   Abs HTML

   This paper presents a novel real-time dynamic framework for quantifying time-series structure in spoken words using spikes. Audio signals are converted into multi-channel spike trains using a biologically-inspired leaky integrate-and-fire (LIF) spike generator. These spike trains are mapped into a function space of infinite dimension, i.e., a Reproducing Kernel Hilbert Space (RKHS) using point-process kernels, where a state-space model learns the dynamics of the multidimensional spike input using gradient descent learning. This kernelized recurrent system is very parsimonious and achieves the necessary memory depth via feedback of its internal states when trained discriminatively, utilizing the full context of the phoneme sequence. A main advantage of modeling nonlinear dynamics using state-space trajectories in the RKHS is that it imposes no restriction on the relationship between the exogenous input and its internal state. We are free to choose the input representation with an appropriate kernel, and changing the kernel does not impact the system nor the learning algorithm. Moreover, we show that this novel framework can outperform both traditional hidden Markov model (HMM) speech processing as well as neuromorphic implementations based on spiking neural network (SNN), yielding accurate and ultra-low power word spotters. As a proof of concept, we demonstrate its capabilities using the benchmark TI-46 digit corpus for isolated-word automatic speech recognition (ASR) or keyword spotting. Compared to HMM using Mel-frequency cepstral coefficient (MFCC) front-end without time-derivatives, our MFCC-KAARMA offered improved performance. For spike-train front-end, spike-KAARMA also outperformed state-of-the-art SNN solutions. Furthermore, compared to MFCCs, spike trains provided enhanced noise robustness in certain low signal-to-noise ratio (SNR) regime.

3. Nearest-Instance-Centroid-Estimation Linear Discriminant Analysis (Nice Lda)

   Singh, Rishabh, Li, Kan, and Principe, Jose C.

   In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2018

   Abs HTML

   We propose a novel cascaded classification technique called the Nearest Instance Centroid Estimation (NICE) LDA algorithm. Our algorithm (inspired from NICE KLMS) performs a cascade combination of two weak classifiers - threshold based class-wise clustering and linear discriminant classification to achieve state-of-the-art results on various high dimensional UCI datasets. We show how our method is more robust towards skewed data and computationally more efficient than previous methods of combining clustering with classification techniques. We also develop an efficient aggregation method based on instance based learning that implements this cascade combination of classifiers in a much simpler manner computationally. We demonstrate that our method of data clustering and LDA implementation, while introducing only one free parameter, leads to results that are similar and often better than those achieved by the state-of-the-art kernel RBF SVMs.

2017

1. Transfer Learning in Adaptive Filters: The Nearest Instance Centroid-Estimation Kernel Least-Mean-Square Algorithm

   Li, Kan, and Príncipe, José C.

   IEEE Transactions on Signal Processing 2017

   Abs HTML

   We propose a novel nearest-neighbors approach to organize and curb the growth of radial basis function network in kernel adaptive filtering (KAF). The nearest-instance-centroid-estimation (NICE) kernel least-mean-square (KLMS) algorithm provides an appropriate time-space tradeoff with good performance. Its centers in the input/feature space are organized by quasi-orthogonal regions for greatly simplified filter evaluation. Instead of using all centers to evaluate/update the function approximation at every new point, a linear search among the iteratively-updated centroids determines the partial function to be used, naturally forming locally-supported partial functionals. Under this framework, partial functionals that compose the adaptive filter are quickly stored/retrieved based on input, each corresponding to a specialized “spatial-band” subfilter. The filter evaluation becomes the update of one of the subfilters, creating a content addressable filter bank (CAFB). This CAFB is incrementally updated for new signal applications with mild constraints, always using the past-learned partial filter sums, opening the door for transfer learning and significant efficiency for new data scenarios, avoiding training from scratch as have been done since the invention of adaptive filtering. Using energy conservation relation, we show the sufficient condition for mean square convergence of the NICE-KLMS algorithm and establish the upper and lower bounds of steady-state excess-mean-square-error (EMSE). Simulations on chaotic time-series prediction demonstrate similar levels of accuracy as existing methods, but with much faster computation involving fewer input samples. Simulations on transfer learning using both synthetic and real-world data demonstrate that NICE CAFB can leverage previously learned knowledge to related task or domain.

2. Automatic plant identification using STEM automata

   Li, Kan, Ma, Ying, and Príncipe, José C.

   In 2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP) 2017

   Abs HTML

   In this paper, we propose a novel approach to automatically identify plant species using dynamics of plant growth and development or spatiotemporal evolution model (STEM). The online kernel adaptive autoregressive-moving-average (KAARMA) algorithm, a discrete-time dynamical system in the kernel reproducing Hilbert space (RKHS), is used to learn plant-development syntactic patterns from feature-vector sequences automatically extracted from 2D plant images, generated by stochastic L-systems. Results show multiclass KAARMA STEM can automatically identify plant species based on growth patterns. Furthermore, finite state machines extracted from trained KAARMA STEM retains competitive performance and are robust to noise. Automatically constructing an L-system or formal grammar to replicate a spatiotemporal structure is an open problem. This is an important first step to not only identify plants but also to generate realistic plant models automatically from observations.

3. Flight dynamics modeling and recognition using finite state machine for automatic insect recognition

   Li, Kan, and Principe, Jose C.

   In 2017 International Joint Conference on Neural Networks (IJCNN) 2017

   Abs HTML

   In this paper, we propose a novel state-based approach to insect flight dynamics modeling and automatic insect recognition (AIR). An emerging technology uses noninvasive laser sensors to capture flight information of flying insects in situ. However current computational entomology methods treat these optical signals as static patterns, rather than time series. We propose to evaluate each insect passage as a nonstationary process involving a sequence of pseudo-acoustic frames and model the short-term flight dynamics using a kernel adaptive autoregressive-moving-average (KAARMA) network. Since flight behavior is both nonlinear and nonstationary in nature, dynamic modeling provides a general framework that fully exploits the transitional and contextual information. Using the kernel adaptive ARMA algorithm and spatial clustering, each flight passage is defined to be an ordered sequence of states in a discretized spatiotemporal space. The state transition trajectories are used to construct a finite state machine (FSM) recognizer. The computational efficiency of the automaton classifier eliminates the need to perform numeric computations and enables realtime performance with far-reaching implications in entomology, agriculture, and disease control and prevention. Results show multiclass KAARMA classifier outperforms the state-of-the-art AIR methods, using support vector machine (SVM), deep learning autoencoder, and batch learning, in identifying Zika vector mosquito Aedes aegypti among five species of flying insects, while using significantly more efficient data representation. The extracted FSM from trained KAARMA network retained competitive performance compared to the current computationally expensive implementations.

4. Automatic insect recognition using optical flight dynamics modeled by kernel adaptive ARMA network

   Li, Kan, and Príncipe, José C.

   In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2017

   Abs HTML

   Automatic insect recognition (AIR), using noninvasive methods in situ, has far-reaching implications in entomology, agriculture, and disease control and prevention. An emerging technology in computational entomology uses flight information captured by laser sensors. Current methods treat these optical signals as static patterns, rather than time series. We propose a novel approach to AIR by evaluating each insect passage as a nonstationary process involving a sequence of pseudo-acoustic frames and modeling the short-term flight dynamics using the kernel adaptive autoregressive-moving average (KAARMA) algorithm. Since flight behavior is both nonlinear and nonstationary in nature, dynamic modeling provides a general framework that fully exploits the transitional and contextual information. Results show KAARMA classifier outperforms the state-of-the-art AIR methods, using support vector machine (SVM), deep-learning autoencoder, and batch learning, in identifying Zika vector mosquito Aedes aegypti among five species of flying insects, while using significantly more efficient data representation.

2016

1. Kernel Learning for Dynamic Texture Synthesis

   You, Xinge, Guo, Weigang, Yu, Shujian, Li, Kan, Príncipe, José C., and Tao, Dacheng

   IEEE Transactions on Image Processing 2016

   Abs HTML

   Dynamic textures (DTs) that represent moving scenes such as flames, smoke, and waves, exhibit fixed dynamics within a period of time and have been successfully modeled using linear dynamic systems (LDS). In this paper, we show that the widely used LDS model can be approximated using a principal component regression (PCR) model with the main advantage of simplicity. Furthermore, to capture the nonlinearity of training frames, we extend traditional PCR to its kernelized version and introduce kernel principal component regression (KPCR) to model and synthesize DTs. To ensure algorithm stability, we remove the standard state model and directly apply the quantized kernel least mean squares algorithm from signal processing domain to approximate the performance achieved with KPCR. We term this improvement kernel adaptive dynamic texture synthesis (KADTS), which also has the benefits of computational and memory efficiency. These advantages make KADTS ideally suited for real-world applications, since the majority of electronic devices, including cell phones and laptops, suffer from limited memory and real-time constraints. We demonstrate, via both theoretical and experimental analyses, the connections between DT synthesis using KPCR and KADTS with a regularization network theory. We also show the superiority of our proposed algorithms for DT synthesis compared with other dynamic system-based benchmarks. MATLAB code is available from our project homepage http://bmal.hust.edu.cn/project/dts.html.

2. A Novel Normalized Sign Algorithm for System Identification Under Impulsive Noise Interference

   Lu, Lu, Zhao, Haiquan, Li, Kan, and Chen, Badong

   Circuits, Systems, and Signal Processing 2016

   Abs HTML

   To overcome the performance degradation of adaptive filtering algorithms in the presence of impulsive noise, a novel normalized sign algorithm (NSA) based on a convex combination strategy, called NSA-NSA, is proposed in this paper. The proposed algorithm is capable of solving the conflicting requirement of fast convergence rate and low steady-state error for an individual NSA filter. To further improve the robustness to impulsive noises, a mixing parameter updating formula based on a sign cost function is derived. Moreover, a tracking weight transfer scheme of coefficients from a fast NSA filter to a slow NSA filter is proposed to speed up the convergence rate. The convergence behavior and performance of the new algorithm are verified by theoretical analysis and simulation studies.

3. Restoring Behavior via Inverse Neurocontroller in a Lesioned Cortical Spiking Model Driving a Virtual Arm

   Dura-Bernal, Salvador, Li, Kan, Neymotin, Samuel A., Francis, Joseph T., Principe, Jose C., and Lytton, William W.

   Frontiers in Neuroscience 2016

   Abs HTML

   Neural stimulation can be used as a tool to elicit natural sensations or behaviors by modulating neural activity. This can be potentially used to mitigate the damage of brain lesions or neural disorders. However, in order to obtain the optimal stimulation sequences, it is necessary to develop neural control methods, for example by constructing an inverse model of the target system. For real brains, this can be very challenging, and often unfeasible, as it requires repeatedly stimulating the neural system to obtain enough probing data, and depends on an unwarranted assumption of stationarity. By contrast, detailed brain simulations may provide an alternative testbed for understanding the interactions between ongoing neural activity and external stimulation. Unlike real brains, the artificial system can be probed extensively and precisely, and detailed output information is readily available. Here we employed a spiking network model of sensorimotor cortex trained to drive a realistic virtual musculoskeletal arm to reach a target. The network was then perturbed, in order to simulate a lesion, by either silencing neurons or removing synaptic connections. All lesions led to significant behvaioral impairments during the reaching task. The remaining cells were then systematically probed with a set of single and multiple-cell stimulations, and results were used to build an inverse model of the neural system. The inverse model was constructed using a kernel adaptive filtering method, and was used to predict the neural stimulation pattern required to recover the pre-lesion neural activity. Applying the derived neurostimulation to the lesioned network improved the reaching behavior performance. This work proposes a novel neurocontrol method, and provides theoretical groundwork on the use biomimetic brain models to develop and evaluate neurocontrollers that restore the function of damaged brain regions and the corresponding motor behaviors.

4. The Kernel Adaptive Autoregressive-Moving-Average Algorithm

   Li, Kan, and Príncipe, José C.

   IEEE Transactions on Neural Networks and Learning Systems 2016

   Abs HTML

   In this paper, we present a novel kernel adaptive recurrent filtering algorithm based on the autoregressive-moving-average (ARMA) model, which is trained with recurrent stochastic gradient descent in the reproducing kernel Hilbert spaces. This kernelized recurrent system, the kernel adaptive ARMA (KAARMA) algorithm, brings together the theories of adaptive signal processing and recurrent neural networks (RNNs), extending the current theory of kernel adaptive filtering (KAF) using the representer theorem to include feedback. Compared with classical feedforward KAF methods, the KAARMA algorithm provides general nonlinear solutions for complex dynamical systems in a state-space representation, with a deferred teacher signal, by propagating forward the hidden states. We demonstrate its capabilities to provide exact solutions with compact structures by solving a set of benchmark nondeterministic polynomial-complete problems involving grammatical inference. Simulation results show that the KAARMA algorithm outperforms equivalent input-space recurrent architectures using first- and second-order RNNs, demonstrating its potential as an effective learning solution for the identification and synthesis of deterministic finite automata.

2015

1. Repairing lesions via kernel adaptive inverse control in a biomimetic model of sensorimotor cortex

   Li, Kan, Dura-Bernal, Salvador, Francis, Joseph T., Lytton, William W., and Príncipe, José C.

   In 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER) 2015

   Abs HTML

   In this paper we propose a kernel adaptive filtering (KAF) approach to repairing lesions via microstimulation in a biomimetic spiking neural network of sensorimotor cortex. The fundamental challenge of designing neuroprosthetics and brain machine interfaces (BMIs) is the decoding of electrical activity of neurons and behavior. For injured or damaged brain, intracranial stimulation has the potential to modulate neural activity to match meaningful and natural response or behavior. In order to optimize the microstimulation sequences, we construct an inverse model of the target system. However, to obtain sufficient learning data, the neural system must be stimulated or probed extensively. For real brains, this is especially challenging and often unfeasible. Here, we demonstrate that by applying KAF to a biomimetic brain and realistic virtual musculoskeletal model, we can repair simulated lesion and drive a virtual arm to perform the correct motor task.

2014

1. Modulation of virtual arm trajectories via microstimulation in a spiking model of sensorimotor cortex

   Dura-Bernal, Salvador, Li, Kan, Brockmeier, Austin J., Kerr, Cliff C., Neymotin, Samuel A., Principe, Jose C., Francis, Joseph T., and Lytton, William W.

   In BMC Neuroscience 2014

   Abs HTML

   Electrical microstimulation can be used to drive neural responses to match meaningful spiking patterns corresponding to natural sensory stimuli or motor behaviors. Optimizing microstimulation sequences requires repeatedly stimulating the neural system to obtain sufficient probing data to construct an inverse model. This is challenging in the real brain where probing time may be limited and plasticity may be induced. Biologically realistic models allow the system to be repeatedly probed and reset, providing a unique test bed for understanding the dynamic interaction between ongoing neural activity and artificially applied stimulation. Here, we employ a biomimetic spiking model (BMM) of sensorimotor cortex which controls a realistic virtual musculoskeletal arm that performs reaching movements. After training the BMM and virtual arm to reach a target, a set of 1536 microstimulation probing sequences were applied to the target sensory population. The output motor population activity and the virtual arm trajectories were recorded (Figure 1). Microstimulation was applied to each target neuron individually as well as to random subsets of multiple neurons simultaneously. Other parameters, such as the start time, duration, and amplitude of the stimulation, were also examined. Results demonstrated that microstimulation can differentially modulate the motor population which excites the arm muscles, and consequently, the hand trajectories. The results of the probing dataset will be used to build an inverse model of the neural system, which will be able to generate microstimulation patterns to repair simulated lesions in the model.

2013

1. Kernel adaptive filtering with confidence intervals

   Li, Kan, Chen, Badong, and Príncipe, José C.

   In The 2013 International Joint Conference on Neural Networks (IJCNN) 2013

   Abs HTML

   Since its introduction, kernel adaptive filtering (KAF) has attracted considerable attention in recent years. Its main advantages include universal nonlinear approximation using kernel methods, linearity with convex learning in the Reproducing Kernel Hilbert Space (RKHS), and online adaptation with moderate complexity. Among its applications, the kernel least mean square (KLMS) algorithm deserves particular attention due to its simplicity and effectiveness for learning complex systems. A major drawback of current implementations of KAF is the lack of a simple determination of the certainty of each estimate. In this paper, we present a novel kernel adaptive filtering architecture with confidence intervals. By introducing an auxiliary filter, the variance of each estimate can be computed using stochastic gradient descent in O(N). Results show that the proposed algorithm produces comparable estimates of the mean and the variance functions, using only a fraction of the computation associated with the Gaussian process (GP) prediction, and is more versatile in the cases of time-varying noise variance or heteroskedasticity.

2010

1. Channels with both random errors and burst erasures: Capacities, LDPC code thresholds, and code performances

   Li, Kan, Kavčić, Aleksandar, Venkataramani, Raman, and Erden, M. Fatih

   In 2010 IEEE International Symposium on Information Theory 2010

   Abs HTML

   We derive the capacities of a class of channels, either memoryless or indecomposable finite-state, that also suffer from bursts of erasures. For such channels, we analyze the performances of low-density parity-check (LDPC) codes and code ensembles under belief propagation (BP) decoding, using density evolution (DE) techniques. Although known LDPC codes perform well in non-erasure-affected channels, their performances are far from the capacities when both random errors and erasures are present. We show that enhancing the codes’ erasure handling using published methods beats, in some instances, the BP thresholds. However, to achieve capacity, codes must be constructed to tackle both effects simultaneously.

2. Construction of Burst-Erasure Efficient LDPC Codes for Use with Belief Propagation Decoding

   Li, K., Kavčić, A., and Erden, M. F.

   In 2010 IEEE International Conference on Communications 2010

   Abs HTML

   In this paper, we consider the optimal permutation of encoded symbols that will result in the maximum burst erasure efficiency for a given low-density parity-check (LDPC) code under belief propagation (BP) decoding. A novel algorithm is proposed to permute columns of the parity-check matrix to achieve high burst erasure correction efficiencies. This method can be applied to any linear code to increase its burst-erasure efficiency while retaining its random-error correction capability. Results show that the algorithm outperforms previous methods.