IEEE Transactions on Neural Networks and Learning Systems

In this article, we present a modification to the state-of-the-art N-BEATS deep learning architecture for the univariate time series point forecasting problem for generating parametric probabilistic forecasts. Next, we propose an extension to this probabilistic N-BEATS architecture to allow optimizing probabilistic forecasts from both a traditional forecast accuracy perspective as well as a forecast stability perspective, where the latter is defined in terms of a change in the forecast distribution for a specific time period caused by updating the probabilistic forecast for this time period when new observations become available (i...

Recent years have witnessed the great success of the applications of graph convolutional networks (GCNs) in various scenarios. However, due to the challenging over-smoothing and over-squashing problems, the ability of GCNs to model information from long-distance nodes has been largely limited. One solution is to aggregate features from different hops of neighborhoods with a linear combination of them followed by a shallow feature transformation. However, we demonstrate that those methods can only achieve a tradeoff between tackling those two problems...

The series includes two parts, articulating the two novel avenues of research on intelligent fault diagnosis (FD) for nonlinear feedback control systems. In Part I of the series, we design a novel FD paradigm by elaborating an invertible neural network (INN) for feedback control systems. With the aid of a left manifold, the core idea behind the INN-based FD scheme is as follows: 1) formulation of residual generator used for FD as a projection of system data onto the null space that has the same dimension as system outputs; 2) in a topological space, elaboration of a homeomorphism that delivers an invertible relationship between system outputs and residual signals when the system input is given; and 3) skillful introduction of both the master and slave objective functions to achieve system/parameter identification with information loseless property...

In this article, we consider the impulsive estimation problem for a specific category of discrete-time complex networks (CNs) characterized by Markovian switching topologies. The measurement outputs of the underlying CNs, transmitted to the observer over wireless networks, are subject to bit rate constraints. To effectively reduce the estimation error and enhance estimation performance, a mode-dependent impulsive observer is proposed that employs the impulse mechanism. The application of stochastic analysis techniques leads to the derivation of a sufficient condition for ensuring the mean-square boundedness of the estimation error dynamics...

Knowledge distillation (KD) has shown great potential for transferring knowledge from a complex teacher model to a simple student model in which the heavy learning task can be accomplished efficiently and without losing too much prediction accuracy. Recently, many attempts have been made by applying the KD mechanism to graph representation learning models such as graph neural networks (GNNs) to accelerate the model's inference speed via student models. However, many existing KD-based GNNs utilize multilayer perceptron (MLP) as a universal approximator in the student model to imitate the teacher model's process without considering the graph knowledge from the teacher model...

Due to the inefficiency of pixel-level annotations, weakly supervised salient object detection with image-category labels (WSSOD) has been receiving increasing attention. Previous works usually endeavor to generate high-quality pseudolabels to train the detectors in a fully supervised manner. However, we find that the detection performance is often limited by two types of noise contained in pseudolabels: 1) holes inside the object or at the edge and outliers in the background and 2) missing object portions and redundant surrounding regions...

The 6-D pose estimation is a critical work essential to achieve reliable robotic grasping. Currently, the prevalent method is reliant on keypoint correspondence. However, this approach hinges on the determination of object keypoint locations, alongside their detection and localization in real scenes. It also employs the random sample consensus (RANSAC)-based perspective-n-point (PnP) algorithm to solve the pose. Yet, it is nondifferentiable and incapable of backpropagation with loss during the training phase...

Embedding the Hamiltonian formalisms into neural networks (NNs) enhances the reliability and precision of data-driven models, in which substantial research has been conducted. However, these approaches require the system to be represented in canonical coordinates, i.e., observed states should be generalized position-momentum pairs, which are typically unknown. This poses limitations when the method is applied to real-world data. Existing methods tackle this challenge through coordinate transformation or designing complex NNs to learn the symplectic phase flow of the state evolution...

Deep multiview clustering provides an efficient way to analyze the data consisting of multiple modalities and features. Recently, the autoencoder (AE)-based deep multiview clustering algorithms have attracted intensive attention by virtue of their rewarding capabilities of extracting inherent features. Nevertheless, most existing methods are still confronted by several problems. First, the multiview data usually contains abundant cross-view information, thus parallel performing an individual AE for each view and directly combining the extracted latent together can hardly construct an informative view-consensus feature space for clustering...

Filter pruning has gained widespread adoption for the purpose of compressing and speeding up convolutional neural networks (CNNs). However, the existing approaches are still far from practical applications due to biased filter selection and heavy computation cost. This article introduces a new filter pruning method that selects filters in an interpretable, multiperspective, and lightweight manner. Specifically, we evaluate the contributions of filters from both individual and overall perspectives. For the amount of information contained in each filter, a new metric called information capacity is proposed...

Graph neural networks (GNNs) have demonstrated remarkable success for semisupervised node classification. However, these GNNs are still limited to the conventionally semisupervised framework and cannot fully leverage the potential value of large numbers of unlabeled samples. The pseudolabeling method in semisupervised learning (SSL) is widely recognized because it can clearly leverage unlabeled samples. Nevertheless, the existing pseudolabeling methods usually utilize a fixed threshold for all classes and only use a portion of unlabeled samples (ones with high prediction confidence), which leads to class imbalance and low data utilization...

Deep neural networks (DNNs) are increasingly being used to learn controllers due to their excellent approximation capabilities. However, their black-box nature poses significant challenges to closed-loop stability guarantees and performance analysis. In this brief, we introduce a structured DNN-based controller for the trajectory tracking control of Lagrangian systems using backing techniques. By properly designing neural network structures, the proposed controller can ensure closed-loop stability for any compatible neural network parameters...

Graph theory-based techniques have recently been adopted for anomaly detection in hyperspectral images (HSIs). However, these methods rely excessively on the relational structure within the constructed graphs and tend to downplay the importance of spectral features in the original HSI. To address this issue, we introduce graph frequency analysis to hyperspectral anomaly detection (HAD), which can serve as a natural tool for integrating graph structure and spectral features. We treat anomaly detection as a problem of graph frequency location, achieved by constructing a beta distribution-based graph wavelet space, where the optimal wavelet can be identified adaptively for anomaly detection...

As artificial intelligence (AI) has already seen numerous successful applications, the upcoming challenge lies in how to realize artificial general intelligence (AGI). Self-learning algorithms can autonomously acquire knowledge and adapt to new, demanding applications, recognized as one of the most effective techniques to overcome this challenge. Although many related studies have been conducted, there is still no comprehensive and systematic review available, nor well-founded recommendations for the application of autonomous intelligent systems, especially autonomous driving...

This article focuses on the optimal consensus control problem for multiagent systems (MASs) with discontinuous constraints. The case of discontinuous constraints is a particular instance of state constraints, which has been studied less but occurs in many practical situations. Due to the discontinuous constraint boundaries, the traditional barrier function-based backstepping methods cannot be used directly. In response to this thorny problem, a novel constraint boundary reconstruction technique is proposed by designing a class of switch-like functions...

Recently, a strong interest has been in multiview high-dimensional data collected through cross-domain or various feature extraction mechanisms. Nonnegative matrix factorization (NMF) is an effective method for clustering these high-dimensional data with clear physical significance. However, existing multiview clustering based on NMF only measures the difference between the elements of the coefficient matrix without considering the spatial structure relationship between the elements. And they often require postprocessing to achieve clustering, making the algorithms unstable...

Agents usually adjust their strategic behaviors based on their own payoff and aspiration in gaming environments. Hence, aspiration-based learning rules play an important role in the evolutionary dynamics in a population of competing agents. However, there exist different options for how to use the aspiration information for specifying the microscopic learning rules. It is also interesting to investigate under what conditions the aspiration-based learning rules can favor the emergence of cooperative behavior in population games...

Nowadays, pruning techniques have drawn attention to convolutional neural networks (CNNs) for reducing the consumption of computation resources. In particular, the Taylor-based method simplifies the evaluation of importance for each filter as the product of the gradient and weight value of the output features, which outperforms other methods in reductions of parameters and floating point operations (FLOPs). However, the Taylor-based method sacrifices too much accuracy when the overall pruning rate is relatively large compared with other pruning algorithms...

Binary neural network (BNN) is an effective approach to reduce the memory usage and the computational complexity of full-precision convolutional neural networks (CNNs), which has been widely used in the field of deep learning. However, there are different properties between BNNs and real-valued models, making it difficult to draw on the experience of CNN composition to develop BNN. In this article, we study the application of binary network to the single-image super-resolution (SISR) task in which the network is trained for restoring original high-resolution (HR) images...

Dynamic treatment regimes (DTRs), which comprise a series of decisions taken to select adequate treatments, have attracted considerable attention in the clinical domain, especially from sepsis researchers. Existing sepsis DTR learning studies are mainly based on offline reinforcement learning (RL) approaches working on electronic healthcare records data. However, a trained policy may choose a treatment different from a human clinician's prescription. Furthermore, most of them do not consider: 1) heterogeneity in sepsis; 2) short-term transitions; and 3) the relationship between a patient's health state and the prescription...