Neural Networks : the Official Journal of the International Neural Network Society

Answering complex First-Order Logic (FOL) query plays a vital role in multi-hop knowledge graph (KG) reasoning. Geometric methods have emerged as a promising category of approaches in this context. However, existing best-performing geometric query embedding (QE) model is still up against three-fold potential problems: (i) underutilization of embedding space, (ii) overreliance on angle information, (iii) uncaptured hierarchy structure. To bridge the gap, we propose a lollipop-like bi-centered query embedding method named LollipopE...

Big torque inputs in controls could increase energy consumption, and big estimated perturbations in observers could produce device damages. Therefore, it would be interesting to propose a constrained control for safe reference tracking and a constrained observer for safe perturbation estimation in robots. Furthermore, the best gains in controls produce a balance between safe reference tracking and save energy consumption. Therefore, it would be interesting to propose a method to find the best gains. In this paper, an observer-based differential evolution constrained control is proposed for safe reference tracking in robots...

To enhance resource efficiency and model deployability of neural networks, we propose a neural-layer architecture based on Householder weighting and absolute-value activating, called Householder-absolute neural layer or simply Han-layer. Compared to a fully connected layer with d-neurons and d outputs, a Han-layer reduces the number of parameters and the corresponding computational complexity from O(d2 ) to O(d). The Han-layer structure guarantees that the Jacobian of the layer function is always orthogonal, thus ensuring gradient stability (i...

The transformer-based model can simultaneously learn the representation for both images and text, providing excellent performance for multimodal applications. Practically, the large scale of parameters may hinder its deployment in resource-constrained devices, creating a need for model compression. To accomplish this goal, recent studies suggest using knowledge distillation to transfer knowledge from a larger trained teacher model to a small student model without any performance sacrifice. However, this only works with trained parameters of the student model by using the last layer of the teacher, which makes the student model easily overfit in the distillation procedure...

Episodic memory, as a type of long-term memory (LTM), is used to learn and store the unique personal experience. Based on the episodic memory biological mechanism, this paper proposes a bionic episodic memory memristive neural network circuit. The proposed memristive neural network circuit includes a neocortical module, a parahippocampal module and a hippocampus module. The neocortical module with the two paths structure is used to receive the sensory signal, and is also used to separate and transmit the spatial information and the non-spatial information involved in the sensory signal...

Graph Transformers (GTs) have achieved impressive results on various graph-related tasks. However, the huge computational cost of GTs hinders their deployment and application, especially in resource-constrained environments. Therefore, in this paper, we explore the feasibility of sparsifying GTs, a significant yet under-explored topic. We first discuss the redundancy of GTs based on the characteristics of existing GT models, and then propose a comprehensive Graph Transformer SParsification (GTSP) framework that helps to reduce the computational complexity of GTs from four dimensions: the input graph data, attention heads, model layers, and model weights...

This paper addresses the predefined-time distributed optimization of nonlinear multi-agent system using a hierarchical control approach. Considering unknown nonlinear functions and external disturbances, we propose a two-layer hierarchical control framework. At the first layer, a predefined-time distributed estimator is employed to produce optimal consensus trajectories. At the second layer, a neural-network-based predefined-time disturbance observer is introduced to estimate the disturbance, with neural networks used to approximate the unknown nonlinear functions...

In network systems, control using minimum nodes or pinning control can be effectively used for stabilization problems to cut down the cost of control. In this paper, we investigate the set stabilization problem of logical control networks. In particular, we study the set stabilization problem of probabilistic Boolean networks (PBNs) and probabilistic Boolean control networks (PBCNs) via controlling minimal nodes. Firstly, an algorithm is given to search for the minimum index set of pinning nodes. Then, based on the analysis of its high computational complexity, we present optimized algorithms with lower computational complexity to ascertain the network control using minimum node sets...

Traditional convolutional neural networks (CNNs) often suffer from high memory consumption and redundancy in their kernel representations, leading to overfitting problems and limiting their application in real-time, low-power scenarios such as seizure detection systems. In this work, a novel cosine convolutional neural network (CosCNN), which replaces traditional kernels with the robust cosine kernel modulated by only two learnable factors, is presented, and its effectiveness is validated on the tasks of seizure detection...

Graph Neural networks (GNNs) have been applied in many scenarios due to the superior performance of graph learning. However, fairness is always ignored when designing GNNs. As a consequence, biased information in training data can easily affect vanilla GNNs, causing biased results toward particular demographic groups (divided by sensitive attributes, such as race and age). There have been efforts to address the fairness issue. However, existing fair techniques generally divide the demographic groups by raw sensitive attributes and assume that are fixed...

In machine learning it is often necessary to assume or know the distribution of the data, however it is difficult to do so in practical applications. Aiming to this problem, this work, we propose a novel distribution-free Bayesian regularized learning framework for semi-supervised learning, which is called Hessian regularized twin minimax probability extreme learning machine (HRTMPELM). In this framework, we attempt to construct two non-parallel hyperplanes by introducing the high separation probability assumption, such that each hyperplane separates samples from one class with maximum probability while moving away from samples from the other class...

Channel Pruning is one of the most widespread techniques used to compress deep neural networks while maintaining their performances. Currently, a typical pruning algorithm leverages neural architecture search to directly find networks with a configurable width, the key step of which is to identify representative subnet for various pruning ratios by training a supernet. However, current methods mainly follow a serial training strategy to optimize supernet, which is very time-consuming. In this work, we introduce PSE-Net, a novel parallel-subnets estimator for efficient channel pruning...

Generative Flow Networks (GFlowNets) aim to generate diverse trajectories from a distribution in which the final states of the trajectories are proportional to the reward, serving as a powerful alternative to reinforcement learning for exploratory control tasks. However, the individual-flow matching constraint in GFlowNets limits their applications for multi-agent systems, especially continuous joint-control problems. In this paper, we propose a novel Multi-Agent generative Continuous Flow Networks (MACFN) method to enable multiple agents to perform cooperative exploration for various compositional continuous objects...

Cropping-and-segmenting pattern parsers often combine diverse inner correlations into a single metric/scheme, resulting in over-generalizations and redundant representations. It is proposed to streamline pattern parsing by using presenting a redundant association elimination network (RAEN) with capsule attention twisters (CATs) and capsule-attention routing agreement (CARA). CATs trim delicate relationships between parts and wholes that are weak and interchangeable. Senior entities can only be updated by primary entities that meet the requirements of inter-part diversity and intra-object cohesiveness...

Expert demonstrations in imitation learning often contain different behavioral modes, e.g., driving modes such as driving on the left, keeping the lane, and driving on the right in the driving tasks. Although most existing multi-modal imitation learning methods allow learning from demonstrations of multiple modes, they have strict constraints on the data of each mode, generally requiring a near data ratio of all modes. Otherwise, it tends to fall into a mode collapse or only learn the data distribution of the mode that has the largest data volume...

This paper addresses the resilient event-triggering adaptive neural network (NN) control problem for networked control systems under mixed cyber attacks. Compared with the conventional event-triggered mechanism (ETM) with constant threshold, a novel resilient ETM is designed to withstand the affect of denial-of-service attacks and conserve communication resources. Different from the energy-bounded deception attacks, an unknown state-dependent nonlinear attack signal is considered in this work. To identify the deception attack, the NN technique is utilized to approximate the unknown attack signal...

Snapshot compressive hyperspectral imaging necessitates the reconstruction of a complete hyperspectral image from its compressive snapshot measurement, presenting a challenging inverse problem. This paper proposes an enhanced deep unrolling neural network, called EDUNet, to tackle this problem. The EDUNet is constructed via the deep unrolling of a proximal gradient descent algorithm and introduces two innovative modules for gradient-driven update and proximal mapping reflectivity. The gradient-driven update module leverages a memory-assistant descent approach inspired by momentum-based acceleration techniques, for enhancing the unrolled reconstruction process and improving convergence...

Representation learning for dynamic networks is designed to learn the low-dimensional embeddings of nodes that can well preserve the snapshot structure, properties and temporal evolution of dynamic networks. However, current dynamic network representation learning methods tend to focus on estimating or generating observed snapshot structures, paying excessive attention to network details, and disregarding distinctions between snapshots with larger time intervals, resulting in less robustness for sparse or noisy networks...

This study presents a solution to the challenges of tracking consensus and guarantee-cost H∞ control in a specific set of second-order multi-agent systems with external disturbances. A proposed event-triggered control method based on periodic sampling data is presented for second-order multi-agent systems that include external disturbances. In contrast to the real-time monitoring of system state information used in the previous event-triggered mechanism, this approach collects system state information through periodic sampling...

The specified convergence time, designated by the user, is highly attractive for many high-demand applications such as industrial robot control, missile guidance, and autonomous vehicles. For the application of neural networks in the field of secure communication and power systems, the importance of prescribed-time synchronization(PTs) and stable performance of the system is more prominent. This paper introduces a prescribed-time controller without the fractional power function and sign function, which can reach synchronization at a prescribed time and greatly reduce the chattering phenomenon of neural networks...