NeuCube for Online Multisensory Data Modelling


The Brain-Like Artificial Intelligence (BLAI) is pioneered by Prof.Nikola Kasabov and here it is applied for modelling multi-sensory data.

This project develops new methods and a system for multi-sensory predictive data modelling in a real time using spiking neural networks and more specifically - the NeuCube computational architecture. The system can be used for various applications where sensory information is measured in a real time, such as: environmental (e.g., seismic); ecological (e.g., climate; pollution); power energy (e.g., wind turbine data); health (patient sensory data for patient rehabilitation at home), and many more.

An increased interest on acquisition, processing and analysis of spatio-temporal stream data that arises from a set of sensors (sensor network) spatially distributed has emerged over the last decade.

Sensor networks have become an interdisciplinary subject including many fields such as wireless networks and communications, protocols, distributed algorithms, signal processing, embedded systems, and information management. They have important applications such as remote environmental monitoring, seismic activity, target tracking, etc.

Spiking neural networks (SNN), the third generation of neural networks, have been proved to incorporate temporal information in communication and computation like real neurons do. They have the potential to communicate and learn spatial and temporal information by sequences of spikes transmitted among spatially located synapses and neurons. They receive and send out individual pulses allowing multiplexing of information like the frequency and amplitude of sound. Both spatial and temporal information can be encoded in an SNN as locations of synapses and neurons, and time of their spiking activity, respectively.

Evolving spatio-temporal data machines (eSTDM), based on neuromorphic brain-like information processing principles, are multi-modular computer systems designed to deal with large and fast spatio/spectro temporal data using SNN as major processing modules. In this context, the NeuCube- as a generic and systematic framework- has being designed for any spatio-temporal stream data problems. The novel methods and algorithms allow modelling large and fast multisensory spatio-temporal data which contain diverse input variables captured by each element of the sensor network.

FIGURE. Multivariate input mapping approaches. Variable vi indicates the output that a sensor can provide, t represents the time, and sn indicates a node in the sensor network. Finally, ni indicates an input neuron which process the output of a sensor sn.

Related Papers and Benchmarking

The proposed methods and systems, when compared with traditional statistical and machine learning methods, showed superior results in the following aspects:

  1. Although traditional methods such as Support Vector Machines (SVM), Multilayer Perceptron neural networks (MLP), Bayesian methods and many more have utilised for studying temporal data, they are inefficient in capturing complex spatio-temporal relationships from stream data. On the other hand, NeuCube can map input variables into spatially located spiking neurons towards modelling large and fast on-line multisensory spatio-temporal stream data;
  2. NeuCube can map one or multiple input variables coming from each node of a sensor network into several SNN. This novel approach allows researchers to simultaneously perceive and analyse the patterns that each variable produces through different levels and combinations of multisensory spatio-temporal stream data;
  3. NeuCube can provide enough information to extract relevant knowledge of the system in contex.

See also some of the related papers:

Kasabov, N. K. (2014). NeuCube: A spiking neural network architecture for mapping, learning and understanding of spatio-temporal brain dataNeural Networks52, 62-76.

N. Kasabov, N. Scott, E.Tu, S. Marks, N.Sengupta, E.Capecci, M.Othman,M. Doborjeh, N.Murli,R.Hartono, J.Espinosa-Ramos, L.Zhou, F.Alvi, G.Wang, D.Taylor, V. Feigin,S. Gulyaev, M.Mahmoudh, Z-G.Hou, J.Yang, Design methodology and selected applications of evolving spatio- temporal data machines in the NeuCube neuromorphic framework, Neural Networks, v.78, 1-14, 2016.

E. Tu, N. Kasabov, J. Yang, Mapping Temporal Variables into the NeuCube Spiking Neural Network Architecture for Improved Pattern Recognition and Predictive Modelling, IEEE Trans. on Neural Networks and Learning Systems, 28 (6), 1305-1317,, 2017 DOI: 10.1109/TNNLS.2016.2536742, 201

R&D System

For this project, an R&D system has been developed based on NeuCube. The system can be obtained subject to licensing agreement.


The developer of this project is:

Dr Israel Espinosa Ramos