On the Synergy of Network Science and Artificial Intelligence

1 Motivation

Most of the science done throughout the human evolution uses the traditional reductionism paradigm, which attempts to explain the behavior of any type of system by summing the behavior of its constituent elements and the interactions between them. Consequently, nowadays we have extremely specialized persons on particular small subfields of science, but we have less and less people which study the whole near us. Personally, I think that such a paradigm is not wrong as it is proven by millenniums of humanity advances in science, but I do believe that it is incomplete. The limitations of reductionism were hinted millenniums ago by the ancient Greeks, i.e. Aristotle wrote in Metaphysics that "The whole is more than the sum of its parts". Mathematically, the above statement can not be true, as the whole should be the sum of its parts. Still, it may be that we do now know all the parts, and in many cases in may be very difficult to even intuiting those parts. A classical example here may be the gravitational waves. Gravity was first postulated by Isaac Newton in the 17th century, but in his theory the gravitational waves could not exists as it assumes that physical interactions propagate at infinite speed. Still, after more than two centuries later, Albert Einstein has intuited and predicted the existence of gravitational waves [Einstein, 1916], and one more century later after a huge number of great technological advancements the humans were able to prove the existence of gravitational waves [Abbott et al., 2016].

A solution to overcome the limitations of reductionism may come from the complex systems paradigm which tries to study the systems and the interactions between them as a whole, focusing on multidisciplinary research, approach first pioneered by the Santa Fe institute [Ledford, 2015]. A complete theory of complexity is also very hard to devise, but Network Science (NS) may offer the required mathematical tools (e.g. complex networks) in a data driven era to overpass the reductionism paradigm [Barabasi, 2012]. Complex networks are graph with non-trivial topological features and it has been found that such features are present in many real world systems [Newman, 2010] belonging to various research fields (e.g. neuroscience, astrophysics, biology, epidemiology, social and communication networks).

At the same time, while the NS community has trying to use Artificial Intelligence (AI) techniques to solve various NS open questions, such as in [Psorakis et al., 2011], the AI community has largely ignored the latest findings in network science. Even more, almost any AI subfield tends to focus just on the latest developments from itself, in line with the reductionism paradigm. Contrary, my Ph.D. research tries to give an warning signal and to bind these fields. Thus, it focuses theoretically towards finding the synergy between NS and AI (herein with a focus on deep learning [LeCun et al., 2015] and swarm intelligence [Bonabeau et al., 2000]) with two long term research goals: (1) to better understand the fundamental principles behind the world near us, which may be modeled in amazing structures of networks of networks at micro and macro-scale, from the vigintillions of interacting atoms in the observable universe to the billions of persons in a social network; and (2) to advance the artificial general intelligence concept. All of these, with the ultimate aim of improving the general well-being of the human society.

2 Achievements

More concretely, up to now, by following the above vision, together with my collaborators, I was able to make fundamental theoretical contributions in both fields (i.e. network science and artificial intelligence), while trying to show that there is a bidirectional relation between them, as follows.

Firstly, we have devised a novel class of deep learning models, such as Factored Four-Way Conditional Restricted Boltzmann Machines (FFW-CRBMs) [Mocanu et al., 2015a] and Disjunctive FFW-CRBMs [Mocanu et al., 2016a], by using four-way multiplicative interactions between the neurons from different layers. Such a construction enabled simultaneously classification and prediction of high dimensional and non linear time series.

Secondly, we have conceived a novel class of similarity measures, e.g. [Mocanu et al., 2014b; Bou Ammar et al., 2014; Mocanu et al., 2015b], by using Restricted Boltzmann Machines (RBMs) or variants of them as density estimators.

Thirdly, we have devised a novel class of sparse RBM models [Mocanu et al., 2016b], inspired by complex network concepts, capable to have faster computational time (e.g. 2 orders of magnitude faster than an RBM with 1000 visible and 1000 hidden neurons) at almost no cost in performance. Moreover, we speculate that variants of this model may be used to analyze directly high-dimensional input data (e.g. over one million dimensions or, in other words, a normal image with a resolution of 1000x1000 pixels) without performing dimensionality reduction;

Fourthly, we have conceived a new class of fully decentralized stochastic methods, e.g. [Mocanu et al., 2014a], inspired by swarm intelligence, to compute the centralities of all nodes and links simultaneously in a complex network. The parallel time complexity of this approach is on the polylogarithmic scale with respect to the number of nodes in the network. To give an impression on the magnitude of the computational problem at hand, if we would consider 1 billion devices that run a Facebook application and would have incorporated a protocol for the aforementioned method, an unloaded network, and a transmission rate of 1 message per millisecond, then the centrality of all network elements (users and their connections) may be computed in less than 9 seconds.

At the same time, the practical applicability of these concepts was not let behind, and we have demonstrated their validity in the context of real-world settings, e.g. image/video quality assessment in communication networks [Mocanu et al., 2014b; 2015b; 2015c], computer vision [Mocanu et al., 2014c; 2015a; 2016a].

3 Conclusion and near future research

However, the ones above represent just a drop in the ocean, and they have a major limitation as they consider just static settings. Thus, as near future research directions, I intend to combine these concepts and extend them to dynamic networks (i.e. which change their topologies over time) and online learning settings, while continue to study the synergy between network science, artificial intelligence, and neuroscience.

Acknowledgments

This research has been partially supported by the European Union's Horizon 2020 project INTER-IoT, grant 687283.