General Theory of Neural Networks

🌈 Abstract

The article discusses a General Theory of Neural Networks that unifies diverse systems, from gene regulatory networks to artificial neural networks, under the concept of Universal Activation Networks (UANs). The key points covered include:

• UANs exhibit common principles of evolvability, generative open-endedness, and computational homomorphism across biological and artificial networks.
• Insights from artificial gene regulatory networks, such as the importance of network topology over implementation details and the concept of critical topology, are explored as a foundation for understanding UANs.
• A series of conjectures are proposed, including the computational nature of UANs, the importance of critical topology, and the evolutionary pruning of networks to their necessary and sufficient structure.
• The article emphasizes the need for rigorous empirical testing to validate the UAN theory and identify universal features across different domains.

🙋 Q&A

[01] Insights from Artificial Gene Regulatory Networks

1. What key insights did the research on artificial gene regulatory networks provide?

• The model developed by von Dassow et al. demonstrated that gene regulatory networks adhere to basic computational principles, and that network topology determines function rather than implementation details.
• Reducing the gene regulatory network to a Boolean network revealed that nature had effectively evolved an intricate system to implement a fundamental Boolean computational circuit.
• The research on the evolution of artificial gene regulatory networks showed that evolution consistently favored sparser network densities, approaching a minimal threshold where the network could still function. This demonstrated the cost of network complexity and the presence of spurious interactions.
• The concept of a "critical topology" was introduced, representing the minimal network structure that retains full functionality and efficiency.

2. What implications did the findings on artificial gene regulatory networks have for understanding biological and artificial networks?

• The research anticipated later developments in machine learning, such as the benefits of pruning artificial neural networks to enhance performance.
• The insights on critical topology and the evolutionary pruning of networks suggest that the same principles may apply to all Universal Activation Networks, bridging the understanding of biological and artificial networks.

[02] Conjectures on Universal Activation Networks

1. What are the key conjectures proposed in the article?

• Universal Activation Networks (UANs) can simulate the function of any other activator networks across biology, physics, and artificial intelligence, suggesting a computational homomorphism.
• UANs operate according to computational principles, not magic, and computation should be the null hypothesis when analyzing these networks.
• The critical topology of a UAN, rather than implementation details, dictates its function.
• In a fully connected UAN network, most interaction weights are spurious and can be pruned to reveal the necessary and sufficient circuit topology (the critical topology).
• Allowing a UAN to evolve its network connectivity will result in the pruning of the network topology to the necessary and sufficient network topology (the critical network).

2. How do these conjectures aim to unify the understanding of biological and artificial networks?

• The conjectures propose common computational principles that can be applied across diverse networks, from gene regulatory networks to artificial neural networks.
• By focusing on the critical topology and the evolutionary pruning of networks, the conjectures suggest a unified framework for understanding the structure and function of these networks, regardless of their biological or artificial origin.
• The conjectures emphasize the importance of identifying universal features and computational principles that can bridge the gaps between traditionally disparate fields.