Growing 3D Artefacts and Functional Machines with Neural Cellular Automata

• Main Authors: Sudhakaran, Shyam, Grbic, Djordje, Li, Siyan, Katona, Adam, Najarro, Elias, Glanois, Claire, Risi, Sebastian
• Format: Journal Article
• Language: English
• Published: 15-03-2021
• Subjects: Computer Science - Learning
• DOI: 10.48550/arxiv.2103.08737

Proceedings of the 2021 Conference on Artificial Life (ALIFE 2021) Neural Cellular Automata (NCAs) have been proven effective in simulating
morphogenetic processes, the continuous construction of complex structures from
very few starting cells. Recent developments in NCAs lie in the 2D domain,
namely reconstructing target images from a single pixel or infinitely growing
2D textures. In this work, we propose an extension of NCAs to 3D, utilizing 3D
convolutions in the proposed neural network architecture. Minecraft is selected
as the environment for our automaton since it allows the generation of both
static structures and moving machines. We show that despite their simplicity,
NCAs are capable of growing complex entities such as castles, apartment blocks,
and trees, some of which are composed of over 3,000 blocks. Additionally, when
trained for regeneration, the system is able to regrow parts of simple
functional machines, significantly expanding the capabilities of simulated
morphogenetic systems. The code for the experiment in this paper can be found
at: https://github.com/real-itu/3d-artefacts-nca.