How are deep mantle processes related to the mapped geological record? How can we reconcile geochemical observations with geophysical inferences? These are first order unanswered questions despite our steady progress in imaging the Earth's internal structure and understanding the high temperature and pressure properties of minerals. To make a breakthrough, we have to understand solid-state convection in the Earth's mantle in much greater detail. Much is known about the physical processes, such as melting and the delicate interaction between thermal and chemical buoyancy, but the parameters that enter their mathematical description are not very well known. Once these parameters are determined, the thermo-chemical evolution of our planet can self-consistently be modelled. The state-of-the-art is to roughly estimate these parameters and qualitatively compare the modelling to some relevant geophysical, geochemical or geological observations. This comparison is not comprehensive and never explains all observables. We propose a radically new approach, where all observables are used together to infer these parameters directly, using a fully non-linear Bayesian inference technique based on neural networks. This will determine for the first time the initial conditions at the Earth's formation, the Earth-like flow parameters essential to model the thermo-chemical evolution of our planet and produce models that are simultaneously consistent with the main different geophysical and geochemical datasets.