Very recently, it has been claimed that the Bayesian paradigm has revolutionized statistical thinking in numerous fields of research, as a considerable amount of novel Bayesian statistical models and estimation algorithms have gained popularity among scientists. Despite of the evident success of the Bayesian approach, there are also many research problems where the computational challenges have so far proven to be too exhaustive to promote wide-spread use of the state-of-the-art Bayesian methodology. In particular, due to significant advances in measurement technologies, e.g. in molecular biology, a constant need for analyzing and modeling very large and complex data sets has emerged on a wide scale during the past decade. Such needs are even anticipated to rapidly increase in near future with the current technological advances. The prevailing situation is therefore somewhat paradoxical, as the theoretical superiority of the Bayesian paradigm as an uncertainty handling framework is widely acknowledged, yet it can be unable to provide practically applicable solutions to complex scientific problems. To resolve this issue, the research project will have a focus on stochastic computational and modeling strategies to develop methods that overcome problems associated with the analysis of highly complex data sets. With these methods we aim to be able to solve a multitude of statistical learning problems for data sets which cannot yet be reliably handled in practice by any of the existing Bayesian tools. Our approaches will build upon recent advances in Bayesian predictive modeling and adaptive stochastic Monte Carlo computation, to create a novel family of parallel interacting learning algorithms. Several significant statistical modeling problems will be considered to demonstrate the potential of the developed methods. Our goal is also to provide implementations of some of the algorithms as freely available software packages to benefit concretely the scientific community.