The goal of the P5 project is an intelligent perimeter proactive surveillance system that works robustly under a wide range of weather and lighting conditions and that has strong privacy preserving features. The system will monitor the region outside the security area of critical buildings and infrastructure, and give early warning if terrestrial or airborne threats are detected. The system will support, rather than replace, a human operator. A low false alarm rate due to animals or other innocuous events, combined with high threat detection sensitivity and privacy standards, are central ambitions of the project. To achieve these goals, a multispectral sensor suite comprising both passive and active sensors is envisaged, i.e., a system based on radar, visual and thermal sensors. The sensor suite will be complemented with advanced algorithms for information fusion, object detection and classification, privacy preservation, and high level modelling of intent and behaviour analysis. The P5 project is strongly user-driven and demonstration of the developed surveillance system will be conducted at two different critical infrastructure sites. Finally, the P5 project will make contributions to evolving standards in detection systems.