An interesting trend in multi-core processors is the Graphics Processing Units (GPUs). GPUs consist of hundreds of very simple processing units specialized in efficient parallel computations. Until the recent past, GPUs were seen and used exclusively for the computations required in the processing of ray tracing for graphical life-like scenes mostly used in computer games. But recently there is considerable interest in utilizing the GPUs for high performance numerical computation. GPUs come at a substantial cost discount with respect to attainable performance, due to the ``price subsidization'' by the huge and profitable market for computer games on gaming consoles and personal computers, especially when compared to the tiny market of high performance scientific computing. The dependence of high performance on a specific GPU architecture and the short life due to short product cycle, impose a very heavy burden on programmers responsible for code optimization and maintenance since it has to be redone every few months. With this work, we propose a methodology and a system we call ``AutoGPU'' to extend the life cycle of optimized numerical codes on GPUs by 1) defining a high-level explicit parallelism language for programming numerical algorithms in the domain of signal and image processing applications, 2) automating the code generation with the use of a special compiler, code templates and algorithm libraries and 3) automating the tuning of performance critical software components using advanced optimization and searching techniques. Despite the lack of tools, there is an explosion of effort to utilize the GPU architecture for general purpose computing tasks with very promising results. With the availability of program generators and optimizers like AutoGPU, we expect the promise of supercomputing capability on a desktop computer to become closer to reality.