Talk Bringing High Performance to Python/Numpy Without Changing a Single Line of Code

Presented by Simon Andreas Frimann Lund in Poster and Demo Session 2012 in room Hall Dupréel

Author: Mads Ruben Burgdorff Kristensen, Simon Andreas Frimann Lund, Troels Blum, Brian Vinter Affiliation: Niels Bohr Institute, University of Copenhagen, Denmark

Recent years have provided a wealth of projects showing that using Python for scientific applications outperforms even popular choices such as Matlab. A major factor driving these successes is the efficient utilization of multi-cores, GPUs for general-purpose computation and scaling computations to clusters. However, often these advances sacrifice some of the high-productivity features of Python by introducing new language constructs, enforcing new language semantics and/or enforcing explicit data types. The result is that the user will have to rewrite existing Python applications to use the Python extension.

In order to use GP-GPUs in Python, a popular approach is to embed CUDA/OpenCL code kernels directly in the Python application. The programming productivity of this approach is better and more readable than C/C++ applications but it is still inferior to native Python code. Furthermore, the approach enforces hardware specific programming and thus requires intimate knowledge of the underlying hardware and the CUDA/OpenCL programming model.

Copenhagen Vector Byte Code (cphVB) strives to provide a high-performance back-end for Numerical Python (NumPy) without reducing the high-productivity of Python/NumPy. Without any involvement of the user, cphVB will transform regular sequential Python/NumPy applications into high-performance applications. The cphVB runtime system is designed to be capable of utilizing a broad range of computing platforms efficiently, e.g. Multi-core CPUs, GPGPUs and clusters of such machines.

cphVB consists of a bridge that translates NumPy array operations into cphVB vector operations. The bridge will send these vector operations to a Vector Engine Manager which is in charge of scheduling operations onto a set of available Vector Engines.

cphVB currently implements a selection of Vector Engines optimized to specific hardware architectures, such as multi-core CPUs and GP-GPUs. These engines can be utilized by Python/NumPy applications without modifying the source-code, the utilization is completely transparent at the application-level.

Thus, cphVB provides a high-productivity, high-performance framework that supports legacy Python/NumPy applications without changing a single line of code.

tagged by
no related entity