Wolfram Compiler

The discrete Fourier transform (DFT) and its specialized case, the number theoretic transform (NTT), are two important mathematical tools having applications in several areas of science and engineering. However, despite their usefulness and utility, their adoption continues to be a challenge as computing the DFT of a signal can be a time-consuming and expensive operation. To speed things up, fast Fourier transform (FFT) algorithms, which are reduced-complexity formulations for computing the DFT …

The fast Fourier Transform (FFT), a reduced-complexity formulation of the Discrete Fourier Transform (DFT), is an important tool in many areas of science and engineering. FFTW is a well-known package that follows this approach and is currently one of the fastest available implementations of the FFT. NVIDIA introduced its version of FFTW called cuFFT that achieves high performance on the GPUs. In this work we present a novel way to map the FFT algorithm on the newly introduced Tensor Cores by …

The popularity of data- and scientific-oriented applications, abundance of on-demand compute resources, and scarcity of domain expert programmers have given rise to high-level scripting languages. These high-level scripting languages offer a fast way to translate ideas into code, but tend to pay a heavy performance overhead. In order to alleviate the performance penalty, each implementation of these languages often offers a compilation path to a subset of the language. In this paper we present …

Driven by deep learning, there has been a surge of specialized processors for matrix multiplication, referred to as Tensor Core Units (TCUs). These TCUs are capable of performing matrix multiplications on small matrices (usually 4 X 4 or 16 X 16) to accelerate HPC and deep learning workloads. Although TCUs are prevalent and promise increase in performance and/or energy efficiency and are heavily used within supercomputers to achieve exascale performance, they suffer from over specialization — …

Driven by deep learning, there has been a surge of specialized processors for matrix multiplication, referred to as Tensor Core Units (TCUs). These TCUs come under the guise of different marketing terms and are capable of performing matrix multiplications on small matrices (usually 4x4 or 16x16) to accelerate the convolutional and recurrent neural networks in deep learning workloads. Although TCUs are prevalent and promise increase in performance and/or energy efficiency, they suffer from over …

Functional algorithmic skeletons promise a high-level programming interface for distributed-memory clusters that free developers from concerns of task decomposition, scheduling, and communication. Unfortunately, prior distributed functional skeleton frameworks do not deliver performance comparable to that achievable in a low-level distributed programming model such as C with MPI and OpenMP, even when used in concert with high-performance array libraries. There are several causes: they do not …

The Wolfram Language is a high level scripting language used in technical, scientific, engineering, mathematical, and computing fields. The language is widely used in practice and, in particular, it is the foundation of Wolfram Mathematica and Wolfram Alpha. This talk describes the Wolfram Language Compiler. The Wolfram Compiler is a long-term project that compiles Wolfram programs. We show how clever use of parametric types, code specialization, and optimization techniques allow one to compile …

The Wolfram Language is a high level scripting language used in technical, scientific, engineering, mathematical, and computing fields. The language is widely used in practice and, in particular, it is the foundation of Wolfram Mathematica and Wolfram Alpha. This talk describes the Wolfram Language Compiler. The Wolfram Compiler is a long-term project that compiles Wolfram programs. We show how clever use of parametric types, code specialization, and optimization techniques allow one to compile the Wolfram Language and achieve performance that rivals hand-tuned C code.

The Wolfram Language is a dynamic untyped language that has a 30-year history. The talk will describe current work in developing a compiler for the Wolfram Language. It will cover the compiler’s architecture, along with techniques to convert the language into low-level code. This includes a type system with sexy-types such as type classes, ad hoc polymorphism, parametric polymorphism and overloading. We will describe our multitiered pipeline developed to optimize Wolfram code, along with …

In this talk we will show case new features and techniques developed for the Mathematica compiler. We will show how the compiler leverages the Homoiconicity of the Wolfram language to perform macro expansion, how we structure the compiler into independent passes (and how you can write your own), how the type system works and how it guides function selection, and finally some performance results of running the compiler across different problems.