Recent & Upcoming Talks

2020

Workshop on Benchmarking Machine Learning Workloads

With evolving system architectures, hardware and software stacks, diverse machine learning (ML) workloads, and data, it is important to understand how these components interact with each other. Well-defined benchmarking procedures help evaluate and reason the performance gains with ML workload-to-system mappings. We welcome all novel submissions in benchmarking machine learning workloads from all disciplines, such as image and speech recognition, language processing, drug discovery, simulations, …

Using Benanza & DLBricks to Inform Optimizations

Benanza and DLBricks are a sustainable way to develop ML benchmarks along with analyses the results to inform and pin-point optimization opportunities. Benanza and DLBricks consist of: a model processor which parses models into an internal representation, a benchmark generator that automatically generates micro-benchmarks given a set of models, a database of benchmark results, and an analyzer that computes the “lower-bound” latency of DL models using the benchmark data and informs optimizations …

Digging Deep into Model Performing using Across Stack Profiling

This talk presents an Across-Stack Profiling (XSP) which is a leveled profiling design that leverages existing profiling tools to give a drill down view of model, system, and hardware bottlenecks. The design does so in spite of the profiling overheads incurred from the profiling. We coupled the profiling capability with an automatic analysis pipeline to systematically characterize over 150 state-of-the-art ML models. Through this characterization, we show that our across-stack profiling solution …

2019

SC 2019 - Across-Stack Profiling and Characterization of State-of-the-Art Machine Learning Models on GPUs

The past few years have seen a surge of using Machine Learning (ML) and Deep Learning (DL) algorithms for traditional HPC tasks such as feature detection, numerical analysis, and graph analytics. While ML and DL help solving HPC tasks, their adoption has been hampered in part because of the complexity of understanding ML/DL and their interactions with systems utilization. Optimizing these algorithms requires characterizing their performance and resource utilization across the hardware/software …

Tutorial at IISWC 2019 - Challenges and Solutions for End-to-End and Across Stack ML Benchmarking

The current landscape of Machine Learning (ML) and Deep Learning (DL) is rife with non-uniform models, frameworks, and system stacks. It lacks standard tools and methodologies to evaluate and profile models or systems. Due to the absence of standard tools, the state of the practice for evaluating and comparing the benefits of proposed AI innovations (be it hardware or software) on end-to-end AI pipelines is both arduous and error-prone — stifling the adoption of the innovations in a rapidly …

Developing in the Wolfram Compiler

The Wolfram Language is a dynamic untyped language that has a 30-year history. The talk will describe current work in developing a compiler for the Wolfram Language. It will cover the compiler’s architecture, along with techniques to convert the language into low-level code. This includes a type system with sexy-types such as type classes, ad hoc polymorphism, parametric polymorphism and overloading. We will describe our multitiered pipeline developed to optimize Wolfram code, along with …

HotChips 2019 - MLModelScope: Evaluate and Profile ML Models at Scale and Across Stack

The current landscape of Machine Learning (ML) and Deep Learning (DL) is rife with non-uniform frameworks, models, and system stacks but lacks standard tools to facilitate the evaluation and measurement of model. Due to the absence of such tools, the current practice for evaluating and comparing the benefits of proposed AI innovations (be it hardware or software) on end-to-end AI pipelines is both arduous and error prone – stifling the adoption of the innovations. We propose MLModelScope …

Tutorial at ASPLOS 2019 - Benchmarking Deep Learning Systems

The current landscape of Machine Learning (ML) and Deep Learning (DL) is rife with non-uniform models, frameworks, and system stacks. It lacks standard tools and methodologies to evaluate and profile models or systems. Due to the absence of standard tools, the state of the practice for evaluating and comparing the benefits of proposed AI innovations (be it hardware or software) on end-to-end AI pipelines is both arduous and error-prone — stifling the adoption of the innovations in a rapidly …

GTC 2019 - MLModelScope: Evaluate and Measure Machine Learning Models

The current landscape of Machine Learning (ML) and Deep Learning (DL) is rife with non-uniform frameworks, models, and system stacks but lacks standard tools to facilitate the evaluation and measurement of models. Due to the absence of such tools, the current practice for evaluating and comparing the benefits of proposed AI innovations (be it hardware or software) on end-to-end AI pipelines is both arduous and error-prone — stifling the adoption of the innovations.

2018

Wolfram Compiler Advances

In this talk we will show case new features and techniques developed for the Mathematica compiler. We will show how the compiler leverages the Homoiconicity of the Wolfram language to perform macro expansion, how we structure the compiler into independent passes (and how you can write your own), how the type system works and how it guides function selection, and finally some performance results of running the compiler across different problems.

MLModelScope: Manage and Manage your ML Models

The emergence of Machine Learning (ML) and Deep Learning (DL) algorithms as a viable programming model has inspired plenty of innovations in the software development, algorithms, and system and architecture communities. Today, many software, hardware, and cloud providers are vying to capture the emerging market and make claims that are not easy to verify. Furthermore, since different communities optimize for different targets and use different terminology, users are overloaded with a lot of …

MLModelScope: Introspect and Manage your ML Models (Best Poster)

The emergence of Machine Learning (ML) and Deep Learning (DL) algorithms as a viable programming model has inspired plenty of innovations in the software development, algorithms, and system and architecture communities. Today, many software, hardware, and cloud providers are vying to capture the emerging market and make claims that are not easy to verify. Furthermore, since different communities optimize for different targets and use different terminology, users are overloaded with a lot of …

2016

Document Introspection System for CognitiVe Research: An Open Application for Cognitive Computing Workload Exploration

The traditional process of software development has revolved around the development, testing, and deployment of monolithic applications. The data-driven nature of Machine Learning and Cognitive Computing applications have disrupted this paradigm. Instead, the development of applications revolves around the processing of training data and experimentation with the learning and inference algorithms that power the application. This in turn has changed the primary activities of software developers …