Differentiable modelling on GPUs

Why to wait hours for computations to complete, when it could take only a few seconds? Tired of prototyping code in an interactive, high-level language and rewriting it in a lower-level language to get high-performance code? Unsure about the applicability of differentiable programming? Or simply curious about parallel and GPU computing and automatic differentiation being game changers in physics-based and data-driven modelling.

This workshop covers trendy areas in modern high-performance computing with examples from geoscientific applications. The physical processes governing natural systems' evolution are often mathematically described as systems of differential equations. In addition, the incredible amount of available data relating to these natural systems increases at unprecedented pace. Understanding and predicting the dynamic of the natural systems requires accurate predictive models built upon constraining parameters from differential equation solvers using a data-driven approach, where fast and accurate solutions require numerical implementations to leverage modern parallel hardware.

The goal of this workshop is to offer an interactive hands-on to solve and constrain systems of differential equations in parallel on GPUs using distributed stencil computations on GPUs and recent development of GPU-ready automatic differentiation (AD) capabilities. We plan to focus on conciseness and efficiency using the ParallelStencil.jl, ImplicitGlobalGrid.jl and Enzyme.jl Julia packages as tools.

The workshop consists of 2 parts:

1. You will learn about parallel and distributed GPU computing and iterative solvers.
2. You will implement a PDE constrained optimisation on GPUs.

By the end of this workshop, you will:

• Have a GPU PDE solver that predicts ice flow and an AD-powered inverse solver to calibrate the forward model;
• Have a Julia code that achieves similar performance to legacy C, CUDA, MPI code or outperforms it;
• Know how the Julia language solves the "two-language problem";
• Be able to leverage the computing power of modern GPU accelerated servers and supercomputers.

We look forward to having you on board and will make sure to foster exchange of ideas and knowledge to provide an as inclusive as possible event.

Workshop GitHub repo: https://github.com/PTsolvers/gpu-workshop-JuliaCon23

Co-authors: Ivan Utkin¹ ², Samuel Omlin¹

¹ ETH Zurich | ² Swiss Federal Institute for Forest, Snow and Landscape Research (WSL)