An Algebra of Array Combinators and its Applications

Research Project, 2021 – 2025

Machine learning happens all around us and drives science. We are also surrounded by hardware that is vulnerable to attack. In hardware and software, important parts correspond to programs operating on arrays.
Folding such programs onto accelerators or into hardware implementations raises similar questions, and is of great practical importance. Improvements are needed.I will develop new data types and combinators that express array (or tensor) access patterns through a structured form of index arithmetic. For linear arrays, we treat indices as vectors of index bits (VIB), that is over GF(2), and design combinators to express how indices may be used With VIB manipulations, rather than arbitrary arithmetic on indices, the resulting accesses and combinations of elements are expressive, yet obey a rich algebra. This view extends to permutations of array elements via the Bit Matrix Multiply and Complement (BMMC) permutations studied by Cormen. We will extend this approach to multidimensional arrays, with particular emphasis on ways to express partitioning. We will develop a new datatype for multidimensional arrays, with associated combinators and their algebra. We will build Domain Specific Languages for expressing partitioning, and will study abstractions beyond simple tiling or sharding. We will demonstrate the results on case studies from the Silveroak project at Google (hardware), and from ML (software) benchmarks such as those considered in the FlexFlow project at Stanford.