6. AI Compiler Backend

In this chapter, we will explore the design of the AI compiler backend. The objective of an AI compiler backend is to enhance the efficiency of AI program execution by optimizing the Intermediate Representation (IR) generated by the compiler frontend. This optimization enables the full utilization of hardware capabilities. The backend achieves this goal by applying optimizations to IR code based on hardware capabilities. Furthermore, it selects suitable operators based on the capabilities of target hardware to execute computations efficiently, while also allocating memory to optimize data reuse and locality. Additionally, the backend often incorporates an operator compiler, which optimizes the execution strategy for code statements associated with operators.

This chapter aims to achieve the following learning objectives:

• Understand the role and architecture of an AI compiler backend.

• Understand typical methods for optimizing computational graphs.

• Understand typical methods for selecting operators.

• Understand typical methods for memory allocation.

• Understand the architecture and functionalities of operator compilers.

• 6.1. Overview
• 6.2. Graph Optimization
  • 6.2.1. Hardware-Agnostic Optimizations
  • 6.2.2. Hardware-Specific Optimizations
• 6.3. Operator Selection
  • 6.3.1. Basic Concepts of Operator Selection
  • 6.3.2. Process of Operator Selection
• 6.4. Memory Allocation
  • 6.4.1. Device Memory
  • 6.4.2. Process of Memory Allocation
  • 6.4.3. Memory Reuse
  • 6.4.4. Optimization Techniques for Memory Allocation
  • 6.4.5. Data Compression
  • 6.4.6. Memory Swap
• 6.5. Operator Compiler
  • 6.5.1. Scheduling Strategy
  • 6.5.2. Combining Scheduling Strategies
  • 6.5.3. Finding Optimized Strategies with Polyhedral Models
  • 6.5.4. Adaptation to Instruction Sets
  • 6.5.5. Expression Ability
• 6.6. Chapter Summary
• 6.7. Further Reading