12. Reinforcement Learning System
Quick search
code
Show Source
GitHub 中文版
Machine Learning Systems: Design and Implementation
Table Of Contents
  • 1. Preface
  • 2. Introduction
    • 2.1. Machine Learning Applications
    • 2.2. Design Objectives of Machine Learning Frameworks
    • 2.3. Machine Learning Framework Architecture
    • 2.4. Application Scenarios of Machine Learning Systems
    • 2.5. Book Organization and Intended Audience
  • 3. Part I Framework Design
  • 4. Programming Model
    • 4.1. Overview
    • 4.2. Machine Learning Workflow
    • 4.3. Neural Network Programming
    • 4.4. Functional Programming
    • 4.5. Bridging Python and C/C++ Functions
    • 4.6. Chapter Summary
  • 5. AI Compiler Frontend
    • 5.1. Overview of AI Compilers
    • 5.2. Overview of AI Compiler Frontends
    • 5.3. Intermediate Representation
    • 5.4. Automatic Differentiation
    • 5.5. Type Systems and Static Analysis
    • 5.6. Frontend Compilation Optimization
    • 5.7. Chapter Summary
    • 5.8. Further Reading
  • 6. AI Compiler Backend
    • 6.1. Overview
    • 6.2. Graph Optimization
    • 6.3. Operator Selection
    • 6.4. Memory Allocation
    • 6.5. Operator Compiler
    • 6.6. Chapter Summary
    • 6.7. Further Reading
  • 7. Hardware Accelerator
    • 7.1. Overview
    • 7.2. Components of Hardware Accelerators
    • 7.3. Programming Methods
    • 7.4. Performance Optimization Methods
    • 7.5. Chapter Summary
  • 8. Distributed Training
    • 8.1. Overview
    • 8.2. Parallelism Methods
    • 8.3. Pipeline Parallelism with Micro-Batching
    • 8.4. Architecture of Machine Learning Clusters
    • 8.5. Collective Communication
    • 8.6. Parameter Server
    • 8.7. Federated Learning
    • 8.8. Training Large Language Models
    • 8.9. Chapter Summary
    • 8.10. Further Reading
  • 9. Model Deployment
    • 9.1. Overview
    • 9.2. Conversion to Inference Model and Model Optimization
    • 9.3. Model Compression
    • 9.4. Advanced Efficient Techniques
    • 9.5. Model Inference
    • 9.6. Security Protection of Models
    • 9.7. Chapter Summary
    • 9.8. Further Reading
  • 10. Part II Application Scenarios
  • 11. Recommender System
    • 11.1. Overview
    • 11.2. System Components
    • 11.3. Recommendation Pipeline
    • 11.4. Model Update
    • 11.5. Supporting Real-time Machine Learning
    • 11.6. Chapter Summary
    • 11.7. Further Reading
  • 12. Reinforcement Learning System
    • 12.1. Overview
    • 12.2. Introduction to Reinforcement Learning
    • 12.3. Single-Node Reinforcement Learning System
    • 12.4. Distributed Reinforcement Learning System
    • 12.5. Multi-agent Reinforcement Learning
    • 12.6. Multi-agent Reinforcement Learning System
    • 12.7. Chapter Summary
  • 13. Robotic System
    • 13.1. Overview of Robotic Systems
    • 13.2. Robot Operating System
    • 13.3. Case Study: Using ROS
    • 13.4. Modern Robot Learning
    • 13.5. Chapter Summary
Machine Learning Systems: Design and Implementation
Table Of Contents
  • 1. Preface
  • 2. Introduction
    • 2.1. Machine Learning Applications
    • 2.2. Design Objectives of Machine Learning Frameworks
    • 2.3. Machine Learning Framework Architecture
    • 2.4. Application Scenarios of Machine Learning Systems
    • 2.5. Book Organization and Intended Audience
  • 3. Part I Framework Design
  • 4. Programming Model
    • 4.1. Overview
    • 4.2. Machine Learning Workflow
    • 4.3. Neural Network Programming
    • 4.4. Functional Programming
    • 4.5. Bridging Python and C/C++ Functions
    • 4.6. Chapter Summary
  • 5. AI Compiler Frontend
    • 5.1. Overview of AI Compilers
    • 5.2. Overview of AI Compiler Frontends
    • 5.3. Intermediate Representation
    • 5.4. Automatic Differentiation
    • 5.5. Type Systems and Static Analysis
    • 5.6. Frontend Compilation Optimization
    • 5.7. Chapter Summary
    • 5.8. Further Reading
  • 6. AI Compiler Backend
    • 6.1. Overview
    • 6.2. Graph Optimization
    • 6.3. Operator Selection
    • 6.4. Memory Allocation
    • 6.5. Operator Compiler
    • 6.6. Chapter Summary
    • 6.7. Further Reading
  • 7. Hardware Accelerator
    • 7.1. Overview
    • 7.2. Components of Hardware Accelerators
    • 7.3. Programming Methods
    • 7.4. Performance Optimization Methods
    • 7.5. Chapter Summary
  • 8. Distributed Training
    • 8.1. Overview
    • 8.2. Parallelism Methods
    • 8.3. Pipeline Parallelism with Micro-Batching
    • 8.4. Architecture of Machine Learning Clusters
    • 8.5. Collective Communication
    • 8.6. Parameter Server
    • 8.7. Federated Learning
    • 8.8. Training Large Language Models
    • 8.9. Chapter Summary
    • 8.10. Further Reading
  • 9. Model Deployment
    • 9.1. Overview
    • 9.2. Conversion to Inference Model and Model Optimization
    • 9.3. Model Compression
    • 9.4. Advanced Efficient Techniques
    • 9.5. Model Inference
    • 9.6. Security Protection of Models
    • 9.7. Chapter Summary
    • 9.8. Further Reading
  • 10. Part II Application Scenarios
  • 11. Recommender System
    • 11.1. Overview
    • 11.2. System Components
    • 11.3. Recommendation Pipeline
    • 11.4. Model Update
    • 11.5. Supporting Real-time Machine Learning
    • 11.6. Chapter Summary
    • 11.7. Further Reading
  • 12. Reinforcement Learning System
    • 12.1. Overview
    • 12.2. Introduction to Reinforcement Learning
    • 12.3. Single-Node Reinforcement Learning System
    • 12.4. Distributed Reinforcement Learning System
    • 12.5. Multi-agent Reinforcement Learning
    • 12.6. Multi-agent Reinforcement Learning System
    • 12.7. Chapter Summary
  • 13. Robotic System
    • 13.1. Overview of Robotic Systems
    • 13.2. Robot Operating System
    • 13.3. Case Study: Using ROS
    • 13.4. Modern Robot Learning
    • 13.5. Chapter Summary

12. Reinforcement Learning System¶

Reinforcement learning (RL) has emerged as a subfield of machine learning that focuses on how autonomous agents can learn to make optimal decisions through interactions with their environments. By designating rewards or penalties for each action taken, RL provides a framework for training intelligent systems to maximize cumulative reward over time.

  • 12.1. Overview
  • 12.2. Introduction to Reinforcement Learning
    • 12.2.1. Background
    • 12.2.2. Reinforcement Learning Components
    • 12.2.3. Markov Decision Process
  • 12.3. Single-Node Reinforcement Learning System
    • 12.3.1. RL System
    • 12.3.2. RLzoo
    • 12.3.3. Other Systems
  • 12.4. Distributed Reinforcement Learning System
    • 12.4.1. Distributed RL Algorithm–A3C
    • 12.4.2. Distributed RL Algorithm–IMPALA
    • 12.4.3. Other Algorithms
    • 12.4.4. Distributed RL System – RLlib
    • 12.4.5. Distributed RL System–Reverb and Acme
  • 12.5. Multi-agent Reinforcement Learning
    • 12.5.1. Multi-agent RL
    • 12.5.2. Game Example
    • 12.5.3. Self-play
  • 12.6. Multi-agent Reinforcement Learning System
    • 12.6.1. Curse of Multi-agent
    • 12.6.2. Complex Game Types
    • 12.6.3. Algorithm Heterogeneity
    • 12.6.4. Hybrid Methods
  • 12.7. Chapter Summary
Previous
11.7. Further Reading
Next
12.1. Overview