Further Reading =============== 1. A Survey on Distributed Machine Learning [1]_ 2. Horovod: fast and easy distributed deep learning in TensorFlow [2]_ 3. GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism [3]_ 4. Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour [4]_ .. [1] https://dl.acm.org/doi/abs/10.1145/3377454 .. [2] https://arxiv.org/abs/1802.05799 .. [3] https://arxiv.org/abs/1811.06965 .. [4] https://arxiv.org/abs/1706.02677