Data parallelism

Data parallelism involves running models in parallel on different devices where each sees a distinct subset of data during training or inference. It accelerates training deep learning models, reducing training time significantly [1].

It is often confused with task parallelism, but both can be applied in tandem make the most of available resources, reduce training and deployment times, and optimize the end-to-end machine learning process for better results and faster development. See task parallelism.