PyTorch - bobbae/gcp GitHub Wiki
PyTorch is an optimized tensor library for deep learning using GPUs and CPUs. It based on Torch library for applications such as computer vision and natural language processing, primarily developed by Facebook's AI Research lab (FAIR).
A number of pieces of deep learning software are built on top of PyTorch, including Tesla Autopilot, Uber's Pyro, HuggingFace's Transformers, PyTorch Lightning, and Catalyst.
PyTorch provides two high-level features:
- Tensor computing (like NumPy) with strong acceleration via graphics processing units (GPU)
- Deep neural networks built on a type-based automatic differentiation system
PyTorch defines a class called Tensor (torch.Tensor) to store and operate on homogeneous multidimensional rectangular arrays of numbers. PyTorch Tensors are similar to NumPy Arrays, but can also be operated on a CUDA-capable Nvidia GPU. PyTorch supports various sub-types of Tensors.
PyTorch on Google Cloud
https://cloud.google.com/blog/topics/developers-practitioners/pytorch-google-cloud-blog-series-recap
PyTorch on Vertex AI
https://medium.com/nlplanet/training-a-pytorch-model-on-gcp-vertex-ai-ed20df97ce14
PyTorch vs TensorFlow
TensorFlow is developed by Google Brain and actively used at Google both for research and production needs. Its closed-source predecessor is called DistBelief.
PyTorch is a cousin of lua-based Torch framework which was developed and used at Facebook. However, PyTorch is not a simple set of wrappers to support popular language, it was rewritten and tailored to be fast and feel native.
https://towardsdatascience.com/pytorch-vs-tensorflow-spotting-the-difference-25c75777377b
PyTorch on GCP Vertex AI
PyTorch/XLA
Cloud TPU VMs
https://cloud.google.com/blog/products/compute/introducing-cloud-tpu-vms
JAX
https://www.assemblyai.com/blog/why-you-should-or-shouldnt-be-using-jax-in-2022/
JAX vs Tensorflow vs PyTorch
https://analyticsindiamag.com/jax-vs-tensorflow-vs-pytorch-a-comparative-analysis/
JAX vs PyTorch: Automatic Differentiation for XGBoost
https://towardsdatascience.com/jax-vs-pytorch-automatic-differentiation-for-xgboost-10222e1404ec
JAX vs Julia
https://kidger.site/thoughts/jax-vs-julia/
PyTorch examples
https://github.com/pytorch/examples
PyTorch Video course
https://www.youtube.com/watch?v=GIsg-ZUy0MY