Category: Blog

  • What is autograd? – Automatic differentiation and optimization with PyTorch

    Welcome file Automatically calculating a gradient The gradient descent method Conclusion In recent years, there has been a marked increase in media and academic attention towards advancements in artificial intelligence, particularly concerning deep neural networks. The most pronounced surge in attention was observed following the release of ChatGPT last November. Although the usage of these…

  • Optimization Techniques in Scientific Computing (Part III)

    Optimization Techniques in Scientific Computing (Part III) Optimization Techniques in Scientific Computing (Part III) Introduction and recap Run code on a GPU First attempt Further vectorization Batch matrix multiplication Final boost Conclusion Introduction and recap In my previous two blogs for optimization techniques in scientific computing, I have talked about concepts such as vectorization and…

  • Optimization Techniques in Scientific Computing (Part II)

    Optimization Techniques in Scientific Computing (Part II) Optimization Techniques in Scientific Computing (Part II) Introduction and recap The first implementation Optimization ideas Parallelism Core-level Node-level Cluster-level Key consideration Introduction and recap In my previous blog, I introduced some straightforward yet valuable optimization techniques. While these techniques are generally suitable for relatively simple problems, such as…