UDL with Julia/Pluto

Contact

Prof. Dr. Claus Möbus

Room: A02 2-226

orcid.org/0000-0003-1640-4168

claus.moebus@uol.de 

-------------------------------------------

Secretary

Manuela Wüstefeld

Room: A02 2-228

Tel: +49 441 / 798-4520

manuela.wuestefeld@uol.de 

-------------------------------------------

UDL with Julia/Pluto

Understanding Deep Learning (UDL) with Julia/Pluto.jl

These Julia/Pluto scripts are my personal learning diary when working through Simon J.D. Prince's book Understanding Deep Learning (UDL). The book is accompanied by skeleton Python scripts to be completed by students to test their acquired knowledge. In contrast, I am developing full Julia/Pluto scripts in a style called literate programming to implement the book's neural network’s’ concepts. Though the scripts are carefully tested they are not production-ready.

I have two main goals. First, I try to improve my personal competence in Julia and second I am testing the quality und utility of Julia’s ecosystem for building neural net models.

I use the following libraries: ADTypes, Combinatorics, ComponentArrays, CUDA, cuDNN, DataFrames, Distributions, Flux, GLM, Latexify, LaTeXStrings, LogExpFunctions, Logistics, Lux,  LuxCUDA, NNlib, Optim, Optimisers,  Plots, Pluto, PlutoUI, Printf, ProgressMeter, Random, SpecialFunctions, Statistics, Symbolics, Zygote.

Here I put special emphasis on LUX.jl and LuxCUDA.jl.

Why Julia and no other languge? Having some background in Fortran, Lisp, Scheme, Prolog, and WebPPL Julia is more appealing to me than other languages. As others say it is a modern, elegant, and powerful language particularly suitabable for scientists (e.g. cognitive scientists) who deal with AI.

                                                                                                                               Claus Möbus

  1. Intro
    1. Julia/Pluto-UDL-Notebook_1.1: Background Mathematics
    2. t-Test as Linear Regression with LUX.jl
    3. Multivariate Linear Regression with LUX.jl
  2. Supervised Learning
    1. Julia/Pluto-UDL-Notebook 2.1
    2. Julia/Pluto-Notebook 2.1 with FLUX.jl
    3. Julia/Pluto-Notebook 2.1 with LUX.jl
  3. Shallow Neural Networks
    1. Shallow Neural Networks I
      1. Julia/Pluto-UDL-Notebook 3.1
      2. Julia/Pluto-Notebook 3.1 with FLUX.jl
      3. Julia/Pluto-Notebook 3.1 with LUX.jl
    2. Shallow Neural Networks II
      1. Julia/Pluto-UDL-Notebook 3.2
      2. Julia/Pluto-Notebook 3.2 with FLUX.jl
      3. Julia/Pluto-Notebook 3.2 with LUX.jl
    3. Shallow Network Regions: Julia/Pluto-UDL-Notebook 3.3
    4. Activation functionst3://page?uid=100426
      1. Julia/Pluto-UDL-Notebook 3.4
      2. Julia/Pluto-UDL-Notebook 3.4 with FLUX.jl
      3. Julia/Pluto-UDL-Notebook 3.4 with LUX.jl
  4. Deep Neural Networks
    1. Composing Networks
      1. Composing I
      2. Composing II
    2. Clipping Functions
    3. Deep Networks
  5. Loss Functions
    1. Identification of a Latent Generator
    2. Maximum Likelihood Estimation (MLE)
    3. MLE & Least Squares Loss
    4. Binary Cross-Entropy Loss
      1. Binary Cross-Entropy Loss I
      2. Binary Cross-Entropy Loss II
    5. Multiclass Cross-Entropy Loss
      1. Multiclass Cross-Entropy Loss I
      2. Multiclass Cross-Entropy Loss II
  6. Fitting Models
    1. (Gradient-Free) Line Search
    2. Gradient Descent
      1. Gradient Descent I
      2. Gradient Descent II
      3. Gradient Descent III
    3. Stochastic Gradient Descent
      1. Stochastic Gradient Descent I
      2. Stochastic Gradient Descent II
      3. Stochastic Gradient Descent III
      4. Stochastic Gradient Descent IV
    4. Momentum
    5. Adam

 

-------------------------------------------------------------------------------------------------------------

This is a draft under the Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0) license. Comments, improvement and issue reports are welcome: claus.moebus(@)uol.de

-------------------------------------------------------------------------------------------------------------

(Changed: 15 Nov 2024)  | 
Zum Seitananfang scrollen Scroll to the top of the page