| layout | title |
|---|---|
page |
Stanford ME 343 / CME 216 homepage |
Machine Learning for Computational Engineering
These is the web site for ME 343/CME 216 Machine Learning in Computational Engineering. This material was created by Eric Darve, with the help of course staff and students.
Extensions can be requested in advance for exceptional circumstances (e.g., travel, sickness, injury, COVID related issues) and for OAE-approved accommodations.
Submissions after the deadline and late by at most 2 days (+48 hours after the deadline) will be accepted with a 10% penalty. No submissions will be accepted 2 days after the deadline.
See Gradescope for all the current assignments and their due dates. Post on Slack if you cannot access the Gradescope class page. We will send you the 6-letter code.
- [Python setup guide](Python Setup Guide)
- Google colab guide
- Introduction to Python notebook
- Numpy demo notebook
- Python inheritance demo notebook
- All the files including required text input files
Module 1-Part 1
- 1.1 Brief introduction to ML; Video; Slides
- 1.2 Examples of machine learning; Video; Slides
- 1.3 Supervised learning; Video; Slides
- 1.4 Machine learning in engineering; Video; Slides
- 1.5 Introduction to SVM; Video; Slides
- [Reading Assignment 1](Reading Assignments/intro_svm)
Module 1-Part 2
- [Python setup guide](Python Setup Guide)
- SVM Python notebook
- 1.6 scikit-learn; Video; Slides
- 1.7 Soft-margin; Video; Slides
- 1.8 Over-fitting; Video; Slides
- 1.9 Training and validation sets; Video; Slides
- 1.10 Kernel trick; Video; Slides
- [Reading Assignment 2](Reading Assignments/svm_kernel)
- [Homework 1](Homework/HW1/HW1 Questions)
Module 2
- TensorFlow Python notebook DNN regression demo
- 2.1 Perceptron; Video; Slides
- 2.2 MLP; Video; Slides
- 2.3 TensorFlow/Keras; Video; Slides
- 2.4 Sequential API; Video; Slides
- 2.5 Functional API; Video; Slides
- Subclassing Python notebook inheritance demo
- 2.6 Subclassing; Video; Slides
- [Reading Assignment 3](Reading Assignments/TensorFlow)
Module 3-Part 1
- Python notebook DNN regularization demo
- 3.1 Loss function; Video; Slides
- 3.2 Cross-entropy; Video; Slides
- 3.3 TensorFlow loss functions; Video; Slides
- 3.4 Backpropagation; Video; Slides
- 3.5 Backpropagation formula; Video; Slides
- 3.6 Learning rate for training; Video; Slides
- 3.7 Empirical method for learning rate; Video; Slides
- 3.8 Overfitting; Video; Slides
- 3.9 DNN initializers; Video; Slides
- 3.10 Regularization; Video; Slides
- [Reading Assignment 4](Reading Assignments/deep_learning)
- [Homework 2](Homework/HW2/HW2 Questions)
Module 3-Part 2
- Saddle point demo notebook
- ADAGRAD notebook
- 3.11 Stochastic Gradient Descent; Video; Slides
- 3.12 Saddle points; Video; Slides
- 3.13 Momentum; Video; Slides
- 3.14 Adagrad; Video; Slides
- 3.15 RMSProp and Adam; Video; Slides
- [Reading Assignment 5](Reading Assignments/optimization)
- [Homework 3](Homework/HW3/HW3 Questions)
Module 3-Part 3
- 3.16 Trust region; Video; Slides
- 3.17 BFGS; Video; Slides
- 3.18 L-BFGS; Video; Slides
- [Reading Assignment 6](Reading Assignments/second_order_optimization)
- [Homework 4](Homework/HW4/HW4 Questions)
Module 4
- Automatic Differentiation in TensorFlow notebook
- Physics Informed Learning with TensorFlow notebook
- 4.1 Physics Informed Machine Learning (PIML); Video; Slides
- 4.2 TensorFlow Automatic Differentiation (TF_AD); Video; Slides
- 4.3 Physics Informed ML using TensorFlow (TF_PIML); Video; Slides
- [Reading Assignment 7](Reading Assignments/PIML)
- [Homework 5](Homework/HW5/HW5 Questions)
Module 5
Automatic differentiation, physics informed learning using ADCME
The slides are assembled into a single PDF file. Each lecture video covers one section in the PDF.
- Module 5, Automatic Differentiation for Computational Engineering; Videos; Slides
- [Reading Assignment 8](Reading Assignments/AD)
Module 6
Inverse modeling using ADCME
- Module 6, Inverse Modeling using ADCME; Videos; Slides
- [Reading Assignment 9](Reading Assignments/Inverse)
- [Homework 6](Homework/HW6/HW6 Questions)
- Deep learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville
- Deep learning with Python by François Chollet
- Hands-on machine learning with Scikit-Learn, Keras, and TensorFlow : concepts, tools, and techniques to build intelligent systems by Aurélien Géron
- Numerical optimization by Jorge Nocedal and Stephen Wright
- Fundamentals of deep learning : designing next-generation machine intelligence algorithms by Nikhil Buduma
- Elements of statistical learning by Trevor Hastie, Robert Tibshirani, and Jerome Friedman
- Deep learning: an introduction for applied mathematicians by Catherine Higham and Desmond Higham
- Machine learning: a probabilistic perspective by Kevin Murphy (in searchworks)
- Deep learning illustrated: a visual, interactive guide to artificial intelligence by Jon Krohn
- Neural networks and deep learning by Michael Nielsen
- Foundations of machine learning by Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar
- Neural networks and learning machines by Simon Haykin
- The matrix cookbook by Kaare Petersen and Michael Pedersen
- Introduction to deep learning: concepts and fundamentals by Laura Graesser
- Introduction to deep learning models with TensorFlow: learn how to work with TensorFlow to create and run a TensorFlow graph, and build a deep learning model by Lucas Adams
- Deep learning with TensorFlow: applications of deep neural networks to machine learning tasks by Jon Krohn
- LeCun, Bengio and Hinton, Deep learning, Nature, 521:436-444, 2015
- Schmidhuber, Deep learning in neural networks: an overview, Neural Networks, 61:85-117, 2015
- Automatic differentiation in machine learning: a survey by Atılım Günes Baydin, Barak Pearlmutter, Alexey Andreyevich Radul, and Jeffrey Mark Siskind
- A review of the adjoint-state method for computing the gradient of a functional with geophysical applications by R.-E. Plessix
- Introduction to Deep Learning, MIT
- fast.ai
- Machine Learning 2014-2015, Oxford, by Nando de Freitas
- ADCME (wiki page) developed by Kailai Xu and Prof. Darve
- DeepXDE developed by Lu Lu and Prof. Karniadakis
- List of books and tutorials on ML
- Online courses
- TensorFlow notebooks by Jon Krohn
- TF2 notebooks by Jon Krohn
- Deep learning illustrated notebooks by Jon Krohn
- TensorFlow playground
- MNIST visualization by Adam Harley
- Distill, a journal for machine learning visualizations