If I can train gpt2 in c, I can claim to kinda understand c.

This repository contains all the resources and code developed through a tailored curriculum designed in conversation with ChatGPT.

Curriculum

The curriculum is structured to cover both fundamental and advanced concepts necessary for mastering C, particularly in the context of implementing and training neural networks. Each chapter of the curriculum corresponds to a specific topic, with corresponding code examples and exercises stored in dedicated folders within this repository.

Chapters

1. C Programming Refresher

   • Topics Covered:
     • Syntax and Basic Constructs
     • Pointers and Memory Management
     • Structures and Unions
     • File Handling
     • Matrix Operations in C
     • Kinda Advance C Usage
   • Folder: 01_C_Refresher

2. Introduction to CUDA

   • Topics Covered:
     • Introduction to CUDA and GPU Computing
     • GPU-Rich Enviroment Setup
     • Basics of CUDA Programming
     • Memory Management in CUDA
     • Kernel Diving and Performance Optimization
     • Error Handling in CUDA
     • Advanced CUDA Features
   • Folder: 02_Intro_to_CUDA

3. Basics of Neural Networks

   • Topics Covered:
     • Neural Network Architecture
     • Forward Propagation
     • Backpropagation
     • Loss Functions
     • Common Activation Functions
   • Folder: 03_Neural_Networks_Basics

4. Deep Dive into Transformer Architecture

   • Topics Covered:
     • Understanding Transformers
     • Positional Encoding
     • Layer Normalization
     • Multi-head Attention
   • Folder: 04_Transformer_Architecture

Current Building Status

• Learning 01_C_Refresher
• Building 01_C_Refresher
• Learnign 02_Intro_to_CUDA