Part 1: Introduction, Theoretical Foundations, and AI Definitions<\/strong><\/h3>\n\n\n\n1. Professor\u2019s Background and Research Focus<\/strong><\/p>\n\n\n\n\n- Academic Roots:<\/strong> The lecturer holds a PhD from the University of Hong Kong (HKU) and has spent six years teaching at his current institution.<\/li>\n\n\n\n
- Theoretical Computer Science:<\/strong> His primary research is in \"Theoretical CS,\" which focuses on the mathematical modeling of computer science problems, designing and analyzing algorithms, and proving their mathematical correctness.<\/li>\n\n\n\n
- Algorithmic Game Theory:<\/strong> He specializes in the intersection of Game Theory and CS. He notes that modern AI milestones\u2014such as Generative Adversarial Networks (GANs) and Large Language Model (LLM) training\u2014borrow heavily from Game Theory concepts.<\/li>\n\n\n\n
- Goal of the Lesson:<\/strong> To move beyond the \"how-to\" of AI and explore the \"why\"<\/strong>\u2014the theory behind neural networks, parameter scaling, and predictive logic.<\/li>\n<\/ul>\n\n\n\n
2. Defining Artificial Intelligence (AI)<\/strong><\/p>\n\n\n\n\n- Core Capabilities:<\/strong> AI is defined by three abilities: Perception<\/strong> (obtaining data), Synthesis<\/strong> (processing existing data), and Inference<\/strong> (using data to generate new information).<\/li>\n\n\n\n
- Inference as the \"Gold Standard\":<\/strong> The lecturer argues that while search engines (Google\/Baidu) can provide known facts, true AI (Generative AI) creates new content\u2014writing code, generating images, and producing video.<\/li>\n\n\n\n
- Machine vs. Human Intelligence:<\/strong> AI is specifically \"Machine Intelligence,\" distinct from human or biological animal intelligence.<\/li>\n<\/ul>\n\n\n\n
3. The Shift in Research Focus<\/strong><\/p>\n\n\n\n\n- Decline of Speech Recognition:<\/strong> Previously a core AI field, speech recognition is now considered a \"basic\" and \"simple\" application. It is integrated into everything from WeChat to TV remotes and is effectively \"solved.\"<\/li>\n\n\n\n
- The Rise of Computer Vision (CV):<\/strong> CV is currently the most popular research area. The lecturer notes that submissions to major conferences (like ICCV) double every few years, with a massive majority of research coming from China. CV is the backbone of robotics, autonomous driving, and facial recognition.<\/li>\n<\/ul>\n\n\n\n
4. Early Historical Milestones<\/strong><\/p>\n\n\n\n\n- Alan Turing (1950s):<\/strong> Proposed the Turing Test<\/strong> to determine if a machine\u2019s intelligence is indistinguishable from a human's. It was a \"text-only\" test, focusing on intellectual similarity rather than physical appearance.<\/li>\n\n\n\n
- Isaac Asimov\u2019s Three Laws of Robotics:<\/strong>\n
\n- Protect:<\/strong> A robot may not injure a human or allow a human to come to harm through inaction.<\/li>\n\n\n\n
- Obey:<\/strong> A robot must obey human orders, unless they conflict with the First Law.<\/li>\n\n\n\n
- Survive:<\/strong> A robot must protect its own existence, unless it conflicts with the first two laws.<\/li>\n<\/ol>\n<\/li>\n\n\n\n
- Modern Relevance (AI Safety):<\/strong> The professor reintroduced these laws into his curriculum in 2026 because of growing concerns about AI Safety<\/strong>. He warns that as AI reaches human-level intelligence, it might lie or manipulate its output to optimize its \"objective function\" (its programmed goals), potentially breaking human-imposed constraints.<\/li>\n<\/ul>\n\n\n\n
\n\n\n\nThis is the end of Part 1. I have covered the academic background, definitions, and the early history of AI. There is still much to cover regarding the \"AI Boom,\" DeepMind\u2019s achievements, Neural Network theory, and Robotics.<\/strong><\/p>\n\n\n\nPart 2: The \"AI Boom,\" Strategic Milestones, and the Rise of Alpha Systems<\/strong><\/h3>\n\n\n\n1. The \"AI Boom\" and the Hardware Revolution<\/strong><\/p>\n\n\n\n\n- The Hardware Bottleneck:<\/strong> Most AI concepts (Neural Networks, NLP, Autonomous Driving) were conceived in the 1950s and 60s. However, they failed to show power because computing power and storage were severely limited.<\/li>\n\n\n\n
- Scale Matters:<\/strong> The lecturer notes that in the 1980s, even if you gathered every hard drive in the world, you couldn't store a single modern Large Language Model (LLM) because the parameters are too vast.<\/li>\n\n\n\n
- Rapid Development (2000\u20132020):<\/strong> This period is defined as the \"AI Boom\" due to exponential growth in hardware (GPUs and memory), allowing researchers to finally implement old theories at scale.<\/li>\n<\/ul>\n\n\n\n
2. Key Milestones in Competitive Games<\/strong><\/p>\n\n\n\n\n- IBM Deep Blue (1997):<\/strong> The first time a machine defeated a world chess champion (Garry Kasparov). This was a major milestone because chess was considered a benchmark for human intelligence.<\/li>\n\n\n\n
- The Inspiration of Demis Hassabis:<\/strong> Hassabis, a child chess prodigy (ranked world #2 in his age group), was deeply inspired by Deep Blue. He eventually founded DeepMind (later acquired by Google) with the goal of solving intelligence itself.<\/li>\n\n\n\n
- AlphaGo (2016):<\/strong> A massive breakthrough because Go is significantly more complex than chess.\n
\n- Action Space:<\/strong> Go has a 19 \\times 19 board with roughly 360 possible moves per turn, creating a search space far too large for \"brute force\" calculation.<\/li>\n\n\n\n
- Strategic Depth:<\/strong> Unlike chess, where pieces have fixed movements, Go requires high-level pattern recognition and intuition.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\n
3. From Playing Games to Solving Science: AlphaZero and AlphaFold<\/strong><\/p>\n\n\n\n\n- AlphaZero and Zero-Knowledge Training:<\/strong> In 2017, DeepMind developed AlphaZero, which didn't learn from human games. Instead, it used \"Zero-Knowledge Training\"<\/strong>\u2014playing against itself millions of times per second.<\/li>\n\n\n\n
- Game Theory & Nash Equilibrium:<\/strong> AlphaZero\u2019s self-play mechanism is rooted in Nash Equilibrium<\/strong>. By constantly finding the best strategy to beat its previous version, the system iterates toward an optimal state.<\/li>\n\n\n\n
- AlphaFold & The 2024 Nobel Prize:<\/strong>\n
\n- DeepMind shifted focus from games to \"meaningful\" science. AlphaFold<\/strong> was designed to predict the 3D folding structure of proteins from a 1D amino acid sequence.<\/li>\n\n\n\n
- Impact:<\/strong> Solving the \"protein folding problem\" is crucial for drug discovery and medical research.<\/li>\n\n\n\n
- The Nobel Prize:<\/strong> In 2024, the Nobel Prize in Chemistry was awarded to Hassabis and his colleagues for AlphaFold, marking 2024 as the \"Nobel Year for AI.\"<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\n
4. The History of Modern AI Giants<\/strong><\/p>\n\n\n\n\n- OpenAI (Founded 2015):<\/strong> Originally founded as a non-profit by Elon Musk and other billionaires with a \\$10 billion pool (intended to serve all of humanity). However, it later transitioned into a for-profit company, leading to a fallout between Musk and OpenAI.<\/li>\n\n\n\n
- Apple & Siri (2011):<\/strong> Siri was the first major AI assistant. However, the lecturer notes that Siri has struggled to keep pace. By 2025\/2026, Apple changed strategy, deciding to power Siri\u2019s core with Google\u2019s Gemini model rather than trying to develop their own LLM alone.<\/li>\n\n\n\n
- DeepMind:<\/strong> Started as a small startup by Hassabis, it was acquired by Google and became the engine for Google's AI (Gemini, AlphaFold, etc.).<\/li>\n<\/ul>\n\n\n\n
\n\n\n\nThis concludes Part 2. We have moved from the early history into the \"Alpha\" era and the scientific breakthroughs of 2024.<\/strong><\/p>\n\n\n\nPart 3: The Evolution of LLMs, the Mystery of Emergence, and the DeepSeek Phenomenon<\/strong><\/h3>\n\n\n\n1. The Trajectory of GPT (Generative Pre-trained Transformer)<\/strong><\/p>\n\n\n\n\n- GPT-1 to GPT-3 (2018\u20132020):<\/strong> OpenAI released these versions sequentially. GPT-1 was a proof of concept with 110 million parameters; GPT-3 exploded to 175 billion. Initially, these were not widely used by the public.<\/li>\n\n\n\n
- ChatGPT (2022):<\/strong> Based on GPT-3.5, this was the \"tipping point.\" Its breakthrough was Contextual Memory<\/strong>. Unlike Siri, which treats every question as a new event, ChatGPT can remember 100+ lines of conversation history and build a \"profile\" of the user\u2019s intent.<\/li>\n\n\n\n
- The Logic of Prediction:<\/strong> The professor explains that the core mechanism of an LLM is simple: Predict the next word.<\/strong> It is \"prediction based on prediction.\" While this sounds fragile (a small error should snowball into a big error), it works surprisingly well at scale.<\/li>\n<\/ul>\n\n\n\n
2. The Concept of \"Intelligence Emergence\" (\u667a\u80fd\u6d8c\u73b0)<\/strong><\/p>\n\n\n\n\n- Non-linear Growth:<\/strong> Intelligence does not increase steadily as models get bigger. Instead, researchers observed that as model size increases, intelligence remains flat for a long time, then suddenly \"emerges\" or jumps exponentially once a certain scale is hit.<\/li>\n\n\n\n
- Explainable AI (XAI):<\/strong> Why this jump happens is currently one of the biggest mysteries in the field. No one truly knows the theoretical reason why scaling parameters leads to sudden \"reasoning\" abilities.<\/li>\n<\/ul>\n\n\n\n
3. Parameters vs. Training Data<\/strong><\/p>\n\n\n\n\n- The Brain Metaphor:<\/strong>\n
\n- Parameters:<\/strong> Represent \"Brain Capacity\" (potential). More parameters equal higher expressive power (2^{n} relation).<\/li>\n\n\n\n
- Training Data:<\/strong> Represent \"Education.\" A large brain is useless without a vast library of knowledge to learn from.<\/li>\n<\/ul>\n<\/li>\n\n\n\n
- Scaling Together:<\/strong> You cannot increase one without the other. A \"smart\" brain with no schooling is as ineffective as a \"normal\" brain trying to memorize the entire internet.<\/li>\n<\/ul>\n\n\n\n
4. The DeepSeek R1 Breakthrough<\/strong><\/p>\n\n\n\n\n- Efficiency over Brute Force:<\/strong> In late 2024\/early 2025, DeepSeek (China) surprised the world. While US companies (OpenAI\/Google) were \"stacking GPUs\" (thousands of NVIDIA A100\/H100 chips), DeepSeek achieved similar performance (beating GPT-4o in benchmarks) using a significantly smaller model size<\/strong> and fewer hardware resources.<\/li>\n\n\n\n
- Hardware Constraints:<\/strong> This was particularly important for China due to GPU export bans. DeepSeek proved that algorithmic innovation could compensate for a lack of raw hardware power.<\/li>\n<\/ul>\n\n\n\n
5. The \"Data Wall\" and Ethical Concerns<\/strong><\/p>\n\n\n\n\n- Exhausting Human Data:<\/strong> We are hitting a limit where humans have literally run out of high-quality text data to feed AI. Models like GPT-5 (released Aug 2025) are estimated to have used nearly all available digital text (32+ Terabytes of training data).<\/li>\n\n\n\n
- Privacy & Intellectual Property:<\/strong> To keep growing, AI companies are accused of \"scraping\" private data (social media, private chats) and copyrighted material (paid novels) without permission.<\/li>\n\n\n\n
- AI Hallucinations:<\/strong> Because AI is just \"predicting,\" it can \"hallucinate\" (lie) to make a sentence sound logical. GPT-5\u2019s primary goal was not higher scores, but Stability and Reliability<\/strong>\u2014reducing these hallucinations to make AI more professional.<\/li>\n<\/ul>\n\n\n\n
6. The Current State (2025\u20132026)<\/strong><\/p>\n\n\n\n\n- Gemini vs. GPT:<\/strong> Google\u2019s Gemini 3.0 is currently competing head-to-head with GPT-5. The professor notes that while GPT is \"strict and objective,\" Gemini is often perceived as \"gentle and cooperative.\"<\/li>\n\n\n\n
- The Breakout Year:<\/strong> 2026 is seen as the year AI matures, moving from benchmarks to high-stability industrial applications.<\/li>\n<\/ul>\n\n\n\n
\n\n\n\nThis ends Part 3. I have covered the technical and historical aspects of LLMs and the current market competition. The next part will focus on Neural Network Theory (Mathematics) and the specific applications like Handwriting Recognition and Robotics.<\/strong><\/p>\n\n\n\nPart 4: Neural Network Theory, Training Mechanics, and Practical Applications<\/strong><\/h3>\n\n\n\n1. The Mathematical Structure of Neural Networks (NN)<\/strong><\/p>\n\n\n\n\n- Biological Inspiration:<\/strong> The Artificial Neural Network (ANN) mimics the human brain's network of neurons, which communicate through chemical and electrical signals to make decisions (e.g., your eyes seeing bright light and signaling your brain to close your eyelids).<\/li>\n\n\n\n
- Basic Components:<\/strong>\n
\n- Neurons:<\/strong> The processing cells.<\/li>\n\n\n\n
- Layers:<\/strong> Input Layer, Hidden Layers (where the \"thinking\" happens), and the Output Layer.<\/li>\n\n\n\n
- Edges & Weights (w):<\/strong> Every connection between neurons has a weight that represents its influence. A positive weight means positive correlation; a negative weight means the opposite.<\/li>\n<\/ul>\n<\/li>\n\n\n\n
- The Power of Nonlinearity:<\/strong>\n
\n- A neuron is mathematically a function: ( y = f(x) ).<\/li>\n\n\n\n
- If the functions were only linear (like a simple weighted average), multiple layers would \"collapse\" into a single linear function, making the network no more powerful than a simple equation.<\/li>\n\n\n\n
- Activation Functions:<\/strong> By using nonlinear functions (like truncated or curved functions), the network can model extremely complex, non-straight-line relationships. This is what makes \"Deep\" learning powerful.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\n
2. The Training Process: Gradient Descent<\/strong><\/p>\n\n\n\n\n- Initialization:<\/strong> We start by giving every edge in the network a random, arbitrary weight. At this stage, the AI is \"guessing\" and will almost always be wrong.<\/li>\n\n\n\n
- Penalty\/Objective Function:<\/strong> When the AI makes an error (e.g., identifies a \"4\" as an \"8\"), the system calculates the \"distance\" between its guess and the correct label.<\/li>\n\n\n\n
- Gradient Descent:<\/strong> This is the core \"learning\" algorithm. It uses calculus to find the exact direction in which to adjust the weights to reduce the error as fast as possible.<\/li>\n\n\n\n
- Convergence:<\/strong> The process of inputting data and adjusting weights is repeated millions of times. Eventually, the weights stabilize, and the AI consistently gives the correct answer. At this point, the model has \"converged.\"<\/li>\n<\/ul>\n\n\n\n
3. Application: Computer Vision and Medical Diagnosis<\/strong><\/p>\n\n\n\n\n- Handwriting Recognition:<\/strong> An image is converted into a vector (e.g., a 28 \\times 28 pixel image becomes a 784-variable input). Each variable represents the brightness of a pixel.<\/li>\n\n\n\n
- Healthcare (Stomach Cancer & Parkinson\u2019s):<\/strong>\n
\n- AI is used to perform highly repetitive tasks, such as scanning thousands of gastroscopy images for signs of cancer.<\/li>\n\n\n\n
- Second-Layer Protection:<\/strong> AI can filter out \"low-risk\" cases, allowing doctors to focus their time and expertise only on \"high-risk\" cases identified by the model.<\/li>\n\n\n\n
- Early Detection:<\/strong> Modern research uses AI to analyze voice patterns or walking gaits to detect early-stage Parkinson\u2019s or Alzheimer\u2019s, which are difficult for human doctors to catch early.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\n
4. Autonomous Driving and Robustness<\/strong><\/p>\n\n\n\n\n- The \"Noise\" Problem:<\/strong> The professor showed a research example where a \"Stop Sign\" was slightly modified with digital \"noise\" invisible to the human eye.<\/li>\n\n\n\n
- Adversarial Attacks:<\/strong> While humans still see a Stop Sign, the AI might misidentify it as something else. This highlights the issue of Robustness<\/strong>\u2014ensuring that AI isn't easily tricked by small changes in environment or weather.<\/li>\n\n\n\n
- University of Macau (UM) Research:<\/strong> The UM lab focuses on \"Internet of Smart Cities\" and \"Smart Buses,\" utilizing technologies like Transfer Learning<\/strong> to help AI make correct decisions during \"small probability events\" (rare, extreme scenarios).<\/li>\n<\/ul>\n\n\n\n
5. The Future of Intelligent Robotics<\/strong><\/p>\n\n\n\n\n- Mechanical vs. Intelligent:<\/strong> Traditional robots follow fixed programs (weld this, turn that). Modern intelligent robots (Humanoids) use \"Large Models\" as their brain to perceive objects and make autonomous decisions.<\/li>\n\n\n\n
- The Three Pillars of Robotics Success:<\/strong>\n
\n- Industrial Foundation:<\/strong> Excellence in hardware and mechanical engineering (e.g., Tesla\u2019s rocket\/car tech).<\/li>\n\n\n\n
- The \"Brain\":<\/strong> Powerful LLMs for reasoning (e.g., DeepSeek, Gemini, GPT).<\/li>\n\n\n\n
- Market Demand:<\/strong> A clear need for automation in labor-intensive sectors.<\/li>\n<\/ol>\n<\/li>\n\n\n\n
- Key Players:<\/strong>\n
\n- Boston Dynamics:<\/strong> The historical leader in robot agility.<\/li>\n\n\n\n
- Tesla (Optimus):<\/strong> Leveraging massive automotive supply chains.<\/li>\n\n\n\n
- Unitree (\u5b87\u6811\u79d1\u6280):<\/strong> A leading Chinese company showing global competitiveness due to strong government support and a huge domestic market.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\n
\n\n\n\nThis concludes the detailed summary of the Day 2 lesson.<\/strong><\/p>\n\n\n\nWe have covered:<\/p>\n\n\n\n
\n- Theory:<\/strong> Game Theory and AI definitions.<\/li>\n\n\n\n
- History:<\/strong> Milestones from Turing to the 2024 Nobel Prizes.<\/li>\n\n\n\n