来源：Deep Neural Nets: 33 years ago and 33 years from now

---

中文摘要 #

本文回顾了LeCun等人1989年发表的手写邮编识别论文,这是最早的端到端反向传播神经网络实际应用之一。作者使用PyTorch重现了该论文的实验,并对比了当时与现代深度学习的差异。通过引入现代技术如Adam优化器、数据增强、Dropout等,成功将测试错误率降低了60%。文章还探讨了深度学习33年来的发展趋势:虽然基本原理相似,但数据集和模型规模已增长约1亿倍,训练速度提升约3000倍。展望未来33年,作者预测基础模型将成为主流,大多数应用将通过微调或提示工程实现,而非从头训练新模型。这反映了AI领域从小规模专用模型向大规模通用模型演进的趋势。

**关键词：**深度学习、神经网络、模型优化、基础模型、技术演进

English Summary #

Deep Neural Nets: 33 years ago and 33 years from now

This article examines LeCun et al.’s 1989 paper on handwritten zip code recognition, one of the earliest real-world applications of end-to-end neural networks trained with backpropagation. The author reproduces the experiment using PyTorch and compares historical and modern deep learning approaches. By incorporating modern techniques like Adam optimizer, data augmentation, and Dropout, the test error rate was reduced by 60%. The paper discusses deep learning’s evolution over 33 years: while fundamental principles remain similar, dataset and model sizes have grown approximately 100 million-fold, with training speeds increasing about 3000-fold. Looking ahead 33 years, the author predicts foundation models will dominate, with most applications achieved through fine-tuning or prompt engineering rather than training new models from scratch. This reflects AI’s evolution from small, specialized models to large, general-purpose ones.

**Keywords: **deep learning, neural networks, model optimization, foundation models, technological evolution