EINNET - Optimizing Tensor Programs with Derivation-Based Transformations

Overview

Optimizing tensor programs for deep neural networks using derivation-based transformations, outperforming existing optimizers by leveraging a larger search space and automatically creating new operators.

Syllabus

• Introduction to Tensor Programs
Overview of Tensor Operations and Structures
Importance in Deep Neural Networks
• Basics of Program Optimization
Traditional Optimization Techniques
Limitations of Current Optimizers
• Derivation-Based Transformations
Concept and Theory
Examples of Derivations in Tensor Programs
• Search Space Exploration
Benefits of a Larger Search Space
Techniques for Effective Exploration
• Automatic Operator Creation
Understanding Novel Operator Generation
Implementing Automatic Creation in Tensor Programs
• Practical Examples and Case Studies
Transformations in Real-World Neural Networks
Comparative Analysis with Existing Optimizers
• Tools and Frameworks
Overview of Tools for Optimization
Hands-on Practice with Optimization Software
• Optimization Challenges and Solutions
Common Obstacles
Innovative Approaches to Problem Solving
• Future Directions in Tensor Program Optimization
Emerging Trends and Technologies
Research Opportunities
• Project and Evaluation
Designing a Tensor Program with Derivation-Based Transformations
Performance Evaluation and Metrics
• Conclusion and Course Wrap-Up
Summary of Key Concepts
Next Steps for Learners in Tensor Optimization Technologies

---

Subjects

Conference Talks