InternEvo
Environment Installation
Quickstart Guide
Training Initialization
Script of Starting Training
Parallel Mode and Principle
Data Load and Procedure
Mixed Precision Training
Mixture-of-Experts
Model Checkpointing
Performance Analysis Tool
Monitor and Alert
Training Example Introduction
Q&A
InternEvo
InternEvo
Edit on GitHub
InternEvo
Environment Setup
Environment Installation
Environment Preparation
Installation through pip
Installation through Source Code
Environment Image
NPU Environment Installation
Quickstart Guide
Quickstart Guide
Installation
Data Preparation
Training Configuration
Start Training
Training Results
Load the training checkpoint and generate.
Long Text Generation
Model Setup
Training Initialization
Argument Parsing
Model Initialization
Dataloader Initialization
Parallel Communication Initialization
Optimizer Initialization
Trainer Initialization
Script of Starting Training
Configuration Parameter Parsing
Initialization process
Start Training Process
Parallel Training
Parallel Mode and Principle
Tensor Parallel
Pipeline Parallel
Data Parallel
ZeRO1.5
2D-Attention
Data Format
Data Load and Procedure
Daterloader Loading Data
Achieve Data From Dataloader
During the Forward process, the data format is:
Mixed Precision
Mixed Precision Training
Implementation Instructions
TF32 Training
Mixture-of-Experts
Mixture-of-Experts
Parameter Settings
Model Training
Model Checkpointing
Model Checkpointing
CheckpointManager
Model loading and saving path format conventions.
Asynchronous upload.
Snapshot Checkpoint
Checkpoint automatic recovery
Manual control of checkpoint storage
Profiler
Performance Analysis Tool
Torch Profiler
Memory Profiler
Monitor
Monitor and Alert
Monitoring
Alerting
Light Monitoring
Example
Training Example Introduction
7B Demo
20B Demo
Q&A
Q&A
Indices and tables
Index
Module Index
Search Page