GE Architecture Documentation【免费下载链接】geGEGraph Engine是面向昇腾的图编译器和执行器提供了计算图优化、多流并行、内存复用和模型下沉等技术手段加速模型执行效率减少模型内存占用。 GE 提供对 PyTorch、TensorFlow 前端的友好接入能力并同时支持 onnx、pb 等主流模型格式的解析与编译。项目地址: https://gitcode.com/cann/geThis documentation set introduces GE (Graph Engine) architecture design from different dimensions,targeting developers who want to contribute code to GE, helping quickly understand the overall project structure, core design decisions, and implementation details of each module.Excluding tests, there are approximately790,000 lines of effective codewith over 8,000 source files:ModuleRatioResponsibilitycompiler34%Graph compilation (optimization, partitioning, building)runtime20%Graph execution (loading, execution, Hybrid)graph_metadef12%IR definition, operator registrationdflow11%Distributed flow frameworkbase7%Infrastructureinc6%Public headersapi4%API interfacesparser3%Model parsingArchitecture OverviewDocumentDescriptionGE Architecture IntroductionSystem architecture overview, AscendIR introduction, compilation optimization, plugin extension mechanismModule Architecture DocumentsDocumentDescriptionAscendIRDetailed design of AscendIR graph intermediate representationCompilerGE Compiler compilation flow, optimization passes, engine partitioning, build stagesRuntimeGE Executor model loading, Sink mode, Hybrid execution, v2 architectureFeature Design DocumentsThe following documents describe cross-module feature designs:DocumentDescriptionDump ModuleDump module overall design: architecture layering, RT1.0/RT2.0 adaptation, HCCL processing, dynamic switchExternal WeightFileConstant feature: weight separation from OM, compile-time Const→FileConstant conversion, RT V1/V2 loading flow, memory management, global weight managerConstant FoldingConstant folding optimization: compile-time constant expression evaluation, dimension calculation, empty tensor replacement, delayed effect mechanism, multi-stage compilation pipelineFusion Pattern PassFusion Pattern Pass mechanism: PatternFusionPass / DecomposePass matching, filtering, replacement, execution stages and Python/C integrationDynamic GearDynamic gear feature: dynamic Batch / dynamic resolution / ND arbitrary dimension modes, gear enumeration, static subgraph generation and runtime dispatchMemory Conflict HandlingMemory conflict protection system: semantic read-write conflict, memory layout conflict, subgraph address isolation, Inplace reuse conflict, multi-stream concurrency managementModel CacheCompilation result persistence mechanism: graph compilation cache, JIT compilation cache, operator model cache three-level system, cache hit and invalidation strategiesProfilingPerformance collection and observability: layered collection architecture (API/Host/Device), on-demand enablement, msprof unified reportingSO in OMOperator self-contained packaging: packaging dependent operator .so files into OM on demand, eliminating runtime dependency on OPP operator packagesTensorMove EliminationTensorMove redundant node elimination optimization: identify and delete redundant memory copy nodes, O3 optimization levelVariable ManagementVariable lifecycle management: registration, memory allocation, format conversion, logical address mapping, serialization/deserialization full flowZero CopyZero copy feature: input zero copy (eliminate H2D), output zero copy (eliminate D2H/D2D), compile-time planning and runtime executionConcat No TaskConcat continuous memory optimization: compile-time identification of continuous input Concat operators, mark as virtual operators to skip Task generation and memory movementGE Local OperatorGE Local engine: dedicated engine for non-computation nodes (Data, Constant, control flow, shape transformation, etc.), zero runtime computation overheadEngineEngine system: plugin-based engine architecture, priority-driven automatic selection, compile-time engine registration and partitioning, runtime dispatchTiling SinkTiling sink feature: move Tiling computation from Host to Device AICPU execution, eliminate Host-Device synchronization overheadGraph SplitterGraph split feature: static/dynamic Shape split, engine-level split, pipeline stage split, JIT incremental splitStatic ExecutorStatic subgraph executor: Task Sink pre-dispatch, DavinciModel loading/execution, hybrid execution mode address refreshDynamic ExecutorRT2.0 dynamic Shape executor: Lowering mechanism, ExecuteGraph, ModelV2Executor, three-subgraph lifecycle, Kernel registration systemStream AllocatorStream allocation feature: logical stream allocation, synchronization event management, physical stream split, stream activation mechanismInferShapeShape inference: OriginShape/StorageShape dual system, compile-time InferShapePass, runtime inference node, symbolic inferenceFormat InferenceFormat inference: OriginFormat anchor propagation, StorageFormat automatic selection, TransData insertion optimizationModule Key Design Principles and Software ConstraintsThe following documents record key design constraints and development standards for features:DocumentDescriptionMemory Module Software ConstraintsStatic/dynamic memory reuse, Allocator threading model, memory release timing, process exit cleanupRT2 Runtime Design PrinciplesRT2 dynamic Shape module design principles: loading/execution rules, performance, compatibility, concurrency, debuggabilityGraph Split Module Design PrinciplesGraph split module design principles: responsibility boundaries, split basis, multi-threading concurrency, debugging logs, compatibility, review checklistStream Allocator Design PrinciplesStatic/dynamic Shape stream allocation design: Pass architecture, stream reuse, Event synchronization, stream activation mechanismStatic Shape Runtime Design PrinciplesStatic Shape module design principles: performance optimization, ArgsFormat, address refresh strategy, memory managementGraph Foundation Structure Design PrinciplesGraph compilation common foundation structure design principles: independence, compatibility, observability, concurrency model, cross-platform consistency【免费下载链接】geGEGraph Engine是面向昇腾的图编译器和执行器提供了计算图优化、多流并行、内存复用和模型下沉等技术手段加速模型执行效率减少模型内存占用。 GE 提供对 PyTorch、TensorFlow 前端的友好接入能力并同时支持 onnx、pb 等主流模型格式的解析与编译。项目地址: https://gitcode.com/cann/ge创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考 CANN/GE架构设计文档
GE Architecture Documentation【免费下载链接】geGEGraph Engine是面向昇腾的图编译器和执行器提供了计算图优化、多流并行、内存复用和模型下沉等技术手段加速模型执行效率减少模型内存占用。 GE 提供对 PyTorch、TensorFlow 前端的友好接入能力并同时支持 onnx、pb 等主流模型格式的解析与编译。项目地址: https://gitcode.com/cann/geThis documentation set introduces GE (Graph Engine) architecture design from different dimensions,targeting developers who want to contribute code to GE, helping quickly understand the overall project structure, core design decisions, and implementation details of each module.Excluding tests, there are approximately790,000 lines of effective codewith over 8,000 source files:ModuleRatioResponsibilitycompiler34%Graph compilation (optimization, partitioning, building)runtime20%Graph execution (loading, execution, Hybrid)graph_metadef12%IR definition, operator registrationdflow11%Distributed flow frameworkbase7%Infrastructureinc6%Public headersapi4%API interfacesparser3%Model parsingArchitecture OverviewDocumentDescriptionGE Architecture IntroductionSystem architecture overview, AscendIR introduction, compilation optimization, plugin extension mechanismModule Architecture DocumentsDocumentDescriptionAscendIRDetailed design of AscendIR graph intermediate representationCompilerGE Compiler compilation flow, optimization passes, engine partitioning, build stagesRuntimeGE Executor model loading, Sink mode, Hybrid execution, v2 architectureFeature Design DocumentsThe following documents describe cross-module feature designs:DocumentDescriptionDump ModuleDump module overall design: architecture layering, RT1.0/RT2.0 adaptation, HCCL processing, dynamic switchExternal WeightFileConstant feature: weight separation from OM, compile-time Const→FileConstant conversion, RT V1/V2 loading flow, memory management, global weight managerConstant FoldingConstant folding optimization: compile-time constant expression evaluation, dimension calculation, empty tensor replacement, delayed effect mechanism, multi-stage compilation pipelineFusion Pattern PassFusion Pattern Pass mechanism: PatternFusionPass / DecomposePass matching, filtering, replacement, execution stages and Python/C integrationDynamic GearDynamic gear feature: dynamic Batch / dynamic resolution / ND arbitrary dimension modes, gear enumeration, static subgraph generation and runtime dispatchMemory Conflict HandlingMemory conflict protection system: semantic read-write conflict, memory layout conflict, subgraph address isolation, Inplace reuse conflict, multi-stream concurrency managementModel CacheCompilation result persistence mechanism: graph compilation cache, JIT compilation cache, operator model cache three-level system, cache hit and invalidation strategiesProfilingPerformance collection and observability: layered collection architecture (API/Host/Device), on-demand enablement, msprof unified reportingSO in OMOperator self-contained packaging: packaging dependent operator .so files into OM on demand, eliminating runtime dependency on OPP operator packagesTensorMove EliminationTensorMove redundant node elimination optimization: identify and delete redundant memory copy nodes, O3 optimization levelVariable ManagementVariable lifecycle management: registration, memory allocation, format conversion, logical address mapping, serialization/deserialization full flowZero CopyZero copy feature: input zero copy (eliminate H2D), output zero copy (eliminate D2H/D2D), compile-time planning and runtime executionConcat No TaskConcat continuous memory optimization: compile-time identification of continuous input Concat operators, mark as virtual operators to skip Task generation and memory movementGE Local OperatorGE Local engine: dedicated engine for non-computation nodes (Data, Constant, control flow, shape transformation, etc.), zero runtime computation overheadEngineEngine system: plugin-based engine architecture, priority-driven automatic selection, compile-time engine registration and partitioning, runtime dispatchTiling SinkTiling sink feature: move Tiling computation from Host to Device AICPU execution, eliminate Host-Device synchronization overheadGraph SplitterGraph split feature: static/dynamic Shape split, engine-level split, pipeline stage split, JIT incremental splitStatic ExecutorStatic subgraph executor: Task Sink pre-dispatch, DavinciModel loading/execution, hybrid execution mode address refreshDynamic ExecutorRT2.0 dynamic Shape executor: Lowering mechanism, ExecuteGraph, ModelV2Executor, three-subgraph lifecycle, Kernel registration systemStream AllocatorStream allocation feature: logical stream allocation, synchronization event management, physical stream split, stream activation mechanismInferShapeShape inference: OriginShape/StorageShape dual system, compile-time InferShapePass, runtime inference node, symbolic inferenceFormat InferenceFormat inference: OriginFormat anchor propagation, StorageFormat automatic selection, TransData insertion optimizationModule Key Design Principles and Software ConstraintsThe following documents record key design constraints and development standards for features:DocumentDescriptionMemory Module Software ConstraintsStatic/dynamic memory reuse, Allocator threading model, memory release timing, process exit cleanupRT2 Runtime Design PrinciplesRT2 dynamic Shape module design principles: loading/execution rules, performance, compatibility, concurrency, debuggabilityGraph Split Module Design PrinciplesGraph split module design principles: responsibility boundaries, split basis, multi-threading concurrency, debugging logs, compatibility, review checklistStream Allocator Design PrinciplesStatic/dynamic Shape stream allocation design: Pass architecture, stream reuse, Event synchronization, stream activation mechanismStatic Shape Runtime Design PrinciplesStatic Shape module design principles: performance optimization, ArgsFormat, address refresh strategy, memory managementGraph Foundation Structure Design PrinciplesGraph compilation common foundation structure design principles: independence, compatibility, observability, concurrency model, cross-platform consistency【免费下载链接】geGEGraph Engine是面向昇腾的图编译器和执行器提供了计算图优化、多流并行、内存复用和模型下沉等技术手段加速模型执行效率减少模型内存占用。 GE 提供对 PyTorch、TensorFlow 前端的友好接入能力并同时支持 onnx、pb 等主流模型格式的解析与编译。项目地址: https://gitcode.com/cann/ge创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考