Sample Usage Guide【免费下载链接】geGEGraph Engine是面向昇腾的图编译器和执行器提供了计算图优化、多流并行、内存复用和模型下沉等技术手段加速模型执行效率减少模型内存占用。 GE 提供对 PyTorch、TensorFlow 前端的友好接入能力并同时支持 onnx、pb 等主流模型格式的解析与编译。项目地址: https://gitcode.com/cann/geFeature DescriptionThis sample demonstrates a custom pass for custom operator AddCustom.This use case targets scenarios where users can access custom operator prototypesThe pass implementation in this example: for AddCustom with one input being 0, perform deletion operation. Provides two verification methods: online inference and ATC offline model compilation. The sample uses eager style API and fusion interface.Directory Structure├── README.md // C sample documentation ├── src │ ├──addcustom_zero_pass.cpp // pass implementation file ├── CMakeLists.txt // build script ├── data | ├──torch_forward.py // torch script for online inference |—— gen_es_api | |——CMakeLists.txt // build script for generating eager style api |—— proto // custom operator prototypes | |——add_custom_proto.cc | |——add_custom_proto.hEnvironment RequirementsCompiler: GCC 7.3.xPython and dependencies: python3.9, pytorch2.1Environment preparation completed.PreparationCreate custom operator project: prerequisite for writing custom pass for custom operators is user has created custom operator project, refer to Custom Operator Graph Integration. During this phase, users need to complete: custom operator implementation, custom operator package compilation and deployment, custom operator adaptation development.Obtain operator prototype: After successful compilation, copy custom operator prototype definitions from build_out/autogen path in custom operator project to proto directory in current project. This sample already has AddCustom custom operator prototype in proto directory, users can replace or add as needed.Program CompilationConfigure environment variables.Run environment setup script:source ${ASCEND_PATH}/set_env.sh${ASCEND_PATH}is cann path under CANN software package installation directory. Replace with actual installation path, e.g.,${INSTALL_PATH}/cann.Modifygen_es_api/CMakeListsfile as needed:Modify custom operator prototype file path: add_library(custom_op_proto SHARED .../proto/your_proto_name.cc)ModifyCMakeLists.txtas needed.ASCEND_PATH: Default software package path. If$ASCEND_HOME_PATHset via set_env.sh, no modification needed.PASS_SO_DIR: Custom fusion pass dynamic library installation directory name, defaultpass_so_dir.target_include_directories: Required header files. For this sample, no modification needed. For custom development, add header files below the example without deleting existing items. If network has custom operators, add custom operator prototype definition headers.target_link_libraries: Required libraries. For this sample, no modification needed. For custom development, add libraries below the example without deleting existing items.Do not link other SOs from software package to avoid compatibility issues during future upgrades.Execute sequentially:mkdir build cd build cmake ..After pass writing completed, run following commands to compile custom pass so and copy compiled dynamic library libadd_zero_pass.so to custom fusion pass directory, where xxx is user-defined directory. Optional parameter-j$(nproc)can be added after make for parallel build tasks,$(nproc)dynamically gets CPU core count.make -j$(nproc) add_custom_zero_pass make installAfter sample verification, run the following command to clean custom pass so installed under CANN package to avoid affecting subsequent UT/ST:make clean_custom_passPass WritingDefine classAddCustomZeroPassinheriting fromPatternFusionPass.Override three functions from base classPatternFusionPass:Patternsdefines matching templates for identifying topologies matching the template in the graph.MeetRequirementsfilters topologies matched by template.Replacementdefines replacement part.RegisterAddCustomZeroPassas custom fusion pass with execution phase BeforeInferShape.VerificationConfigure environment variables.Run environment setup script:source ${ASCEND_PATH}/set_env.sh${ASCEND_PATH}is cann path under CANN software package installation directory. Replace with actual installation path, e.g.,${INSTALL_PATH}/cann.Online inferenceSet environment variable to dump model graph during compilation:export DUMP_GE_GRAPH1Enter data directory and execute .py file for online inference:python torch_forward.pyLog shows:Define pattern for AddCustomZeroPass Define MeetRequirements for AddCustomZeroPass Define replacement for AddCustomZeroPassView resultsAfter execution, series of .pdtxt files generated in directory. Compare the following dump graphs:ge_onnx_xxxxx_PreRunBegin.pdtxtdump graph before executionge_onnx_xxxxx_RunCustomPassBeforeInferShape.pdtxtcustom pass dump graph before InferShape executionModel optimized as expected, i.e., add-zero nodes removed.If results not as expected, set following environment variables (for atc command, also add parameter--logdebug) to print logs to screen for troubleshooting.export ASCEND_SLOG_PRINT_TO_STDOUT1 #print logs to screen export ASCEND_GLOBAL_LOG_LEVEL0 #log level debug【免费下载链接】geGEGraph Engine是面向昇腾的图编译器和执行器提供了计算图优化、多流并行、内存复用和模型下沉等技术手段加速模型执行效率减少模型内存占用。 GE 提供对 PyTorch、TensorFlow 前端的友好接入能力并同时支持 onnx、pb 等主流模型格式的解析与编译。项目地址: https://gitcode.com/cann/ge创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考