Development Debugging Issue Records

Python

Python binding related, move to python/README.md

Error report when modifying weights loaded using safetensors

Weights loaded in safetensors mode are directly mmapped to the file, opened in read-only mode, and cannot be modified. To modify the weights, clone a Tensor, modify it, and replace.

Ascend development environment

Compilation error, linking error, undefined reference

Compilation requires root privileges

Vscode terminal lacks path prompts, some common command errors, inability to input, etc.

The terminal opened by default in Vscode is incomplete, requiring manual creation of a new terminal, selecting Bash

cmake method

  • Specific steps

    • Create a build directory in the root directory, copy cmake/config.cmake to this directory

      mkdir build
cp cmake/config.cmake build
cd build

    • Edit build/config.cmake to customize compilation options (the author’s custom compilation options: path/cmake/config_ascendcl.cmake)

    • cmake

      cmake ../ -DCMAKE_CXX_COMPILER=g++ -DCMAKE_SKIP_RPATH=TRUE

On the development server, models and third-party library resources are placed in /home/resource, all colleagues who can log in to this server have permission to access this directory

  • Model resources: /home/resource/model_zoo

  • Third-party library resources: /home/resource/third_party

    • Local terminal activation:

      • export LD_LIBRARY_PATH=/home/ascenduserdg01/github/nndeploy/build:$LD_LIBRARY_PATH

      • export LD_LIBRARY_PATH=/home/resource/third_party/onnxruntime-linux-aarch64-1.20.1/lib:$LD_LIBRARY_PATH

    • Always effective

      • echo ‘export LD_LIBRARY_PATH=/home/ascenduserdg01/github/nndeploy/build:$LD_LIBRARY_PATH’ >> ~/.bashrc

      • echo ‘export LD_LIBRARY_PATH=/home/resource/third_party/onnxruntime-linux-aarch64-1.20.1/lib:$LD_LIBRARY_PATH’ >> ~/.bashrc

      • source ~/.bashrc

  • Image resources: /home/resource/data_set

  • Under this development server, all developers have permission to this directory, specific execution commands are as follows:

    sudo chown -R :nndeploy /home/resource/
sudo chmod -R g+rwx /home/resource/

About the protobuf conflict issue caused by third-party tokenizer during compilation

Problem description

After direct compilation, executing the relevant demo’s executable file will cause a segment fault, gdb inspection results are as follows

raymond@ascenduserdg01:~/workspace/nndeploy/build$ gdb ./nndeploy_demo_dag
...
Thread 1 "nndeploy_demo_d" received signal SIGSEGV, Segmentation fault.
0x0000ffffecb9c744 in google::protobuf::Arena::AllocateAlignedNoHook(unsigned long) () from /lib/aarch64-linux-gnu/libprotobuf.so.23
(gdb) bt
#0  0x0000ffffecb9c744 in google::protobuf::Arena::AllocateAlignedNoHook(unsigned long) () from /lib/aarch64-linux-gnu/libprotobuf.so.23
#1  0x0000ffffecc50d08 in google::protobuf::FileDescriptorProto* google::protobuf::Arena::CreateMaybeMessage<google::protobuf::FileDescriptorProto>(google::protobuf::Arena*) ()
   from /lib/aarch64-linux-gnu/libprotobuf.so.23
#2  0x0000ffffecc25d70 in google::protobuf::EncodedDescriptorDatabase::Add(void const*, int) () from /lib/aarch64-linux-gnu/libprotobuf.so.23
#3  0x0000ffffecbf8ac8 in google::protobuf::DescriptorPool::InternalAddGeneratedFile(void const*, int) () from /lib/aarch64-linux-gnu/libprotobuf.so.23
#4  0x0000ffffecc6aaa0 in google::protobuf::internal::AddDescriptors(google::protobuf::internal::DescriptorTable const*) () from /lib/aarch64-linux-gnu/libprotobuf.so.23
#5  0x0000ffffecb8e478 in ?? () from /lib/aarch64-linux-gnu/libprotobuf.so.23
#6  0x0000fffff7fc7624 in call_init (env=0xfffffffff338, argv=0xfffffffff328, argc=1, l=<optimized out>) at ./elf/dl-init.c:70
#7  call_init (l=<optimized out>, argc=1, argv=0xfffffffff328, env=0xfffffffff338) at ./elf/dl-init.c:26
#8  0x0000fffff7fc772c in _dl_init (main_map=0xfffff7fff370, argc=1, argv=0xfffffffff328, env=0xfffffffff338) at ./elf/dl-init.c:117
#9  0x0000fffff7fd9cc8 in _dl_start_user () from /lib/ld-linux-aarch64.so.1
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Root cause analysis

The issue occurs in /lib/aarch64-linux-gnu/libprotobuf.so.23, so inspect the dynamic links and symbol table of nndeploy_demo_dag

raymond@ascenduserdg01:~/workspace/nndeploy/build$ ldd nndeploy_demo_dag | grep proto
	libprotobuf.so.23 => /lib/aarch64-linux-gnu/libprotobuf.so.23 (0x0000ffffa5990000)
raymond@ascenduserdg01:~/workspace/nndeploy/build$ nm nndeploy_demo_dag | grep protobuf | grep ' T ' | head -n 5
0000000000135bb0 T _ZN2pb11CppFeaturesC1EPN6google8protobuf5ArenaE
0000000000135bd0 T _ZN2pb11CppFeaturesC1EPN6google8protobuf5ArenaERKS0_
0000000000135bb0 T _ZN2pb11CppFeaturesC2EPN6google8protobuf5ArenaE
0000000000135bd0 T _ZN2pb11CppFeaturesC2EPN6google8protobuf5ArenaERKS0_
000000000003ea60 T _ZN6google8protobuf10DescriptorC1Ev

It was found that the demo not only links to the dynamic library of protobuf, but because some symbol definitions already exist in protobuf, it should also link to the static library of protobuf; but the previously built results ran without any issues, the segment fault occurred after adding the third-party’s tokenizer in the build, and the tokenizer links to the static library of sentencepiece

target_link_libraries(tokenizers_cpp PRIVATE tokenizers_c sentencepiece-static ${TOKENIZERS_CPP_LINK_LIBS})

sentencepiece uses protobuf-lite during compilation, so the third-party’s tokenizer library contains protobuf-lite, leading to a conflict between protobuf-lite and protobuf

Temporary solutions

  • First method: The system needs to install libprotobuf, directly skip the compilation of protobuf in third_party, the demo’s executable file only links to the dynamic library of protobuf

  • Only modify protobuf.cmake as follows:

include(ExternalProject)
include(cmake/util.cmake)

if (ENABLE_NNDEPLOY_PROTOBUF STREQUAL "OFF")
elseif (ENABLE_NNDEPLOY_PROTOBUF STREQUAL "ON")
  option(protobuf_BUILD_TESTS "" OFF)
  option(protobuf_MSVC_STATIC_RUNTIME "" ${ONNX_USE_MSVC_STATIC_RUNTIME})

  find_package(PkgConfig REQUIRED)
  pkg_check_modules(PROTOBUF REQUIRED protobuf)

  find_package(Protobuf REQUIRED)
  if(Protobuf_FOUND)
    include_directories(${Protobuf_INCLUDE_DIRS})
    set(NNDEPLOY_THIRD_PARTY_LIBRARY ${NNDEPLOY_THIRD_PARTY_LIBRARY} ${Protobuf_LIBRARIES})
  else()
    set(LIBS libprotobuf)
    add_subdirectory_if_no_target(${PROJECT_SOURCE_DIR}/third_party/protobuf ${LIBS})
    include_directories(${PROBUF_ROOT}/src)
    set(NNDEPLOY_THIRD_PARTY_LIBRARY ${NNDEPLOY_THIRD_PARTY_LIBRARY} ${LIBS})
  endif()
else()
endif()

  • Second method: The system installs libprotobuf-lite, when compiling the tokenizer, link to libprotobuf-lite.so, the demo’s executable file links to both the dynamic library of protobuf and the dynamic library of protobuf-lite

  • Only modify tokenizer_cpp.cmake as follows:

include(ExternalProject)

if (ENABLE_NNDEPLOY_PLUGIN_TOKENIZER_CPP STREQUAL "OFF")
elseif (ENABLE_NNDEPLOY_PLUGIN_TOKENIZER_CPP STREQUAL "ON")
  set(TOKENZIER_CPP_PATH third_party/tokenizers-cpp)
  add_subdirectory(${TOKENZIER_CPP_PATH} tokenizers)
  include_directories(${TOKENZIER_CPP_PATH}/include)
  set(NNDEPLOY_THIRD_PARTY_LIBRARY ${NNDEPLOY_THIRD_PARTY_LIBRARY} tokenizers_cpp)   

  find_library(PROTOBUF_LITE_LIB protobuf-lite REQUIRED)
  if (PROTOBUF_LITE_LIB)
      message(STATUS "Found protobuf-lite: ${PROTOBUF_LITE_LIB}")
      set(NNDEPLOY_THIRD_PARTY_LIBRARY ${NNDEPLOY_THIRD_PARTY_LIBRARY} ${PROTOBUF_LITE_LIB})   
  else()
      message(FATAL_ERROR "protobuf-lite not found! Please install")
  endif()

else()
endif()

Explanation

  • Both solutions are workarounds, requiring the installation of the relevant protobuf on the system

  • For the first method, the third-party’s protobuf is directly skipped during compilation, so the third-party’s protobuf seems meaningless

  • For the second method, there is a risk, in the case where both dynamic and static libraries are linked, the dynamic library will be preferred, but it’s still uncertain whether conflicts will occur, especially when the versions of the dynamic and static libraries are inconsistent

Proxy failure, github connection issues, etc.

Unstable github connection issues

When setting up the development environment on the Ascend server, we encounter issues like repository cloning from github.com or other behaviors being particularly slow. For this phenomenon, there are currently two methods to handle:

  1. Clone the repository using SSH connection.

  2. By mounting a proxy on the server, make HTTPS services more convenient.

Currently, I can only temporarily use the first method to access the service, the second method seems to have encountered some restrictions waiting for someone to solve.

For the first method, you can mainly refer to GitHub’s article titled ‘connecting-to-github-with-ssh’. Then, after going through the process, you will find that you still cannot use SSH to connect to GitHub, ‘ssh -T git@github.com’ cannot get the expected result; due to this reason, the author found that the default port 20 is restricted, and port 443 needs to be used. You need to add code in the root directory $HOME/.ssh/config, then you can proceed.

Host github.com
    Hostname ssh.github.com
    Port 443
    User git

In addition, you also need to add the following code in $HOME/.gitconfig to let the subsequent submodules execute according to SSH.

[url "ssh://git@github.com/"]
	insteadOf = https://github.com/

Using the SSH solution alone can solve the latency issue.

For the second method, you can configure the proxy on your own server in two ways:

  1. According to the article’s method, download the specified package (for Accend, download this ARM version of clash). Proceed as described in the article, but you may encounter the issue of missing MMDB files, which needs to be solved by yourself. The way I solved it was to copy the Country.MMDB file from the second method (the MMDB file contains common website to IP key-value pairs).

  2. Download the clash-for-linux repository and start the proxy according to the README; Note: These two are the methods the author used during testing, using Linux’s clash method, you need to purchase a subscription yourself. So far, the author has gone through the process, but seems to have encountered some constraints (this method has been verified on multiple servers): alt text

The author suspects whether the server only has port 443 and some necessary ports available, other ports are not, because if the proxy uses port 52984, while in the first method, SSH uses the default port 20. The author has also tried most methods online to get a result, including but not limited to manually compiling git, etc…

Enter the system’s default shell as a solution to sh:

Temporary method: Enter the server’s default bash: sudo vim /etc/passwd Change the end of your user to /usr/bin/bash