#include <default_llm_infer.h>
Inheritance diagram for nndeploy::llm::DefaultLlmInferParam:
Collaboration diagram for nndeploy::llm::DefaultLlmInferParam:
Public Member Functions | |
| virtual base::Status | deserialize (rapidjson::Value &json) override |
| virtual base::Status | serialize (rapidjson::Value &json, rapidjson::Document::AllocatorType &allocator) override |
 Public Member Functions inherited from nndeploy::base::Param | |
| base::Status | addDropdownParam (const std::string &dropdown_param, const std::vector< std::string > &dropdown_values) |
| base::Status | addIoParam (const std::string &io_param) |
| base::Status | addRequiredParam (const std::string &required_param) |
| base::Status | addUiParam (const std::string &ui_param) |
| base::Status | clearDropdownParams () |
| base::Status | clearIoParams () |
| base::Status | clearRequiredParams () |
| base::Status | clearUiParams () |
| virtual base::Status | deserialize (const std::string &json_str) |
| virtual base::Status | get (const std::string &key, base::Any &any) |
| std::map< std::string, std::vector< std::string > > | getDropdownParams () |
| std::vector< std::string > | getIoParams () |
| std::vector< std::string > | getRequiredParams () |
| std::vector< std::string > | getUiParams () |
| virtual base::Status | loadFile (const std::string &path) |
| Param () | |
| base::Status | removeDropdownParam (const std::string &dropdown_param) |
| base::Status | removeIoParam (const std::string &io_param) |
| base::Status | removeRequiredParam (const std::string &required_param) |
| base::Status | removeUiParam (const std::string &ui_param) |
| virtual base::Status | saveFile (const std::string &path) |
| virtual std::string | serialize () |
| virtual base::Status | set (const std::string &key, base::Any &any) |
| base::Status | setDropdownParams (const std::map< std::string, std::vector< std::string >> &dropdown_params) |
| base::Status | setIoParams (const std::vector< std::string > &io_params) |
| base::Status | setRequiredParams (const std::vector< std::string > &required_params) |
| base::Status | setUiParams (const std::vector< std::string > &ui_params) |
| virtual | ~Param () |
Public Attributes | |
| base::DataType | attention_mask_data_type_ = base::dataTypeOf<float>() |
| std::string | attention_type_ = "full" |
| std::shared_ptr< EmbeddingParam > | embedding_param_ = nullptr |
| std::shared_ptr< inference::InferenceParam > | inference_param_ = nullptr |
| base::InferenceType | inference_type_ = base::kInferenceTypeOnnxRuntime |
| bool | is_embedding_ = false |
| std::vector< int32_t > | kv_init_shape_ |
| int | layer_nums_ = 24 |
| int | max_seq_len_ = 2048 |
| Public Attributes inherited from nndeploy::base::Param | |
| std::map< std::string, std::vector< std::string > > | dropdown_params_ |
| std::vector< std::string > | io_params_ |
| std::vector< std::string > | required_params_ |
| std::vector< std::string > | ui_params_ |
Detailed Description
Definition at line 31 of file default_llm_infer.h.
Member Function Documentation
◆ deserialize()
|
inlineoverridevirtual |
Reimplemented from nndeploy::base::Param.
Definition at line 97 of file default_llm_infer.h.
Here is the call graph for this function:
◆ serialize()
|
inlineoverridevirtual |
Reimplemented from nndeploy::base::Param.
Definition at line 46 of file default_llm_infer.h.
Here is the call graph for this function:
Member Data Documentation
◆ attention_mask_data_type_
| base::DataType nndeploy::llm::DefaultLlmInferParam::attention_mask_data_type_ = base::dataTypeOf<float>() |
Definition at line 42 of file default_llm_infer.h.
◆ attention_type_
| std::string nndeploy::llm::DefaultLlmInferParam::attention_type_ = "full" |
Definition at line 43 of file default_llm_infer.h.
◆ embedding_param_
| std::shared_ptr<EmbeddingParam> nndeploy::llm::DefaultLlmInferParam::embedding_param_ = nullptr |
Definition at line 34 of file default_llm_infer.h.
◆ inference_param_
| std::shared_ptr<inference::InferenceParam> nndeploy::llm::DefaultLlmInferParam::inference_param_ = nullptr |
Definition at line 37 of file default_llm_infer.h.
◆ inference_type_
| base::InferenceType nndeploy::llm::DefaultLlmInferParam::inference_type_ = base::kInferenceTypeOnnxRuntime |
Definition at line 36 of file default_llm_infer.h.
◆ is_embedding_
| bool nndeploy::llm::DefaultLlmInferParam::is_embedding_ = false |
Definition at line 33 of file default_llm_infer.h.
◆ kv_init_shape_
| std::vector<int32_t> nndeploy::llm::DefaultLlmInferParam::kv_init_shape_ |
Definition at line 41 of file default_llm_infer.h.
◆ layer_nums_
| int nndeploy::llm::DefaultLlmInferParam::layer_nums_ = 24 |
Definition at line 39 of file default_llm_infer.h.
◆ max_seq_len_
| int nndeploy::llm::DefaultLlmInferParam::max_seq_len_ = 2048 |
Definition at line 40 of file default_llm_infer.h.
The documentation for this struct was generated from the following file:
- /home/docs/checkouts/readthedocs.org/user_builds/nndeploy-zh/checkouts/latest/plugin/include/nndeploy/llm/default_llm_infer.h
