[After #607][api/single] Support executorch-llama #574
Conversation
anyj0527
requested review from
jaeyun-jung,
again4you,
myungjoo,
gichan-jang,
songgot and
jijoongmoon
as code owners
|
📝 TAOS-CI Version: 1.5.20200925. Thank you for submitting PR #574. Please a submit 1commit/1PR (one commit per one PR) policy to get comments quickly from reviewers. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://ci.nnstreamer.ai/. |
anyj0527
force-pushed
the
single-llama
branch
2 times, most recently
from
83c9879 to
ed9a143
Compare
| ml_tensors_info_set_tensor_type (in_info, 0, ML_TENSOR_TYPE_UINT8); | ||
| ml_tensors_info_set_tensor_dimension (in_info, 0, dim); | ||
|
|
||
| ml_tensors_data_create (in_info, &in_data); |
There was a problem hiding this comment.
How about checking if in_data is created properly?
status = ml_tensors_data_create (in_info, &in_data);
EXPECT_EQ (status, ML_ERROR_NONE);
| ml_tensors_info_set_tensor_dimension (in_info, 0, dim); | ||
|
|
||
| ml_tensors_data_create (in_info, &in_data); | ||
| ml_tensors_data_set_tensor_data (in_data, 0, prompt.c_str (), prompt.size () + 1); |
There was a problem hiding this comment.
Let's check the result, how about you?
EXPECT_EQ (status, ML_ERROR_NONE);
| @@ -1083,6 +1090,11 @@ ml_single_open_custom (ml_single_h * single, ml_single_preset * info) | |||
| status = ML_ERROR_STREAMS_PIPE; | |||
| goto error; | |||
| } | |||
| /* handle flexible single */ | |||
| if (info->nnfw == ML_NNFW_TYPE_EXECUTORCH_LLAMA) { | |||
There was a problem hiding this comment.
I think if there are more flexible filter, we may revise this part later.
The llama2c filter I created doesn't seem to be scalable due to API limitations, so I won't add it now.
| @@ -75,6 +75,7 @@ typedef enum { | |||
| ML_NNFW_TYPE_NCNN = 18, /**< Tencent ncnn (Since 9.0) */ | |||
| ML_NNFW_TYPE_TENSORRT = 19, /**< NVidia Tensor-RT (Since 9.0) */ | |||
| ML_NNFW_TYPE_QNN = 20, /**< Qualcomm QNN (Qualcomm® AI Engine Direct) (Since 9.0) */ | |||
| ML_NNFW_TYPE_EXECUTORCH_LLAMA = 21, /**< ExecuTorch Llama runner */ | |||
|
Please do not merge this until tizen gbm release is done! |
- Let single API handle flexible filter. Signed-off-by: Yongjoo Ahn <yongjoo1.ahn@samsung.com>
- Add a enum for new nnfw `EXECUTORCH_LLAMA` Signed-off-by: Yongjoo Ahn <yongjoo1.ahn@samsung.com>
anyj0527
force-pushed
the
single-llama
branch
from
ed9a143 to
8f8b2e4
Compare
anyj0527
changed the title
- Let single API handle executorch-llama as flexible filter thus do invoke-dynamic. Signed-off-by: Yongjoo Ahn <yongjoo1.ahn@samsung.com>
- Add a simple test to show how to use executorch-llama with single api Signed-off-by: Yongjoo Ahn <yongjoo1.ahn@samsung.com>
anyj0527
force-pushed
the
single-llama
branch
from
8f8b2e4 to
cd1847b
Compare
EXECUTORCH_LLAMA