You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi, big guys!
Recently I tried to run phi3 on NPU inference using DirectML as backend EP under Onnxruntime framework, but I failed!
I loaded the model and tokenizer through the Model class and Tokenizer class, but from the C# API given in the official Onnxruntime docs(https://onnxruntime.ai/docs/genai/api/csharp.html), I can't find any API about being able to set up the NPU, can someone expand on this in detail to explain how to load the model onto the NPU?(It's best if you can explain it through a demo)
Or instead do you know if there are demos on loading Phi3 or other LLM models to NPU?
Looking forward to your reply!
Below is the code I used to load the model and tokenizer using the OnnxruntimeGenai-directml library and enter the question to be answered.
The text was updated successfully, but these errors were encountered:
Hi, big guys!
Recently I tried to run phi3 on NPU inference using DirectML as backend EP under Onnxruntime framework, but I failed!
I loaded the model and tokenizer through the Model class and Tokenizer class, but from the C# API given in the official Onnxruntime docs(https://onnxruntime.ai/docs/genai/api/csharp.html), I can't find any API about being able to set up the NPU, can someone expand on this in detail to explain how to load the model onto the NPU?(It's best if you can explain it through a demo)
Or instead do you know if there are demos on loading Phi3 or other LLM models to NPU?
Looking forward to your reply!
Below is the code I used to load the model and tokenizer using the OnnxruntimeGenai-directml library and enter the question to be answered.
The text was updated successfully, but these errors were encountered: