Optimize AWS Inferentia utilization with FastAPI and PyTorch models on Amazon EC2 Inf1 & Inf2 instances
AWS Machine Learning
JULY 24, 2023
If the model changes on the server side, the client has to know and change its API call to the new endpoint accordingly. If you’re using a different AMI (Amazon Linux 2023, Base Ubuntu etc.), script in the fastapi and trace-model folders use this to create Docker images. install the CLI tools by following this guide.
Let's personalize your content