inf2 vs inf1 difference in bert output

0

Hi,

I have some code which generates a shape of torch.Size([1, 512, 1024] when calling bert on inf1. I have compiled the model for inf2. However the same code on inf2 produces a shape of torch.Size([1, 2]) . What could be the issue? Has anyone faced anything similar?

Topics

Machine Learning & AI Compute

Tags

AWS Inferentia AWS Neuron

Language

English

rePost-User-6132983

asked a year ago342 views

1 Answer

Newest
Most votes
Most comments

Are these answers helpful? Upvote the correct answer to help the community benefit from your knowledge.

0

This really is not enough information to get going with, please share the test code that produces this result

answered a year ago

Relevant content

neuron compiling bert model for inferentia on tf2
rePost-User-6132983
asked 2 years ago
Difference in pytorch BERT model output vs neuron
rePost-User-6132983
asked 2 years ago
Not able to compile to NEFF, the BERT model from neuron tutorial
AWS-User-0668339
asked 2 years ago
neuron compiling a bert model
rePost-User-6132983
asked 2 years ago
How do I troubleshoot issues with hosted zones in Route 53 that have the same domain names in different AWS accounts?
AWS OFFICIALUpdated a year ago
How can I determine which SageMaker notebook instance made a particular API call if all the instances use the same IAM role?
AWS OFFICIALUpdated 3 years ago
What are some of the best practices for importing existing resources into a CloudFormation stack?
AWS OFFICIALUpdated 5 months ago
How can I deploy an Amazon SageMaker model to a different AWS account?
AWS OFFICIALUpdated 10 months ago
New NLP/CV Examples to Get Started on AWS Inferentia and AWS Trainium
EXPERT
Kamran Khan
published 2 years ago
Train large language model using Hugging Face and AWS Trainium
EXPERT
Kamran Khan
published 2 years ago