Hello experts,
I am new to AI Core product, and I am trying to make some scenarios, but I faced problem that is stopping me for a while now.
Background:
I have been doing tutorial about SAP BTP AI Core: https://developers.sap.com/group.ai-core-get-started-basics.html
Everything was working fine till the tutorial “Make Predictions for House Prices with SAP AI Core”: https://developers.sap.com/tutorials/ai-core-deploy.html
There is step about to loading trained model to make predictions (I will refer to this later):
model = pickle.load(open (‘/mnt/models/model.pkl’,’rb’))
Then there is the rest of the code for creating endpoints and using this model for prediction.
Further configuration I did correct (Docker, yaml etc.), according to tutorial.
The service started but was not working (error 500 when calling endpoint).
My tests and analysis:
I couldn’t find any potential problem for that, I rechecked everything and the web service from tutorial was still not working.
I decided to create my own model and configuration for a very simple example of model which is instance of class making addition (very simple one, just for tests).
But this time I created some more endpoints for this web service to check if problem is only with the one using (and loading) model:
Again, everything was working except the part of using the model, so:
- The AWS S3 Bucket has this model (I check it manually using console) and path is correct.
- Python code is fine because I successfully tested it locally.
- Git with YAML file is correct because the service is running and other endpoints working.
- All actions in AI Core (deployments, executions etc.) were also successful.
I implemented more functionalities into this webservice, so I was able to confirm that all other endpoints were working correctly and only the one loading model was returning error 500.
One of the functionalities was defined as this:
And the output was:
AI Core Server: __pycache__|main.py|requirements.txt|
So basically, there is no other folder (should be /mnt) or file in this location… Of course, I don’t know how exactly AI Core works internally, so maybe this approach for testing is wrong, but I also don’t know how else I can check this, when I see this log:
“INFO:root:Successfully copied s3://hcp-… to /mnt/models”
The location is 100% correct, I can see (and copy) this model using direct access to my AWS S3 Bucket. I did this and deployed locally, and it is working fine.
My conclusion is that the model is not copied successfully (the path doesn’t even exist), or I am making some error in the implementation of this functionality.
Question:s
- Is my approach for testing this correct (should this python function return all files and folders for my instance of AI Core in root location /)?
- How else I can debug/analyze this problem? Have you ever faced a similar issue?
- Is it possible that the free version of AI Core has some limitations like this or a hidden non-deterministic bug?
I fount very similar problem here:
https://community.sap.com/t5/technology-q-a/sap-ai-core-error-while-prediction/qaq-p/12769557
This problem has not been resolved on that forum as I see. I followed hints from there but with no results.
Best regards