The Cohere AI SDK is not compatible with the GitHub Cohere model.

[anchenyi]

Select Topic Area

Bug

Body

I copied and pasted the Python Cohere AI SDK sample from the GitHub model playground (https://github.com/marketplace/models/azureml-cohere/cohere-command-a/playground/code?target=https://github.com), but encountered a 404 error when running it locally.

Cohere AI SDK version: 5.16.1

Error:
File "C:\Users\anchenyi\AppData\Local\Programs\Python\Python311\Lib\site-packages\cohere\raw_base_client.py", line 971, in chat
raise ApiError(status_code=_response.status_code, headers=dict(_response.headers), body=_response.text)
cohere.core.api_error.ApiError: headers: {'azureml-destination-deployment': 'dep-coh-cmd-a-250413074009', 'azureml-destination-endpoint': 'cohere-command-a-eus2-ep', 'azureml-destination-logical-group': '', 'azureml-destination-model-group': 'cohere-eu2-cmd-a-scaleset', 'azureml-destination-region': 'eastus2', 'azureml-model-deployment': 'dep-coh-cmd-a-250413074009', 'azureml-model-group': 'cohere-eu2-cmd-a-scaleset', 'azureml-model-session': 'dep-coh-cmd-a-250413074009', 'azureml-ms-model-error-comprehensive': 'model_error-404-', 'azureml-served-by-cluster': 'vienna-eastus2-01', 'content-type': 'text/plain; charset=utf-8', 'date': 'Mon, 21 Jul 2025 03:48:52 GMT', 'ms-azureml-model-error-reason': 'model_error', 'ms-azureml-model-error-statuscode': '404', 'request-context': 'appId=', 'strict-transport-security': 'max-age=31536000; includeSubDomains; preload', 'vary': 'Origin,Accept-Encoding', 'x-content-type-options': 'nosniff', 'x-request-id': '5d46db39-6b17-4ecc-a46b-268c72c7c290', 'x-request-time': '0.035', 'content-length': '9', 'x-github-backend': 'Kubernetes', 'x-github-request-id': 'D20D:C4B26:14335E:1BE9F3:687DB8A3', 'server': 'github.com'}, status_code: 404, body: NOT FOUND

[github-actions[bot]]

💬 Your Product Feedback Has Been Submitted 🎉

Thank you for taking the time to share your insights with us! Your feedback is invaluable as we build a better GitHub experience for all our users.

Here's what you can expect moving forward ⏩

• Your input will be carefully reviewed and cataloged by members of our product teams. 
  • Due to the high volume of submissions, we may not always be able to provide individual responses.
  • Rest assured, your feedback will help chart our course for product improvements.
• Other users may engage with your post, sharing their own perspectives or experiences. 
• GitHub staff may reach out for further clarification or insight. 
  • We may 'Answer' your discussion if there is a current solution, workaround, or roadmap/changelog post related to the feedback.

Where to look to see what's shipping 👀

• Read the Changelog for real-time updates on the latest GitHub features, enhancements, and calls for feedback.
• Explore our Product Roadmap, which details upcoming major releases and initiatives.

What you can do in the meantime 💻

• Upvote and comment on other user feedback Discussions that resonate with you.
• Add more information at any point! Useful details include: use cases, relevant labels, desired outcomes, and any accompanying screenshots.

As a member of the GitHub community, your participation is essential. While we can't promise that every suggestion will be implemented, we want to emphasize that your feedback is instrumental in guiding our decisions and priorities.

Thank you once again for your contribution to making GitHub even better! We're grateful for your ongoing support and collaboration in shaping the future of our platform. ⭐

[LindseyB]

Did you also generate the GITHUB_TOKEN? And just so I'm making sure I can test on my end with the exact example you have would you mind pasting your exact code your ran on your end?

Additionally, on the 17th we made a change to no longer support https://models.inference.ai.azure.com and now only support https://models.github.ai which you can read more about in our changelog.

I ran the following code successfully with python 3.9.6 on a mac

import os
from azure.ai.inference import ChatCompletionsClient
from azure.ai.inference.models import SystemMessage, UserMessage
from azure.core.credentials import AzureKeyCredential

endpoint = "https://models.github.ai/inference"
model = "cohere/cohere-command-a"
token = os.environ["GITHUB_TOKEN"]

client = ChatCompletionsClient(
    endpoint=endpoint,
    credential=AzureKeyCredential(token),
)

response = client.complete(
    messages=[
        SystemMessage(""),
        UserMessage("What is the capital of France?"),
    ],
    temperature=0.8,
    top_p=0.1,
    max_tokens=2048,
    model=model
)

print(response.choices[0].message.content)

[anchenyi]

Hi @LindseyB, I have set the env var and used the new endpoint. The example you gave is azure ai inference SDK not Cohere AI SDK. Please try the Cohere AI SDK in the dropdown menu. Thanks!

[LindseyB]

Thanks for the response I'm able to recreate the error, for now the only work around is to use the azure AI inference SDK, but I'll let our team know about the current issue.