Which models should I compare?

Start with the models you can select in your build. The announced provider list does not establish which individual models are available to you.

What should I record during a comparison?

Keep the build and model names, the shared prompt and code, your evaluation criteria, and any difference in context each candidate received.

Can I use one test for every coding task?

Use a separate comparison for each type of work that matters to you. An explanation result cannot settle how well a model handles troubleshooting.

What if my task needs Agent Mode?

Confirm support for the capability you need before comparing outputs. Preview support varies across models.

How to Choose an AI Model in Visual Studio

Choose an AI model in Visual Studio by starting with the work you need done. Compare candidates on the same task, using criteria you set before reading their answers: accuracy, useful detail, and a clear next step. The September 2026 Visual Studio update encourages developers to choose a model that fits their workflow.¹

Your choice also depends on access and feature support. Bring Your Own Model is in preview, and some models do not support every Agent Mode capability. Keep each comparison tied to the task, models, and Visual Studio build you use.¹

What model choice means in Visual Studio

The September 2026 update describes Bring Your Own Model as enabled by default in Visual Studio Community, Professional, and Enterprise. It lists Microsoft Foundry, OpenAI, Anthropic, and Ollama as supported providers. That provider list does not tell you which individual models are available in your build.¹

The announced setup route starts in Chat’s model picker, where you can add or manage a model before connecting a provider. The update also says you can use Bring Your Own Model without signing in to GitHub. Check the options shown in your installed build before making a shortlist.¹

Write down your Visual Studio build, the model names you can select, and whether your task needs Agent Mode. If a candidate lacks a capability the task requires, exclude it from that comparison. This keeps the decision focused on models that can perform the same work.¹

Choose based on the coding task

Start with work you can inspect. A Visual Studio Blog post identifies understanding unfamiliar codebases and troubleshooting as uses for AI development tools. For an explanation task, ask each candidate to describe the same function and trace an important path through it. For troubleshooting, provide the same error and relevant code, then compare the proposed diagnostic steps.²

Choose criteria that fit the task. For an explanation, check whether the response separates what the code shows from what it infers, identifies relevant methods, and makes the control flow understandable. For troubleshooting, check whether the proposed cause follows from the evidence you supplied and whether the next step could confirm it.

Check feature support before judging answer quality. If you intend to use Agent Mode, confirm that each candidate supports the capability your workflow needs. Support is not uniform across models in the preview, so a good answer in Chat alone does not establish suitability for an Agent Mode task.¹

Select and test a model

Open Chat’s model picker to add or manage a model, then connect its provider, following the route described in the September 2026 update. Check the labels in your installed build as you proceed. Record the build and model names with your results so you can revisit the choice when your options change.¹

Try a controlled explanation exercise with two models you can access. Select one short function or class. Give both candidates the same code and request: explain its purpose, trace one important path, identify assumptions, and name any uncertainty. Keep the instructions and surrounding context the same. If one candidate receives more code context, note that difference.

Review both responses against the code. Mark verifiable statements, unsupported guesses, missing steps, and points that require a follow-up question. Choose the candidate that better meets your criteria for this task. If troubleshooting also matters to you, run a separate comparison using the same error and code for both candidates.

Check limits before committing

Confirm which individual models you can use in your installed build. The September 2026 announcement names supported providers and identifies Bring Your Own Model as a preview feature; those facts alone do not establish a model’s current availability or your eligibility to use it. Include your build when sharing a shortlist with teammates.¹

Check the pricing, usage limits, and data handling terms that apply to your intended provider and account before sending project code. The available research does not establish those terms. The ability to use Bring Your Own Model without a GitHub sign-in does not settle questions about cost or data handling.¹

Test the feature path you expect to use, especially if it involves Agent Mode. Some preview models lack capabilities there. A useful shortlist records whether you can select each model, whether it supports the required workflow, and how its output held up against the same test task.¹

Start with the models available in your build, compare them on identical code and prompts, and inspect their answers against the source. Then confirm workflow support and applicable terms. Choose the model that meets those checks for the task at hand.¹