How to Choose an MCP Server: A Decision Framework Built from 6,000+ Servers

How to Choose an MCP Server: A Decision Framework Built from 6,000+ Servers

TL;DR

• The directory lists 6,775 MCP servers, but only 28.7% have a runnable install command in their README, 21.8% have no clear license, and 20.4% have been dormant for one to two years.
• Choosing one is not about picking the most-starred repo. It is three steps: a five-minute repo screen → reading the tool definitions like an API → a 30-minute test with read-only credentials.
• The security review collapses into one sentence: assume the server can see data it should not, then verify that it cannot.

Why selection is harder than discovery

The MCP ecosystem no longer has a supply problem. It has a “which of these actually works” problem.

We analysed public data for 5,945 MCP servers in the directory (full report). Five numbers are worth memorising before you evaluate anything:

Then there is the counter-intuitive part: stars are close to useless as a quality signal. 62.8% of servers have fewer than 100 stars and the median is 64, while the top 1% of repos hold 47.5% of all stars. A 40-star server can be a team’s production dependency; a 10,000-star repo can be a single Hacker News spike.

The default stance should be: trust no single metric; verify with a process.

Start with three questions

Before opening any repository, write these three lines. If you cannot, you are not ready to evaluate.

1. What specific job should it do? Not “connect to a database”, but “let the agent query the orders and refunds tables read-only and aggregate by day”. The narrower the job, the easier it is to call something good enough.
2. What data can it see or change? Read-only or write? One table or the whole instance? Can it reach the filesystem, environment variables, or internal network addresses? This boundary determines how strict the security review must be.
3. Who owns it when it breaks? Personal experiments can be loose. Once it is in a team workflow, someone must own upgrades, audits, and rollback.

Step 1 — A five-minute screen

Open the repo and check six things in order. Any failure is a pass. Five minutes is enough.

# Four fields, one command
gh repo view owner/repo --json pushedAt,licenseInfo,latestRelease,stargazerCount
gh api repos/owner/repo/issues?state=open\&per_page=5 --jq '.[].updated_at'

A useful rule of thumb: if the README takes ten minutes to explain how to start the server, it probably does not belong in production. That is not a knock on the author; it means installability was never treated as a feature.

Step 2 — Read the tool definitions like an API

The real interface of an MCP server is the set of tools it exposes. The README is for humans; the tool definitions are for the model, and they decide whether the thing is usable.

Look at the tools/list response (or the schemas in the source) and check five things.

1. Tool count Past roughly 20 tools, be careful: definitions consume context, and the chance of a model picking the wrong one among near-duplicates rises quickly. Good implementations group tools, offer a two-step “search then call” entry point, or let you enable a subset.

2. Do the descriptions state the boundaries? Weak: Query the database. Strong: Run a read-only SQL query against the analytics replica. Rejects writes, DDL, and statements without a LIMIT. Returns at most 500 rows.

The second tells the model what it can do, what it cannot, and what happens at the limit.

3. Is the parameter schema strict? Types, required fields, enums, defaults. The more free-form strings a tool takes, the more the model invents.

4. Are destructive operations annotated? The MCP spec supports readOnlyHint, destructiveHint, idempotentHint, and related annotations. A delete tool without destructiveHint: true gives the client no way to ask for confirmation first. That is a design defect.

5. What happens on failure? A structured error (code, readable message, suggested fix), or a stack trace? Is a retry safe? Are large result sets paginated?

// A tool definition you can trust
{
  "name": "search_orders",
  "description": "Search orders by customer email or order id. Read-only. Returns at most 50 rows per page; use cursor for the next page.",
  "annotations": { "readOnlyHint": true, "openWorldHint": false },
  "inputSchema": {
    "type": "object",
    "properties": {
      "email": { "type": "string", "format": "email" },
      "cursor": { "type": "string" }
    },
    "additionalProperties": false
  }
}

Step 3 — Security review, ordered by blast radius

MCP connects external systems to a model, which brings external risk inside. Check from the most destructive failure mode down:

1. Least privilege. A read-only token, a restricted schema, a dedicated account. Use a replica if one exists. Grant the narrowest scope that still does the job.
2. Secret handling. Environment variables are the floor, not the ceiling. Is there secret-manager support? Can a key leak into logs or error messages?
3. Arbitrary code execution. eval, exec, installing dependencies at runtime, downloading and running scripts — each one needs a specific justification.
4. Egress control. A server that can fetch any URL hands the model an SSRF and exfiltration channel. Prefer an allowlist of domains.
5. Prompt injection through tool output. A sentence on a web page, in an issue, or in an email can instruct the agent to call another tool. Prefer servers that isolate or label untrusted content and do not treat it as instructions.
6. Supply chain. Who publishes the npm package, when was it last updated, is there a postinstall, and does the dependency tree contain anything unexpected?
7. Audit and kill switches. Can every tool call be logged? Can dangerous tools be disabled individually? No audit trail, no production.

For remote servers, add three questions: what OAuth scopes does it request, where is the token stored, and which region does the data land in.

Step 4 — A 30-minute test with three tasks

Reading code tells you whether the author cares. Running it tells you whether it works.

Use a test account with read-only credentials in an isolated environment, and do three things:

Record four numbers: success rate, P95 latency, token cost per call, readability of errors. Then measure time to roll back: edit the config, restart the client, rotate the credential. Run it once before you need it.

A worked example: six candidates, one survivor

Goal: let the agent query the analytics database read-only, to answer “new users by channel last week”.

D wins. It has fewer than 100 stars, but it passed both the permission-boundary and error-handling tests. That is the point: stars measure exposure, not usability.

Ten red flags

Stop and reconsider if you see any of these:

1. No copy-paste install command in the README.
2. No LICENSE, or NOASSERTION.
3. Requests admin rights or full-database read/write without explaining why.
4. curl | bash in the install path, or a postinstall that downloads a binary.
5. Vague tool descriptions with free-form string parameters.
6. Delete, transfer, or send actions with no destructiveHint.
7. A year without commits, and unanswered issues.
8. API keys in URLs or logs.
9. Raw tool output concatenated into the system prompt with no isolation.
10. No versions at all — only the main branch.

Preferences by use case

Build or adopt?

Adopt by default. Build only when at least one of these is true:

• The target is an internal system with no existing server;
• The permission model is unusual (row-level, field-level) and no existing server can enforce it;
• You need two or three tools, and a large general-purpose server would only add attack surface;
• You need audit, rate limiting, or staged rollout, and the open-source option has none.

The cost of building is not the protocol — the SDKs handle transport and handshakes well. It is tests, error handling, permissions, and long-term maintenance. A 200-line server usually needs 400 lines of tests and docs before it is safe to ship.

The middle path is fork with a minimal patch: keep the upstream tool definitions, change only permissions and egress, and keep the diff small enough to rebase.

Making it stick in a team

Individuals choose with judgement; teams need rules. Four mechanisms do most of the work:

1. An allowlist — only reviewed servers reach shared configs.
2. Version pinning — upgrades go through a change process.
3. Audit logs — who called which tool, with which arguments, in which session.
4. An owner — one person per server, responsible for upgrades, incidents, and retirement.

Five common mistakes

• More stars means better. No. Stars track exposure, not use. 62.8% of servers have fewer than 100 stars.
• More tools means more capable. No. Tool definitions consume context, and selection cost grows with count.
• Official means safe. No. Official repos can also lack a license, tests, or least privilege.
• Remote is always easier. No. Remote removes deployment but adds auth, data residency, and an availability dependency.
• Install once and forget. No. 20.4% of servers are already dormant for one to two years; replacement is normal.

The minimum viable process

If you remember one thing, remember the order:

1. Five minutes on the repo: last commit, release, license, install command.
2. Ten minutes on tool definitions: count, descriptions, schemas, destructive annotations.
3. Thirty minutes of hands-on testing with read-only credentials, on three tasks.
4. Only then talk about production.

The ecosystem is still moving fast enough that today’s best choice may be unmaintained in six months. That is why the durable asset is not a particular server — it is a repeatable way to decide. With one, the next evaluation takes forty-five minutes instead of a weekend.

---

*The data in this article comes from MCPMarkets’ analysis of 5,945 public MCP servers. The full method and its limitations are in the October 2026 ecosystem report. Browse every listing in the MCP server directory, or filter by use case in developer tools.