FA

fancydirty/ai-companion-benchmarks-2026

Developer tools
0 stars 0 forks Quality 30 Trend 8

Open dataset and reproducible evaluation logs from (https://thecompanionlab.com/).

Overview

Open dataset and reproducible evaluation logs from (https://thecompanionlab.com/). Between 16 and 27 October 2026, we tested (Secrets AI, Nomi AI, SpicyChat AI, Ourdream AI, Kindroid AI, Xotic AI, GirlfriendGPT, Candy AI, DreamGF, and EroLove) using: 1. in identical order (empathy/venting, planting two memory facts, character-specific humor, pushback against a terrible financial idea, a memory quiz three turns later, and a contextual selfie request). 2. , recording the exact merchant descriptor strings printed on the card notifications, renewal pricing, token/coin meters, and cancellation dark patterns. 3. to identify the underlying LLMs, image checkpoints (Z-Image Turbo, Seedream, WAI-Illustrious-SDXL), and video engines (Wan, OmniHuman, Oh.xyz). 4. rebuilt screen-by-screen so readers can inspect each app's onboarding before handing over an email address. (Nine, not ten: EroLove has no character creator, which is a product decision rather than a gap.) 5.

README

AI Companion App Benchmarks, Bank Statement Descriptors & LLM Stack Dataset (2026)

Open dataset and reproducible evaluation logs from The Companion Lab (https://thecompanionlab.com/).

Between 16 and 27 October 2026, we tested 10 AI companion and AI girlfriend apps (Secrets AI, Nomi AI, SpicyChat AI, Ourdream AI, Kindroid AI, Xotic AI, GirlfriendGPT, Candy AI, DreamGF, and EroLove) using:

  1. The same 6 standardized conversation prompts in identical order (empathy/venting, planting two memory facts, character-specific humor, pushback against a terrible financial idea, a memory quiz three turns later, and a contextual selfie request).
  2. 8 paid subscriptions on our own bank card, recording the exact merchant descriptor strings printed on the card notifications, renewal pricing, token/coin meters, and cancellation dark patterns.
  3. Client-side JavaScript bundle and image/video EXIF inspection to identify the underlying LLMs, image checkpoints (Z-Image Turbo, Seedream, WAI-Illustrious-SDXL), and video engines (Wan, OmniHuman, Oh.xyz).
  4. 9 interactive, zero-signup character creator replicas rebuilt screen-by-screen so readers can inspect each app’s onboarding before handing over an email address. (Nine, not ten: EroLove has no character creator, which is a product decision rather than a gap.)
  5. A read of EroLove’s 179 KB inline chat JavaScript and its undocumented API (23 Oct 2026), followed by a 48-hour re-test on 26 Oct 2026 confirming that EroLove patched all four initial defects we reported.

Quick Navigation & Primary Hubs


1. Real Bank Statement Descriptors (bank-statement-descriptors.csv)

Every merchant string below comes from a real payment made on our own card in October 2026.

Merchant Descriptor on Card Notice App Charged Checkout Promise Discretion & Billing Page Link Safety & Cancellation Audit
S LABS INC 604-3295224 CA US Secrets AI $19.99 Appears as S LABS INC. on your statement S LABS INC Statement Evidence Is Secrets AI Safe & Legit?
NDAI.CC 514-2860020 DE US SpicyChat AI $19.95 (Nothing stated on plans page) NDAI.CC Statement Evidence Is SpicyChat AI Safe & Legit?
NDAI.CHAT 514-2860020 DE US GirlfriendGPT $15.00 NDAI.CHAT NDAI.CHAT Statement Evidence Is GirlfriendGPT Safe & Legit?
DREAMSTUDIOS.CO Limassol CY Ourdream AI $19.99 Dream Studio (in site code) DREAMSTUDIOS.CO Statement Evidence Is Ourdream AI Safe & Legit?
EverAI candy.ai MT Candy AI $9.99 EverAI (omits candy.ai domain) EverAI candy.ai Statement Evidence Is Candy AI Safe & Legit?
Dream AI dreamgf.ai RO DreamGF $12.99 DreamAI (rebills at $25.98/mo) Dream AI dreamgf.ai Statement Evidence Is DreamGF Safe & Legit?
Xotic.ai Tallinn EE Xotic AI $24.99 Xotic AI Xotic.ai Statement Evidence Is Xotic AI Safe & Legit?

Note on the three apps without a descriptor: We tested Nomi AI on its free plan (Is Nomi AI Safe?); we paid $13.99 on the web for Kindroid AI Standard (Is Kindroid AI Safe?) without recording the card notification line; and EroLove gave us Premium for this test, so we never paid and no charge appeared (Is EroLove AI Safe?). EroLove also takes crypto only, so its subscription does not auto-renew.


2. October 2026 Ranked Scorecard (benchmarks-october-2026.json)

Rank App Overall Conversation Creator Free Plan Price Clarity Monthly Price (Web) No-Signup Creator Replica
#1 Secrets AI 8.3 / 10 9 9 7 8 $19.99 ($9.17/mo yr) Try Secrets Creator
#2 Nomi AI 7.8 / 10 8 6 9 8 $15.99 ($8.33/mo yr) Try Nomi Creator
#3 SpicyChat AI 7.5 / 10 7 7 8 8 $19.95 (no coin meter) Try SpicyChat Creator
#4 Ourdream AI 7.0 / 10 7 9 6 6 $19.99 ($9.99/mo yr) Try Ourdream Creator
#5 Kindroid AI 6.8 / 10 9 7 5 6 $13.99 web / $15.99 iOS Try Kindroid Creator
#6 Xotic AI 5.5 / 10 8 7 2 5 $24.99 (1,200 XOT) Try Xotic Creator
#7 GirlfriendGPT 5.5 / 10 8 7 3 4 $15.00 (5,000 msgs) Try GirlfriendGPT Creator
#8 Candy AI 4.8 / 10 5 8 2 4 $9.99 first mo, then $13.99 Try Candy Creator
#9 DreamGF 3.5 / 10 5 4 2 3 $12.99 first mo, then $25.98 Try DreamGF Creator
#10 EroLove 7.0 / 10 8 n/a 5 8 $12.99 ($6.50/mo yr), crypto only No creator, by design

Ranking rule, stated because row #10 looks out of order. An app we could score in fewer than four areas is always listed last, whatever its average. EroLove’s 7.0 is the mean of three scores (8, 5, 8 after the 26 Oct re-test), not four, because it has no character creator at all — the cast is written and closed. Scoring that as zero would read a deliberate product decision as a failure, so the app goes last and says why, rather than being silently compared on an axis it does not have.


3. Head-to-Head 1v1 Comparisons (/compare/)

Each comparison page includes a 12-row feature and pricing table with explicit winner calls plus the verbatim 6-message chat transcripts side by side:


4. Competitor & Unfiltered Alternatives Guides (/alternatives/)

For readers switching from filtered apps or apps with aggressive token meters, we mapped our 10 tested apps against the specific pain points of each platform:


5. Deep-Dive Technical Research Articles (/blog/)


6. EroLove: 179 KB Client Bundle Teardown (23 Oct) & 48-Hour Fix Verification (26 Oct 2026)

Update (26 October 2026 — All 4 Issues Fixed & Re-Tested): Within 48 hours of our initial 23 October teardown, EroLove’s engineering team shipped fixes for all four defects below. We re-ran our tests on a live account on 26 October 2026 and verified every fix in both the UI and the network responses. With the 800/day message cap now printed clearly on the plans page, EroLove’s price-clarity score moved from 7/10 to 8/10, raising its overall score from 6.7/10 to 7.0/10. EroLove also tied Secrets AI at the top of our 10-app persistent memory test for its editable, plain-English What they know / Our story memory panel.

EroLove is the only app here we originally tested by reading its client code (179 KB on 23 Oct, 177 KB on 26 Oct after stripping dormant upload code) as well as clicking through the interface:

Initial Finding (23 Oct 2026) What We Saw on 23 Oct 26 Oct 2026 Patch & Our Re-Test Verification
1. 30 pages promised a user-upload feature that returned 404 POST /api/chat/upload returned 404 {"error":"not_found"} (retired 14 Oct), while ~30 FAQ blocks across category, comparison, scenario pages and /privacy still promised user photo uploads. Fixed & Verified (26 Oct): EroLove removed the stale upload FAQ copy across all 9 languages (HTML and .md twins) and stripped sendPhoto / fileToJpegDataUrl / attachBtn from the client bundle.
2. Memory extraction drifted into Estonian/French In an all-English session (default_language: "en"), Anna’s extracted facts came back in Estonian and Rae’s in French, while editing and pinning worked in English. Fixed & Verified (26 Oct): Extractor now pins the session language. In our 5-message English re-test with Yuna on 26 Oct, all extracted memory notes and Our story entries were written in clean English.
3. A content-blocked photo showed the same message as a server outage 2 of 5 photo requests returned {"status":"blocked"}, which the UI mapped to the generic "Couldn't take a photo right now — try a bit later" string (0 sparks charged). Fixed & Verified (26 Oct): Client bundle now branches s.status === "blocked" ? "photoBlocked" : "photoFail", displaying "This photo goes against our content rules, so it wasn't made. No sparks spent — ask for a different one."
4. “Unlimited messages” had an unprinted 800/day cap /api/me carried dailyLimit: 800 on Premium while the plans page said "Unlimited messages". Fixed & Verified (26 Oct): /pricing, /account/plan, /press and /llms.txt now explicitly state "Unlimited messages (up to 800 a day)". Our Rae + Kit room test also confirmed a multi-companion room turn increments usedToday by only 1.

Full teardown and 26 October re-test logs: EroLove Review: The Best Memory I’ve Used, and Four Bugs They Fixed in 48 Hours.


7. Machine-readable access

Every page on the site has a plain-markdown twin at the same path plus .md, and the whole dataset is available as one file. Both are generated from the same source as the HTML, so a crawler cannot receive a version of these findings without the caveats attached.

What Where
Summary with a structured Facts block https://thecompanionlab.com/llms.txt
Unabridged dataset: every score with its observation, every price row with its source https://thecompanionlab.com/llms-full.txt
Markdown twin of any page append .md — e.g. the EroLove review as markdown
This repo’s data files bank-statement-descriptors.csv, benchmarks-october-2026.json

robots.txt names 30 AI crawlers individually (including CCBot, OAI-SearchBot, ClaudeBot and PerplexityBot) with Content-Signal: search=yes, ai-train=yes, ai-input=yes.


License

Data files (bank-statement-descriptors.csv, benchmarks-october-2026.json) are released under CC BY 4.0. Please attribute The Companion Lab.

View the full README on GitHub

Install

This server does not publish a one-line install command.

Open the repository installation guide New to MCP? Start with the mcpskill.org guide →