tornlux/unrealbridge

UnrealBridge

Give your AI Agent the ability to control and edit Unreal Engine.

---

UnrealBridge is an Unreal Engine editor bridging layer built for AI Agents. It provides a typed operation surface for core scenarios such as animation-asset introspection, reactive event subscription, asset search and reference analysis, and automatic layout of Blueprint graphs. The Agent issues queries and modifications against a locally running editor instance; every change takes effect in real time, is bounded by the transaction system, and is undoable.

Highlights

• AST-based hallucination defense. Before any script reaches UE, bridge_preflight.py parses it as Python AST and validates every unreal.UnrealBridge*Library.fn(...) call against an auto-generated manifest (26 libraries × 1382 UFUNCTIONs) — catching unknown function/library names (with did-you-mean), wrong positional arg counts, unknown kwargs, and non-existent bridge-enum members without ever round-tripping to the editor. A second layer redirects raw AssetRegistry / GameplayStatics usage patterns to their bridge equivalents and tracks each returned value’s type so attribute access on a str or SoftObjectPath doesn’t silently misbehave; on a real AttributeError from a UE object the bridge calls back into UE Python, lists that live class’s reflected UPROPERTYs, and emits a paste-ready correction (auto-handles snake_case ↔ PascalCase mismatches). A third layer ships a kwargs-only Python wrapper module so positional-arg-order errors are structurally impossible. Together these dropped a fresh-context agent’s bridge-call failure rate from 24% → 16% across A/B validation runs — protection that prompt-only “look-up-before-call” rules in SKILL.md had failed to deliver.

• Deep asset-structure introspection + author-level write ops. UnrealBridgeAnimLibrary covers full queries over AnimBP state machines, AnimGraph nodes, linked layers, slots, curves, Sequence / Montage / BlendSpace, and the skeleton tree — paired with a full suite of write ops: building an ABP from scratch, adding / removing states / transitions / condition rules, creating and wiring AnimGraph nodes, auto-layout of both the state machine and AnimGraph. UnrealBridgeAssetLibrary goes beyond keyword search with forward-dependency and reverse-reference analysis, surfacing a complete dependency view to the Agent. Compared with basic CRUD wrappers or schemes that require hand-assembled reflection calls, this level of structured capability is available out of the box.

• Deliverable Control Rig and retargeting workflows. UnrealBridgeRigLibrary authors Control Rig hierarchies and connected RigVM graphs, evaluates them on transient instances, configures IK Rig solvers/goals/chains, builds IK Retargeter op stacks/mappings/poses/profiles, initializes the real retarget processor, batch-retargets animation assets, and samples output for root spikes, foot sliding/penetration, and joint discontinuities. Type/property discovery and compile/processor diagnostics keep these workflows on official editor-controller paths rather than private-data mutation.

• Deliverable Niagara VFX workflows. UnrealBridgeNiagaraLibrary discovers engine templates and scripts; authors Systems, Emitters, stack modules/inputs, User parameters, renderers, materials and bindings; compiles and audits the result; and simulates transient previews with per-emitter particle/memory readback. Production presets cover moving Ribbon/Dynamic Beam/Static Beam weapon trails, directional and radial sparks, layered explosions with a real shockwave/light layer, and material-synchronized dissolve/disintegration particles.

• Reactive event subscription. The Agent can subscribe to GAS events, attribute changes, actor lifecycle, AnimNotify, input, timers, and editor-side asset-change events. When the specified event fires, the bridge calls back proactively — no polling needed. This is a scenario that a pure request / response protocol cannot cover.

• Agent control surface at PIE runtime. UnrealBridgeGameplayLibrary provides aggregated world observation, navigation pathfinding, and input operations for movement / look / jump — suitable for AI-behavior validation, automated testing, and in-game NPC prototyping.

• Blueprint graph quality toolchain. More than just auto-layout: auto_layout_graph’s pin_aligned strategy reads live Slate geometry to align exec rails, straighten_exec_chain snaps the main rail, collapse_nodes_to_function extracts subgraphs, lint_blueprint scans by fixed rules for orphans / unnamed nodes / oversized functions / uncommented large graphs, and add_comment_box + preset palette (Section / Validation / Danger / Network / UI / Debug / Setup) partition graphs for readability. AnimGraph and state machines get dedicated auto_layout_anim_graph / auto_layout_state_machine (the latter recurses into each state’s inner graph + every transition rule graph).

• Native Python execution. 26 UnrealBridge*Library surfaces expose 1382 UFUNCTIONs in total, covering common subsystems; un-wrapped capabilities are reachable directly through the native unreal.* API. Compared to fixed-tool-list MCP schemes or reflection protocols that expose only a single call command, this design strikes a balance between flexibility and structure. Every level write op is wrapped in FScopedTransaction and supports standard Undo / Redo.

Architecture

flowchart LR
    Agent["AI Agent"]

    subgraph Host["Agent host"]
      CLI["bridge.py"]
      Pre["AST preflight(local — rejects bad callsbefore TCP send)"]
      Mani[("bridge_manifest.json26 libs · 1382 UFUNCTIONs")]
    end

    Gen["tools/gen_manifest.pyscans C++ headers"]

    subgraph UE["Unreal Editor 5.3+"]
      Disc["FUnrealBridgeDiscoveryUDP responder"]
      Server["FUnrealBridgeServerTCP · length-prefixed JSON"]
      Reactive["UnrealBridgeReactiveSubsystem+ 10 event adapters"]
      Exec["IPythonScriptPlugin::ExecPythonCommandEx(GameThread)"]
      Wrap["unreal_bridgekwargs-only wrapper(optional safer surface)"]
      Libs["26× UnrealBridge*Library"]
      Engine["UEditor · UWorld · Assets"]
    end

    Agent --> CLI
    CLI -- "AST gate" --> Pre
    Pre -. "lookup" .-> Mani

    Gen -- "writes" --> Mani
    Gen -- "writes" --> Wrap

    CLI -- "UDP probesmulticast + loopback" --> Disc
    CLI -- "TCP / JSON(port from discovery)" --> Server
    Server -- "RPC script" --> Exec
    Engine -. "delegate fires" .-> Reactive
    Reactive -- "handler script" --> Exec

    Exec -- "user code calls" --> Libs
    Exec -. "or via" .-> Wrap
    Wrap --> Libs
    Libs --> Engine

Quick Start

1. Clone the repo

git clone https://github.com//UnrealBridge.git
cd UnrealBridge

2. 🚨 Run link_agents_skills.bat (one-time)

Required for Codex / Gemini CLI / OpenCode / Cursor. Skip if you only use Claude Code.

The skill source of truth lives at .claude/skills/. This script creates an NTFS junction at .agents/skills/ so every Agent runtime following the Agent Skills open standard sees the same content. Junctions can’t be committed (Windows git limitation), so each clone has to materialize it locally — once.

link_agents_skills.bat

Mac / Linux equivalent: ln -sfn .claude/skills .agents/skills (run from repo root).

3. Install the plugin and matching skills

Pass the UE project root to the canonical sync command:

sync_project.bat D:\Path\To\YourProject

One invocation mirrors Plugin/UnrealBridge/ and the matching .claude/skills/unreal-bridge/ into the project’s plugin, .agents, and .claude destinations. Local build outputs are retained. Each destination is stamped with the source commit. The local source-repository path is stored under the project’s ignored Saved/UnrealBridge/ directory, allowing the reload helpers to repeat the same version-locked sync without publishing a machine-local path. sync_plugin.bat is retained as a legacy command-name alias and accepts the same project-root argument.

4. Build & launch

Open the .uproject and let UE rebuild the plugin automatically, or run the project’s Build.bat from the command line. Launch the editor — the plugin starts the server at PostEngineInit. You’re good once LogUnrealBridge: Listening on 127.0.0.1: shows up in the log (the port is OS-assigned; the client finds it via paired multicast + loopback probes — no manual config needed).

5. Verify

python .claude/skills/unreal-bridge/scripts/bridge.py ping
# → pong
python .claude/skills/unreal-bridge/scripts/bridge.py exec \
  "import unreal; print(unreal.UnrealBridgeLevelLibrary.get_level_summary())"

Claude Code integration (optional)

Copy the skill somewhere Claude Code can discover it:

cp -r .claude/skills/unreal-bridge ~/.claude/skills/            # user-wide
# or into the target project's own .claude/skills/

For rebuild_relaunch.py to auto-relaunch the editor, set one of:

setx UNREAL_EDITOR_EXE "C:\Program Files\Epic Games\UE_5.7\Engine\Binaries\Win64\UnrealEditor.exe"
setx UE_ROOT            "C:\Program Files\Epic Games\UE_5.7"

Quick usage

Once the skill is installed, drop any of these into a Claude Code session:

• “List every PointLight in the current level.”
• “Move the PlayerStart up by 200 units.”
• “Compile /Game/Blueprints/BP_Character and tell me whether it has errors.”
• “Show me the state machines inside /Game/Animations/ABP_Hero.”
• “Create an ABP on SK_Mannequin with an Idle / Walk / Run state machine driven by a Speed variable (>10 enters Walk, >200 enters Run), then layer a Slot + LayeredBoneBlend in the outer graph for an upper-body overlay.”

The Agent reads SKILL.md, picks the right UnrealBridge*Library function, calls it through bridge.py, and reports back.

Usage

CLI

bridge.py ping
bridge.py exec "print('hello from UE')"
bridge.py exec-file my_script.py

Flags (all optional — the common case works with no flags):

• --project= — disambiguate when >1 editors are running
• --endpoint=host:port — skip discovery, connect directly (also env UNREAL_BRIDGE_ENDPOINT)
• --token= — only when the server binds non-loopback (also env UNREAL_BRIDGE_TOKEN)
• --timeout (default 30s), --json, --discovery-timeout= (default 800)

bridge.py list-editors sends a probe and lists every editor that answered — handy for multi-editor setups.

From Python inside UE

import unreal

summary = unreal.UnrealBridgeLevelLibrary.get_level_summary()
print(summary)

lights = unreal.UnrealBridgeLevelLibrary.find_actors_by_class(
    "/Script/Engine.PointLight", 50
)
print(len(lights), "point lights")

Two reload loops

python .claude/skills/unreal-bridge/scripts/hot_reload.py        # body-only edits
python .claude/skills/unreal-bridge/scripts/rebuild_relaunch.py  # reflection changes

Installed skill copies auto-detect their containing UE project and the source repository recorded by sync_project.bat. From a source checkout, pass --project-dir D:\Path\To\YourProject; use --sync-source only when moving the source checkout invalidated the recorded path.

Bridge libraries

Protocol

Two channels:

1. UDP discovery on port 9876. The client sends the same request-id-bearing probe to LAN multicast 239.255.42.99 and local loopback 127.0.0.1, then de-duplicates replies by editor PID. Multicast preserves LAN/multi-editor discovery; loopback keeps local discovery reliable when Windows, a VPN, or a virtual NIC drops multicast loopback. Every editor replies with its bound TCP address + port + token fingerprint. Multiple editors coexist via SO_REUSEADDR.

2. TCP data on the port the editor reports in its discovery response (OS-assigned; 127.0.0.1 by default). Length-prefixed JSON:

Request :  [4-byte big-endian length][{"id","script","timeout","token?"}]
Response:  [4-byte big-endian length][{"id","success","output","error"}]
Ping    :  {"id","command":"ping"}  →  pong

Token auth kicks in automatically when the server binds non-loopback; the client reads the token from /Saved/UnrealBridge/token.txt and includes it in every request.

Scripts run on the GameThread; captured stdout and stderr are separated by the special __UB_ERR__ sentinel.

Server config (CLI / env / EditorPerProjectUserSettings.ini [UnrealBridge])

Repository layout

UnrealBridge/
├── Plugin/UnrealBridge/         # UE 5.3+ Editor plugin (C++)
│   ├── Source/UnrealBridge/     #   TCP server + bridge libraries
│   └── Content/Python/          #   Helpers auto-loaded into UE's Python env
├── .claude/skills/unreal-bridge/
│   ├── scripts/                 # bridge.py, hot_reload.py, rebuild_relaunch.py
│   └── references/              # Per-library API docs
├── docs/                        # Design notes and plans
├── tools/                       # Standalone helpers
├── sync_project.bat             # Mirror plugin + matching skills into a UE project
└── sync_plugin.bat              # Legacy alias for sync_project.bat

Requirements

• Unreal Engine 5.3+ with PythonScriptPlugin and GameplayAbilities (both ship with the engine). The matrix at tools/build_matrix.py verifies clean BuildPlugin against 5.3.2 / 5.4.4 / 5.5.4 / 5.6.1 / 5.7.1 / 5.8.0; some libraries (Chooser / PoseSearch / Material / Navigation / StateTree / Smart Object / Control Rig, IK, and Niagara), UMG MVVM, and a few standalone UFUNCTIONs require 5.7+, while lower versions retain safe StateTree/Smart Object/Rig/Niagara stubs and UMG MVVM no-ops; a handful of inline shims cover 5.3 and 5.8 — see docs/version-compatibility.md. UE 5.2 and earlier are not supported.
• Windows 10/11 — the plugin itself is portable, but paths inside the helper scripts are hard-coded Windows-style
• Python 3.9+ on PATH
• Visual Studio 2022 with the UE workload — for plugin compilation. Toolchain notes:
  • 5.5 / 5.6 / 5.7 / 5.8 work with current MSVC (verified on 14.44.35207, VS 17.14).
  • 5.3 / 5.4 require MSVC with _MSC_VER ≤ 1939 (verified on 14.38.33130, VS 17.8). Newer MSVCs hit an engine-side C4668: '__has_feature' is not defined in ConcurrentLinearAllocator.h that 5.3 / 5.4’s UBT promotes to a hard error via /we4668; 5.5+ guarded the macro and dropped the promotion. To verify 5.3 / 5.4 on a machine with both toolchains installed: temporarily pin 14.38.33130 in %APPDATA%\Unreal Engine\UnrealBuildTool\BuildConfiguration.xml, run the affected matrix entries, then restore the prior value. Do not patch engine source.
  • 5.8 source build (vs. Launcher install) may need the UBA executor disabled if the engine snapshot is missing the UbaDetours.dll content normally fetched by Setup.bat. Configure via engines.local.json’s per-engine env block: "env": { "UnrealBuildTool_BuildConfiguration__bAllowUBAExecutor": "false" }. UBT falls back to the local ParallelExecutor.
• Claude Code CLI — optional, only if you use the bundled skill

Safety

• Every level-edit op is wrapped in FScopedTransaction — Ctrl+Z in the editor reverts anything the bridge did.
• The TCP server binds to 127.0.0.1 only; it is not reachable from the network.

License

MIT — see LICENSE.

---