sharpai/deepcamera

---

---

🗺️ Roadmap

• [x] Skill architecture — pluggable SKILL.md interface for all capabilities
• [x] Skill Store UI — browse, install, and configure skills from Aegis
• [x] AI/LLM-assisted skill installation — community-contributed skills installed and configured via AI agent
• [x] GPU / NPU / CPU (AIPC) aware installation — auto-detect hardware, install matching frameworks, convert models to optimal format
• [x] Hardware environment layer — shared env_config.py for auto-detection + model optimization across NVIDIA, AMD, Apple Silicon, Intel, and CPU
• [ ] Skill development — 19 skills across 10 categories, actively expanding with community contributions

🧩 Skill Catalog

Each skill is a self-contained module with its own model, parameters, and communication protocol. See the Skill Development Guide and Platform Parameters to build your own.

✅ Ready · 🧪 Testing · 📐 Planned

Registry: All skills are indexed in skills.json for programmatic discovery.

Detection & Segmentation Skills

Detection and segmentation skills process visual data from camera feeds — detecting objects, segmenting regions, or analyzing scenes. All skills use the same JSONL stdin/stdout protocol: Aegis writes a frame to a shared volume, sends a frame event on stdin, and reads detections from stdout. Every detection skill is interchangeable from Aegis’s perspective.

graph TB
    CAM["📷 Camera Feed"] --> GOV["Frame Governor (5 FPS)"]
    GOV --> |"frame.jpg → shared volume"| PROTO["JSONL stdin/stdout Protocol"]

    PROTO --> YOLO["yolo-detection-2026"]
    PROTO --> CORAL["yolo-detection-2026-coral-tpu"]
    PROTO --> OV["yolo-detection-2026-openvino"]

    subgraph Backends["Skill Backends"]
        YOLO --> ENV["env_config.py auto-detect"]
        ENV --> TRT["NVIDIA → TensorRT"]
        ENV --> CML["Apple Silicon → CoreML"]
        ENV --> OVIR["Intel → OpenVINO IR"]
        ENV --> ONNX["AMD / CPU → ONNX"]

        CORAL --> LITERT["ai-edge-litert + libedgetpu"]
        LITERT --> TPU["Coral USB → Edge TPU delegate"]
        LITERT --> CPU1["No TPU → CPU fallback"]

        OV --> OVSDK["OpenVINO SDK"]
        OVSDK --> NCS2["Intel NCS2 USB"]
        OVSDK --> IGPU["Intel iGPU / Arc"]
        OVSDK --> CPU2["CPU fallback"]
    end

    YOLO --> |"stdout: detections"| AEGIS["Aegis IPC → Live Overlay + Alerts"]
    CORAL --> |"stdout: detections"| AEGIS
    OV --> |"stdout: detections"| AEGIS

• Unified protocol — each skill creates its own Python venv or Docker container, but Aegis sees the same JSONL interface regardless of backend
• Coral TPU uses ai-edge-litert (LiteRT) with the libedgetpu delegate — supports Python 3.9–3.13 on Linux, macOS, and Windows
• Same output — Aegis sees identical JSONL from all skills, so detection overlays, alerts, and forensic analysis work with any backend

LLM-Assisted Skill Installation

Skills are installed by an autonomous LLM deployment agent — not by brittle shell scripts. When you click “Install” in Aegis, a focused mini-agent session reads the skill’s SKILL.md manifest and figures out what to do:

1. Probe — reads SKILL.md, requirements.txt, and package.json to understand what the skill needs
2. Detect hardware — checks for NVIDIA (CUDA), AMD (ROCm), Apple Silicon (MPS), Intel (OpenVINO), or CPU-only
3. Install — runs the right commands (pip install, npm install, system packages) with the correct backend-specific dependencies
4. Verify — runs a smoke test to confirm the skill loads before marking it complete
5. Determine launch command — figures out the exact run_command to start the skill and saves it to the registry

This means community-contributed skills don’t need a bespoke installer — the LLM reads the manifest and adapts to whatever hardware you have. If something fails, it reads the error output and tries to fix it autonomously.

🚀 Getting Started with SharpAI Aegis

The easiest way to run DeepCamera’s AI skills. Aegis connects everything — cameras, models, skills, and you.

• 📷 Connect cameras in seconds — add RTSP/ONVIF cameras, webcams, or iPhone cameras for a quick test
• 🤖 Built-in local LLM & VLM — llama-server included, no separate setup needed
• 📦 One-click skill deployment — install skills from the catalog with AI-assisted troubleshooting
• 🔽 One-click HuggingFace downloads — browse and run Qwen, DeepSeek, SmolVLM, LLaVA, MiniCPM-V
• 📊 Find the best VLM for your machine — benchmark models on your own hardware with HomeSec-Bench
• 💬 Talk to your guard — via Telegram, Discord, or Slack. Ask what happened, tell it what to watch for, get AI-reasoned answers with footage.

🎯 YOLO 2026 — Real-Time Object Detection

State-of-the-art detection running locally on any hardware, fully integrated as a DeepCamera skill.

YOLO26 Models

YOLO26 (Jan 2026) eliminates NMS and DFL for cleaner exports and lower latency. Pick the size that fits your hardware:

All models detect 80+ COCO classes: people, vehicles, animals, everyday objects.

Hardware Acceleration

The shared env_config.py auto-detects your GPU and converts the model to the fastest native format — zero manual setup:

Aegis Skill Integration

Detection runs as a parallel pipeline alongside VLM analysis — never blocks your AI agent:

Camera → Frame Governor → detect.py (JSONL) → Aegis IPC → Live Overlay
                5 FPS           ↓
                          perf_stats (p50/p95/p99 latency)

• 🖱️ Click to setup — one button in Aegis installs everything, no terminal needed
• 🤖 AI-driven environment config — autonomous agent detects your GPU, installs the right framework (CUDA/ROCm/CoreML/OpenVINO), converts models, and verifies the setup
• 📺 Live bounding boxes — detection results rendered as overlays on RTSP camera streams
• 📊 Built-in performance profiling — aggregate latency stats (p50/p95/p99) emitted every 50 frames
• ⚡ Auto start — set auto_start: true to begin detecting when Aegis launches

📖 Full Skill Documentation →

🔒 Privacy — Depth Map Anonymization

Watch your cameras without seeing faces, clothing, or identities. The depth-estimation skill transforms live feeds into colorized depth maps using Depth Anything v2 — warm colors for nearby objects, cool colors for distant ones.

Camera Frame ──→ Depth Anything v2 ──→ Colorized Depth Map ──→ Aegis Overlay
   (live)          (0.5 FPS)           warm=near, cool=far      (privacy on)

• 🛡️ Full anonymization — depth_only mode hides all visual identity while preserving spatial activity
• 🎨 Overlay mode — blend depth on top of original feed with adjustable opacity
• ⚡ Rate-limited — 0.5 FPS frontend capture + backend scheduler keeps GPU load minimal
• 🧩 Extensible — new privacy skills (blur, pixelation, silhouette) can subclass TransformSkillBase

Runs on the same hardware acceleration stack as YOLO detection — CUDA, MPS, ROCm, OpenVINO, or CPU.

📖 Full Skill Documentation → · 📖 README →

📊 HomeSec-Bench — How Secure Is Your Local AI?

HomeSec-Bench is a 143-test security benchmark that measures how well your local AI performs as a security guard. It tests what matters: Can it detect a person in fog? Classify a break-in vs. a delivery? Resist prompt injection? Route alerts correctly at 3 AM?

Run it on your own hardware to know exactly where your setup stands.

Results: Local vs. Cloud vs. Hybrid

Running on a Mac M1 Mini 8GB: local Qwen3.5-4B scores 39/54 (72%), cloud GPT-5.2 scores 46/48 (96%), and the hybrid config reaches 53/54 (98%). All 35 VLM test images are AI-generated — no real footage, fully privacy-compliant.

📄 Read the Paper · 🔬 Run It Yourself · 📋 Test Scenarios

---

📦 More Applications

---

🤝 Support & Community

• 💬 Slack Community — help, discussions, and camera setup assistance
• 🐛 GitHub Issues — technical support and bug reports
• 🏢 Commercial Support — pipeline optimization, custom models, edge deployment

Contributions