The backend agents build with

Multimodal database, orchestration, and serving in one file

Quickstart | Documentation | CLI | Cloud | Discord | For agents

Build AI apps on video, images, audio and documents in seconds. A table holds references to your files next to regular columns: a path or URL you insert stays where it is, and files you upload or Pixeltable produces go to its media store, on local disk or in your own S3, GCS or Azure bucket. The work you want done on them, like pulling the audio out of a video, transcribing it or embedding a summary, is a column too. Insert a row and Pixeltable runs that work, stores the results and indexes them, with no embed-and-upsert job to run. Change a prompt later and only the summaries rerun; the transcripts you already paid for stay.

That replaces the stack most of us end up with: a bucket for the files, a database for the metadata, a vector database for search, an orchestrator for the model calls, an API on top, and the scripts that keep it all in sync. Here it’s one file, app.py.

What it looks like

A small video API: upload a video with an audio track, get a summary back, then search across everything you uploaded. This is all of app.py:

import pixeltable as pxt
from pixeltable.functions import openai
from pixeltable.functions.video import extract_audio
from pixeltable.serving import FastAPIRouter

TableModel = pxt.model_base()
embed = openai.embeddings.using(model='text-embedding-3-small')

class Videos(TableModel, name='videos'):
    video: pxt.Video  # the only column you insert; the rest run on insert

    audio = extract_audio(video, format='mp3')
    transcript = openai.transcriptions(audio, model='whisper-1')
    summary = openai.chat_completions(
        model='gpt-4o-mini',
        messages=[
            {'role': 'system', 'content': 'Summarize this video in two sentences.'},
            {'role': 'user', 'content': transcript.text},
        ],
    ).choices[0].message.content.astype(pxt.String)

    __indexes__ = [pxt.EmbeddingIndex(summary, embedding=embed)]

@pxt.query
def search(query: str):
    sim = Videos.summary.similarity(string=query)
    return Videos.order_by(sim, asc=False).limit(3).select(Videos.video, Videos.summary)

api = FastAPIRouter(name='video-api')
api.add_insert_route(Videos, path='/videos', uploadfile_inputs=[Videos.video], outputs=[Videos.summary])
api.add_query_route(path='/search', query=search)

pip install -U 'pixeltable[serve]' openai
# the first pxt command starts a local daemon that runs Pixeltable's Postgres and reads your environment once
export OPENAI_API_KEY=sk-...        # set it first, or run pxt daemon restart after
pxt init                            # makes this folder a project
pxt schema update app.py my_app     # creates the table and its index
pxt service update app.py my_app    # starts the service and prints its URL

# PORT is the one pxt service update printed
curl -F [email protected] http://127.0.0.1:PORT/videos
# {"summary": "..."}
curl -X POST http://127.0.0.1:PORT/search -H 'Content-Type: application/json' -d '{"query": "pricing"}'
# {"rows": [{"video": "...", "summary": "..."}]}

Each column you compute says which columns it reads, so Pixeltable knows what to run and in what order. In practice:

• Upload 100 more videos and only those 100 get processed.
• Insert 10,000 videos and the process doing the insert runs the model calls: the service for a route, your Python process for a script. It keeps many requests in flight and stays under the provider’s rate limits, read from the provider’s responses where it reports them or set in config.toml (600 requests a minute per provider if you set nothing), and it retries rate-limited requests with backoff. On Pixeltable Cloud you set CPU, memory and worker counts in the project config.
• Change the summary prompt, run pxt schema update and pxt service update again, and new uploads use it. pxt recompute my_app/videos summary then redoes the old summaries without touching the transcripts you already paid for.
• Bulk-load from Python with on_error='ignore' and a failed model call leaves its error on that row instead of stopping the batch. pxt recompute my_app/videos transcript --errors-only reruns just those rows.
• Every change to a table is versioned. pxt history shows it, pxt revert undoes it.

How it works explains the rules behind each of these.

Why we built it

We kept seeing that same stack in every multimodal AI project, and each piece is fine on its own. The trouble is in between: a sync script that hopes the bucket and the database still match, and a vector database still serving embeddings of last week’s summaries. Most of the code was plumbing, and so were most of the bugs.

We’re database people (Pixeltable comes from the creators of Apache Parquet and Impala), so our answer was to move the work into the database. The engine tracks what depends on what, so recomputing the right rows is its job, not yours.

It won’t make your models better. Media is still big, model calls still fail and evaluation is still on you. The longer version is on Why Pixeltable.

How it’s different

If you don’t want a schema, Pixeltable is the wrong tool. Coming from Pinecone, Postgres, LangGraph or Airflow? Migrate shows what each one becomes.

What you can build

• RAG over documents, with no re-chunk or re-embed scripts when you add or remove documents
• Video and image pipelines: frames, transcripts, detections
• Agents whose tool calls are rows you can query later

Rather not start from a blank file?

pxt service example --out app.py   # a working app: a table, computed columns and HTTP routes
pxt schema example                 # prints every construct a schema can use

For complete apps, browse awesome-pixeltable-apps: each one opens in GitHub Codespaces.

Install

pip install -U 'pixeltable[serve]'

Python 3.11 or later. The quickstart goes from an empty folder to your first POST. Rather start in a notebook? The 10-minute tour needs no CLI. HTTP serving covers uploads and calling your routes from TypeScript. Self-hosting covers your own FastAPI app and running without HTTP.

Pixeltable Cloud is for running this in production: the same app.py on a hosted database, with media in a managed bucket and CPU, memory and worker counts set in the project config. The Cloud guide goes from pxt login to a running service. To try it with no account, pxt new gives you a free Cloud database for about 48 hours and prints a link to keep it. Plans and prices are on the pricing page.

Using a coding agent?

npx skills add pixeltable/pixeltable-skill
# or, from the root of your repo, with only Python:
pxt skills install

The skill teaches your agent to write app.py the way it’s written above. With one file, the agent writes the whole backend and you review one diff instead of a change in every system.

Rather install a plugin? The same skill ships as one for Claude, Cursor and ChatGPT. To let the agent look at your tables while it works, add the developer MCP server. It runs on your machine, with tools to inspect tables, run queries and use a Python REPL against your local or Cloud database. AI coding agents covers setup for each tool and the hosted MCP servers for the docs and your Cloud org.

Contributing

Bug reports and PRs are welcome: CONTRIBUTING.md. Questions go to Discord or GitHub Discussions. Pixeltable is Apache 2.0.