# pgvector inside the daemon

The goal was to run **pgvector's actual math**, not a reimplementation, so results match pgvector by construction. pgvector is a PostgreSQL extension written in C; Arkivector compiles its code into the daemon.

## What is reused

From pgvector 0.8.7 (`daemon/src/pgvector/`):

- the four types (`vector`, `halfvec`, `sparsevec`, `bit`): structs, text input/output with pgvector's error messages, casts and normalization;
- every distance kernel (L2, inner product, cosine, L1, Hamming, Jaccard), including the half-precision conversion;
- the HNSW graph: element and neighbour structures, `HnswSearchLayer`, neighbour selection, the in-memory build (`InsertTupleInMemory`) and the 0.8 iterative scan.

Disk and SQL code (pages, WAL, buffer manager, the index access method) is compiled out.

## Reuse modes

Every copied file is listed in [`PROVENANCE.md`](https://github.com/DikshitRJ/arkivector/blob/main/daemon/src/pgvector/PROVENANCE.md) with a mode:

| Mode | Meaning |
| :---: | :--- |
| V | Verbatim copy, byte for byte |
| F | Selected functions copied unchanged (generated by `tools/pgvector_copy.py`) |
| S | Copy with shims: only `#ifndef ARKIVECTOR` guards around disk code and lines marked `/* ARKIVECTOR: … */` |
| P | Algorithm port (e.g. the in-memory scan driver `hnswscan_mem.c`) |

CI job "Copied pgvector/PostgreSQL code unmodified" (`tools/check_provenance.py`) checks V files byte for byte, regenerates F files and diffs S files against upstream after removing the marked regions. Updating pgvector means re-copying and rerunning the parity suite.

## The compatibility layer

pgvector expects PostgreSQL around it. `daemon/src/pgcompat/` provides it:

| PostgreSQL API | In Arkivector |
| :--- | :--- |
| `palloc`, memory contexts | Per-request and per-batch arenas |
| `ereport(ERROR)`, `PG_TRY` | `setjmp`/`longjmp` per thread; the error becomes an API error, and LWLocks taken inside the `PG_TRY` are released on unwind |
| `fmgr` (`DirectFunctionCall`, `FunctionCall`) | Calls pgvector functions the way PostgreSQL does |
| LWLocks, spinlocks | pthread-based locks with a per-thread held-lock list |
| `List`, `pairingheap`, `simplehash`, `relptr`, float formatting | Verbatim PostgreSQL 17 source files |

Upstream PostgreSQL files are also checked by the provenance job.

## Proving parity

The parity suite (`daemon/test/parity/`) runs about 2,000 random pairs per type and operator through both the daemon's code and a real PostgreSQL with pgvector 0.8.7, and checks that every distance is bit-identical or within float32 summation rounding, and that text output and error messages are byte-identical. It runs in CI on every change.

## Licences

The copied code keeps pgvector's PostgreSQL License (`LICENSE.pgvector`) and PostgreSQL's (`LICENSE.postgresql`); `arkivectord --licenses` prints them.

---
Web page: https://arkivector.dikshitrj.in/docs/architecture/pgvector · All docs on one page: https://arkivector.dikshitrj.in/docs.md · Whole site: https://arkivector.dikshitrj.in/agents.md