feat(extension-pgvector): support variable length embeddings - #30234
feat(extension-pgvector): support variable length embeddings#30234RyanGarber wants to merge 2 commits into
Conversation
Signed-off-by: Ryan <ryanmichaelgarber@gmail.com>
|
|
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yml Review profile: CHILL Plan: Advanced Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
Included review availability: Your plan provides up to 10 included reviews per hour; 8 remain after this review. 📝 WalkthroughWalkthroughThe pgvector extension now supports Changespgvector variable-dimension support
Priority: ⬇️ Low Estimated code review effort: 3 (Moderate) | ~20 minutes Merge Risk: ⚪ Minimal · up to The change adds optional dimensions for pgvector columns while retaining fixed-dimension behavior. No unresolved merge-readiness risk is identified. Sequence Diagram(s)sequenceDiagram
participant Author as vector()
participant Descriptor as PgVectorDescriptor
participant Column as pgVectorColumn()
participant Codec as PgVectorCodec
Author->>Descriptor: Create empty typeParams
Descriptor->>Column: Provide undimensioned column metadata
Column->>Codec: Create codec with undefined length
Codec-->>Column: Encode and decode vectors of varying lengths
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Full details: Docstring CoverageExplanation Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 5 functions across 7 files. (1 skipped: 1 unsupported.)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
🟡 Changes recommended
Variable-dimension vectors currently skip dimension bounds validation in the runtime codec, which can allow oversized/invalid payloads before Postgres rejects them.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds variable-length pgvector column authoring to Prisma Next’s pgvector extension by introducing an overload that maps to Postgres’ undimensioned vector type, while preserving existing fixed-dimension behavior and validation.
Changes:
- Add
vector()/pgVectorColumn()authoring surfaces for undimensioned vectors (nolengthintypeParams), plus descriptor/runtime support for optionallength. - Update codec typing/rendering so emitted
contract.d.tscan referenceVector(undimensioned) orVector<N>(dimensioned). - Expand coverage with new unit/integration tests and update READMEs for the new optionality.
File summaries
| File | Description |
|---|---|
| packages/9-public/@prisma/orm-extension-pgvector/test/extension-tarball.test.ts | Adds an installed-package smoke test ensuring variable-dimension vectors work end-to-end. |
| packages/9-public/@prisma/orm-extension-pgvector/README.md | Updates public entrypoint docs to mention vector() and variable dimensions. |
| packages/3-extensions/pgvector/test/variable-dimensions.test.ts | New tests covering authoring, params validation, native type expansion, and encode/decode for variable dimensions. |
| packages/3-extensions/pgvector/test/codec-render-output-type.test.ts | Adds coverage for renderOutputType when length is omitted (render Vector). |
| packages/3-extensions/pgvector/test/codec-conformance.integration.test.ts | Adds integration conformance cases for {} typeParams (variable-dimension vectors). |
| packages/3-extensions/pgvector/src/exports/column-types.ts | Introduces the vector() overload and keeps existing vector(n) validation for fixed dimensions. |
| packages/3-extensions/pgvector/src/core/codecs.ts | Makes length optional in params schema and codec; renders Vector vs Vector<N> accordingly; adds variable-dim column helper. |
| packages/3-extensions/pgvector/src/core/authoring.ts | Makes the PSL authoring length argument optional for pgvector.Vector. |
| packages/3-extensions/pgvector/README.md | Updates internal docs to describe the new optional length behavior and contract typing output. |
Review details
Suppressed comments (1)
packages/3-extensions/pgvector/src/core/codecs.ts:114
- When
lengthis omitted (variable-dimension vectors), the runtime codec no longer enforces pgvector’s dimension bounds. This makes it possible to encode/decode empty vectors or extremely large vectors (e.g. > VECTOR_MAX_DIM) that will likely be rejected by Postgres and can create very large wire payloads before failing. Consider validatingvalue.lengthis within [1, VECTOR_MAX_DIM] even whenthis.lengthis undefined, while still skipping the exact-length match in that mode.
if (this.length !== undefined && value.length !== this.length) {
throw pgVectorError(
code,
`Vector length mismatch: expected ${this.length}, got ${value.length}`,
{
- Files reviewed: 9/9 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…EADME Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Linked issue
n/a
Summary
The pgvector extension currently requires a fixed length for all pgvector columns, but many real-world use cases involve lengths that cannot be known. For example, many apps today allow users to supply their own API keys and models. The requirement of a length at contract time prevents that use case entirely.
This commit simply adds a second overload to the existing vector column type that cleanly maps to the non-dimensioned
vectortype in Postgres. All existing validation for fixed-length vectors remain in place, so existing behavior is completely unchanged. This change simply adds a new option.Testing performed
Skill update
Checklist
git commit -s) per the DCO. The DCO status check will block merge if any commit is missing aSigned-off-by:trailer.n/aif the change is doc-only / refactor with no behavioural delta).TML-NNNN: <sentence-case title>form (Linear ticket prefix + concise title naming the concrete deliverable). See.claude/skills/create-pr/SKILL.mdfor the full convention.n/a — internal only).Notes for the reviewer
This change is mission-critical for my work. Until it is released, I must use my own custom fork of this extension for migrations to work. Given it also just aligns the extension type closer to the official Postgres type and has zero breaking changes, I very much hope to see it in the coming RCs!
Summary by CodeRabbit
New Features
vector().vector(N)continue to validate vector lengths.Documentation