Configuration reference#

This page documents all configuration options for plone.pgcatalog, including Zope configuration, environment variables, GenericSetup profiles, and dependency information.

zope.conf settings#

plone.pgcatalog requires zodb-pgjsonb as its ZODB storage backend. The storage is configured in zope.conf:

%import zodb_pgjsonb

<zodb_db main>
  <pgjsonb>
    dsn dbname=zodb host=localhost port=5432 user=zodb password=zodb
  </pgjsonb>
</zodb_db>

Using environment variables in zope.conf#

ZConfig supports variable substitution via the %define directive and the ${} syntax. Combined with environment variables, this keeps secrets out of the configuration file:

%import zodb_pgjsonb

<zodb_db main>
  <pgjsonb>
    dsn dbname=${ZODB_DB:zodb} host=${ZODB_HOST:localhost} port=${ZODB_PORT:5432} user=${ZODB_USER:zodb} password=${ZODB_PASSWORD:zodb}
    blob-dir ${ZODB_BLOB_DIR:/var/plone/blobs}
  </pgjsonb>
</zodb_db>

The ${VAR:default} syntax falls back to the value after the colon when the environment variable is not set. This works for any ZConfig directive, not just dsn. See the ZConfig documentation for details.

plone.pgcatalog itself is autodiscovered via z3c.autoinclude and does not need a separate %import directive.

Environment variables#

Variable

Default

Description

PGCATALOG_BM25_LANGUAGES

(none)

Comma-separated ISO 639-1 codes, or "auto" to detect from portal_languages. Controls which per-language BM25 columns are created. Only relevant when VectorChord-BM25 extensions are installed.

PGCATALOG_TIKA_URL

(none)

Tika server URL, for example http://localhost:9998. Enables async text extraction from binary content (PDFs, Office docs, images). When set, the queue table and merge function are created at startup. See Enable Tika text extraction.

PGCATALOG_TIKA_CONTENT_TYPES

common office/PDF/image types

Comma-separated MIME types to send to Tika. Default includes PDF, MS Office, OpenDocument, RTF, and common image formats. Matching is case-insensitive and ignores MIME parameters such as ; charset=utf-8, unless an entry spells a parameter out. Checked twice: when a job is queued, and again when the worker claims it, before the blob is fetched. A claimed job whose type is not allowed becomes skipped. The standalone worker reads this same variable and must be given the same value; if unset, it uses the default list, which includes image types, and logs a warning at startup.

PGCATALOG_TIKA_METADATA_FIELDS

dc:title,dc:description,dc:subject,dc:creator,meta:keyword

Comma-separated Tika metadata keys whose values are merged into searchable_text alongside the extracted body text. Only the container document’s metadata is used, not that of embedded documents. Applies to the in-process and the standalone worker.

PGCATALOG_TIKA_INPROCESS

(none)

Set to true, 1, or yes to start the extraction worker as a daemon thread inside the Zope process. Requires PGCATALOG_TIKA_URL.

PGCATALOG_SLOW_QUERY_MS

10

Threshold in milliseconds for slow query detection. Queries exceeding this are logged as warnings and recorded in the pgcatalog_slow_queries table for analysis via the ZMI Slow Queries tab. Set to 0 to disable.

PGCATALOG_QUERY_CACHE_SIZE

200

Max cached query results per process. Set to 0 to disable. Invalidated when pgcatalog_change_seq changes (only on catalog writes, not on unrelated ZODB commits). The least recently used entry is evicted first.

PGCATALOG_QUERY_CACHE_MAX_ROWS

100000

Max rows over all cached query results per process. A cached row holds only zoid and path (about 330 bytes), so the default bounds the cache at roughly 32 MiB. The least recently used entries are evicted to make room.

PGCATALOG_QUERY_CACHE_MAX_ENTRY_ROWS

10000

Query results with more rows are not cached.

PGCATALOG_QUERY_CACHE_TTR

60

Time-to-round in seconds for datetime values in cache keys. Controls cache key granularity for effectiveRange queries. Higher values = more cache hits but slightly stale effectiveRange.

PGCATALOG_UNDO_FOLLOWUP_LIMIT

2000

Maximum number of undo follow-up queue rows processed at the end of the request that undid a transaction (only with a history-preserving storage). Rows beyond it stay hidden until the next drain; a warning gives the count. See Catalog after undo (undo.process_queue).

PGCATALOG_IDX_BATCH

500

Number of brains whose idx and meta are loaded together on the first metadata access. Only the block holding the accessed brain is loaded, so a page showing 20 of 30 000 results loads one block, not the whole result set.

PGCATALOG_PREFETCH_BATCH

100

Number of objects to prefetch when brain.getObject() is called. Set to 0 to disable. Requires zodb-pgjsonb >= 1.8.0. In addition, automatic refs prefetch on load() is registered at startup when zodb-pgjsonb >= 1.9.2 is available (uses idx IS NOT NULL filter to limit prefetch to cataloged content).

ZODB_TEST_DSN

dbname=zodb_test host=localhost port=5433 user=zodb password=zodb

DSN for test database (tests only).

BM25_TEST_DSN

dbname=zodb_test host=localhost port=5434 user=zodb password=zodb

DSN for BM25 integration tests (tests only).

Standalone worker environment variables#

These variables configure the pgcatalog-tika-worker CLI when running as a standalone process (outside Zope). The in-process worker ignores them.

The standalone worker also reads PGCATALOG_TIKA_CONTENT_TYPES and PGCATALOG_TIKA_METADATA_FIELDS from the table above. Give it the same PGCATALOG_TIKA_CONTENT_TYPES as the Zope processes.

Variable

Default

Description

TIKA_WORKER_DSN

(required)

PostgreSQL connection string.

TIKA_WORKER_URL

(required)

Tika server URL.

TIKA_WORKER_POLL_INTERVAL

5

Seconds between polls when idle (LISTEN/NOTIFY provides instant wakeup).

TIKA_WORKER_HTTP_TIMEOUT

120

Seconds to wait for a Tika HTTP response. Raise it for OCR of large scanned PDFs, which can exceed the default.

TIKA_WORKER_MAX_EMBEDDED_RESOURCES

1000

Maximum number of embedded documents Tika parses per job, sent as the X-Tika-MaxEmbeddedResources header. Bounds the size of the JSON response, which has one entry per embedded document. A response larger than 32 MiB fails the job.

TIKA_WORKER_S3_BUCKET

(none)

S3 bucket name for S3-tiered blobs.

TIKA_WORKER_S3_ENDPOINT_URL

(none)

S3 endpoint URL (for Garage or compatible).

TIKA_WORKER_S3_REGION

(none)

S3 region name.

TIKA_WORKER_S3_ACCESS_KEY

(none)

S3 access key id. When unset, boto3 falls back to its default credential chain (AWS_* environment variables, ~/.aws, IAM role).

TIKA_WORKER_S3_SECRET_KEY

(none)

S3 secret access key. See TIKA_WORKER_S3_ACCESS_KEY.

GenericSetup profile#

plone.pgcatalog ships a GenericSetup profile (default) that configures Plone to use the PostgreSQL-backed catalog.

  • setuphandlers.py: Replaces portal_catalog with PlonePGCatalogTool, preserving addon index definitions via automatic snapshot and restore.

  • metadata.xml: Profile version 1.

The profile is applied automatically when installing the add-on through the Plone control panel or via a dependency declaration in another profile.

ZCML registration#

All ZCML registrations are loaded automatically via z3c.autoinclude. The following registrations are made:

  • PlonePGCatalogTool registered as IPGCatalogTool utility.

  • IDatabaseOpenedWithRoot subscriber for startup initialization (schema creation, index registry sync, connection pool setup).

  • IPubEnd subscriber that releases request-scoped PostgreSQL connections at the end of each HTTP request.

  • IPubSuccess subscriber that recatalogs what an undo hid, after the request that did the undo has committed (see Catalog after undo (undo.process_queue)).

Python dependencies#

Package

Purpose

psycopg[binary,pool]>=3.1

PostgreSQL adapter with connection pooling.

orjson>=3.9

Fast JSONB deserialization.

Products.CMFPlone

Plone framework.

zodb-pgjsonb>=1.18.0

ZODB storage backend (provides the object_state table). v1.8.0 adds load_multiple() for batch prefetch; v1.9.2 adds register_prefetch_refs_expr() for automatic refs prefetch on load(); v1.17.0 keeps catalog columns on plain writes; v1.18.0 adds the undone() hook that keeps the catalog consistent after undo.

Optional dependencies#

Package

Extra

Purpose

VectorChord-BM25 (vchord_bm25)

—

BM25 ranking extension for PostgreSQL. Enables relevance-ranked full-text search as an alternative to tsvector ranking.

pg_tokenizer

—

Text tokenization for BM25 (language-specific stemmers and vocabulary mapping).

httpx>=0.24

tika

HTTP client for Tika communication. Required for text extraction.

boto3>=1.26

tika-s3

AWS SDK for S3-tiered blob access. Only needed when blobs are stored in S3.

Install extras with: pip install plone.pgcatalog[tika] or pip install plone.pgcatalog[tika-s3].

Console scripts#

Command

Description

pgcatalog-tika-worker

Standalone text extraction worker. Requires TIKA_WORKER_DSN and TIKA_WORKER_URL environment variables. See Enable Tika text extraction.

Docker images#

Image

Use Case

postgres:17

Standard PostgreSQL with tsvector-based full-text ranking.

tensorchord/vchord-suite:pg17-latest

PostgreSQL with VectorChord-BM25 and pg_tokenizer pre-installed.

apache/tika:latest

Apache Tika server for text extraction from PDFs, Office docs, and images. Stateless, no persistent storage needed.

PostgreSQL tuning recommendations#

For SSD-based deployments (most modern setups), set random_page_cost = 1.1 in postgresql.conf (default is 4.0, tuned for spinning disks). This helps the planner prefer index scans over bitmap scans for complex multi-field queries. Verified on production: ~2x improvement for navigation queries without affecting simple queries.

For large Plone sites, configure the ZODB cache generously:

<zodb_db main>
  cache-size 70000
  cache-size-bytes 500MB
  <pgjsonb>
    dsn ...
  </pgjsonb>
</zodb_db>

The default ZODB cache (5000 objects) is too small for sites with many content objects. A site with 14,000 events showed 5-6 second warm-cache page loads with the default, dropping to 0.8 seconds with 70,000 objects.