Tutorial: migrate from ZCatalog to plone.pgcatalog

What you will do

Take an existing Plone 6 site that uses the standard ZCatalog and migrate all catalog data to PostgreSQL. After this tutorial, your site will use plone.pgcatalog for all catalog operations while the standard Plone API continues to work unchanged.

Prerequisites

  • An existing Plone 6 site

  • The site must already use zodb-pgjsonb as its ZODB storage backend

  • PostgreSQL 14+ accessible (tested with 17)

  • Admin access to the Plone site

Note

plone.pgcatalog requires zodb-pgjsonb because it stores catalog data as extra columns in the same object_state table that holds ZODB objects. If your site uses FileStorage or RelStorage with MySQL, you will need to migrate to zodb-pgjsonb first.

Step 1: back up your database

Before making any changes, create a full backup of your PostgreSQL database. You will use this to roll back if anything goes wrong.

pg_dump -h localhost -p 5433 -U zodb zodb > backup_before_pgcatalog.sql

Adjust the host, port, and credentials to match your environment.

Step 2: install plone.pgcatalog

uv pip install plone.pgcatalog

The package is autodiscovered by Plone via z3c.autoinclude. No ZCML slug or manual include is needed.

Step 3: apply the GenericSetup profile

You have two options: use the Plone web UI, or run a zconsole script. The result is the same.

Option A: via the Plone add-on installer

Log in as a Manager user 2. Go to Site Setup > Add-ons 3. Find plone.pgcatalog in the list and click Install

Option B: via a zconsole script

Create a file called migrate.py:

import transaction
from zope.component.hooks import setSite
from plone import api

SITE_ID = "Plone"  # adjust to your site ID

site = app[SITE_ID]  # noqa: F821 -- app injected by zconsole
setSite(site)

catalog = api.portal.get_tool("portal_catalog")
print(f"Before: catalog class = {catalog.__class__.__name__}")
print(f"Before: {len(catalog)} objects indexed")

setup = api.portal.get_tool("portal_setup")
setup.runAllImportStepsFromProfile("profile-plone.pgcatalog:default")
transaction.commit()

# Re-fetch -- the catalog object was replaced by the install step
catalog = api.portal.get_tool("portal_catalog")
print(f"After:  catalog class = {catalog.__class__.__name__}")
print(f"After:  {len(list(catalog.indexes()))} ZCatalog indexes registered")

Run it:

.venv/bin/zconsole run instance/etc/zope.conf migrate.py

What the profile does

The GenericSetup profile performs these changes:

  1. Snapshots existing indexes – captures all index definitions and metadata columns from the current catalog, including any addon-provided indexes

  2. Replaces portal_catalog – replaces the standard catalog tool with PlonePGCatalogTool (based on UniqueObject + Folder, not ZCatalog)

  3. Restores catalog indexes – re-applies essential Plone indexes, then restores addon indexes from the snapshot so no index definitions are lost

  4. Removes orphaned ZCTextIndex lexicons – no longer needed since full-text search is handled by PostgreSQL tsvector

  5. Applies DDL schema – creates the necessary columns, GIN indexes, and PostgreSQL functions on the object_state table

  6. Rebuilds the catalog – runs clearFindAndRebuild() to traverse the site and index all existing content into PostgreSQL

Tip

The old ZCatalog’s BTree data (the actual indexed values) becomes unreferenced in ZODB after migration. Run a ZODB pack after migration to reclaim the space.

Step 4: verify the rebuild

The install step automatically runs clearFindAndRebuild(), so all existing content should already be indexed in PostgreSQL. If the automatic rebuild failed (check the Zope log for warnings), you can run it manually.

Create a file called rebuild.py:

import time
import transaction
from zope.component.hooks import setSite
from AccessControl.SecurityManagement import newSecurityManager
from AccessControl.users import SimpleUser
from plone import api

SITE_ID = "Plone"  # adjust to your site ID

site = app[SITE_ID]  # noqa: F821
setSite(site)

admin = SimpleUser("admin", "", ["Manager"], [])
newSecurityManager(None, admin)

catalog = api.portal.get_tool("portal_catalog")

print("Rebuilding catalog (clearFindAndRebuild) ...")
t0 = time.time()
catalog.clearFindAndRebuild()
transaction.commit()
elapsed = time.time() - t0
print(f"  Completed in {elapsed:.1f}s")
print(f"  {len(catalog)} objects indexed")

Run it:

.venv/bin/zconsole run instance/etc/zope.conf rebuild.py

Note

Expected timing: approximately 15 ms per object. A site with 10,000 objects takes about 2.5 minutes. Large sites with 100,000+ objects may take 25 minutes or more.

Step 5: verify the migration

Check object counts

Start Zope and visit portal_catalog in the ZMI (Zope Management Interface). The Catalog tab should show:

  • The correct number of indexed objects

  • All expected indexes listed

  • All expected metadata columns listed

Test search from Python

Create a file called verify.py:

from zope.component.hooks import setSite
from plone import api

SITE_ID = "Plone"

site = app[SITE_ID]  # noqa: F821
setSite(site)

catalog = api.portal.get_tool("portal_catalog")

results = catalog(portal_type="Document")
print(f"Found {len(results)} documents")

results = catalog(SearchableText="test")
print(f"Full-text search found {len(results)} results")

# Verify brains have expected attributes
if results:
    brain = results[0]
    print(f"  Title: {brain.Title}")
    print(f"  Path:  {brain.getPath()}")

Run it:

.venv/bin/zconsole run instance/etc/zope.conf verify.py

Compare with PostgreSQL

Connect to PostgreSQL directly and confirm the counts match:

SELECT COUNT(*) FROM object_state WHERE path IS NOT NULL;

This count should match the object count shown in the ZMI.

Rollback strategy

If anything goes wrong, you can restore the original state.

  1. Restore the database backup:

    psql -h localhost -p 5433 -U zodb zodb < backup_before_pgcatalog.sql

  2. Uninstall plone.pgcatalog:

    uv pip uninstall plone-pgcatalog

  3. Restart Zope – the original CatalogTool class will be restored from the database, and the site will operate as before.

Troubleshooting

If you run into issues after migration, here are the most common problems and how to resolve them.

“NotImplementedError: PlonePGCatalogTool.getAllBrains() is not supported”

Code that calls ZCatalog internal methods needs updating. PlonePGCatalogTool does not inherit from ZCatalog and blocks methods that have no PostgreSQL equivalent. See ZCatalog compatibility for blocked methods and their replacements.

Object count is zero after migration

The install step runs clearFindAndRebuild() automatically, but if that failed (check the Zope log), run it manually as described in Step 4 above.

Addon indexes are missing

The install step snapshots existing indexes before replacing the catalog. If addons were installed after applying the plone.pgcatalog profile, their indexes will not be in the snapshot. To fix this, either:

  • Re-apply the addon’s catalog.xml via GenericSetup, or

  • Restart Zope – sync_from_catalog() runs at startup and discovers indexes from portal_catalog._catalog.indexes

“DeprecationWarning: portal_catalog.search() is deprecated”

Replace catalog.search(...) with catalog.searchResults(...). See ZCatalog compatibility for all deprecated methods.

Full-text search returns no results

Run clearFindAndRebuild() to populate the searchable_text tsvector column. Individual reindexObject() calls update the idx JSONB but only rebuild searchable_text when all indexes are reindexed.

Permission errors after migration

plone.pgcatalog declares the same permissions as ZCatalog, so existing role mappings carry over. If you still see Unauthorized, check the Permissions page to verify which permission protects the method you are calling.

What you learned

  • Migration requires zodb-pgjsonb as the ZODB backend (same PostgreSQL database)

  • The GenericSetup profile replaces the catalog tool class and applies DDL schema

  • clearFindAndRebuild() populates PostgreSQL from existing content objects

  • The standard Plone catalog API works unchanged after migration

  • A database backup provides a safe rollback path

Next steps