Table of Contents
Vector Search: text and image similarity
Vector Search finds galleries by the meaning of their text or by a similar image. For example, a text query may find galleries about a house near the sea even when the descriptions use different words. Image similarity lets you show thumbs that look like a selected image.
The workflow has two parts:
- Create a vector. The model service converts text or an image into a vector, a set of numbers, and returns it to your script installation. The script saves the result in its own local MariaDB database.
- Find similar vectors. Your script installation searches the saved vectors in its own database, applying the block filters. The model service neither stores your gallery database nor performs this search.
A new text search phrase first needs a vector: the script obtains it from the model service, then searches for similar vectors in the local database. An external service adds network latency to the first step. Searching with an existing gallery or thumb vector uses your database without contacting the service. Perceived similarity depends on the model and the source data.
Requirements
- MariaDB 11.8 or newer. Ordinary rotation continues to work on an unsupported database, and the settings explain why Vector Search is unavailable.
- A running model service that the script can reach over HTTP or HTTPS. It can run on the PHP server or on a separate server; Python is not required on the PHP server.
- Ready vectors for the galleries or thumbs you want to find.
Running your own service is preferable: the shared service's per-IP and request-size restrictions do not apply, and you control your own limits. Hosting it on the PHP server removes external network latency; with sufficient resources, vector creation will usually be faster. Computation speed still depends on CPU, memory and load. When choosing a server, verify that AVX2 is exposed to the operating system and container.
The service is supplied separately as the services/embedding/ directory. To build it from that directory, use Docker and Compose:
docker compose build docker compose up -d \ --no-build
The initial build needs internet access; the completed image contains the models for subsequent startup. The supplied Compose file publishes port 8080 only on 127.0.0.1. For access through a domain or from another server, ask the administrator to configure nginx as a reverse proxy: external HTTPS on port 443 forwards to the container's internal port, for example 127.0.0.1:8080. This is the recommended setup for external access: nginx handles the certificate and access limits, while the container port remains local. Set Model Service URL to the nginx HTTPS address without /embed. When PHP and the service share a server, direct access through http://127.0.0.1:8080 is also available.
Hosted service vector.smartcj.com
You can connect the script to https://vector.smartcj.com instead of running your own Docker service. You do not need to install Python or models on your server. The service currently accepts requests without API keys.
Set Model Service URL to https://vector.smartcj.com without /embed. Click Test connection, check the result for all three models and save the settings. Changing only the service address does not require rebuilding existing vectors when the models remain the same.
| Model | Input | Vector dimensions |
|---|---|---|
intfloat/multilingual-e5-small | Text | 384 |
intfloat/multilingual-e5-base | Text | 768 |
google/siglip2-base-patch16-224 | Images | 768 |
Each IP is limited to one request per second to /embed, with a burst buffer of up to five requests and excess requests delayed. Up to two concurrent requests are allowed per IP. Sites sharing one outbound IP share these limits.
The maximum JSON request size is 1 MiB, including Base64. Four images of 50 KB each produce approximately 270 KB of JSON, comfortably within this limit. This is an example batch size, not a separate four-image restriction: the API accepts up to eight items as long as the entire request fits within 1 MiB.
For a 429 response, reduce the request rate and concurrency, and wait before retrying. For 413, reduce image sizes or split the batch.
Calling the API directly
All models use POST /embed; select the model in the model field. Set purpose to query for search or document for indexing. Use GET /health to check readiness.
Send the User-Agent: vector-client/1.0 header. Cloudflare blocks Python urllib's default User-Agent with error 1010 on this service.
Example text search request:
api=https://vector.smartcj.com curl --fail-with-body \ "$api/embed" \ -A 'vector-client/1.0' \ --json '{ "model": "intfloat/multilingual-e5-small", "purpose": "query", "inputs": [ {"text":"Mountains and sea"} ] }'
The response contains a vectors array in input order. For text indexing, send purpose: “document” and the fields alt, description, the arrays tags, categories, models and, optionally, string values in custom_fields. The text field is used for search queries. The service adds E5 prefixes automatically.
For images, select SigLIP and send an image_base64 field in each inputs item: Base64-encoded file bytes without a data: prefix. The service does not download image URLs. Image responses also include an errors array: a successful vector has a null error; an invalid image returns an error code and null in place of its vector.
Example POST request using image.jpg from the current directory. Python builds the complete JSON with the Base64-encoded image, then curl sends it to the service. For indexing, change query to document.
api=https://vector.smartcj.com request=$(mktemp) trap 'rm -f "$request"' EXIT python3 - <<'PYTHON' > "$request" import base64 import json from pathlib import Path payload = { "model": ( "google/" "siglip2-base-patch16-224" ), "purpose": "query", "inputs": [{ "image_base64": base64.b64encode( Path("image.jpg") .read_bytes() ).decode() }] } print(json.dumps(payload)) PYTHON mime=application/json curl --fail-with-body \ "$api/embed" \ -A 'vector-client/1.0' \ -H "Content-Type: $mime" \ --data-binary "@$request"
e5-small and e5-base
e5-small is a lighter text model with 384-number vectors. e5-base is a larger model with 768-number vectors and requires more computation resources. It may produce better results on your data, but compare it using your own examples.
An e5-base vector contains twice as many numbers and occupies twice the space in the same storage format. The database must store larger vectors and a larger vector index; searches need more memory and computation. This does not mean the entire database doubles in size: it also stores descriptions and other data.
The models use different vector spaces. Compare text vectors from the same model. Changing Vector Model clears text vectors and queues their rebuild.
Quick start
- Open Rotation - Settings - Vector Search.
- Set Model Service URL to the service's base address, including its port. For example, use
http://127.0.0.1:8080when the service and script run on the same server. For external access, use the HTTPS address of nginx configured by your administrator in front of the service. Do not append/embed. Click Test connection and wait for the service to become ready. - For text, choose Vector Model:
intfloat/multilingual-e5-smallorintfloat/multilingual-e5-base. The default ise5-small. Image similarity uses the fixed SigLIP 2 Base 224 model. - In Vector Text Fields, choose the data to compare: alt, description, tags, categories, models and any relevant custom fields. When sites share description data, the site that owns those data selects the fields; the other sites inherit its selection.
- Turn on Enable Text Indexing, Enable Image Indexing, or both, and save the settings.
- Under Global Progress, click Start processing for the required source. The image source is named
rot_thumbs. Processing runs through the normal rotation cron; refresh the page to see progress. Image progress also shows Saved, Skipped, active time and Images/s.
The switches control indexing. With a type set to Off, the script stops calculating its new vectors and running its background jobs. Searches can still use ready vectors. To process an existing collection, use Start processing; enabling a switch alone does not schedule a complete scan.
The service address, text model and Prioritize accuracy are shared settings for linked sites. Changing the text model clears text vectors and queues their rebuild for all applicable sources. Changing Vector Text Fields queues a rebuild for the corresponding data owner.
Choosing a search mode
| Task | Tag attributes | Compared data |
|---|---|---|
| Search by the meaning of a phrase | search=“…” vector_search=“true” | The phrase and gallery text |
| Galleries with similar text | vector_similar_text=CURRENT_ITEM | The current gallery's text and other gallery text |
| Galleries with similar images | vector_similar_gallery_id=CURRENT_ITEM | The current gallery's primary thumb and indexed thumbs |
| Images similar to a specific thumb | vector_similar_thumb_id=123 | The specified thumb ID and indexed thumbs |
The former parameter names are no longer supported. Update the corresponding attributes in your templates.
Use one vector mode per block. Text and images are compared separately. CURRENT_ITEM takes the current gallery from the normal gallery-page URL.
In the examples, <!–URL–> links to the gallery's original URL. If your site uses custom rewrites for internal gallery pages, use the usual link from your template instead. Replace ID 1 with the ID you need.
Search by the meaning of a phrase
<thumb num=1-20 search="seaside house" vector_search="true"> <a href="<!--URL-->"> <img src="<!--THUMB_URL-->" alt="<!--ALT-->" /> </a> </thumb>
The script first sends the search phrase to the model service and receives its vector. It then searches the local database for similar saved gallery text vectors. A new phrase therefore needs a working service; an external service adds network latency to vector creation compared with a local service. The database search runs on your server. When comparing speed, also account for caching as described below.
Galleries with similar text
<thumb num=1-12 vector_similar_text=1> <a href="<!--URL-->"> <img src="<!--THUMB_URL-->" alt="<!--ALT-->" /> </a> </thumb>
The example uses gallery ID 1. For the current gallery, set vector_similar_text=CURRENT_ITEM. During indexing, the service already converted the gallery text into a vector and the script saved it in the local database. This search uses that existing vector without contacting the model service and excludes the source gallery from the results.
This mode supports Similar Gallery Tag Bonus. It promotes candidates that share rare tags with the source gallery. Its range is 0 to 1, the default is 0.1, and 0 disables the bonus. It applies to text-based similar galleries; phrase queries and image similarity do not use it.
Galleries with similar images
<thumb num=1-12 vector_similar_gallery_id=1> <a href="<!--URL-->"> <img src="<!--THUMB_URL-->" alt="<!--ALT-->" /> </a> </thumb>
The example uses gallery ID 1. For the current gallery, set vector_similar_gallery_id=CURRENT_ITEM. During indexing, the service converts images into vectors and the script saves them in the local database. This search takes the existing vector of the primary thumb, the main group's best_thumb for the selected site, and finds similar vectors locally without calling the service.
Each result displays the indexed thumb that matched the query. This can be a different frame from the gallery's usual primary thumb. The source gallery is excluded from the results.
Images similar to a specific thumb
<thumb num=1-12 vector_similar_thumb_id=1> <a href="<!--URL-->"> <img src="<!--THUMB_URL-->" alt="<!--ALT-->" /> </a> </thumb>
Here, 1 is a positive thumb ID, not a gallery ID. Use this mode to find images similar to a particular frame. It uses active thumbs from ordinary galleries available to the selected site. The source thumb vector and candidate vectors were saved in the local database during indexing; this search does not call an external service.
For vector_similar_thumb_id, CURRENT_ITEM also means the primary thumb of the current gallery on its gallery page, rather than the current item in a thumbnail loop. It selects the same image source as vector_similar_gallery_id=CURRENT_ITEM. Numeric values differ: one parameter takes a gallery ID, the other a specific thumb ID.
vector_similar_gallery_id=CURRENT_ITEM— the current gallery's primary thumb.vector_similar_gallery_id=123— the primary thumb of gallery ID 123.vector_similar_thumb_id=123— the specific thumb with ID 123.vector_similar_thumb_id=CURRENT_ITEM— the current gallery's primary thumb.
Positive integer IDs and CURRENT_ITEM may be written with ordinary straight quotes or without quotes. For CURRENT_ITEM, the script resolves the current gallery from the URL by ID or slug. If that gallery or its ready vector is unavailable, the block falls back to ordinary selection using its other attributes.
In the admin area, choose Search in - Similar images (gallery ID) or Search in - Similar images (thumb ID) and enter the corresponding ID.
How results are selected
Site availability and the block's filters apply first. Only items with ready vectors participate in the search.
For text-based similar galleries, the selected candidates may receive a bonus for shared rare tags, followed by configured sponsor mixing. The final order can therefore differ from ordering by vector distance alone.
Image similarity selects thumbs and returns one result per gallery:
- For a block requesting N galleries, the search selects N + 20 candidate thumbs ordered by vector distance.
- For each gallery, it keeps the nearest thumb among the returned candidates.
- It shows up to N distinct galleries in vector-distance order. The tag bonus and sponsor mixing do not apply.
For example, num=1-12 selects up to 32 candidate thumbs. If those thumbs belong to only eight galleries, the block shows eight results. There is no extra query to fill the remaining positions. Successful empty or short results remain empty or short.
Later reranking cannot add an item that was missed during candidate selection.
Prioritize accuracy
The checkbox is under Rotation - Settings - Vector Search and is off by default. It applies to text and image searches across linked sites.
- Off: fast approximate search using the HNSW vector index is allowed. Text search explicitly selects that index; for image similarity, the database chooses the plan.
- On: the vector index is excluded from the search. The database compares distances against all eligible ready vectors. This can increase query time and database load.
Here, “accuracy” refers to how the nearest vectors are found. The perceived similarity of images or text still depends on the model, the data and subsequent result ordering.
Changing this option changes retrieval; the vectors themselves do not need recalculating. The block's filters and gallery selection rules described above still apply.
Why HNSW can return different results
Imagine looking for a coffee shop in an unfamiliar city. You could walk every street, or ask passers-by and follow their directions. The latter narrows the search faster, but there could be another suitable coffee shop just outside your route.
HNSW uses links between nearby vectors to narrow the search. It can miss an eligible item. Differences in results and speed depend on the data, filters and load; there is no fixed guarantee for every site.
The star represents the query, dots represent ready vectors, and green rings mark returned matches. The left panel illustrates a nearby vector being missed; the right panel checks all eligible vectors. This illustrates the principle rather than measurements from a particular database.
Choosing a setting
Start with the default, Off. If differences in the returned galleries matter, compare both modes on your own data. Check the load under normal visitor traffic as well as individual requests.
Filters also affect query time: a rare tag and a very common tag can produce different loads even with the same database size. Enabling the option does not promise a particular response time.
For a technical check: On adds IGNORE INDEX(embedding) to the SQL query. With Off, text search uses FORCE INDEX(embedding), while image similarity lets the database choose its plan.
Caching and result updates
Vector Search follows the ordinary cache settings for queries, template blocks and pages. The SQL queries for the two accuracy modes have separate cache keys, but an already cached block or page can keep showing the previous results until its cache expires.
After changing Prioritize accuracy, replacing thumbs or rebuilding vectors, use the normal cache-clearing controls to check the updated result immediately. When comparing speed, account for whether the request runs against the database or returns a cached result.
Troubleshooting
Start processing was clicked, but progress does not change
Check that indexing is enabled for the relevant type, the rotation cron is running and the model service is reachable. Refresh the page and check Latest Errors. Individual invalid images are skipped; service unavailability pauses processing so the next rotation run can retry.
The model service is temporarily unavailable
Similar-gallery searches using saved text or image vectors continue without contacting the model service. The service is needed to calculate new vectors and vectors for new text search phrases.
The source gallery or thumb has no vector yet
In a template, the script falls back to ordinary selection using the block's remaining options. This differs from a successful search with zero matches, which stays empty. In the admin area, an unavailable image search displays a diagnostic and an empty result.
See also: Template tags, Rotation settings. Русский: Vector Search.


