TrueNAS Scale / Docker Compose Samples

This document provides Compose examples to bring up the external services used by the Nextcloud + Elasticsearch Discovery File Explorer:

Elasticsearch (single-node) + Kibana
Apache Tika (for text extraction)
Qdrant (vector database for collections/embeddings UI)

These samples are suitable for local dev or TrueNAS Scale (Apps) adaptation. Harden for production (auth, TLS, resource limits).

1) Elasticsearch + Kibana

Notes:

Single node with discovery.type=single-node
Security disabled for dev. ENABLE AUTH/TLS IN PROD.
Set ELASTICSEARCH_URL accordingly (example: https://elastic.fortura.cc for your testing cluster)

version: "3.9"
services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.12.2
    container_name: es
    environment:
      - discovery.type=single-node
      - xpack.security.enabled=false
      - ES_JAVA_OPTS=-Xms1g -Xmx1g
    ports:
      - "9200:9200"
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9200/_cluster/health"]
      interval: 10s
      timeout: 5s
      retries: 10

  kibana:
    image: docker.elastic.co/kibana/kibana:8.12.2
    container_name: kibana
    environment:
      - ELASTICSEARCH_HOSTS=http://elasticsearch:9200
    ports:
      - "5601:5601"
    depends_on:
      elasticsearch:
        condition: service_healthy

Environment wiring in .env.local:

ELASTICSEARCH_URL=http://localhost:9200     # or your remote, e.g., https://elastic.fortura.cc
ELASTICSEARCH_INDEX=files
ELASTICSEARCH_ALIAS=files_current

Initialize index:

npm run create:index

2) Apache Tika

Use Tika for server-side plain text extraction during ingestion.

version: "3.9"
services:
  tika:
    image: apache/tika:latest-full
    container_name: tika
    ports:
      - "9998:9998"
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9998"]
      interval: 10s
      timeout: 5s
      retries: 10

Environment wiring:

TIKA_BASE_URL=http://localhost:9998

3) Qdrant

Qdrant stores vectors. For your deployment, you provided:

Domain: https://vectors.biohazardvfx.com (domain over API)
API Key: set in .env.local as QDRANT_API_KEY

Local Compose:

version: "3.9"
services:
  qdrant:
    image: qdrant/qdrant:v1.9.2
    container_name: qdrant
    ports:
      - "6333:6333"  # REST
      - "6334:6334"  # gRPC
    volumes:
      - qdrant_data:/qdrant/storage
volumes:
  qdrant_data: {}

Environment wiring:

QDRANT_URL=http://localhost:6333
QDRANT_API_KEY=               # leave blank for local/no-auth; set for remote

TrueNAS Scale Notes

Translate the Compose services above into TrueNAS “Apps” or Helm charts.
For Elasticsearch:
- Persist data volumes on a dataset with adequate IOPS
- Enable security in production (xpack, TLS)
- Add resource limits and JVM tuning for heap
For Qdrant:
- Persist /qdrant/storage to a dataset
- Configure authentication if exposed externally (reverse proxy + auth)
For Tika:
- Stateless; consider auto-restart policy
Networking & DNS:
- Ensure the app pods (Next.js app) can reach ES/Tika/Qdrant service hostnames/ports.
Outbound access:
- If Sentry is used, allow outbound network for DSN ingestion.

Application Wiring Summary

.env.local (example used for your testing)

# Nextcloud
NEXTCLOUD_BASE_URL=https://nextcloud.biohazardvfx.com
NEXTCLOUD_USERNAME=admin
NEXTCLOUD_APP_PASSWORD=*** (do not commit)
NEXTCLOUD_ROOT_PATH=/remote.php/dav/files/admin

# Elasticsearch
ELASTICSEARCH_URL=https://elastic.fortura.cc
ELASTICSEARCH_INDEX=files
ELASTICSEARCH_ALIAS=files_current

# Apache Tika
TIKA_BASE_URL=http://localhost:9998

# Qdrant
QDRANT_URL=https://vectors.biohazardvfx.com
QDRANT_API_KEY=*** (do not commit)

# Optional embeddings (not required now)
OPENAI_API_BASE=
OPENAI_API_KEY=
OPENAI_EMBEDDING_MODEL=text-embedding-3-large
EMBEDDING_DIM=1536

# Sentry
SENTRY_DSN=

Operational Flow

Bring up ES/Tika/Qdrant (local or TrueNAS Scale).
Ensure .env.local points to your services.
Create ES index and alias:
```
npm run create:index
```

Ingest Nextcloud → Tika → ES:

npx tsx -r dotenv/config -r tsconfig-paths/register scripts/ingest-nextcloud.ts

Optional subtree:

npx tsx -r dotenv/config -r tsconfig-paths/register scripts/ingest-nextcloud.ts -- --root=/remote.php/dav/files/admin/SomeFolder

Run the app locally:
```
npm run dev
```
Use the UI to browse Nextcloud, search, edit Markdown, manage tags/history, and open Qdrant page for collections/points and embeddings visualization.

Security Reminders

Do not expose dev ES (no-auth) to the internet.
Store secrets in .env.local and never commit them.
Configure TLS and auth for production services (reverse proxies, mTLS, etc.).

4.8 KiB Raw Blame History