# TrueNAS Scale / Docker Compose Samples

This document provides Compose examples to bring up the external services used by the Nextcloud + Elasticsearch Discovery File Explorer:
- Elasticsearch (single-node) + Kibana
- Apache Tika (for text extraction)
- Qdrant (vector database for collections/embeddings UI)

These samples are suitable for local dev or TrueNAS Scale (Apps) adaptation. Harden for production (auth, TLS, resource limits).

## 1) Elasticsearch + Kibana

Notes:
- Single node with `discovery.type=single-node`
- Security disabled for dev. ENABLE AUTH/TLS IN PROD.
- Set `ELASTICSEARCH_URL` accordingly (example: https://elastic.fortura.cc for your testing cluster)

```yaml
version: "3.9"
services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.12.2
    container_name: es
    environment:
      - discovery.type=single-node
      - xpack.security.enabled=false
      - ES_JAVA_OPTS=-Xms1g -Xmx1g
    ports:
      - "9200:9200"
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9200/_cluster/health"]
      interval: 10s
      timeout: 5s
      retries: 10

  kibana:
    image: docker.elastic.co/kibana/kibana:8.12.2
    container_name: kibana
    environment:
      - ELASTICSEARCH_HOSTS=http://elasticsearch:9200
    ports:
      - "5601:5601"
    depends_on:
      elasticsearch:
        condition: service_healthy
```

Environment wiring in `.env.local`:
```
ELASTICSEARCH_URL=http://localhost:9200     # or your remote, e.g., https://elastic.fortura.cc
ELASTICSEARCH_INDEX=files
ELASTICSEARCH_ALIAS=files_current
```

Initialize index:
```
npm run create:index
```

## 2) Apache Tika

Use Tika for server-side plain text extraction during ingestion.

```yaml
version: "3.9"
services:
  tika:
    image: apache/tika:latest-full
    container_name: tika
    ports:
      - "9998:9998"
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9998"]
      interval: 10s
      timeout: 5s
      retries: 10
```

Environment wiring:
```
TIKA_BASE_URL=http://localhost:9998
```

## 3) Qdrant

Qdrant stores vectors. For your deployment, you provided:
- Domain: https://vectors.biohazardvfx.com (domain over API)
- API Key: set in `.env.local` as `QDRANT_API_KEY`

Local Compose:
```yaml
version: "3.9"
services:
  qdrant:
    image: qdrant/qdrant:v1.9.2
    container_name: qdrant
    ports:
      - "6333:6333"  # REST
      - "6334:6334"  # gRPC
    volumes:
      - qdrant_data:/qdrant/storage
volumes:
  qdrant_data: {}
```

Environment wiring:
```
QDRANT_URL=http://localhost:6333
QDRANT_API_KEY=               # leave blank for local/no-auth; set for remote
```

## TrueNAS Scale Notes

- Translate the Compose services above into TrueNAS “Apps” or Helm charts.
- For Elasticsearch:
  - Persist data volumes on a dataset with adequate IOPS
  - Enable security in production (xpack, TLS)
  - Add resource limits and JVM tuning for heap
- For Qdrant:
  - Persist `/qdrant/storage` to a dataset
  - Configure authentication if exposed externally (reverse proxy + auth)
- For Tika:
  - Stateless; consider auto-restart policy
- Networking & DNS:
  - Ensure the app pods (Next.js app) can reach ES/Tika/Qdrant service hostnames/ports.
- Outbound access:
  - If Sentry is used, allow outbound network for DSN ingestion.

## Application Wiring Summary

`.env.local` (example used for your testing)
```
# Nextcloud
NEXTCLOUD_BASE_URL=https://nextcloud.biohazardvfx.com
NEXTCLOUD_USERNAME=admin
NEXTCLOUD_APP_PASSWORD=*** (do not commit)
NEXTCLOUD_ROOT_PATH=/remote.php/dav/files/admin

# Elasticsearch
ELASTICSEARCH_URL=https://elastic.fortura.cc
ELASTICSEARCH_INDEX=files
ELASTICSEARCH_ALIAS=files_current

# Apache Tika
TIKA_BASE_URL=http://localhost:9998

# Qdrant
QDRANT_URL=https://vectors.biohazardvfx.com
QDRANT_API_KEY=*** (do not commit)

# Optional embeddings (not required now)
OPENAI_API_BASE=
OPENAI_API_KEY=
OPENAI_EMBEDDING_MODEL=text-embedding-3-large
EMBEDDING_DIM=1536

# Sentry
SENTRY_DSN=
```

## Operational Flow

1) Bring up ES/Tika/Qdrant (local or TrueNAS Scale).
2) Ensure `.env.local` points to your services.
3) Create ES index and alias:
   ```
   npm run create:index
   ```
4) Ingest Nextcloud → Tika → ES:
   ```
   npx tsx -r dotenv/config -r tsconfig-paths/register scripts/ingest-nextcloud.ts
   ```
   Optional subtree:
   ```
   npx tsx -r dotenv/config -r tsconfig-paths/register scripts/ingest-nextcloud.ts -- --root=/remote.php/dav/files/admin/SomeFolder
   ```
5) Run the app locally:
   ```
   npm run dev
   ```
6) Use the UI to browse Nextcloud, search, edit Markdown, manage tags/history, and open Qdrant page for collections/points and embeddings visualization.

## Security Reminders

- Do not expose dev ES (no-auth) to the internet.
- Store secrets in `.env.local` and never commit them.
- Configure TLS and auth for production services (reverse proxies, mTLS, etc.).