diff --git a/.gitignore b/.gitignore index e2caa13..294772f 100644 --- a/.gitignore +++ b/.gitignore @@ -5,3 +5,4 @@ __pycache__/ *.pyc *.db *.partial +AGENTS.local.md diff --git a/AGENTS.md b/AGENTS.md deleted file mode 100644 index bb98f2c..0000000 --- a/AGENTS.md +++ /dev/null @@ -1,104 +0,0 @@ -# Importarr Agent Instructions - -Importarr is owned as a Linux-ops-managed service repository. Treat this checkout as the source of truth for application code, deployment files, and local service installs. - -## Repository Source Of Truth - -- Work from `/srv/opencode-workspace/importarr` for Importarr code and deploy changes. -- The live service is installed on `dgsserver1` from a repo checkout at `/opt/importarr/repo`; gmk1 installs are not the public Importarr service. -- Do not edit host-local legacy scripts as the normal workflow: - - `/usr/local/sbin/importarr-status.py` - - `/usr/local/sbin/manual-media-import.py` -- If an emergency hotfix is made outside the repo, backport it here immediately and reinstall from the repo. - -## Seamless Feature Workflow - -When asked to implement an Importarr feature, fix, UI change, deployment change, or operational behavior change: - -1. Inspect `git status --short --branch` before editing. -2. Implement the smallest correct repo change. -3. Run the narrowest useful verification, normally: - ```sh - .venv/bin/pytest -q - ``` - If shell deploy scripts changed, also run: - ```sh - sh -n deploy/systemd-install.sh && sh -n deploy/repo-upgrade.sh - ``` -4. Inspect `git diff` and ensure no secrets, raw `.env`, tokens, databases, or private material are included. -5. Commit and push completed Importarr changes by default unless the user explicitly asks not to publish or verification is blocked. -6. Install/restart on `dgsserver1` from the repository so the public service matches the repo: - ```sh - ssh -p 2222 opencode@dgsserver1 'sudo -n sh /opt/importarr/repo-upgrade.sh' - ``` -7. Verify the live service: - ```sh - make verify-live - ``` -8. If the live install fails, inspect `systemctl status importarr.service` and `journalctl -u importarr.service`; fix the repo, commit/push the fix, reinstall, and verify again. - -## Gitea Issue Completion Workflow - -When the user points an agent at an Importarr Gitea issue and asks to solve it, the expected end-to-end flow is mandatory unless the user explicitly says not to deploy: - -1. Read the issue first, including comments and acceptance criteria. -2. Implement the smallest correct repo change on a dedicated issue branch when appropriate. -3. Run the narrowest useful tests/checks. -4. Commit and push the completed change to `main` or merge/push the issue branch as instructed. -5. Update the Gitea issue with what changed and the verification that ran, then close it when solved. -6. Update the `dgsserver1` install from the pushed repo state: - ```sh - ssh -p 2222 opencode@dgsserver1 'sudo -n sh /opt/importarr/repo-upgrade.sh' - ``` -7. Restart/verify the `dgsserver1` service from the repo-managed install and confirm the public URL: - ```sh - ssh -p 2222 opencode@dgsserver1 'sudo -n systemctl status importarr.service --no-pager -l' - curl -fsS https://importarr.delphas.dk/health - ``` - -If the worker timer is intentionally paused, do not re-enable it unless the issue explicitly includes worker scheduling or the user approves. - -## Install Model - -- The service virtualenv lives at `/opt/importarr/venv` on `dgsserver1`. -- The status UI systemd unit runs `/opt/importarr/venv/bin/importarr-status` from the packaged repo install. -- The manual worker systemd unit runs `/opt/importarr/venv/bin/manual-media-import` from the packaged repo install. -- The package is installed from the repository into the venv using normal wheel/package install, not editable install. -- Do **not** use editable install for the system service: the unprivileged `importarr` user may not be able to read `/srv/opencode-workspace/importarr`, causing `ModuleNotFoundError` at startup. -- `/opt/importarr/repo-upgrade.sh` is the pull-and-reinstall helper for machines that should follow pushed `main`. - -## Local Commands - -```sh -make test -make install-systemd -make upgrade-local -make repo-upgrade -make verify-live -``` - -`make repo-upgrade` is for pulling already-pushed changes with `git pull --ff-only`. It refuses to run with uncommitted repo changes. - -## Runtime Defaults On This Host - -- Public URL: `https://importarr.delphas.dk/` -- dgsserver1 local URL: `http://127.0.0.1:8095/` -- Health: `http://127.0.0.1:8095/health` -- Status: `http://127.0.0.1:8095/api/status` -- Systemd service: `importarr.service` -- Env file: `/etc/importarr/importarr.env` -- SQLite state: `/var/lib/importarr/importarr.db` - -## Secret Handling - -- Prefer `*_FILE` settings for secrets, for example: - - `IMPORTARR_SAB_API_KEY_FILE` - - `IMPORTARR_AUTH_TOKEN_FILE` - - `IMPORTARR_RADARR_API_KEY_FILE` - - `IMPORTARR_SONARR_API_KEY_FILE` -- Never commit real env files, API keys, tokens, private keys, service databases, or backup data. -- Template files may list variable names with placeholder values or commented examples only. - -## Linux Ops Follow-Through - -For changes that materially alter the live service setup, ports, routes, monitoring, backup coverage, or host ownership, also follow the linux-ops documentation/systems-overview update rules. Do not mix unrelated pre-existing uncommitted changes from `linux-ops-docs` or `systems-overview` into Importarr commits. diff --git a/Makefile b/Makefile index c543a05..2e13fcf 100644 --- a/Makefile +++ b/Makefile @@ -1,9 +1,11 @@ PYTHON ?= .venv/bin/python PIP ?= .venv/bin/pip SERVICE ?= importarr.service -LIVE_URL ?= http://127.0.0.1:8095 +IMPORTARR_PREFIX ?= /opt/importarr +IMPORTARR_REPO_DIR ?= $(CURDIR) +IMPORTARR_URL ?= http://127.0.0.1:8765 -.PHONY: test install-systemd install-from-repo upgrade-local repo-upgrade verify-live +.PHONY: test install-systemd install-from-repo repo-upgrade verify test: $(PYTHON) -m pytest @@ -12,16 +14,12 @@ install-systemd: sudo -n sh deploy/systemd-install.sh install-from-repo: - sudo -n /opt/importarr/venv/bin/pip install --upgrade /srv/opencode-workspace/importarr - -upgrade-local: - sudo -n /opt/importarr/venv/bin/pip install --upgrade /srv/opencode-workspace/importarr - sudo -n systemctl restart $(SERVICE) + sudo -n $(IMPORTARR_PREFIX)/venv/bin/pip install --upgrade $(IMPORTARR_REPO_DIR) repo-upgrade: - sudo -n sh /opt/importarr/repo-upgrade.sh + sudo -n IMPORTARR_PREFIX=$(IMPORTARR_PREFIX) IMPORTARR_REPO_DIR=$(IMPORTARR_REPO_DIR) sh deploy/repo-upgrade.sh -verify-live: - curl -fsS $(LIVE_URL)/health - curl -fsS $(LIVE_URL)/api/status - curl -fsS $(LIVE_URL)/api/preview +verify: + curl -fsS $(IMPORTARR_URL)/health + curl -fsS $(IMPORTARR_URL)/api/status + curl -fsS $(IMPORTARR_URL)/api/preview diff --git a/README.md b/README.md index 3c1349c..6bb5c08 100644 --- a/README.md +++ b/README.md @@ -2,17 +2,17 @@ Importarr is an Arr-style service for importing manually categorized SABnzbd downloads after SAB reports final completion. It owns one SAB category, defaults to `manual`, and refuses to import transient Direct Unpack paths or jobs still in SAB queue/post-processing. -## New-machine install +## Install Importarr is intended to feel like a small Arr service: deploy the container or systemd service, edit one env file, point SABnzbd category `manual` at the same completed-download path, then open the web UI. ### Docker Compose, recommended ```sh -mkdir -p /opt/importarr/config -cd /opt/importarr -curl -fsSLO https://gitea.delphas.dk/daniels/importarr/raw/branch/main/deploy/docker-compose.example.yml -curl -fsSLo importarr.env https://gitea.delphas.dk/daniels/importarr/raw/branch/main/deploy/importarr.env.example +mkdir -p importarr/config +cd importarr +curl -fsSLO https://example.com/importarr/deploy/docker-compose.example.yml +curl -fsSLo importarr.env https://example.com/importarr/deploy/importarr.env.example ${EDITOR:-vi} importarr.env docker compose -f docker-compose.example.yml --env-file importarr.env up -d ``` @@ -26,25 +26,14 @@ docker build -t importarr:local . ### systemd / pip install ```sh -git clone https://gitea.delphas.dk/daniels/importarr.git +git clone https://example.com/importarr.git cd importarr sudo sh deploy/systemd-install.sh sudo ${EDITOR:-vi} /etc/importarr/importarr.env sudo systemctl start importarr.service ``` -The installer creates the `importarr` system user when needed, installs a virtualenv at `/opt/importarr/venv`, and installs the package from the checked-out repository. The repository is therefore the source of truth: pull or edit the repo, reinstall/restart from the repo, and the service runs the package built from that code. - -For local upgrades from a checked-out repo on dgsserver1, use the repo workflow instead of editing live scripts: - -```sh -cd /srv/opencode-workspace/importarr -.venv/bin/python -m pytest -git status --short --branch -sudo -n /opt/importarr/venv/bin/pip install --upgrade /srv/opencode-workspace/importarr -sudo -n systemctl restart importarr.service -make verify-live -``` +The installer creates the `importarr` system user when needed, installs a virtualenv, and installs the package from the checked-out repository. Override install paths with `IMPORTARR_*` variables if the defaults do not fit your environment. For a machine that should stay current with the repository, use the installed repo-upgrade helper: @@ -54,7 +43,7 @@ sudo -n sh /opt/importarr/repo-upgrade.sh The helper refuses to run when the checkout has uncommitted changes, then performs `git pull --ff-only`, reinstalls the package from the repo, restarts `importarr.service`, and prints service status. Use it after changes have been committed and pushed to `main`. -Release-worthy changes should be committed, tagged with SemVer (`v0.1.1`, `v0.2.0`, ...), pushed with tags, then reinstalled from the tagged checkout or artifact. Do not hand-edit `/usr/local/sbin/importarr-status.py` or `/usr/local/sbin/manual-media-import.py` except for a documented emergency hotfix that is immediately backported here. +Release-worthy changes should be committed, tagged with SemVer (`v0.1.1`, `v0.2.0`, ...), pushed with tags, then installed from the tagged checkout or artifact. ### Required setup @@ -96,46 +85,15 @@ pytest uvicorn importarr.main:app --reload ``` -## Migration notes for dgsserver1 +## Operations -Export the existing script settings into `IMPORTARR_*` env vars, add historical folders as explicit manual batches, run a dry-run/inspection through `/api/preview`, then switch the systemd service or Compose route after the ready set matches expectations. +Importarr intentionally does not document private deployment topology, hostnames, reverse proxies, monitoring, backups, or operator workflows in this repository. Keep those details in your own ops runbooks. -On dgsserver1, the packaged service is the only intended active entrypoint after cutover. Keep `manual-media-import.timer` disabled unless a repo-managed worker/timer replaces it later. +For a systemd install, prefer a normal package install from the checked-out repo over an editable install so the service user does not need read access to your development checkout. -## Repository-as-install workflow - -Importarr should not drift into host-local scripts. Treat the checked-out repository as the install source: - -1. Make changes in `/srv/opencode-workspace/importarr`. -2. Run tests: `make test`. -3. Commit and push the repo change. -4. Install/restart from the same repo: `make upgrade-local` for local changes, or `make repo-upgrade` to pull the latest pushed `main` and restart. -5. Verify the live service: `make verify-live`. - -Do not edit `/usr/local/sbin/importarr-status.py`, `/usr/local/sbin/manual-media-import.py`, or files copied out of the repo as the normal workflow. If an emergency live hotfix is unavoidable, backport it to this repository immediately and run the repo install workflow again. - -### Local install lessons learned - -- The live systemd service runs as the unprivileged `importarr` user. -- Do not install the system service with `pip install --editable /srv/opencode-workspace/importarr`; that can fail at startup if the service user cannot read the workspace checkout. -- The supported local service install is a normal package install from the repo into `/opt/importarr/venv`: +If the service fails, standard systemd diagnostics are usually enough: ```sh -sudo -n /opt/importarr/venv/bin/pip install --upgrade /srv/opencode-workspace/importarr -sudo -n systemctl restart importarr.service -``` - -- `deploy/systemd-install.sh`, `make upgrade-local`, and `/opt/importarr/repo-upgrade.sh` already use this supported model. -- After every implementation task that should affect the live local service, run: - -```sh -make upgrade-local -make verify-live -``` - -- If `make verify-live` fails, check: - -```sh -sudo -n systemctl --no-pager --full status importarr.service -sudo -n journalctl -u importarr.service -n 120 --no-pager +sudo systemctl --no-pager --full status importarr.service +sudo journalctl -u importarr.service -n 120 --no-pager ``` diff --git a/deploy/docker-compose.example.yml b/deploy/docker-compose.example.yml index f33f06f..8b6e200 100644 --- a/deploy/docker-compose.example.yml +++ b/deploy/docker-compose.example.yml @@ -6,7 +6,7 @@ services: - "8765:8765" volumes: - ./config:/config - - /srv/scrypted/sabnzbd-data/downloads/manual:/data/downloads/manual + - /path/to/downloads/manual:/data/downloads/manual - /path/to/movies:/data/movies - /path/to/tv:/data/tv restart: unless-stopped diff --git a/deploy/importarr.env.example b/deploy/importarr.env.example index 7e4a3fd..2f69623 100644 --- a/deploy/importarr.env.example +++ b/deploy/importarr.env.example @@ -1,16 +1,16 @@ -IMPORTARR_SAB_URL=http://127.0.0.1:8080 +IMPORTARR_SAB_URL=http://sabnzbd:8080 # Prefer *_FILE for secrets. Plain env vars still work for local/dev installs. # IMPORTARR_SAB_API_KEY=change-me # IMPORTARR_SAB_API_KEY_FILE=/etc/importarr/sab-api-key IMPORTARR_SAB_CATEGORY=manual -IMPORTARR_DOWNLOAD_ROOT=/srv/scrypted/sabnzbd-data/downloads/manual -IMPORTARR_MOVIES_ROOT=/srv/media/movies -IMPORTARR_TV_ROOT=/srv/media/tv -IMPORTARR_STATE_PATH=/var/lib/importarr/importarr.db +IMPORTARR_DOWNLOAD_ROOT=/data/downloads/manual +IMPORTARR_MOVIES_ROOT=/data/movies +IMPORTARR_TV_ROOT=/data/tv +IMPORTARR_STATE_PATH=/config/importarr.db IMPORTARR_LOG_LEVEL=info # IMPORTARR_AUTH_TOKEN=change-me # IMPORTARR_AUTH_TOKEN_FILE=/etc/importarr/auth-token IMPORTARR_BIND_HOST=0.0.0.0 -IMPORTARR_BIND_PORT=8095 +IMPORTARR_BIND_PORT=8765 IMPORTARR_POLL_SECONDS=60 -IMPORTARR_REPO_DIR=/srv/opencode-workspace/importarr +# IMPORTARR_REPO_DIR=/path/to/importarr diff --git a/deploy/manual-media-import-failure.service b/deploy/manual-media-import-failure.service index 5fc77fd..fadad3a 100644 --- a/deploy/manual-media-import-failure.service +++ b/deploy/manual-media-import-failure.service @@ -3,4 +3,4 @@ Description=Critical alert when manual media importer fails [Service] Type=oneshot -ExecStart=/usr/bin/python3 /usr/local/sbin/ha-critical-notify.py service-recovery "manual-media-import.service failed" --host dgsserver1 --resource manual-media-import --details "Check journalctl -u manual-media-import.service and /var/log/manual-media-import.log" +ExecStart=/usr/bin/logger -t importarr "manual-media-import.service failed; check journalctl -u manual-media-import.service" diff --git a/deploy/repo-upgrade.sh b/deploy/repo-upgrade.sh index 6555d10..b318daf 100644 --- a/deploy/repo-upgrade.sh +++ b/deploy/repo-upgrade.sh @@ -12,9 +12,10 @@ if [ -f "$ENV_FILE" ]; then . "$ENV_FILE" fi -REPO_DIR=${IMPORTARR_REPO_DIR:-/srv/opencode-workspace/importarr} +PREFIX=${IMPORTARR_PREFIX:-/opt/importarr} +REPO_DIR=${IMPORTARR_REPO_DIR:-$(pwd)} SERVICE=${IMPORTARR_SERVICE:-importarr.service} -VENV=${IMPORTARR_VENV:-/opt/importarr/venv} +VENV=${IMPORTARR_VENV:-$PREFIX/venv} if [ ! -d "$REPO_DIR/.git" ]; then echo "Importarr repo not found at $REPO_DIR" >&2 @@ -33,7 +34,7 @@ git pull --ff-only "$VENV/bin/pip" install --upgrade "$REPO_DIR" GIT_SHA="$(git rev-parse --short=12 HEAD 2>/dev/null || printf development)" BUILD_DATE="$(date -u +%Y-%m-%dT%H:%M:%SZ)" -cat > /opt/importarr/build.env < "$PREFIX/build.env" < None: + if not HOME_ASSISTANT_WEBHOOK: + return item_type = str(item.get("item_type") or "Movie") name = item.get("title") or dest.parent.name payload_body: dict[str, object] = { @@ -906,7 +903,7 @@ def source_tag_for(path: Path) -> str: for root in DOWNLOAD_ROOTS: try: path.relative_to(root) - return "manual" if root == MANUAL_DOWNLOADS else "danish-legacy" + return "manual" if root == MANUAL_DOWNLOADS else "legacy" except ValueError: continue return "manual"