Author SHA1 Message Date
daniels 939dc9819d Require synced review site 2026-07-30 19:47:35 +02:00
daniels 65c96bbc4b Migrate UI to React and Tailwind Refs #32 2026-07-30 14:54:46 +02:00
daniels fb955adaf8 Add persistent local review stack 2026-07-30 14:36:45 +02:00
daniels c5cc0901c5 Fix remaining queue races Refs #6 2026-07-30 10:40:39 +02:00
daniels 7bdab60d6f Fix queue claim concurrency Refs #6 2026-07-30 10:32:09 +02:00
daniels 4e3d021652 Fix queue review findings Refs #6 2026-07-30 10:05:57 +02:00
daniels a088d840e3 Implement persistent queue engine Refs #6 2026-07-30 09:01:25 +02:00
daniels 72562eab95 Target manual importer from UI controls #31 2026-07-29 21:37:20 +02:00
daniels 7be54f9f3c Show self-update availability in UI #25 2026-07-29 21:21:05 +02:00
daniels 006db01930 Fix import queue UI controls #31 2026-07-29 21:02:18 +02:00
daniels 96f077ec76 Add release-aware self update #25 2026-07-29 20:44:41 +02:00
daniels 5c205fff13 Clarify Importarr light dark themes #30 2026-07-29 20:42:23 +02:00
daniels b2c34ef995 Add light and dark UI palettes #30 2026-07-29 20:33:00 +02:00
daniels da4df50205 Tighten Importarr UI #30 2026-07-29 20:28:09 +02:00
daniels 2a7c39bf6a Make UI SABnzbd-style #30 2026-07-29 20:19:33 +02:00
daniels 5752e9fb2f Add API update controls #22 2026-07-29 20:11:26 +02:00
daniels 6cdf1f5d49 Improve desktop jobs queue layout #29 2026-07-29 20:06:03 +02:00
daniels 7c50def0c9 Display build dates in local time 2026-07-29 20:04:37 +02:00
daniels 2156989b4b Improve mobile jobs queue #29 2026-07-29 19:52:51 +02:00
daniels d15ea13cb3 Make UI mobile responsive #29 2026-07-29 18:58:05 +02:00
daniels 5829623a9e Add per-item run now action #26 2026-07-29 15:19:31 +02:00
daniels c2ccb0d4bb Clear stale ignored SAB rows
Drop ignored SAB records before each sync so ownership remapping removes old decisions.
2026-07-29 15:16:39 +02:00
daniels f9eb633e19 Map SAB storage root safely
Only treat configured manual storage as Importarr-owned and map SAB container paths to local paths.
2026-07-29 15:15:15 +02:00
daniels ce82a405c5 Treat manual storage as Importarr-owned
Use SAB storage under the configured download root as ownership even when SAB reports category '*'.
2026-07-29 15:12:04 +02:00
daniels 22a1fc5522 Add settings connection tests 2026-07-29 15:11:26 +02:00
daniels 581934f7b5 Refresh stale SAB category ignores
Drop old ignored SAB rows before resync so fixed category parsing can take effect.
2026-07-29 15:08:32 +02:00
daniels 2ff670a9ae Read SAB cat as category
Treat SAB history cat/category fields equivalently for #20.
2026-07-29 15:06:37 +02:00
daniels 57c266dfa8 Add settings dialog for Arr connections #21 2026-07-29 14:57:38 +02:00
daniels a443909d64 Fix FastAPI index rendering
Use the current TemplateResponse signature and cover the live controls page.
2026-07-29 14:48:48 +02:00
daniels f4d151f9fe Remove legacy Importarr status UI
Run the service through the FastAPI entrypoint only to avoid split UI paths.
2026-07-29 14:46:25 +02:00
daniels 1cafe2b45a Build jobs queue UI for issue #8 2026-07-29 14:45:31 +02:00
40 changed files with 7714 additions and 705 deletions
+15
View File
@@ -0,0 +1,15 @@
.git
.venv
.review-data
deploy/importarr.review.env
.pytest_cache
.mypy_cache
.ruff_cache
.tox
**/__pycache__
**/*.pyc
**/*.pyo
*.egg-info
.coverage
htmlcov
frontend/node_modules
+3
View File
@@ -6,3 +6,6 @@ __pycache__/
*.db *.db
*.partial *.partial
AGENTS.local.md AGENTS.local.md
.review-data/
deploy/importarr.review.env
frontend/node_modules/
+7
View File
@@ -0,0 +1,7 @@
# Repository Instructions
The Importarr review site at <http://172.20.30.35:18765/> must always reflect the local working-tree code.
- After making local code changes, run `deploy/review/review.sh update` from the repository root.
- Before finishing, verify that `http://172.20.30.35:18765/` responds successfully.
- If deployment or URL verification fails, report the failure clearly; do not claim the review site is current.
+8
View File
@@ -1,8 +1,16 @@
FROM node:22-alpine AS frontend
WORKDIR /build/frontend
COPY frontend/package*.json ./
RUN npm ci
COPY frontend ./
RUN npm run build
FROM python:3.12-slim AS runtime FROM python:3.12-slim AS runtime
WORKDIR /app WORKDIR /app
COPY pyproject.toml README.md LICENSE ./ COPY pyproject.toml README.md LICENSE ./
COPY importarr ./importarr COPY importarr ./importarr
COPY --from=frontend /build/importarr/static ./importarr/static
ARG IMPORTARR_VERSION=0.1.0 ARG IMPORTARR_VERSION=0.1.0
ARG IMPORTARR_BUILD_DATE=unknown ARG IMPORTARR_BUILD_DATE=unknown
ARG IMPORTARR_GIT_SHA=unknown ARG IMPORTARR_GIT_SHA=unknown
+54
View File
@@ -43,6 +43,8 @@ sudo -n sh /opt/importarr/repo-upgrade.sh
The helper refuses to run when the checkout has uncommitted changes, then performs `git pull --ff-only`, reinstalls the package from the repo, restarts `importarr.service`, and prints service status. Use it after changes have been committed and pushed to `main`. The helper refuses to run when the checkout has uncommitted changes, then performs `git pull --ff-only`, reinstalls the package from the repo, restarts `importarr.service`, and prints service status. Use it after changes have been committed and pushed to `main`.
Installed deployments can expose the same operation through the authenticated API. `GET /api/control/update-check` queries the latest release from `IMPORTARR_UPDATE_RELEASE_URL` (default: this repository's Gitea latest-release API) and compares it with the running `IMPORTARR_VERSION`. `POST /api/control/update` performs the same check and only runs the update command when a newer release tag exists. Configure `IMPORTARR_UPDATE_COMMAND` when the default `sh deploy/repo-upgrade.sh` is not correct for the service working directory. The web UI Start, Stop, and Restart controls target `manual-media-import.service` by default; configure `IMPORTARR_START_COMMAND`, `IMPORTARR_STOP_COMMAND`, or `IMPORTARR_RESTART_COMMAND` when those defaults need a wrapper such as sudo.
Release-worthy changes should be committed, tagged with SemVer (`v0.1.1`, `v0.2.0`, ...), pushed with tags, then installed from the tagged checkout or artifact. Release-worthy changes should be committed, tagged with SemVer (`v0.1.1`, `v0.2.0`, ...), pushed with tags, then installed from the tagged checkout or artifact.
### Required setup ### Required setup
@@ -71,6 +73,14 @@ Release-worthy changes should be committed, tagged with SemVer (`v0.1.1`, `v0.2.
- `GET /api/manual-batches` - `GET /api/manual-batches`
- `POST /api/manual-batches` with `{ "path": "relative/or/absolute/path" }` - `POST /api/manual-batches` with `{ "path": "relative/or/absolute/path" }`
- `DELETE /api/manual-batches/{id}` - `DELETE /api/manual-batches/{id}`
- `POST /api/control/start`
- `POST /api/control/pause`
- `POST /api/control/stop`
- `POST /api/control/cancel-current`
- `POST /api/control/restart`
- `GET /api/control/update-check`
- `POST /api/control/update`
- `POST /api/queue-items/{id}/action` with `{ "action": "retry|ignore|remove" }`
- `POST /api/import/run-now` - `POST /api/import/run-now`
Set `IMPORTARR_AUTH_TOKEN_FILE` or `IMPORTARR_AUTH_TOKEN` to require `Authorization: Bearer <token>` for write endpoints. Set `IMPORTARR_AUTH_TOKEN_FILE` or `IMPORTARR_AUTH_TOKEN` to require `Authorization: Bearer <token>` for write endpoints.
@@ -81,10 +91,54 @@ Set `IMPORTARR_AUTH_TOKEN_FILE` or `IMPORTARR_AUTH_TOKEN` to require `Authorizat
python3.12 -m venv .venv python3.12 -m venv .venv
. .venv/bin/activate . .venv/bin/activate
pip install -e '.[test]' pip install -e '.[test]'
npm --prefix frontend install
npm --prefix frontend run build
pytest pytest
uvicorn importarr.main:app --reload uvicorn importarr.main:app --reload
``` ```
The web UI is a Vite React application styled with Tailwind CSS. Its local
shadcn-style component primitives use Radix UI for dialogs and composition, and
lucide-react for icons. Run `npm --prefix frontend run dev` for Vite's development
server (it proxies API calls to port 8765), or build before running FastAPI so the
production assets are written to `importarr/static`.
### Persistent local review environment
The review Compose stack builds the current working tree, including uncommitted
UI/API changes, and stays running for pre-commit or pre-push inspection. It is
separate from production: the web port is bound to localhost by default, state and
sample media live under the ignored `.review-data/` directory, external Arr/SAB
services are not required, and service-control/update commands are safe no-ops.
```sh
deploy/review/review.sh up # build current files and start in background
deploy/review/review.sh update # rebuild changed files and recreate as needed
deploy/review/review.sh status
deploy/review/review.sh logs # follow logs; Ctrl-C leaves the stack running
deploy/review/review.sh stop # stop containers, preserving them and data
deploy/review/review.sh down # remove containers/network, preserving data
deploy/review/review.sh reset # remove stack and all local review data
```
On first use the wrapper copies `deploy/importarr.review.env.example` to the
ignored `deploy/importarr.review.env`. Adjust `REVIEW_PORT` there if port 18765
is occupied, then review `http://127.0.0.1:18765/`. For review from a trusted
internal network, set `REVIEW_BIND_ADDRESS` to the host's LAN address and use
that address in the URL. Do not use `0.0.0.0` or expose this review stack to an
untrusted network; the safe default is `127.0.0.1`. To exercise imports without
real integrations, place disposable folders in `.review-data/downloads/`; movie
and TV destinations are `.review-data/movies/` and `.review-data/tv/`.
Each checkout gets its own Compose project and image name, so worktrees do not
replace each other's containers or images. Read-only and cleanup commands do not
create the local env file when it is absent.
Agent workflow: run tests, run `update`, confirm `status` reports healthy, and
leave the stack running for the reviewer. Reviewer workflow: inspect the UI and
API, use `logs` when needed, and use `down` after review (or `reset` when the
saved review state is no longer useful). Re-run `update` after every working-tree
change that should be reviewed.
## Operations ## Operations
Importarr intentionally does not document private deployment topology, hostnames, reverse proxies, monitoring, backups, or operator workflows in this repository. Keep those details in your own ops runbooks. Importarr intentionally does not document private deployment topology, hostnames, reverse proxies, monitoring, backups, or operator workflows in this repository. Keep those details in your own ops runbooks.
+22
View File
@@ -0,0 +1,22 @@
services:
importarr:
build:
context: ..
dockerfile: Dockerfile
image: ${REVIEW_IMAGE:-importarr-review:local}
env_file:
- ${REVIEW_SERVICE_ENV_FILE:-importarr.review.env.example}
ports:
- "${REVIEW_BIND_ADDRESS:-127.0.0.1}:${REVIEW_PORT:-18765}:8765"
volumes:
- ../.review-data/config:/config
- ../.review-data/downloads:/data/downloads/manual
- ../.review-data/movies:/data/movies
- ../.review-data/tv:/data/tv
restart: unless-stopped
healthcheck:
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8765/health', timeout=2)"]
interval: 10s
timeout: 3s
retries: 6
start_period: 5s
+3
View File
@@ -3,6 +3,9 @@ IMPORTARR_SAB_URL=http://sabnzbd:8080
# IMPORTARR_SAB_API_KEY=change-me # IMPORTARR_SAB_API_KEY=change-me
# IMPORTARR_SAB_API_KEY_FILE=/etc/importarr/sab-api-key # IMPORTARR_SAB_API_KEY_FILE=/etc/importarr/sab-api-key
IMPORTARR_SAB_CATEGORY=manual IMPORTARR_SAB_CATEGORY=manual
# SAB may report storage paths from inside its container; set this when that
# differs from the local host path Importarr scans in IMPORTARR_DOWNLOAD_ROOT.
# IMPORTARR_SAB_STORAGE_ROOT=/data/downloads/manual
IMPORTARR_DOWNLOAD_ROOT=/data/downloads/manual IMPORTARR_DOWNLOAD_ROOT=/data/downloads/manual
IMPORTARR_MOVIES_ROOT=/data/movies IMPORTARR_MOVIES_ROOT=/data/movies
IMPORTARR_TV_ROOT=/data/tv IMPORTARR_TV_ROOT=/data/tv
+23
View File
@@ -0,0 +1,23 @@
# Copied to importarr.review.env by deploy/review/review.sh. No secrets or
# external services are required for review.
REVIEW_PORT=18765
# Keep this on loopback unless review access from a trusted network is needed.
REVIEW_BIND_ADDRESS=127.0.0.1
IMPORTARR_BIND_HOST=0.0.0.0
IMPORTARR_BIND_PORT=8765
IMPORTARR_STATE_PATH=/config/importarr.db
IMPORTARR_DOWNLOAD_ROOT=/data/downloads/manual
IMPORTARR_MOVIES_ROOT=/data/movies
IMPORTARR_TV_ROOT=/data/tv
IMPORTARR_SAB_URL=http://127.0.0.1:9
IMPORTARR_SAB_CATEGORY=review
IMPORTARR_POLL_SECONDS=3600
IMPORTARR_LOG_LEVEL=info
# UI control operations must not control host services or update this checkout.
IMPORTARR_START_COMMAND=/bin/true
IMPORTARR_STOP_COMMAND=/bin/true
IMPORTARR_RESTART_COMMAND=/bin/true
IMPORTARR_UPDATE_COMMAND=/bin/true
IMPORTARR_UPDATE_RELEASE_URL=http://127.0.0.1:9/releases/latest
IMPORTARR_UPDATE_CHECK_TIMEOUT_SECONDS=1
+2 -2
View File
@@ -1,12 +1,12 @@
[Unit] [Unit]
Description=Importarr manual media importer status UI Description=Importarr manual media importer web UI
After=network-online.target After=network-online.target
Wants=network-online.target Wants=network-online.target
[Service] [Service]
EnvironmentFile=-/etc/importarr/importarr.env EnvironmentFile=-/etc/importarr/importarr.env
EnvironmentFile=-/opt/importarr/build.env EnvironmentFile=-/opt/importarr/build.env
ExecStart=/opt/importarr/venv/bin/importarr-status ExecStart=/opt/importarr/venv/bin/importarr
Restart=on-failure Restart=on-failure
RestartSec=5s RestartSec=5s
User=root User=root
+2
View File
@@ -32,6 +32,8 @@ fi
git fetch --prune origin git fetch --prune origin
git pull --ff-only git pull --ff-only
"$VENV/bin/pip" install --upgrade "$REPO_DIR" "$VENV/bin/pip" install --upgrade "$REPO_DIR"
install -m 0644 "$REPO_DIR/deploy/importarr.service" /etc/systemd/system/importarr.service
systemctl daemon-reload
GIT_SHA="$(git rev-parse --short=12 HEAD 2>/dev/null || printf development)" GIT_SHA="$(git rev-parse --short=12 HEAD 2>/dev/null || printf development)"
BUILD_DATE="$(date -u +%Y-%m-%dT%H:%M:%SZ)" BUILD_DATE="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
cat > "$PREFIX/build.env" <<EOF cat > "$PREFIX/build.env" <<EOF
+108
View File
@@ -0,0 +1,108 @@
#!/bin/sh
set -eu
script_dir=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
repo_dir=$(CDPATH= cd -- "$script_dir/../.." && pwd)
compose_file="$repo_dir/deploy/docker-compose.review.yml"
env_file="$repo_dir/deploy/importarr.review.env"
env_example="$repo_dir/deploy/importarr.review.env.example"
data_dir="$repo_dir/.review-data"
repo_name=$(printf '%s' "${repo_dir##*/}" | tr '[:upper:]' '[:lower:]' | tr -c 'a-z0-9_-' '-')
repo_id=$(printf '%s' "$repo_dir" | cksum | awk '{print $1}')
project_name="importarr-review-${repo_name}-${repo_id}"
review_image="${project_name}-importarr:review"
usage() {
echo "Usage: $0 {up|update|status|logs|stop|down|reset}" >&2
exit 2
}
command -v docker >/dev/null 2>&1 || {
echo "docker is required" >&2
exit 1
}
if docker info >/dev/null 2>&1; then
docker_with_sudo=false
elif command -v sudo >/dev/null 2>&1 && sudo -n docker info >/dev/null 2>&1; then
docker_with_sudo=true
else
echo "cannot access the Docker daemon with docker or sudo -n docker" >&2
exit 1
fi
compose() {
selected_env_file=$env_example
if [ -f "$env_file" ]; then
selected_env_file=$env_file
fi
if [ "$docker_with_sudo" = true ]; then
sudo -n env REVIEW_IMAGE="$review_image" REVIEW_SERVICE_ENV_FILE="$selected_env_file" \
docker compose --project-name "$project_name" --project-directory "$repo_dir/deploy" \
--env-file "$selected_env_file" -f "$compose_file" "$@"
else
REVIEW_IMAGE="$review_image" REVIEW_SERVICE_ENV_FILE="$selected_env_file" \
docker compose --project-name "$project_name" --project-directory "$repo_dir/deploy" \
--env-file "$selected_env_file" -f "$compose_file" "$@"
fi
}
ensure_env_file() {
if [ ! -f "$env_file" ]; then
cp "$env_example" "$env_file"
echo "Created $env_file from the safe review defaults."
fi
}
remove_data() {
if rm -rf "$data_dir" 2>/dev/null && [ ! -e "$data_dir" ]; then
return
fi
if [ "$docker_with_sudo" = true ]; then
sudo -n rm -rf "$data_dir"
else
echo "cannot remove root-owned review data without passwordless sudo: $data_dir" >&2
exit 1
fi
}
case "${1:-}" in
up|update)
ensure_env_file
mkdir -p "$data_dir/config" "$data_dir/downloads" "$data_dir/movies" "$data_dir/tv"
compose up --detach --build --wait
;;
status)
compose ps
container_id=$(compose ps --quiet importarr)
[ -n "$container_id" ] || {
echo "review service is not running" >&2
exit 1
}
health=$(if [ "$docker_with_sudo" = true ]; then
sudo -n docker inspect --format '{{if .State.Health}}{{.State.Health.Status}}{{else}}none{{end}}' "$container_id"
else
docker inspect --format '{{if .State.Health}}{{.State.Health.Status}}{{else}}none{{end}}' "$container_id"
fi)
[ "$health" = healthy ] || {
echo "review service is not healthy (status: $health)" >&2
exit 1
}
;;
logs)
compose logs --follow --tail=200
;;
stop)
compose stop
;;
down)
compose down --remove-orphans
;;
reset)
compose down --remove-orphans
remove_data
echo "Removed review data: $data_dir"
;;
*) usage ;;
esac
+10
View File
@@ -0,0 +1,10 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="theme-color" content="#252B42" />
<title>Importarr</title>
</head>
<body><div id="root"></div><script type="module" src="/src/main.jsx"></script></body>
</html>
+5604
View File
File diff suppressed because it is too large Load Diff
+32
View File
@@ -0,0 +1,32 @@
{
"name": "importarr-ui",
"private": true,
"version": "0.1.0",
"type": "module",
"scripts": {
"dev": "vite",
"build": "vite build",
"test": "vitest run"
},
"dependencies": {
"@radix-ui/react-dialog": "^1.1.14",
"@radix-ui/react-slot": "^1.2.3",
"class-variance-authority": "^0.7.1",
"clsx": "^2.1.1",
"lucide-react": "^0.468.0",
"react": "^18.3.1",
"react-dom": "^18.3.1",
"tailwind-merge": "^2.6.0"
},
"devDependencies": {
"@testing-library/jest-dom": "^6.6.3",
"@testing-library/react": "^16.1.0",
"@vitejs/plugin-react": "^4.3.4",
"autoprefixer": "^10.4.20",
"postcss": "^8.4.49",
"tailwindcss": "^3.4.17",
"vite": "^6.0.5",
"vitest": "^2.1.8",
"jsdom": "^25.0.1"
}
}
+1
View File
@@ -0,0 +1 @@
export default { plugins: { tailwindcss: {}, autoprefixer: {} } };
+23
View File
@@ -0,0 +1,23 @@
import * as DialogPrimitive from "@radix-ui/react-dialog";
import { Slot } from "@radix-ui/react-slot";
import { cva } from "class-variance-authority";
import { X } from "lucide-react";
import { cn } from "../lib/utils";
const buttonVariants = cva("inline-flex h-9 items-center justify-center gap-2 rounded-md px-4 text-sm font-medium transition-colors focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-secondary disabled:pointer-events-none disabled:opacity-50", {
variants: { variant: { default: "bg-primary text-white hover:bg-secondary", secondary: "bg-secondary text-white hover:bg-primary", outline: "border bg-card hover:bg-input", ghost: "hover:bg-input", destructive: "bg-primary text-white hover:bg-secondary" }, size: { default: "h-9 px-4", icon: "h-9 w-9 p-0", sm: "h-8 px-3" } },
defaultVariants: { variant: "default", size: "default" }
});
export function Button({ className, variant, size, asChild = false, ...props }) { const Comp = asChild ? Slot : "button"; return <Comp className={cn(buttonVariants({ variant, size }), className)} {...props} />; }
export function Input({ className, ...props }) { return <input className={cn("flex h-9 w-full rounded-md border bg-input px-3 py-1 text-sm outline-none placeholder:text-muted-foreground focus:ring-2 focus:ring-secondary", className)} {...props} />; }
export function Card({ className, ...props }) { return <section className={cn("rounded-lg border bg-card shadow-sm", className)} {...props} />; }
export function CardHeader({ className, ...props }) { return <div className={cn("flex flex-col gap-1.5 p-6", className)} {...props} />; }
export function CardTitle({ className, ...props }) { return <h2 className={cn("text-lg font-semibold", className)} {...props} />; }
export function CardContent({ className, ...props }) { return <div className={cn("p-6 pt-0", className)} {...props} />; }
export const Dialog = DialogPrimitive.Root;
export const DialogTrigger = DialogPrimitive.Trigger;
export const DialogClose = DialogPrimitive.Close;
export function DialogContent({ className, children, ...props }) { return <DialogPrimitive.Portal><DialogPrimitive.Overlay className="fixed inset-0 z-40 bg-background/80 backdrop-blur-sm" /><DialogPrimitive.Content className={cn("fixed left-1/2 top-1/2 z-50 max-h-[90vh] w-[calc(100%-2rem)] max-w-2xl -translate-x-1/2 -translate-y-1/2 overflow-y-auto rounded-lg border bg-card p-6 shadow-xl", className)} {...props}>{children}<DialogPrimitive.Close className="absolute right-4 top-4 rounded-sm text-muted-foreground hover:text-foreground" aria-label="Close"><X className="h-4 w-4" /></DialogPrimitive.Close></DialogPrimitive.Content></DialogPrimitive.Portal>; }
export function DialogHeader({ className, ...props }) { return <div className={cn("mb-4 space-y-1.5", className)} {...props} />; }
export function DialogTitle({ className, ...props }) { return <DialogPrimitive.Title className={cn("text-lg font-semibold", className)} {...props} />; }
export function Badge({ className, ...props }) { return <span className={cn("inline-flex rounded-full bg-primary px-2 py-0.5 text-xs font-medium text-white", className)} {...props} />; }
+33
View File
@@ -0,0 +1,33 @@
@tailwind base;
@tailwind components;
@tailwind utilities;
@layer base {
:root {
color-scheme: light;
--background: #F8FAFC;
--card: #FFFFFF;
--input: #FFFFFF;
--primary: #4A43EC;
--secondary: #7171FF;
--accent: #2AD1ED;
--foreground: #1A1D2E;
--muted-foreground: #64748B;
--border: #E2E8F0;
}
.dark {
color-scheme: dark;
--background: #252B42;
--card: #303753;
--input: #23283B;
--primary: #4A43EC;
--secondary: #7171FF;
--accent: #42ECF5;
--foreground: #FFFFFF;
--muted-foreground: #8B95B7;
--border: #3D4668;
}
* { @apply border-border; }
body { @apply m-0 min-w-0 bg-background text-foreground antialiased; }
button, input { font: inherit; }
}
+3
View File
@@ -0,0 +1,3 @@
import { clsx } from "clsx";
import { twMerge } from "tailwind-merge";
export const cn = (...inputs) => twMerge(clsx(inputs));
+51
View File
@@ -0,0 +1,51 @@
import React, { useEffect, useState } from "react";
import { createRoot } from "react-dom/client";
import { AlertTriangle, Clock, Download, FolderPlus, History, Info, Menu, Moon, Pause, Play, RefreshCw, Settings, Square, Sun, Trash2, XCircle, Zap } from "lucide-react";
import "./globals.css";
import { Badge, Button, Card, CardContent, CardHeader, CardTitle, Dialog, DialogContent, DialogHeader, DialogTitle, DialogTrigger, Input } from "./components/ui";
const groups = ["sab_processing", "ready", "importing", "failed", "ignored_category", "manual_batch", "completed"];
const labels = { sab_processing: "SAB processing", ready: "Ready", importing: "Importing", failed: "Failed", ignored_category: "Ignored category", manual_batch: "Manual batch", completed: "Completed" };
const bytes = value => { let n=Number(value||0), i=0; const units=["B","KB","MB","GB","TB"]; if(!n)return "size unknown"; while(n>=1024&&i<4){n/=1024;i++} return `${n.toFixed(n>=10||!i?0:1)} ${units[i]}`; };
const duration = value => { const s=Math.max(0,Math.floor(Number(value||0))); return s>=60?`${Math.floor(s/60)}m ${String(s%60).padStart(2,"0")}s`:`${s}s`; };
async function request(url, options={}) {
const response=await fetch(url, options.body ? {...options,headers:{"content-type":"application/json",...options.headers},body:JSON.stringify(options.body)} : options);
if(!response.ok){ const error=await response.json().catch(()=>({detail:response.statusText})); throw new Error(typeof error.detail==="string"?error.detail:"Request failed"); }
return response.status===204?null:response.json();
}
function Modal({ trigger, title, children }) { return <Dialog><DialogTrigger asChild>{trigger}</DialogTrigger><DialogContent><DialogHeader><DialogTitle>{title}</DialogTitle></DialogHeader>{children}</DialogContent></Dialog>; }
function Field({ label, ...props }) { return <label className="grid gap-2 text-sm font-medium">{label}<Input {...props}/></label>; }
export function SettingsDialog({ status, refresh }) {
const [result,setResult]=useState({});
const [urls,setUrls]=useState({sab_url:"",radarr_url:"",sonarr_url:""});
useEffect(()=>setUrls({sab_url:status?.sab_url||"",radarr_url:status?.radarr_url||"",sonarr_url:status?.sonarr_url||""}),[status?.sab_url,status?.radarr_url,status?.sonarr_url]);
const submit=async e=>{ e.preventDefault(); try { await request("/api/settings",{method:"POST",body:Object.fromEntries(new FormData(e.currentTarget))}); e.currentTarget.reset(); await refresh(); } catch(error){ alert(error.message); } };
const test=async (service,form)=>{ const prefix=service==="sabnzbd"?"sab":service; setResult(r=>({...r,[service]:"Testing…"})); try { const data=await request("/api/settings/test-connection",{method:"POST",body:{service,url:form.elements[`${prefix}_url`].value,api_key:form.elements[`${prefix}_api_key`].value}}); setResult(r=>({...r,[service]:data.message})); } catch(error){setResult(r=>({...r,[service]:error.message}))} };
return <><Modal title="Settings" trigger={<Button variant="ghost" className="w-full justify-start"><Settings/> Settings</Button>}><form onSubmit={submit} className="grid gap-6">{[["sab","sabnzbd","SABnzbd"],["radarr","radarr","Radarr"],["sonarr","sonarr","Sonarr"]].map(([prefix,service,label])=><fieldset className="grid gap-3 rounded-md border p-4" key={service}><legend className="px-2 font-semibold">{label}</legend><Field label={`${label} URL`} name={`${prefix}_url`} type="url" required={prefix==="sab"} value={urls[`${prefix}_url`]} onChange={e=>setUrls(current=>({...current,[e.target.name]:e.target.value}))}/><Field label="API token" name={`${prefix}_api_key`} type="password" autoComplete="off" placeholder={status?.[`${prefix}_api_key_configured`]?"Configured; enter replacement":"API token"}/><div className="flex items-center gap-3"><Button type="button" variant="outline" onClick={e=>test(service,e.currentTarget.form)}>Test connection</Button><span className="text-sm text-muted-foreground">{result[service]}</span></div></fieldset>)}<p className="text-sm text-muted-foreground">Blank token fields clear stored tokens.</p><Button>Save settings</Button></form></Modal><Button variant="ghost" className="w-full justify-start" onClick={async()=>{try{await request("/api/import/run-now",{method:"POST",body:{force:true}});await refresh()}catch(error){alert(error.message)}}}><Zap/> Force run now</Button></>;
}
function ManualBatches({ refresh }) { const [batches,setBatches]=useState([]); const load=()=>request("/api/manual-batches").then(setBatches).catch(()=>{}); return <Modal title="Manual batches" trigger={<Button variant="ghost" className="w-full justify-start" onClick={load}><FolderPlus/> Manual batches</Button>}><form className="flex flex-col gap-3 sm:flex-row" onSubmit={async e=>{e.preventDefault();try{await request("/api/manual-batches",{method:"POST",body:{path:e.currentTarget.path.value}});e.currentTarget.reset();load();refresh()}catch(error){alert(error.message)}}}><Input name="path" required placeholder="Folder under download root"/><Button>Add batch</Button></form><div className="mt-4 grid gap-2">{batches.length?batches.map(b=><div className="rounded-md border p-3 text-sm" key={b.id}><strong>#{b.id} · {b.status}</strong><p className="break-all text-muted-foreground">{b.path}</p></div>):<p className="text-muted-foreground">No manual batches.</p>}</div></Modal>; }
export function App(){
const [status,setStatus]=useState(null), [jobs,setJobs]=useState([]), [update,setUpdate]=useState(null), [menu,setMenu]=useState(false);
const [dark,setDark]=useState(()=>localStorage.getItem("importarr-theme")!=="light" && (localStorage.getItem("importarr-theme")==="dark"||matchMedia("(prefers-color-scheme: dark)").matches));
const refresh=async()=>{ try { const [s,j]=await Promise.all([request("/api/status"),request("/api/jobs")]); setStatus(s);setJobs((j.jobs||[]).map(job=>({...job,reason:[job.reason,job.source_type==="sab"&&`SAB ${job.sab_status||"—"}${job.sab_category?` · ${job.sab_category}`:""}`,job.batch_id&&`batch ${job.batch_id}`].filter(Boolean).join(" · ")})));document.title=`${j.jobs?.length?`${j.jobs.length} jobs`:"Idle"} · Importarr`; } catch(error){ console.error(error); } };
useEffect(()=>{document.documentElement.classList.toggle("dark",dark);localStorage.setItem("importarr-theme",dark?"dark":"light")},[dark]);
useEffect(()=>{refresh();const id=setInterval(refresh,2000);request("/api/control/update-check").then(setUpdate).catch(()=>{});return()=>clearInterval(id)},[]);
const post=async(url,body)=>{try{await request(url,{method:"POST",body});await refresh()}catch(error){alert(error.message)}};
const control=action=>{if(action==="cancel-current"&&!confirm("Cancel the current import job?"))return;post(`/api/control/${action}`)};
const current=status?.current, currentName=typeof current==="object"?(current.file||current.name):current;
return <div className="min-h-screen bg-background">
<header className="sticky top-0 z-30 border-b bg-card/95 backdrop-blur"><div className="mx-auto flex max-w-7xl items-center gap-3 p-4"><Download className="h-7 w-7 text-accent"/><div><h1 className="text-xl font-bold">Importarr</h1><p className="text-xs capitalize text-muted-foreground">{status?.control?.queue_mode||"Connecting"}</p></div><div className="ml-auto hidden max-w-md truncate text-sm text-muted-foreground sm:block">{currentName||"No active import"}</div><Button size="icon" aria-label="Start imports" onClick={()=>control("start")}><Play/></Button><Button size="icon" variant="outline" aria-label="Pause imports" onClick={()=>control("pause")}><Pause/></Button><div className="relative"><Button size="icon" variant="outline" aria-label="Open menu" onClick={()=>setMenu(!menu)}><Menu/></Button>{menu&&<Card className="absolute right-0 mt-2 w-60 p-2"><SettingsDialog status={status} refresh={refresh}/><ManualBatches refresh={refresh}/><Modal title="Service info" trigger={<Button variant="ghost" className="w-full justify-start"><Info/> Service info</Button>}><dl className="grid grid-cols-[auto_1fr] gap-x-4 gap-y-2 text-sm">{status&&Object.entries({Version:status.build.version,"Build date":status.build.build_date,"Git SHA":status.build.git_sha,"SAB URL":status.sab_url,"Download root":status.download_root,"Movies root":status.movies_root,"TV root":status.tv_root,"Write auth":status.auth_enabled?"enabled":"disabled"}).map(([k,v])=><React.Fragment key={k}><dt className="font-medium">{k}</dt><dd className="break-all text-muted-foreground">{String(v)}</dd></React.Fragment>)}</dl></Modal><Button variant="ghost" className="w-full justify-start" onClick={()=>setDark(!dark)}>{dark?<Sun/>:<Moon/>}{dark?"Light":"Dark"} theme</Button><Button variant="ghost" className="w-full justify-start" onClick={()=>control("stop")}><Square/> Stop queue</Button><Button variant="ghost" className="w-full justify-start" onClick={()=>control("cancel-current")}><XCircle/> Cancel current</Button><Button variant="ghost" className="w-full justify-start" onClick={()=>post("/api/import/run-now",{force:true})}><Zap/> Force run now</Button></Card>}</div></div></header>
<main className="mx-auto grid max-w-7xl gap-4 p-4 sm:p-6">
{update?.update_available&&<Card className="border-secondary"><CardContent className="flex flex-wrap items-center gap-3 p-4"><AlertTriangle className="text-accent"/><strong>Version {update.latest_version} is available</strong><Button className="ml-auto" onClick={()=>confirm("Update Importarr now?")&&post("/api/control/update")}>Update now</Button></CardContent></Card>}
<div className="grid grid-cols-2 gap-4 lg:grid-cols-5">{[["Queue mode",status?.control?.queue_mode],["Current import",currentName||"Idle"],["Imported",status?.imported_total],["Failed",status?.failed_total],["Queue items",status?.queue_total]].map(([label,value])=><Card key={label}><CardContent className="p-4"><strong className="block truncate text-lg capitalize">{value??"—"}</strong><span className="text-sm text-muted-foreground">{label}</span></CardContent></Card>)}</div>
<Card><CardHeader className="flex-row items-center justify-between"><div><CardTitle>Current import</CardTitle><p className="text-sm text-muted-foreground">{currentName?`Running ${duration(current.elapsed_seconds)}`:"Waiting for an active import"}</p></div><Clock className="text-accent"/></CardHeader><CardContent><p className="mb-3 break-all text-sm">{currentName||"No active copy."}</p><progress className="h-2 w-full accent-accent" max="100" value={typeof current==="object"?current.percent||0:0}/>{currentName&&<p className="mt-2 text-xs text-muted-foreground">{Number(current.percent||0).toFixed(1)}% · {bytes(current.bytes_copied)} / {bytes(current.total_bytes)}</p>}</CardContent></Card>
<Card><CardHeader className="flex-row items-center gap-3"><History className="text-accent"/><div><CardTitle>Queue and history</CardTitle><p className="text-sm text-muted-foreground">Grouped by processing state</p></div></CardHeader><CardContent className="grid gap-6">{jobs.length?groups.map(group=>{const items=jobs.filter(j=>j.group===group);return items.length?<section key={group}><h3 className="mb-3 flex items-center gap-2 font-semibold">{labels[group]} <Badge>{items.length}</Badge></h3><div className="grid gap-2">{items.map(j=><article className="grid gap-3 rounded-md border bg-input p-4 lg:grid-cols-[minmax(0,1fr)_auto] lg:items-center" key={j.id}><div className="min-w-0"><strong className="block break-words">{j.name}</strong><p className="break-all text-xs text-muted-foreground">{j.source_type} · attempts {j.attempt_count||0} · {j.relative_path||j.storage||j.source_id}</p><div className="mt-2 flex flex-wrap items-center gap-2"><Badge>{j.state}</Badge><span className="text-xs text-muted-foreground">{j.reason}</span></div></div><div className="flex gap-2">{j.can_run_now&&<Button size="icon" title="Run now" onClick={()=>post(`/api/queue-items/${j.id}/action`,{action:"run-now"})}><Play/></Button>}{j.can_retry&&<Button size="icon" variant="outline" title="Retry" onClick={()=>post(`/api/queue-items/${j.id}/action`,{action:"retry"})}><RefreshCw/></Button>}{j.can_ignore&&<Button size="icon" variant="outline" title="Ignore" onClick={()=>confirm("Ignore this queue item?")&&post(`/api/queue-items/${j.id}/action`,{action:"ignore"})}><XCircle/></Button>}{j.can_remove&&<Button size="icon" variant="outline" title="Remove" onClick={()=>confirm("Remove this queue item?")&&post(`/api/queue-items/${j.id}/action`,{action:"remove"})}><Trash2/></Button>}</div></article>)}</div></section>:null}):<p className="text-muted-foreground">No queue items.</p>}</CardContent></Card>
</main></div>
}
const root=document.getElementById("root");
if(root) createRoot(root).render(<React.StrictMode><App/></React.StrictMode>);
+29
View File
@@ -0,0 +1,29 @@
import "@testing-library/jest-dom/vitest";
import { fireEvent, render, screen, waitFor } from "@testing-library/react";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { SettingsDialog } from "./main";
describe("Importarr UI behavior", () => {
beforeEach(() => vi.stubGlobal("fetch", vi.fn()));
afterEach(() => vi.restoreAllMocks());
it("hydrates settings URLs when status arrives asynchronously", async () => {
const { rerender } = render(<SettingsDialog status={null} refresh={vi.fn()} />);
fireEvent.click(screen.getByRole("button", { name: "Settings" }));
expect(screen.getByLabelText("SABnzbd URL")).toHaveValue("");
rerender(<SettingsDialog status={{ sab_url: "http://sab", radarr_url: "http://radarr", sonarr_url: "http://sonarr" }} refresh={vi.fn()} />);
await waitFor(() => expect(screen.getByLabelText("SABnzbd URL")).toHaveValue("http://sab"));
expect(screen.getByLabelText("Radarr URL")).toHaveValue("http://radarr");
expect(screen.getByLabelText("Sonarr URL")).toHaveValue("http://sonarr");
});
it("posts a forced global run", async () => {
fetch.mockResolvedValue({ ok: true, status: 200, json: async () => ({}) });
render(<SettingsDialog status={null} refresh={vi.fn()} />);
fireEvent.click(screen.getByRole("button", { name: "Force run now" }));
await waitFor(() => expect(fetch).toHaveBeenCalledWith("/api/import/run-now", expect.objectContaining({ method: "POST", body: JSON.stringify({ force: true }) })));
});
});
+16
View File
@@ -0,0 +1,16 @@
/** @type {import('tailwindcss').Config} */
export default {
darkMode: ["class"],
content: ["./index.html", "./src/**/*.{js,jsx}"],
theme: {
extend: {
colors: {
background: "var(--background)", foreground: "var(--foreground)",
card: "var(--card)", input: "var(--input)", primary: "var(--primary)",
secondary: "var(--secondary)", accent: "var(--accent)",
muted: { foreground: "var(--muted-foreground)" }, border: "var(--border)"
}
}
},
plugins: []
};
+14
View File
@@ -0,0 +1,14 @@
import { defineConfig } from "vite";
import react from "@vitejs/plugin-react";
import { resolve } from "node:path";
export default defineConfig({
plugins: [react()],
base: "/static/",
server: { proxy: { "/api": "http://127.0.0.1:8765", "/health": "http://127.0.0.1:8765" } },
build: {
outDir: resolve(import.meta.dirname, "../importarr/static"),
emptyOutDir: true,
rollupOptions: { output: { entryFileNames: "assets/app.js", assetFileNames: "assets/app.[ext]" } }
}
});
+19 -2
View File
@@ -8,14 +8,31 @@ from . import __version__
def build_info() -> dict[str, str]: def build_info() -> dict[str, str]:
build_date = os.getenv("IMPORTARR_BUILD_DATE", "development")
return { return {
"name": "Importarr", "name": "Importarr",
"version": os.getenv("IMPORTARR_VERSION", __version__), "version": os.getenv("IMPORTARR_VERSION", __version__),
"build_date": os.getenv("IMPORTARR_BUILD_DATE", "development"), "build_date": local_timestamp(build_date),
"git_sha": os.getenv("IMPORTARR_GIT_SHA", "development"), "git_sha": os.getenv("IMPORTARR_GIT_SHA", "development"),
"python": platform.python_version(), "python": platform.python_version(),
"started_at": STARTED_AT, "started_at": STARTED_AT,
} }
STARTED_AT = datetime.now(UTC).isoformat(timespec="seconds") def local_timestamp(value: str) -> str:
if value == "development":
return value
normalized = value.removesuffix("Z") + "+00:00" if value.endswith("Z") else value
try:
timestamp = datetime.fromisoformat(normalized)
except ValueError:
return value
if timestamp.tzinfo is None:
timestamp = timestamp.replace(tzinfo=UTC)
return timestamp.astimezone().isoformat(timespec="seconds")
STARTED_AT = datetime.now(UTC).astimezone().isoformat(timespec="seconds")
+24
View File
@@ -1,6 +1,7 @@
from __future__ import annotations from __future__ import annotations
import os import os
import shlex
from pathlib import Path from pathlib import Path
from pydantic import BaseModel, Field from pydantic import BaseModel, Field
@@ -10,6 +11,7 @@ class Settings(BaseModel):
sab_url: str = "http://sabnzbd:8080" sab_url: str = "http://sabnzbd:8080"
sab_api_key: str | None = None sab_api_key: str | None = None
sab_category: str = "manual" sab_category: str = "manual"
sab_storage_root: Path | None = None
download_root: Path = Path("/data/downloads/manual") download_root: Path = Path("/data/downloads/manual")
movies_root: Path = Path("/data/movies") movies_root: Path = Path("/data/movies")
tv_root: Path = Path("/data/tv") tv_root: Path = Path("/data/tv")
@@ -20,6 +22,13 @@ class Settings(BaseModel):
sonarr_url: str | None = None sonarr_url: str | None = None
sonarr_api_key: str | None = None sonarr_api_key: str | None = None
auth_token: str | None = None auth_token: str | None = None
start_command: list[str] = Field(default_factory=lambda: ["systemctl", "start", "manual-media-import.service"])
stop_command: list[str] = Field(default_factory=lambda: ["systemctl", "stop", "manual-media-import.service"])
restart_command: list[str] = Field(default_factory=lambda: ["systemctl", "restart", "manual-media-import.service"])
update_command: list[str] = Field(default_factory=lambda: ["sh", "deploy/repo-upgrade.sh"])
update_release_url: str = "https://gitea.delphas.dk/api/v1/repos/daniels/importarr/releases/latest"
update_check_timeout_seconds: int = Field(default=15, ge=1)
control_command_timeout_seconds: int = Field(default=120, ge=1)
bind_host: str = "127.0.0.1" bind_host: str = "127.0.0.1"
bind_port: int = 8765 bind_port: int = 8765
poll_seconds: int = Field(default=60, ge=5) poll_seconds: int = Field(default=60, ge=5)
@@ -30,6 +39,7 @@ class Settings(BaseModel):
sab_url=os.getenv("IMPORTARR_SAB_URL", cls.model_fields["sab_url"].default), sab_url=os.getenv("IMPORTARR_SAB_URL", cls.model_fields["sab_url"].default),
sab_api_key=_env_secret("IMPORTARR_SAB_API_KEY"), sab_api_key=_env_secret("IMPORTARR_SAB_API_KEY"),
sab_category=os.getenv("IMPORTARR_SAB_CATEGORY", "manual"), sab_category=os.getenv("IMPORTARR_SAB_CATEGORY", "manual"),
sab_storage_root=Path(os.getenv("IMPORTARR_SAB_STORAGE_ROOT")) if os.getenv("IMPORTARR_SAB_STORAGE_ROOT") else None,
download_root=Path(os.getenv("IMPORTARR_DOWNLOAD_ROOT", "/data/downloads/manual")), download_root=Path(os.getenv("IMPORTARR_DOWNLOAD_ROOT", "/data/downloads/manual")),
movies_root=Path(os.getenv("IMPORTARR_MOVIES_ROOT", "/data/movies")), movies_root=Path(os.getenv("IMPORTARR_MOVIES_ROOT", "/data/movies")),
tv_root=Path(os.getenv("IMPORTARR_TV_ROOT", "/data/tv")), tv_root=Path(os.getenv("IMPORTARR_TV_ROOT", "/data/tv")),
@@ -40,6 +50,13 @@ class Settings(BaseModel):
sonarr_url=os.getenv("IMPORTARR_SONARR_URL"), sonarr_url=os.getenv("IMPORTARR_SONARR_URL"),
sonarr_api_key=_env_secret("IMPORTARR_SONARR_API_KEY"), sonarr_api_key=_env_secret("IMPORTARR_SONARR_API_KEY"),
auth_token=_env_secret("IMPORTARR_AUTH_TOKEN"), auth_token=_env_secret("IMPORTARR_AUTH_TOKEN"),
start_command=_env_command("IMPORTARR_START_COMMAND", ["systemctl", "start", "manual-media-import.service"]),
stop_command=_env_command("IMPORTARR_STOP_COMMAND", ["systemctl", "stop", "manual-media-import.service"]),
restart_command=_env_command("IMPORTARR_RESTART_COMMAND", ["systemctl", "restart", "manual-media-import.service"]),
update_command=_env_command("IMPORTARR_UPDATE_COMMAND", ["sh", "deploy/repo-upgrade.sh"]),
update_release_url=os.getenv("IMPORTARR_UPDATE_RELEASE_URL", "https://gitea.delphas.dk/api/v1/repos/daniels/importarr/releases/latest"),
update_check_timeout_seconds=int(os.getenv("IMPORTARR_UPDATE_CHECK_TIMEOUT_SECONDS", "15")),
control_command_timeout_seconds=int(os.getenv("IMPORTARR_CONTROL_COMMAND_TIMEOUT_SECONDS", "120")),
bind_host=os.getenv("IMPORTARR_BIND_HOST", "127.0.0.1"), bind_host=os.getenv("IMPORTARR_BIND_HOST", "127.0.0.1"),
bind_port=int(os.getenv("IMPORTARR_BIND_PORT", "8765")), bind_port=int(os.getenv("IMPORTARR_BIND_PORT", "8765")),
poll_seconds=int(os.getenv("IMPORTARR_POLL_SECONDS", "60")), poll_seconds=int(os.getenv("IMPORTARR_POLL_SECONDS", "60")),
@@ -61,3 +78,10 @@ def _env_secret(name: str) -> str | None:
if file_value: if file_value:
return Path(file_value).read_text(encoding="utf-8").strip() return Path(file_value).read_text(encoding="utf-8").strip()
return os.getenv(name) return os.getenv(name)
def _env_command(name: str, default: list[str]) -> list[str]:
value = os.getenv(name)
if not value:
return default
return shlex.split(value)
+9 -2
View File
@@ -27,10 +27,14 @@ class Importer:
target_root = self.tv_root if _looks_like_tv(source) else self.movies_root target_root = self.tv_root if _looks_like_tv(source) else self.movies_root
return _unique_path(target_root / source.name) return _unique_path(target_root / source.name)
def import_file(self, source: Path, should_cancel: Callable[[], bool] | None = None) -> ImportResult: def import_file(self, source: Path, should_cancel: Callable[[], bool] | None = None, on_progress: Callable[[int, int], None] | None = None) -> ImportResult:
target = self.target_for(source) target = self.target_for(source)
target.parent.mkdir(parents=True, exist_ok=True) target.parent.mkdir(parents=True, exist_ok=True)
partial = target.with_name(target.name + ".partial") partial = target.with_name(target.name + ".partial")
total = source.stat().st_size
copied = 0
if on_progress:
on_progress(copied, total)
try: try:
with source.open("rb") as src, partial.open("wb") as dst: with source.open("rb") as src, partial.open("wb") as dst:
while True: while True:
@@ -40,12 +44,15 @@ class Importer:
if not chunk: if not chunk:
break break
dst.write(chunk) dst.write(chunk)
copied += len(chunk)
if on_progress:
on_progress(copied, total)
dst.flush() dst.flush()
os.fsync(dst.fileno()) os.fsync(dst.fileno())
except ImportCancelled: except ImportCancelled:
partial.unlink(missing_ok=True) partial.unlink(missing_ok=True)
raise raise
if partial.stat().st_size != source.stat().st_size: if partial.stat().st_size != total:
raise IOError("partial copy size mismatch") raise IOError("partial copy size mismatch")
partial.rename(target) partial.rename(target)
source.unlink() source.unlink()
+446 -64
View File
@@ -1,13 +1,18 @@
from __future__ import annotations from __future__ import annotations
import json
import os
from pathlib import Path from pathlib import Path
import subprocess
import threading
import time
from typing import Annotated from typing import Annotated
import uvicorn import uvicorn
from fastapi import Depends, FastAPI, Header, HTTPException, Request import httpx
from fastapi.responses import HTMLResponse from fastapi import Depends, FastAPI, Header, HTTPException
from fastapi.responses import FileResponse
from fastapi.staticfiles import StaticFiles from fastapi.staticfiles import StaticFiles
from fastapi.templating import Jinja2Templates
from pydantic import BaseModel from pydantic import BaseModel
from .build_info import build_info from .build_info import build_info
@@ -20,9 +25,15 @@ from .state import State
settings = Settings.from_env() settings = Settings.from_env()
state = State(settings.state_path) state = State(settings.state_path)
templates = Jinja2Templates(directory=str(Path(__file__).parent / "templates")) STATIC_DIR = Path(__file__).parent / "static"
app = FastAPI(title="Importarr") app = FastAPI(title="Importarr")
app.mount("/static", StaticFiles(directory=str(Path(__file__).parent / "static")), name="static") app.mount("/static", StaticFiles(directory=str(STATIC_DIR)), name="static")
WORKER_ID = f"importarr-{os.getpid()}"
WORKER_SHUTDOWN_TIMEOUT_SECONDS = 5.0
_worker_thread: threading.Thread | None = None
_worker_stop = threading.Event()
MAX_RETRY_ATTEMPTS = 3
RETRY_DELAY_SECONDS = 60
class ManualBatchCreate(BaseModel): class ManualBatchCreate(BaseModel):
@@ -37,6 +48,51 @@ class QueueControlRequest(BaseModel):
mode: str mode: str
class QueueItemActionRequest(BaseModel):
action: str
class ControlCommandResponse(BaseModel):
status: str
command: list[str]
returncode: int
stdout: str
stderr: str
class UpdateCheckResponse(BaseModel):
status: str
current_version: str
latest_version: str | None
update_available: bool
release_url: str | None = None
class AppSettingsUpdate(BaseModel):
sab_url: str
sab_api_key: str | None = None
radarr_url: str | None = None
radarr_api_key: str | None = None
sonarr_url: str | None = None
sonarr_api_key: str | None = None
class ConnectionTestRequest(BaseModel):
service: str
url: str
api_key: str | None = None
def load_ui_settings() -> None:
for key in ("sab_url", "sab_api_key", "radarr_url", "radarr_api_key", "sonarr_url", "sonarr_api_key"):
stored = state.get_app_state(key)
if stored is not None:
setattr(settings, key, stored or None)
load_ui_settings()
def require_write_auth(authorization: Annotated[str | None, Header()] = None) -> None: def require_write_auth(authorization: Annotated[str | None, Header()] = None) -> None:
if not settings.auth_token: if not settings.auth_token:
return return
@@ -49,15 +105,19 @@ def health() -> dict[str, str]:
return {"status": "ok", "name": "Importarr", "version": build_info()["version"]} return {"status": "ok", "name": "Importarr", "version": build_info()["version"]}
@app.get("/", response_class=HTMLResponse) @app.get("/", response_class=FileResponse)
def index(request: Request) -> HTMLResponse: def index() -> FileResponse:
return templates.TemplateResponse("index.html", {"request": request, "status": status(), "batches": state.list_manual_batches()}) return FileResponse(STATIC_DIR / "index.html")
@app.get("/api/status") @app.get("/api/status")
def status() -> dict[str, object]: def status() -> dict[str, object]:
history = state.list_history() history = state.list_history()
control = control_status() control = control_status()
queue_items = state.list_queue_items(active_only=False)
queue_counts: dict[str, int] = {}
for row in queue_items:
queue_counts[row["state"]] = queue_counts.get(row["state"], 0) + 1
return { return {
"app": "Importarr", "app": "Importarr",
"build": build_info(), "build": build_info(),
@@ -66,19 +126,85 @@ def status() -> dict[str, object]:
"movies_root": str(settings.movies_root), "movies_root": str(settings.movies_root),
"tv_root": str(settings.tv_root), "tv_root": str(settings.tv_root),
"sab_url": settings.sab_url, "sab_url": settings.sab_url,
"sab_api_key_configured": bool(settings.sab_api_key),
"radarr_url": settings.radarr_url or "",
"radarr_api_key_configured": bool(settings.radarr_api_key),
"sonarr_url": settings.sonarr_url or "",
"sonarr_api_key_configured": bool(settings.sonarr_api_key),
"auth_enabled": bool(settings.auth_token), "auth_enabled": bool(settings.auth_token),
"bind": f"{settings.bind_host}:{settings.bind_port}", "bind": f"{settings.bind_host}:{settings.bind_port}",
"manual_batches": len(state.list_manual_batches(active_only=True)), "manual_batches": len(state.list_manual_batches(active_only=True)),
"imported_total": sum(1 for row in history if row["status"] == "imported"), "imported_total": sum(1 for row in history if row["status"] == "imported"),
"failed_total": sum(1 for row in history if row["status"] == "failed"), "failed_total": sum(1 for row in history if row["status"] == "failed"),
"queue_total": len(queue_items),
"queue_counts": queue_counts,
"current": control["current"], "current": control["current"],
"control": control, "control": control,
} }
@app.get("/api/settings")
def get_ui_settings() -> dict[str, object]:
return {
"sab_url": settings.sab_url,
"sab_api_key_configured": bool(settings.sab_api_key),
"radarr_url": settings.radarr_url or "",
"radarr_api_key_configured": bool(settings.radarr_api_key),
"sonarr_url": settings.sonarr_url or "",
"sonarr_api_key_configured": bool(settings.sonarr_api_key),
}
@app.post("/api/settings")
def update_ui_settings(payload: AppSettingsUpdate, _: None = Depends(require_write_auth)) -> dict[str, object]:
sab_url = payload.sab_url.strip()
if not sab_url:
raise HTTPException(status_code=400, detail="SAB URL is required")
values = {
"sab_url": sab_url,
"sab_api_key": (payload.sab_api_key or "").strip(),
"radarr_url": (payload.radarr_url or "").strip(),
"radarr_api_key": (payload.radarr_api_key or "").strip(),
"sonarr_url": (payload.sonarr_url or "").strip(),
"sonarr_api_key": (payload.sonarr_api_key or "").strip(),
}
for key, value in values.items():
state.set_app_state(key, value)
setattr(settings, key, value or None)
settings.sab_url = sab_url
return get_ui_settings()
@app.post("/api/settings/test-connection")
async def test_connection(payload: ConnectionTestRequest, _: None = Depends(require_write_auth)) -> dict[str, object]:
service = payload.service.strip().lower()
url = payload.url.strip().rstrip("/")
api_key = (payload.api_key or "").strip() or None
if service not in {"sabnzbd", "radarr", "sonarr"}:
raise HTTPException(status_code=400, detail="service must be sabnzbd, radarr, or sonarr")
if not url:
raise HTTPException(status_code=400, detail="URL is required")
if api_key is None:
api_key = getattr(settings, f"{service if service != 'sabnzbd' else 'sab'}_api_key")
try:
if service == "sabnzbd":
data = await SabnzbdClient(url, api_key).queue()
return {"ok": True, "service": service, "message": f"Connected to SABnzbd; {len(data.get('queue', {}).get('slots', []))} queued jobs visible."}
headers = {"X-Api-Key": api_key} if api_key else {}
async with httpx.AsyncClient(timeout=15) as client:
response = await client.get(f"{url}/api/v3/system/status", headers=headers)
response.raise_for_status()
data = response.json()
name = str(data.get("appName") or service.title())
version = str(data.get("version") or "unknown version")
return {"ok": True, "service": service, "message": f"Connected to {name} {version}."}
except Exception as exc:
return {"ok": False, "service": service, "message": f"Connection failed: {exc.__class__.__name__}"}
def control_status() -> dict[str, object]: def control_status() -> dict[str, object]:
mode = state.get_app_state("queue_mode", "running") or "running" mode = state.get_app_state("queue_mode", "running") or "running"
current = state.get_app_state("current_job") current = current_job_status()
cancel_requested = (state.get_app_state("cancel_requested", "false") or "false") == "true" cancel_requested = (state.get_app_state("cancel_requested", "false") or "false") == "true"
return { return {
"queue_mode": mode, "queue_mode": mode,
@@ -88,6 +214,21 @@ def control_status() -> dict[str, object]:
} }
def current_job_status() -> dict[str, object] | str:
raw = state.get_app_state("current_job") or ""
if not raw:
return ""
try:
data = json.loads(raw)
except json.JSONDecodeError:
return raw
if isinstance(data, dict):
started_at = float(data.get("started_at") or time.time())
data["elapsed_seconds"] = max(0, int(time.time() - started_at))
return data
return raw
def queue_accepting_new_jobs() -> bool: def queue_accepting_new_jobs() -> bool:
return (state.get_app_state("queue_mode", "running") or "running") == "running" return (state.get_app_state("queue_mode", "running") or "running") == "running"
@@ -103,8 +244,49 @@ def consume_cancel_request() -> bool:
return True return True
def set_current_job(name: str | None) -> None: def set_current_job(name: str | None, *, bytes_copied: int = 0, total_bytes: int = 0, started_at: float | None = None) -> float:
state.set_app_state("current_job", name or "") started = started_at or time.time()
if not name:
state.set_app_state("current_job", "")
return started
percent = round((bytes_copied / total_bytes * 100), 2) if total_bytes else 0
state.set_app_state(
"current_job",
json.dumps({"file": name, "name": Path(name).name if name else name, "bytes_copied": bytes_copied, "total_bytes": total_bytes, "percent": percent, "started_at": started}),
)
return started
def ensure_worker_running() -> None:
global _worker_thread
if _worker_thread and _worker_thread.is_alive():
return
_worker_stop.clear()
state.release_stale_claims()
_worker_thread = threading.Thread(target=_worker_loop, name="importarr-queue-worker", daemon=True)
_worker_thread.start()
def stop_worker(*, wait: bool = False) -> bool:
_worker_stop.set()
thread = _worker_thread
if wait and thread and thread.is_alive() and thread is not threading.current_thread():
thread.join(WORKER_SHUTDOWN_TIMEOUT_SECONDS)
return not thread or not thread.is_alive()
def _worker_loop() -> None:
importer = Importer(settings.movies_root, settings.tv_root)
while not _worker_stop.is_set():
try:
if queue_accepting_new_jobs():
sync_manual_queue()
item = state.claim_next_queue_item(WORKER_ID) if queue_accepting_new_jobs() else None
if item is not None:
_import_queue_item(item, importer, from_worker=True)
except Exception:
state.upsert_queue_item(source_type="system", source_id="queue-worker", name="Queue worker", state="failed", reason="worker loop error")
_worker_stop.wait(settings.poll_seconds)
@app.post("/api/control/queue") @app.post("/api/control/queue")
@@ -114,6 +296,9 @@ def set_queue_control(payload: QueueControlRequest, _: None = Depends(require_wr
state.set_app_state("queue_mode", payload.mode) state.set_app_state("queue_mode", payload.mode)
if payload.mode == "running": if payload.mode == "running":
state.set_app_state("cancel_requested", "false") state.set_app_state("cancel_requested", "false")
ensure_worker_running()
elif payload.mode == "stopped":
stop_worker()
return control_status() return control_status()
@@ -121,7 +306,8 @@ def set_queue_control(payload: QueueControlRequest, _: None = Depends(require_wr
def start_queue(_: None = Depends(require_write_auth)) -> dict[str, object]: def start_queue(_: None = Depends(require_write_auth)) -> dict[str, object]:
state.set_app_state("queue_mode", "running") state.set_app_state("queue_mode", "running")
state.set_app_state("cancel_requested", "false") state.set_app_state("cancel_requested", "false")
return control_status() ensure_worker_running()
return {"control": control_status(), "command_result": _run_control_command(settings.start_command)}
@app.post("/api/control/pause") @app.post("/api/control/pause")
@@ -133,7 +319,9 @@ def pause_queue(_: None = Depends(require_write_auth)) -> dict[str, object]:
@app.post("/api/control/stop") @app.post("/api/control/stop")
def stop_queue(_: None = Depends(require_write_auth)) -> dict[str, object]: def stop_queue(_: None = Depends(require_write_auth)) -> dict[str, object]:
state.set_app_state("queue_mode", "stopped") state.set_app_state("queue_mode", "stopped")
return control_status() state.set_app_state("cancel_requested", "true")
stop_worker()
return {"control": control_status(), "command_result": _run_control_command(settings.stop_command)}
@app.post("/api/control/cancel-current") @app.post("/api/control/cancel-current")
@@ -142,6 +330,92 @@ def cancel_current(_: None = Depends(require_write_auth)) -> dict[str, object]:
return control_status() return control_status()
@app.post("/api/control/restart")
def restart_service(_: None = Depends(require_write_auth)) -> dict[str, object]:
return _run_control_command(settings.restart_command)
@app.post("/api/control/update")
def update_service(_: None = Depends(require_write_auth)) -> dict[str, object]:
update = check_update_available()
if not update["update_available"]:
return {**update, "command": settings.update_command, "stdout": "", "stderr": ""}
result = _run_control_command(settings.update_command)
return {**update, "command_result": result}
@app.get("/api/control/update-check")
def update_check(_: None = Depends(require_write_auth)) -> dict[str, object]:
return check_update_available()
def check_update_available() -> dict[str, object]:
current = build_info()["version"]
try:
with httpx.Client(timeout=settings.update_check_timeout_seconds) as client:
response = client.get(settings.update_release_url, headers={"Accept": "application/json"})
response.raise_for_status()
release = response.json()
except Exception as exc:
raise HTTPException(status_code=502, detail=f"release check failed: {exc.__class__.__name__}") from exc
latest = str(release.get("tag_name") or release.get("name") or "").strip()
if not latest:
raise HTTPException(status_code=502, detail="release check failed: latest release has no tag_name")
payload = UpdateCheckResponse(
status="update_available" if _is_newer_version(latest, current) else "current",
current_version=current,
latest_version=latest,
update_available=_is_newer_version(latest, current),
release_url=release.get("html_url"),
)
return payload.model_dump()
def _is_newer_version(candidate: str, current: str) -> bool:
candidate_version = _version_key(candidate)
current_version = _version_key(current)
if candidate_version is None or current_version is None:
return candidate.lstrip("vV") != current.lstrip("vV") and current in {"", "development"}
return candidate_version > current_version
def _version_key(value: str) -> tuple[int, ...] | None:
normalized = value.strip().lstrip("vV").split("-", 1)[0]
parts = normalized.split(".")
if not parts or any(not part.isdigit() for part in parts):
return None
return tuple(int(part) for part in parts)
def _run_control_command(command: list[str]) -> dict[str, object]:
if not command:
raise HTTPException(status_code=500, detail="control command is not configured")
try:
result = subprocess.run(
command,
check=False,
capture_output=True,
text=True,
timeout=settings.control_command_timeout_seconds,
)
except subprocess.TimeoutExpired as exc:
raise HTTPException(status_code=504, detail=f"control command timed out after {exc.timeout} seconds") from exc
except OSError as exc:
raise HTTPException(status_code=500, detail=f"control command failed to start: {exc.__class__.__name__}") from exc
payload = ControlCommandResponse(
status="ok" if result.returncode == 0 else "failed",
command=command,
returncode=result.returncode,
stdout=result.stdout[-4000:],
stderr=result.stderr[-4000:],
).model_dump()
if result.returncode != 0:
raise HTTPException(status_code=500, detail=payload)
return payload
@app.get("/api/manual-batches") @app.get("/api/manual-batches")
def manual_batches() -> list[dict[str, object]]: def manual_batches() -> list[dict[str, object]]:
if queue_accepting_new_jobs(): if queue_accepting_new_jobs():
@@ -173,6 +447,35 @@ def history() -> list[dict[str, object]]:
return state.list_history() return state.list_history()
@app.post("/api/queue-items/{item_id}/action")
def queue_item_action(item_id: int, payload: QueueItemActionRequest, _: None = Depends(require_write_auth)) -> dict[str, object]:
item = state.get_queue_item(item_id)
if item is None:
raise HTTPException(status_code=404, detail="queue item not found")
if payload.action == "retry":
retry_state = "ready" if item["source_type"] in {"manual", "sab"} else "detected"
if not state.transition_queue_item_if_unclaimed(item_id, {"failed", "skipped"}, retry_state, "retry requested"):
raise HTTPException(status_code=409, detail="queue item is currently being imported or changed")
elif payload.action == "run-now":
claimed = state.claim_queue_item(item_id, WORKER_ID, {"ready", "failed", "retrying"})
if claimed is None:
raise HTTPException(status_code=409, detail="queue item is not available to run now")
imported = _import_queue_item(claimed, Importer(settings.movies_root, settings.tv_root), force=True, from_worker=True)
updated = state.get_queue_item(item_id)
return {"status": "imported" if imported else "updated", "imported": imported, "item": serialize_queue_item(updated or item)}
elif payload.action == "ignore":
if not state.transition_queue_item_if_unclaimed(item_id, {str(item["state"])}, "skipped", "ignored by user"):
raise HTTPException(status_code=409, detail="queue item is currently being imported or changed")
elif payload.action == "remove":
if not state.delete_queue_item_if_unclaimed(item_id):
raise HTTPException(status_code=409, detail="queue item is currently being imported")
return {"status": "removed", "id": item_id}
else:
raise HTTPException(status_code=400, detail="action must be retry, run-now, ignore, or remove")
updated = state.get_queue_item(item_id)
return {"status": "updated", "item": serialize_queue_item(updated or item)}
@app.get("/api/jobs") @app.get("/api/jobs")
async def jobs() -> dict[str, object]: async def jobs() -> dict[str, object]:
return await preview() return await preview()
@@ -183,7 +486,7 @@ async def preview() -> dict[str, object]:
if queue_accepting_new_jobs(): if queue_accepting_new_jobs():
await sync_queue() await sync_queue()
jobs = queue_jobs() jobs = queue_jobs()
return {"sab_status": "ok", "jobs": jobs, "would_import": sum(1 for row in jobs if row["state"] in {"ready", "manual_batch"}), "control": control_status()} return {"sab_status": "ok", "jobs": jobs, "groups": group_jobs(jobs), "would_import": sum(1 for row in jobs if row["state"] in {"ready", "manual_batch"}), "control": control_status()}
async def sync_queue() -> None: async def sync_queue() -> None:
@@ -197,31 +500,84 @@ async def sync_queue() -> None:
return return
slots = data.get("history", {}).get("slots", []) slots = data.get("history", {}).get("slots", [])
for item in slots: for item in slots:
readiness = classify_history_item(item, active, settings.sab_category, settings.download_root) readiness = classify_history_item(item, active, settings.sab_category, settings.download_root, sab_storage_root=settings.sab_storage_root)
job_id = str(item.get("nzo_id") or item.get("nzoid") or item.get("name") or "") job_id = str(item.get("nzo_id") or item.get("nzoid") or item.get("name") or "")
if not job_id: if not job_id:
continue continue
if readiness.ready and readiness.storage: if readiness.ready and readiness.storage:
for video in scan_videos(readiness.storage): for video in scan_videos(readiness.storage):
state.upsert_queue_item(source_type="sab", source_id=str(video.path), source_path=video.path, name=video.path.name, state="ready", reason=readiness.reason, relative_path=str(video.relative_path), size=video.size, job_id=job_id) state.upsert_queue_item(source_type="sab", source_id=str(video.path), source_path=video.path, name=video.path.name, state="ready", reason=readiness.reason, relative_path=str(video.relative_path), size=video.size, job_id=job_id, sab_category=str(item.get("category") or item.get("cat") or ""))
else: else:
state.upsert_queue_item(source_type="sab", source_id=job_id, source_path=readiness.storage, name=str(item.get("name") or job_id), state=readiness.state, reason=readiness.reason, job_id=job_id) pending_state = "waiting_for_sab" if readiness.state not in {"failed", "skipped"} else readiness.state
state.upsert_queue_item(source_type="sab", source_id=job_id, source_path=readiness.storage, name=str(item.get("name") or job_id), state=pending_state, reason=readiness.reason, job_id=job_id, sab_category=str(item.get("category") or item.get("cat") or ""))
def queue_jobs() -> list[dict[str, object]]: def queue_jobs() -> list[dict[str, object]]:
return [ return [serialize_queue_item(item) for item in state.list_queue_items(active_only=False) if item["source_type"] != "system"]
{
"name": item["name"],
"state": item["state"], def serialize_queue_item(item: dict[str, object]) -> dict[str, object]:
"reason": item["reason"], state_name = str(item["state"])
"relative_path": item["relative_path"], source_type = str(item["source_type"])
"storage": item["source_path"], return {
"size": item["size"], "id": item["id"],
"source_type": item["source_type"], "name": item["name"],
} "state": state_name,
for item in state.list_queue_items() "group": job_group(state_name, source_type),
if item["source_type"] != "system" "reason": item["reason"],
] "relative_path": item["relative_path"],
"storage": item["source_path"],
"size": item["size"],
"source_type": source_type,
"source_id": item["source_id"],
"job_id": item["job_id"],
"batch_id": item["batch_id"],
"first_seen_at": item["first_seen_at"],
"updated_at": item["updated_at"],
"completed_at": item["completed_at"],
"attempt_count": item.get("attempt_count", 0),
"claimed_by": item.get("claimed_by"),
"next_retry_at": item.get("next_retry_at"),
"sab_status": state_name if source_type == "sab" else None,
"sab_category": item.get("sab_category") if source_type == "sab" else None,
"can_run_now": state_name in {"ready", "failed", "retrying"} and not item.get("claimed_by"),
"can_retry": state_name in {"failed", "skipped"},
"can_ignore": state_name not in {"imported", "skipped", "importing"} and not item.get("claimed_by"),
"can_remove": not item.get("claimed_by"),
}
def job_group(state_name: str, source_type: str) -> str:
if source_type == "manual":
return "manual_batch"
if state_name in {"detected", "waiting_for_sab"}:
return "sab_processing"
if state_name == "ready":
return "ready"
if state_name == "retrying":
return "failed"
if state_name in {"importing", "copying"}:
return "importing"
if state_name == "failed":
return "failed"
if state_name == "skipped":
return "ignored_category"
if state_name == "imported":
return "completed"
return "sab_processing"
def group_jobs(jobs: list[dict[str, object]]) -> list[dict[str, object]]:
labels = {
"sab_processing": "SAB processing",
"ready": "Ready",
"importing": "Importing",
"failed": "Failed",
"ignored_category": "Ignored category",
"manual_batch": "Manual batch",
"completed": "Completed",
}
return [{"key": key, "label": label, "jobs": [job for job in jobs if job["group"] == key]} for key, label in labels.items()]
def manual_batch_jobs() -> list[dict[str, object]]: def manual_batch_jobs() -> list[dict[str, object]]:
@@ -238,7 +594,7 @@ def sync_manual_queue() -> None:
for video in scan_videos(Path(batch["path"])): for video in scan_videos(Path(batch["path"])):
source_id = str(video.path) source_id = str(video.path)
seen.add(source_id) seen.add(source_id)
state.upsert_queue_item(source_type="manual", source_id=source_id, source_path=video.path, name=video.path.name, state="manual_batch", relative_path=str(video.path.relative_to(root)), size=video.size, batch_id=batch["id"]) state.upsert_queue_item(source_type="manual", source_id=source_id, source_path=video.path, name=video.path.name, state="ready", reason="manual batch detected", relative_path=str(video.path.relative_to(root)), size=video.size, batch_id=batch["id"])
state.remove_missing_manual_items(batch["id"], seen) state.remove_missing_manual_items(batch["id"], seen)
@@ -263,30 +619,58 @@ async def _import_ready_sab_jobs(importer: Importer, force: bool = False) -> int
for item in data.get("history", {}).get("slots", []): for item in data.get("history", {}).get("slots", []):
if consume_cancel_request(): if consume_cancel_request():
break break
readiness = classify_history_item(item, active, settings.sab_category, settings.download_root, force_status=force) readiness = classify_history_item(item, active, settings.sab_category, settings.download_root, force_status=force, sab_storage_root=settings.sab_storage_root)
if readiness.storage is None or (not readiness.ready and not force): if readiness.storage is None or (not readiness.ready and not force):
continue continue
job_id = str(item.get("nzo_id") or item.get("nzoid") or item.get("name") or "")
sab_category = str(item.get("category") or item.get("cat") or "")
for video in scan_videos(readiness.storage): for video in scan_videos(readiness.storage):
if consume_cancel_request(): if consume_cancel_request():
return imported return imported
set_current_job(str(video.path)) row = state.upsert_queue_item(source_type="sab", source_id=str(video.path), source_path=video.path, name=video.path.name, state="ready", reason=readiness.reason, relative_path=str(video.relative_path), size=video.size, job_id=job_id, sab_category=sab_category)
try: claimed = state.claim_queue_item(row["id"], WORKER_ID, {"ready", "failed", "retrying"})
result = importer.import_file(video.path, should_cancel=consume_cancel_request) if claimed is not None:
state.add_history(result.source, result.target, "imported", result.bytes) imported += _import_queue_item(claimed, importer, force=force, from_worker=True)
state.mark_queue_item("sab", str(video.path), "imported")
imported += 1
except ImportCancelled:
state.add_history(video.path, video.path, "cancelled", 0, "cancelled")
state.mark_queue_item("sab", str(video.path), "skipped", "cancelled")
return imported
except Exception as exc:
state.add_history(video.path, video.path, "failed", 0, exc.__class__.__name__)
state.mark_queue_item("sab", str(video.path), "failed", exc.__class__.__name__)
finally:
set_current_job(None)
return imported return imported
def _import_queue_item(item: dict[str, object], importer: Importer, *, force: bool = False, from_worker: bool = False) -> int:
if item["source_type"] not in {"sab", "manual"} or item["state"] not in {"ready", "manual_batch", "failed", "importing", "retrying", "waiting_for_sab"}:
return 0
source_path = item.get("source_path")
if not source_path:
state.mark_queue_item_result(str(item["source_type"]), str(item["source_id"]), "failed", "missing source path", increment_attempts=True)
return 0
source = Path(str(source_path))
if not source.exists():
state.mark_queue_item_result(str(item["source_type"]), str(item["source_id"]), "failed", "missing source file", increment_attempts=True)
return 0
if not from_worker:
state.mark_queue_item(str(item["source_type"]), str(item["source_id"]), "importing", item.get("reason"))
started = set_current_job(str(source))
try:
result = importer.import_file(source, should_cancel=consume_cancel_request, on_progress=lambda copied, total: set_current_job(str(source), bytes_copied=copied, total_bytes=total, started_at=started))
state.add_history(result.source, result.target, "imported", result.bytes)
state.mark_queue_item_result(str(item["source_type"]), str(item["source_id"]), "imported", increment_attempts=True)
return 1
except ImportCancelled:
state.add_history(source, source, "cancelled", 0, "cancelled")
state.mark_queue_item_result(str(item["source_type"]), str(item["source_id"]), "skipped", "cancelled", increment_attempts=True)
return 0
except Exception as exc:
state.add_history(source, source, "failed", 0, exc.__class__.__name__)
attempts = int(item.get("attempt_count") or 0) + 1
if attempts >= MAX_RETRY_ATTEMPTS:
state.mark_queue_item_result(str(item["source_type"]), str(item["source_id"]), "failed", exc.__class__.__name__, increment_attempts=True)
else:
next_state = "retrying" if from_worker or force else "failed"
retry_delay = RETRY_DELAY_SECONDS if next_state == "retrying" else None
state.mark_queue_item_result(str(item["source_type"]), str(item["source_id"]), next_state, exc.__class__.__name__, increment_attempts=True, next_retry_seconds=retry_delay)
return 0
finally:
set_current_job(None)
def _import_manual_batches(importer: Importer) -> int: def _import_manual_batches(importer: Importer) -> int:
if queue_accepting_new_jobs(): if queue_accepting_new_jobs():
sync_manual_queue() sync_manual_queue()
@@ -297,26 +681,24 @@ def _import_manual_batches(importer: Importer) -> int:
for item in items: for item in items:
if consume_cancel_request(): if consume_cancel_request():
return imported return imported
source = Path(item["source_path"]) claimed = state.claim_queue_item(item["id"], WORKER_ID, {"ready", "failed", "retrying"})
set_current_job(str(source)) if claimed is not None:
try: imported += _import_queue_item(claimed, importer, force=True, from_worker=True)
result = importer.import_file(source, should_cancel=consume_cancel_request) if not scan_videos(path) and not state.batch_has_active_items(batch["id"]):
state.add_history(result.source, result.target, "imported", result.bytes)
state.mark_queue_item("manual", item["source_id"], "imported")
imported += 1
except ImportCancelled:
state.add_history(source, source, "cancelled", 0, "cancelled")
state.mark_queue_item("manual", item["source_id"], "skipped", "cancelled")
return imported
except Exception as exc:
state.add_history(source, source, "failed", 0, exc.__class__.__name__)
state.mark_queue_item("manual", item["source_id"], "failed", exc.__class__.__name__)
finally:
set_current_job(None)
if not scan_videos(path):
state.complete_manual_batch(batch["id"]) state.complete_manual_batch(batch["id"])
return imported return imported
@app.on_event("startup")
def startup_queue_worker() -> None:
ensure_worker_running()
@app.on_event("shutdown")
def shutdown_queue_worker() -> None:
if stop_worker(wait=True):
state.release_stale_claims()
def run() -> None: def run() -> None:
uvicorn.run("importarr.main:app", host=settings.bind_host, port=settings.bind_port, reload=False) uvicorn.run("importarr.main:app", host=settings.bind_host, port=settings.bind_port, reload=False)
+21 -9
View File
@@ -26,25 +26,37 @@ def has_transient_part(path: Path) -> bool:
return any(part in TRANSIENT_PARTS or any(token in part for token in TRANSIENT_PARTS) for part in path.parts) return any(part in TRANSIENT_PARTS or any(token in part for token in TRANSIENT_PARTS) for part in path.parts)
def classify_history_item(item: dict[str, Any], active_nzo_ids: set[str], category: str, download_root: Path, force_status: bool = False) -> Readiness: def classify_history_item(item: dict[str, Any], active_nzo_ids: set[str], category: str, download_root: Path, force_status: bool = False, sab_storage_root: Path | None = None) -> Readiness:
nzo_id = str(item.get("nzo_id") or item.get("nzoid") or "") nzo_id = str(item.get("nzo_id") or item.get("nzoid") or "")
if not force_status and nzo_id and nzo_id in active_nzo_ids: if not force_status and nzo_id and nzo_id in active_nzo_ids:
return Readiness("processing", "SAB job is still present in queue") return Readiness("processing", "SAB job is still present in queue")
if str(item.get("category") or "") != category: item_category = str(item.get("category") or item.get("cat") or "")
return Readiness("ignored", "SAB category is not owned by Importarr")
status = str(item.get("status") or "") status = str(item.get("status") or "")
storage_value = str(item.get("storage") or "")
storage = Path(storage_value).resolve() if storage_value else None
root = download_root.resolve()
sab_root = (sab_storage_root or download_root).resolve()
storage_in_local_root = bool(storage and (storage == root or root in storage.parents))
storage_in_sab_root = bool(storage and (storage == sab_root or sab_root in storage.parents))
storage_in_root = storage_in_local_root or storage_in_sab_root
if item_category != category and not storage_in_root:
return Readiness("ignored", "SAB category/storage is not owned by Importarr", storage)
if not force_status and status == "Failed": if not force_status and status == "Failed":
return Readiness("failed", "SAB history reports failure") return Readiness("failed", "SAB history reports failure")
if not force_status and (status in NOT_READY_STATUSES or status != "Completed"): if not force_status and (status in NOT_READY_STATUSES or status != "Completed"):
return Readiness("processing", f"SAB status is {status or 'unknown'}") return Readiness("processing", f"SAB status is {status or 'unknown'}")
storage_value = str(item.get("storage") or "") if storage is None:
if not storage_value:
return Readiness("unknown", "SAB completed item has no final storage") return Readiness("unknown", "SAB completed item has no final storage")
storage = Path(storage_value).resolve() if not storage_in_root:
root = download_root.resolve()
if storage != root and root not in storage.parents:
return Readiness("ignored", "SAB storage is outside configured download root", storage) return Readiness("ignored", "SAB storage is outside configured download root", storage)
if storage_in_sab_root and not storage_in_local_root:
storage = root / storage.relative_to(sab_root)
if has_transient_part(storage): if has_transient_part(storage):
return Readiness("processing", "SAB storage path contains transient unpack/admin marker", storage) return Readiness("processing", "SAB storage path contains transient unpack/admin marker", storage)
reason = "forced despite SAB status" if force_status and status != "Completed" else "SAB completed in owned category with final storage" if force_status and status != "Completed":
reason = "forced despite SAB status"
elif item_category != category:
reason = "SAB completed inside Importarr download root"
else:
reason = "SAB completed in owned category with final storage"
return Readiness("ready", reason, storage) return Readiness("ready", reason, storage)
+232 -51
View File
@@ -1,21 +1,37 @@
from __future__ import annotations from __future__ import annotations
import sqlite3 import sqlite3
import threading
from pathlib import Path from pathlib import Path
from typing import Any from typing import Any
ACTIVE_QUEUE_STATES = {
"detected",
"waiting_for_sab",
"ready",
"importing",
"retrying",
}
TERMINAL_QUEUE_STATES = {"imported", "failed", "skipped"}
class State: class State:
def __init__(self, path: Path): def __init__(self, path: Path):
self.path = path self.path = path
self.path.parent.mkdir(parents=True, exist_ok=True) self.path.parent.mkdir(parents=True, exist_ok=True)
self._lock = threading.RLock()
self.conn = sqlite3.connect(self.path, check_same_thread=False) self.conn = sqlite3.connect(self.path, check_same_thread=False)
self.conn.row_factory = sqlite3.Row self.conn.row_factory = sqlite3.Row
self.conn.execute("pragma journal_mode=WAL")
self.conn.execute("pragma busy_timeout = 5000")
self.migrate() self.migrate()
def migrate(self) -> None: def migrate(self) -> None:
self.conn.executescript( with self._lock:
""" self.conn.executescript(
"""
create table if not exists manual_batches ( create table if not exists manual_batches (
id integer primary key autoincrement, id integer primary key autoincrement,
path text not null unique, path text not null unique,
@@ -46,57 +62,84 @@ class State:
size integer not null default 0, size integer not null default 0,
batch_id integer, batch_id integer,
job_id text, job_id text,
sab_category text,
attempt_count integer not null default 0,
next_retry_at text,
last_error text,
claimed_by text,
claimed_at text,
first_seen_at text not null default current_timestamp, first_seen_at text not null default current_timestamp,
updated_at text not null default current_timestamp, updated_at text not null default current_timestamp,
completed_at text, completed_at text,
unique(source_type, source_id) unique(source_type, source_id)
); );
""" """
) )
self.conn.commit() columns = {row["name"] for row in self.conn.execute("pragma table_info(import_queue_items)")}
if "sab_category" not in columns:
self.conn.execute("alter table import_queue_items add column sab_category text")
if "attempt_count" not in columns:
self.conn.execute("alter table import_queue_items add column attempt_count integer not null default 0")
if "next_retry_at" not in columns:
self.conn.execute("alter table import_queue_items add column next_retry_at text")
if "last_error" not in columns:
self.conn.execute("alter table import_queue_items add column last_error text")
if "claimed_by" not in columns:
self.conn.execute("alter table import_queue_items add column claimed_by text")
if "claimed_at" not in columns:
self.conn.execute("alter table import_queue_items add column claimed_at text")
self.conn.commit()
def get_app_state(self, key: str, default: str | None = None) -> str | None: def get_app_state(self, key: str, default: str | None = None) -> str | None:
row = self.conn.execute("select value from app_state where key = ?", (key,)).fetchone() with self._lock:
return row["value"] if row else default row = self.conn.execute("select value from app_state where key = ?", (key,)).fetchone()
return row["value"] if row else default
def set_app_state(self, key: str, value: str) -> None: def set_app_state(self, key: str, value: str) -> None:
self.conn.execute( with self._lock:
"insert into app_state(key, value) values (?, ?) on conflict(key) do update set value=excluded.value", self.conn.execute(
(key, value), "insert into app_state(key, value) values (?, ?) on conflict(key) do update set value=excluded.value",
) (key, value),
self.conn.commit() )
self.conn.commit()
def add_manual_batch(self, path: Path) -> dict[str, Any]: def add_manual_batch(self, path: Path) -> dict[str, Any]:
self.conn.execute("insert or ignore into manual_batches(path) values (?)", (str(path),)) with self._lock:
self.conn.commit() self.conn.execute("insert or ignore into manual_batches(path) values (?)", (str(path),))
self.conn.commit()
return self.get_manual_batch_by_path(path) return self.get_manual_batch_by_path(path)
def get_manual_batch_by_path(self, path: Path) -> dict[str, Any]: def get_manual_batch_by_path(self, path: Path) -> dict[str, Any]:
row = self.conn.execute("select * from manual_batches where path = ?", (str(path),)).fetchone() with self._lock:
return dict(row) row = self.conn.execute("select * from manual_batches where path = ?", (str(path),)).fetchone()
return dict(row)
def list_manual_batches(self, active_only: bool = False) -> list[dict[str, Any]]: def list_manual_batches(self, active_only: bool = False) -> list[dict[str, Any]]:
sql = "select * from manual_batches" sql = "select * from manual_batches"
if active_only: if active_only:
sql += " where status = 'active'" sql += " where status = 'active'"
sql += " order by created_at desc" sql += " order by created_at desc"
return [dict(row) for row in self.conn.execute(sql)] with self._lock:
return [dict(row) for row in self.conn.execute(sql)]
def delete_manual_batch(self, batch_id: int) -> None: def delete_manual_batch(self, batch_id: int) -> None:
self.conn.execute("delete from manual_batches where id = ?", (batch_id,)) with self._lock:
self.conn.execute("delete from import_queue_items where batch_id = ? and source_type = 'manual'", (batch_id,)) self.conn.execute("delete from manual_batches where id = ?", (batch_id,))
self.conn.commit() self.conn.execute("delete from import_queue_items where batch_id = ? and source_type = 'manual'", (batch_id,))
self.conn.commit()
def complete_manual_batch(self, batch_id: int) -> None: def complete_manual_batch(self, batch_id: int) -> None:
self.conn.execute("update manual_batches set status='completed', completed_at=current_timestamp where id=?", (batch_id,)) with self._lock:
self.conn.commit() self.conn.execute("update manual_batches set status='completed', completed_at=current_timestamp where id=?", (batch_id,))
self.conn.commit()
def add_history(self, source: Path, target: Path, status: str, bytes_count: int = 0, error: str | None = None) -> None: def add_history(self, source: Path, target: Path, status: str, bytes_count: int = 0, error: str | None = None) -> None:
self.conn.execute( with self._lock:
"insert into import_history(source,target,status,bytes,error,completed_at) values (?,?,?,?,?,case when ? in ('imported','failed') then current_timestamp else null end)", self.conn.execute(
(str(source), str(target), status, bytes_count, error, status), "insert into import_history(source,target,status,bytes,error,completed_at) values (?,?,?,?,?,case when ? in ('imported','failed') then current_timestamp else null end)",
) (str(source), str(target), status, bytes_count, error, status),
self.conn.commit() )
self.conn.commit()
def upsert_queue_item( def upsert_queue_item(
self, self,
@@ -111,49 +154,187 @@ class State:
size: int = 0, size: int = 0,
batch_id: int | None = None, batch_id: int | None = None,
job_id: str | None = None, job_id: str | None = None,
sab_category: str | None = None,
preserve_finished_state: bool = True,
) -> dict[str, Any]: ) -> dict[str, Any]:
self.conn.execute( with self._lock:
""" self.conn.execute(
insert into import_queue_items(source_type, source_id, source_path, name, state, reason, relative_path, size, batch_id, job_id) """
values (?,?,?,?,?,?,?,?,?,?) insert into import_queue_items(source_type, source_id, source_path, name, state, reason, relative_path, size, batch_id, job_id, sab_category)
values (?,?,?,?,?,?,?,?,?,?,?)
on conflict(source_type, source_id) do update set on conflict(source_type, source_id) do update set
source_path=excluded.source_path, source_path=excluded.source_path,
name=excluded.name, name=excluded.name,
state=excluded.state, state=case
reason=excluded.reason, when ? and import_queue_items.state in ('imported','failed','skipped','importing','retrying') then import_queue_items.state
else excluded.state
end,
reason=case
when ? and import_queue_items.state in ('imported','failed','skipped','importing','retrying') then coalesce(import_queue_items.reason, excluded.reason)
else excluded.reason
end,
relative_path=excluded.relative_path, relative_path=excluded.relative_path,
size=excluded.size, size=excluded.size,
batch_id=excluded.batch_id, batch_id=excluded.batch_id,
job_id=excluded.job_id, job_id=excluded.job_id,
sab_category=excluded.sab_category,
updated_at=current_timestamp, updated_at=current_timestamp,
completed_at=case when excluded.state in ('imported','failed','skipped') then current_timestamp else null end next_retry_at=case
when excluded.state = 'retrying' then coalesce(import_queue_items.next_retry_at, excluded.next_retry_at)
when excluded.state = 'ready' then null
else import_queue_items.next_retry_at
end,
completed_at=case
when ? and import_queue_items.state in ('imported','failed','skipped') then import_queue_items.completed_at
when excluded.state in ('imported','failed','skipped') then current_timestamp
else null
end
""", """,
(source_type, source_id, str(source_path) if source_path else None, name, state, reason, relative_path, size, batch_id, job_id), (source_type, source_id, str(source_path) if source_path else None, name, state, reason, relative_path, size, batch_id, job_id, sab_category, preserve_finished_state, preserve_finished_state, preserve_finished_state),
) )
self.conn.commit() self.conn.commit()
row = self.conn.execute("select * from import_queue_items where source_type = ? and source_id = ?", (source_type, source_id)).fetchone() row = self.conn.execute("select * from import_queue_items where source_type = ? and source_id = ?", (source_type, source_id)).fetchone()
return dict(row) return dict(row)
def mark_queue_item(self, source_type: str, source_id: str, state: str, reason: str | None = None) -> None: def mark_queue_item(self, source_type: str, source_id: str, state: str, reason: str | None = None) -> None:
self.conn.execute( with self._lock:
"update import_queue_items set state=?, reason=?, updated_at=current_timestamp, completed_at=case when ? in ('imported','failed','skipped') then current_timestamp else completed_at end where source_type=? and source_id=?", self.conn.execute(
(state, reason, state, source_type, source_id), "update import_queue_items set state=?, reason=?, updated_at=current_timestamp, claimed_by=null, claimed_at=null, next_retry_at=case when ?='ready' then null else next_retry_at end, completed_at=case when ? in ('imported','failed','skipped') then current_timestamp else completed_at end where source_type=? and source_id=?",
) (state, reason, state, state, source_type, source_id),
self.conn.commit() )
self.conn.commit()
def claim_next_queue_item(self, worker_id: str) -> dict[str, Any] | None:
with self._lock:
row = self.conn.execute(
"""
update import_queue_items
set state='importing', claimed_by=?, claimed_at=current_timestamp, updated_at=current_timestamp
where id = (
select id from import_queue_items
where claimed_by is null
and (state = 'ready' or (state = 'retrying' and (next_retry_at is null or next_retry_at <= current_timestamp)))
order by case state when 'ready' then 0 else 1 end, updated_at asc, id asc
limit 1
) and claimed_by is null
returning *
""",
(worker_id,),
).fetchone()
self.conn.commit()
return dict(row) if row else None
def release_stale_claims(self) -> int:
with self._lock:
cursor = self.conn.execute(
"update import_queue_items set state='retrying', claimed_by=null, claimed_at=null, updated_at=current_timestamp where state='importing'"
)
self.conn.commit()
return cursor.rowcount
def claim_queue_item(self, item_id: int, worker_id: str, allowed_states: set[str]) -> dict[str, Any] | None:
placeholders = ",".join("?" for _ in allowed_states)
with self._lock:
row = self.conn.execute(
f"update import_queue_items set state='importing', claimed_by=?, claimed_at=current_timestamp, updated_at=current_timestamp where id=? and state in ({placeholders}) and claimed_by is null returning *",
(worker_id, item_id, *allowed_states),
).fetchone()
self.conn.commit()
return dict(row) if row else None
def transition_queue_item_if_unclaimed(self, item_id: int, allowed_states: set[str], new_state: str, reason: str) -> bool:
placeholders = ",".join("?" for _ in allowed_states)
with self._lock:
cursor = self.conn.execute(
f"update import_queue_items set state=?, reason=?, updated_at=current_timestamp, next_retry_at=null, completed_at=case when ? in ('failed','skipped') then current_timestamp else null end where id=? and state in ({placeholders}) and claimed_by is null",
(new_state, reason, new_state, item_id, *allowed_states),
)
self.conn.commit()
return cursor.rowcount > 0
def mark_queue_item_result(
self,
source_type: str,
source_id: str,
state: str,
reason: str | None = None,
*,
increment_attempts: bool = False,
next_retry_seconds: int | None = None,
) -> None:
with self._lock:
self.conn.execute(
"""
update import_queue_items
set state=?,
reason=?,
last_error=case when ? in ('failed','retrying','skipped') then ? else null end,
attempt_count=attempt_count + ?,
next_retry_at=case
when ? = 'retrying' and ? is not null then datetime('now', '+' || ? || ' seconds')
when ? in ('ready','imported','failed','skipped') then null
else next_retry_at
end,
claimed_by=null,
claimed_at=null,
updated_at=current_timestamp,
completed_at=case when ? in ('imported','failed','skipped') then current_timestamp else null end
where source_type=? and source_id=?
""",
(state, reason, state, reason, 1 if increment_attempts else 0, state, next_retry_seconds, next_retry_seconds, state, state, source_type, source_id),
)
self.conn.commit()
def delete_queue_item(self, item_id: int) -> bool:
with self._lock:
cursor = self.conn.execute("delete from import_queue_items where id = ?", (item_id,))
self.conn.commit()
return cursor.rowcount > 0
def delete_queue_item_if_unclaimed(self, item_id: int) -> bool:
with self._lock:
cursor = self.conn.execute("delete from import_queue_items where id = ? and claimed_by is null", (item_id,))
self.conn.commit()
return cursor.rowcount > 0
def delete_queue_items_by_state(self, source_type: str, state: str, reason: str | None = None) -> int:
with self._lock:
if reason is None:
cursor = self.conn.execute("delete from import_queue_items where source_type = ? and state = ?", (source_type, state))
else:
cursor = self.conn.execute("delete from import_queue_items where source_type = ? and state = ? and reason = ?", (source_type, state, reason))
self.conn.commit()
return cursor.rowcount
def get_queue_item(self, item_id: int) -> dict[str, Any] | None:
with self._lock:
row = self.conn.execute("select * from import_queue_items where id = ?", (item_id,)).fetchone()
return dict(row) if row else None
def remove_missing_manual_items(self, batch_id: int, source_ids: set[str]) -> None: def remove_missing_manual_items(self, batch_id: int, source_ids: set[str]) -> None:
rows = self.conn.execute("select source_id from import_queue_items where source_type='manual' and batch_id=?", (batch_id,)).fetchall() with self._lock:
for row in rows: rows = self.conn.execute("select source_id, state, claimed_by from import_queue_items where source_type='manual' and batch_id=?", (batch_id,)).fetchall()
if row["source_id"] not in source_ids: for row in rows:
self.conn.execute("delete from import_queue_items where source_type='manual' and source_id=?", (row["source_id"],)) if row["source_id"] not in source_ids and row["state"] not in TERMINAL_QUEUE_STATES and not row["claimed_by"]:
self.conn.commit() self.conn.execute("delete from import_queue_items where source_type='manual' and source_id=? and claimed_by is null", (row["source_id"],))
self.conn.commit()
def batch_has_active_items(self, batch_id: int) -> bool:
with self._lock:
row = self.conn.execute(
"select 1 from import_queue_items where batch_id = ? and source_type='manual' and state not in ('imported','failed','skipped') limit 1",
(batch_id,),
).fetchone()
return row is not None
def list_queue_items(self, active_only: bool = True) -> list[dict[str, Any]]: def list_queue_items(self, active_only: bool = True) -> list[dict[str, Any]]:
sql = "select * from import_queue_items" sql = "select * from import_queue_items"
if active_only: if active_only:
sql += " where state not in ('imported','failed','skipped')" sql += " where state not in ('imported','failed','skipped')"
sql += " order by updated_at desc, id desc" sql += " order by updated_at desc, id desc"
return [dict(row) for row in self.conn.execute(sql)] with self._lock:
return [dict(row) for row in self.conn.execute(sql)]
def list_history(self, limit: int = 100) -> list[dict[str, Any]]: def list_history(self, limit: int = 100) -> list[dict[str, Any]]:
return [dict(row) for row in self.conn.execute("select * from import_history order by id desc limit ?", (limit,))] with self._lock:
return [dict(row) for row in self.conn.execute("select * from import_history order by id desc limit ?", (limit,))]
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
-1
View File
@@ -1 +0,0 @@
body{font-family:system-ui,sans-serif;margin:0;background:#111827;color:#e5e7eb}header,main{max-width:1100px;margin:auto;padding:1rem}.topbar{display:flex;justify-content:space-between;gap:1rem;align-items:center;background:#0f172a}.build{text-align:right}.build strong{font-size:1.2rem}.cards{display:grid;grid-template-columns:repeat(auto-fit,minmax(12rem,1fr));gap:1rem}.cards article,.panel{background:#1f2937;border-radius:.75rem;padding:1rem;margin-top:1rem}strong{display:block;font-size:2rem}span,small,dd{color:#9ca3af}table{width:100%;border-collapse:collapse;background:#1f2937;margin-top:1rem}th,td{padding:.6rem;border-bottom:1px solid #374151;text-align:left}input,button{padding:.6rem;border-radius:.4rem;border:1px solid #374151}button{background:#38bdf8;color:#082f49;font-weight:700}.danger{background:#f87171;color:#450a0a}.controls{display:flex;gap:.5rem;flex-wrap:wrap}.inline-form{display:flex;gap:.5rem;flex-wrap:wrap}.inline-form input[name=path]{min-width:min(100%,28rem);flex:1}.info{display:grid;grid-template-columns:10rem 1fr;gap:.4rem 1rem}.info dt{font-weight:700}.info dd{margin:0;overflow-wrap:anywhere}.state{background:#0f172a;border:1px solid #374151;border-radius:999px;padding:.15rem .5rem}
+12
View File
@@ -0,0 +1,12 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="theme-color" content="#252B42" />
<title>Importarr</title>
<script type="module" crossorigin src="/static/assets/app.js"></script>
<link rel="stylesheet" crossorigin href="/static/assets/app.css">
</head>
<body><div id="root"></div></body>
</html>
-496
View File
@@ -1,496 +0,0 @@
#!/usr/bin/env python3
from __future__ import annotations
import html
import json
import os
import subprocess
import time
import urllib.parse
import urllib.request
from datetime import datetime, timedelta, timezone
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
SERVICE = "manual-media-import.service"
TIMER = "manual-media-import.timer"
LOG = Path("/var/log/manual-media-import.log")
IMPORTER_STATUS = Path("/run/manual-media-import/status.json")
MANUAL_BATCHES = Path("/var/lib/importarr/manual-batches.json")
QUEUE_ROOTS = {
"manual": Path(os.getenv("IMPORTARR_DOWNLOAD_ROOT", "/data/downloads/manual")),
"legacy": Path(os.getenv("IMPORTARR_LEGACY_DOWNLOAD_ROOT", "/data/downloads/legacy")),
}
VIDEO_EXT = {".mkv", ".mp4", ".m4v", ".avi", ".mov", ".wmv", ".mpg", ".mpeg", ".ts", ".m2ts", ".webm"}
SAB_CONFIG = Path(os.getenv("IMPORTARR_SABNZBD_CONFIG", "/config/sabnzbd/sabnzbd.ini"))
SAB_API = os.getenv("IMPORTARR_SABNZBD_URL", "http://sabnzbd:8080/api")
LONG_RUNTIME_SECONDS = 25 * 60
HIGH_MEMORY_BYTES = 8 * 1024**3
STALE_QUEUE_SECONDS = 6 * 60 * 60
def run(args: list[str]) -> str:
return subprocess.run(args, check=False, text=True, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL).stdout
def systemctl_show(unit: str) -> dict[str, str]:
data = {}
for line in run(["systemctl", "show", unit, "--no-pager"]).splitlines():
if "=" in line:
key, value = line.split("=", 1)
data[key] = value
return data
def int_value(value: str | None) -> int | None:
try:
return int(value or "")
except ValueError:
return None
def timestamp_to_iso(usec: str | None) -> str | None:
value = int_value(usec)
if not value or value <= 0:
return None
return datetime.fromtimestamp(value / 1_000_000, tz=timezone.utc).isoformat()
def monotonic_runtime_seconds(service: dict[str, str]) -> int | None:
started = int_value(service.get("ActiveEnterTimestampMonotonic"))
if not started:
main_pid = int_value(service.get("MainPID"))
if not main_pid:
return None
etimes = run(["ps", "-o", "etimes=", "-p", str(main_pid)]).strip()
return int_value(etimes)
boot_ns = time.clock_gettime_ns(time.CLOCK_BOOTTIME)
runtime = int((boot_ns / 1000 - started) / 1_000_000)
return max(runtime, 0)
def scan_queue(root: Path) -> dict[str, object]:
files = dirs = bytes_total = 0
processing_files = processing_dirs = processing_bytes = 0
oldest = newest = None
top_level: list[dict[str, object]] = []
if not root.exists():
return {"path": str(root), "exists": False, "files": 0, "dirs": 0, "bytes": 0, "oldest": None, "newest": None, "topLevel": [], "items": [], "processingFiles": 0, "processingDirs": 0, "processingBytes": 0, "processing": [], "processingItems": []}
top_level_map: dict[Path, dict[str, object]] = {}
processing_map: dict[Path, dict[str, object]] = {}
ready_items: list[dict[str, object]] = []
processing_items: list[dict[str, object]] = []
for dirpath, dirnames, filenames in os.walk(root):
for filename in filenames:
path = Path(dirpath) / filename
if path.suffix.lower() not in VIDEO_EXT or "sample" in filename.lower() or "sample" in str(path.parent).lower():
continue
try:
st = path.stat()
except FileNotFoundError:
continue
oldest = st.st_mtime if oldest is None else min(oldest, st.st_mtime)
newest = st.st_mtime if newest is None else max(newest, st.st_mtime)
try:
rel = path.relative_to(root)
except ValueError:
rel = path
top = root / rel.parts[0] if rel.parts else path
sab = sab_state_for(top.name)
transient = top.name.startswith(('_UNPACK_', '__UNPACK__', '_FAILED_', '_ADMIN_'))
is_processing = transient or (sab and sab.get("ready") is False and sab.get("status") != "manual")
relative_dir = str(Path(*rel.parts[:-1])) if len(rel.parts) > 1 else ""
item_label = path.name if not relative_dir else f"{relative_dir} / {path.name}"
state = str(sab.get("state") if sab else ("unpacking" if transient else "ready"))
file_item = {"name": path.name, "label": item_label, "release": top.name, "relativeDir": relative_dir, "path": str(path), "bytes": st.st_size, "mtime": datetime.fromtimestamp(st.st_mtime, tz=timezone.utc).isoformat(), "state": state}
if is_processing:
processing_files += 1
processing_bytes += st.st_size
processing_items.append(file_item)
item = processing_map.setdefault(top, {"name": top.name, "type": "dir" if top.is_dir() else "file", "files": 0, "bytes": 0, "mtime": None})
item["files"] = int(item["files"]) + 1
item["bytes"] = int(item["bytes"]) + st.st_size
item["mtime"] = datetime.fromtimestamp(st.st_mtime, tz=timezone.utc).isoformat()
continue
files += 1
bytes_total += st.st_size
ready_items.append(file_item)
item = top_level_map.setdefault(top, {"name": top.name, "type": "dir" if top.is_dir() else "file", "files": 0, "bytes": 0, "mtime": None})
item["files"] = int(item["files"]) + 1
item["bytes"] = int(item["bytes"]) + st.st_size
item["mtime"] = datetime.fromtimestamp(st.st_mtime, tz=timezone.utc).isoformat()
dirs = sum(1 for item in top_level_map.values() if item["type"] == "dir")
processing_dirs = sum(1 for item in processing_map.values() if item["type"] == "dir")
top_level = sorted(top_level_map.values(), key=lambda item: str(item["name"]).lower())
processing = sorted(processing_map.values(), key=lambda item: str(item["name"]).lower())
return {
"path": str(root),
"exists": True,
"files": files,
"dirs": dirs,
"bytes": bytes_total,
"oldest": datetime.fromtimestamp(oldest, tz=timezone.utc).isoformat() if oldest else None,
"newest": datetime.fromtimestamp(newest, tz=timezone.utc).isoformat() if newest else None,
"topLevel": top_level[:100],
"items": ready_items[:500],
"processingFiles": processing_files,
"processingDirs": processing_dirs,
"processingBytes": processing_bytes,
"processing": processing[:100],
"processingItems": processing_items[:500],
}
def read_logs(limit: int = 100) -> list[dict[str, object]]:
if not LOG.exists():
return []
lines = LOG.read_text(errors="replace").splitlines()[-max(1, min(limit, 1000)):]
records = []
for line in lines:
try:
records.append(json.loads(line))
except json.JSONDecodeError:
records.append({"level": "RAW", "msg": line})
return records
def read_summaries() -> list[dict[str, object]]:
if not LOG.exists():
return []
summaries = []
for line in LOG.read_text(errors="replace").splitlines():
try:
record = json.loads(line)
except json.JSONDecodeError:
continue
if record.get("msg") == "summary":
summaries.append(record)
return summaries
def read_importer_status() -> dict[str, object] | None:
if not IMPORTER_STATUS.exists():
return None
def infer_current_from_logs(logs: list[dict[str, object]]) -> dict[str, object] | None:
for record in reversed(logs):
if record.get("level") != "MOVE" or record.get("msg") not in {"moving", "moving sidecar"}:
continue
src = record.get("src")
dest = record.get("dest")
if not src or not dest:
continue
src_path = Path(str(src))
partial = Path(str(dest) + ".partial")
total = None
copied = None
try:
total = src_path.stat().st_size
except FileNotFoundError:
pass
try:
copied = partial.stat().st_size
except FileNotFoundError:
copied = None
percent = round((copied / total * 100), 2) if copied is not None and total else None
return {"phase": "copying", "src": str(src), "dest": str(dest), "partial": str(partial), "bytes_copied": copied, "bytes_total": total, "percent": percent, "media_type": record.get("media_type"), "source_tag": record.get("source_tag"), "kind": "inferred"}
return None
try:
return json.loads(IMPORTER_STATUS.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError):
return None
def queue_status() -> dict[str, object]:
global _SAB_HISTORY_CACHE
_SAB_HISTORY_CACHE = None
roots = {name: scan_queue(path) for name, path in QUEUE_ROOTS.items()}
return {
"roots": roots,
"files": sum(int(r["files"]) for r in roots.values()),
"dirs": sum(int(r["dirs"]) for r in roots.values()),
"bytes": sum(int(r["bytes"]) for r in roots.values()),
"processingFiles": sum(int(r["processingFiles"]) for r in roots.values()),
"processingDirs": sum(int(r["processingDirs"]) for r in roots.values()),
"processingBytes": sum(int(r["processingBytes"]) for r in roots.values()),
}
def sab_api_key() -> str | None:
try:
import re
m = re.search(r"^api_key\s*=\s*(\S+)", SAB_CONFIG.read_text(errors="replace"), re.M)
return m.group(1) if m else None
except OSError:
return None
def sab_history() -> list[dict[str, object]]:
key = sab_api_key()
if not key:
return []
try:
q = urllib.parse.urlencode({"mode": "history", "output": "json", "limit": 200, "apikey": key})
data = json.load(urllib.request.urlopen(f"{SAB_API}?{q}", timeout=10))
return data.get("history", {}).get("slots", [])
except Exception:
return []
_SAB_HISTORY_CACHE: list[dict[str, object]] | None = None
def sab_state_for(folder_name: str) -> dict[str, object] | None:
global _SAB_HISTORY_CACHE
if _SAB_HISTORY_CACHE is None:
_SAB_HISTORY_CACHE = sab_history()
normalized = folder_name.removeprefix("_UNPACK_").removeprefix("__UNPACK__")
for item in _SAB_HISTORY_CACHE:
name = str(item.get("name") or "")
if name != normalized and name != folder_name:
continue
status = str(item.get("status") or "")
storage = str(item.get("storage") or "")
action = str(item.get("action_line") or "")
category = str(item.get("category") or item.get("cat") or "")
owned = category == "manual"
ready = owned and status == "Completed" and bool(storage) and "_UNPACK_" not in storage
state = "ready" if ready else ("ignored category " + category if not owned else (status.lower() if status else "sab pending"))
if action:
state = action
return {"ready": ready, "owned": owned, "category": category, "status": status, "storage": storage, "state": state}
return None
def read_manual_batches() -> list[str]:
try:
raw = json.loads(MANUAL_BATCHES.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError):
return []
return [str(x) for x in raw] if isinstance(raw, list) else []
def write_manual_batches(items: list[str]) -> None:
MANUAL_BATCHES.parent.mkdir(parents=True, exist_ok=True)
MANUAL_BATCHES.write_text(json.dumps(sorted(set(items)), indent=2), encoding="utf-8")
def add_manual_batch(value: str) -> tuple[bool, str]:
value = value.strip().strip("/")
if not value:
return False, "missing folder"
base = QUEUE_ROOTS["manual"].resolve()
path = (base / value).resolve() if not value.startswith("/srv/") else Path(value).resolve()
if not (path == base or path.is_relative_to(base)):
return False, "folder must be under manual downloads"
if not path.is_dir():
return False, "folder does not exist"
items = read_manual_batches()
items.append(str(path))
write_manual_batches(items)
return True, str(path)
def status() -> dict[str, object]:
service = systemctl_show(SERVICE)
timer = systemctl_show(TIMER)
logs = read_logs(300)
all_logs = read_logs(1000)
summaries = read_summaries()
last_summary = summaries[-1] if summaries else None
cutoff_1h = datetime.now() - timedelta(hours=1)
cutoff_24h = datetime.now() - timedelta(hours=24)
processed_1h = 0
processed_24h = 0
processed_total = 0
runs_1h = 0
runs_24h = 0
runs_total = 0
for summary in summaries:
moved = int(summary.get("moved") or 0)
processed_total += moved
runs_total += 1
try:
ts = datetime.fromisoformat(str(summary.get("ts")))
except ValueError:
ts = None
if ts and ts >= cutoff_1h:
processed_1h += moved
runs_1h += 1
if ts and ts >= cutoff_24h:
processed_24h += moved
runs_24h += 1
queue = queue_status()
running = service.get("ActiveState") == "activating" or service.get("SubState") in {"start", "running"}
current = read_importer_status()
if running and (not current or current.get("phase") == "done"):
current = infer_current_from_logs(all_logs)
runtime = monotonic_runtime_seconds(service) if running else None
memory_current = int_value(service.get("MemoryCurrent"))
memory_peak = int_value(service.get("MemoryPeak"))
warnings = []
if runtime and runtime > LONG_RUNTIME_SECONDS:
warnings.append("manual-media-import.service has been running longer than 25 minutes")
if memory_peak and memory_peak > HIGH_MEMORY_BYTES:
warnings.append("manual-media-import.service peak memory is over 8 GiB")
if last_summary and int(last_summary.get("errors") or 0) > 0:
warnings.append("last importer summary reported errors")
now = time.time()
for name, root in queue["roots"].items():
oldest = root.get("oldest")
if oldest:
try:
age = now - datetime.fromisoformat(str(oldest)).timestamp()
if age > STALE_QUEUE_SECONDS:
warnings.append(f"{name} queue contains files older than 6 hours")
except ValueError:
pass
health_state = "warning" if warnings else ("running" if running else service.get("Result", "unknown"))
return {
"name": "Importarr",
"service": {
"unit": SERVICE,
"activeState": service.get("ActiveState"),
"subState": service.get("SubState"),
"result": service.get("Result"),
"running": running,
"mainPid": int_value(service.get("MainPID")),
"startedAt": timestamp_to_iso(service.get("ActiveEnterTimestampUSec")),
"runtimeSeconds": runtime,
"memoryCurrentBytes": memory_current,
"memoryPeakBytes": memory_peak,
},
"timer": {
"unit": TIMER,
"activeState": timer.get("ActiveState"),
"subState": timer.get("SubState"),
"lastTrigger": timestamp_to_iso(timer.get("LastTriggerUSec")),
"nextElapse": timestamp_to_iso(timer.get("NextElapseUSecRealtime")),
},
"queue": queue,
"lastSummary": last_summary,
"processed": {"last1h": processed_1h, "last24h": processed_24h, "total": processed_total, "runsLast1h": runs_1h, "runsLast24h": runs_24h, "runsTotal": runs_total},
"current": current,
"manualBatches": read_manual_batches(),
"health": {"state": health_state, "warnings": warnings},
}
def fast_health() -> dict[str, object]:
service = systemctl_show(SERVICE)
timer = systemctl_show(TIMER)
running = service.get("ActiveState") == "activating" or service.get("SubState") in {"start", "running"}
runtime = monotonic_runtime_seconds(service) if running else None
warnings = []
if runtime and runtime > LONG_RUNTIME_SECONDS:
warnings.append("manual-media-import.service has been running longer than 25 minutes")
return {"state": "warning" if warnings else ("running" if running else service.get("Result", "unknown")), "warnings": warnings, "service": service.get("ActiveState"), "timer": timer.get("ActiveState")}
def page() -> bytes:
s = status()
logs = read_logs(80)
def esc(value: object) -> str:
return html.escape("" if value is None else str(value))
def gib(value: object) -> str:
return "" if value is None else f"{int(value) / 1024**3:.2f} GiB"
def secs(value: object) -> str:
if value is None:
return ""
seconds = int(value)
return f"{seconds // 60}m {seconds % 60}s"
def job_name(path: object) -> str:
if not path:
return ""
name = Path(str(path)).name
return name[:37] + "..." if len(name) > 40 else name
title_icon = "📥"
title_runtime = secs(s["service"]["runtimeSeconds"])
current = s.get("current") or {}
progress = current.get("percent")
full_current_name = Path(str(current.get("src", ""))).name if current.get("src") else ""
current_src_path = Path(str(current.get("src", ""))) if current.get("src") else None
current_top_name = ""
current_relative_dir = ""
if current_src_path:
for root in QUEUE_ROOTS.values():
try:
rel = current_src_path.relative_to(root)
current_top_name = rel.parts[0] if rel.parts else current_src_path.name
current_relative_dir = str(Path(*rel.parts[:-1])) if len(rel.parts) > 1 else ""
break
except ValueError:
continue
current_name = job_name(current.get("src"))
title = f"{title_icon} {progress}% {title_runtime} - {current_name}" if progress is not None and current_name else f"{title_icon} - idle"
warnings = s["health"]["warnings"]
warning_html = "".join(f"<p class='warn'>{esc(w)}</p>" for w in warnings) or "<p class='ok'>None</p>"
log_text = "\n".join(esc(f"[{r.get('ts','')}] {r.get('level','')} {r.get('msg','')} {json.dumps(r, ensure_ascii=False)}") for r in logs)
source_target_html = f"<p class='tiny'><span>Source:</span> {esc(current.get('src'))}</p><p class='tiny'><span>Target:</span> {esc(current.get('dest'))}</p>" if current else ""
progress_html = f"<p class='filename'><strong>{esc(full_current_name)}</strong><br><span class='muted'>{esc(current_relative_dir)}</span></p><p>{esc(progress)}% · runtime {title_runtime}</p><progress max='100' value='{esc(progress or 0)}'></progress><p>{esc(current.get('phase'))} · {gib(current.get('bytes_copied'))} / {gib(current.get('bytes_total'))}</p>{source_target_html}" if current else "<p>—</p>"
current_row = f"<tr class='active'><td>▶</td><td class='filename'><strong>{esc(full_current_name)}</strong><br><span class='muted'>{esc(current_relative_dir)}</span></td><td>{esc(progress)}%</td><td><progress max='100' value='{esc(progress or 0)}'></progress></td><td>{title_runtime}</td></tr>" if current else ""
current_path = str(current_src_path) if current_src_path else ""
processing_items = [item for item in s["queue"]["roots"]["manual"]["processingItems"] if item["path"] != current_path]
ready_items = [item for item in s["queue"]["roots"]["manual"]["items"] if item["path"] != current_path]
processing_rows = "".join(f"<tr class='processing'><td>⏳</td><td class='filename'><strong>{esc(item['name'])}</strong><br><span class='muted'>{esc(item.get('relativeDir') or item['release'])}</span></td><td>unpacking</td><td>{gib(item.get('bytes'))}</td><td>waiting</td></tr>" for item in processing_items[:30])
ready_rows = "".join(f"<tr><td>◷</td><td class='filename'><strong>{esc(item['name'])}</strong><br><span class='muted'>{esc(item.get('relativeDir') or item['release'])}</span></td><td>ready</td><td>{gib(item.get('bytes'))}</td><td>queued</td></tr>" for item in ready_items[:30])
queue_rows = processing_rows + ready_rows or "<tr><td>✓</td><td colspan='4'>No video files waiting</td></tr>"
history = [r for r in reversed(logs) if r.get("level") == "MOVE" and r.get("msg") == "moving"][:12]
history_rows = "".join(f"<tr><td>✓</td><td class='filename'>{esc(Path(str(r.get('dest',''))).name)}</td><td>{esc(r.get('media_type',''))}</td><td colspan='2'>{esc(r.get('ts',''))}</td></tr>" for r in history) or "<tr><td>—</td><td colspan='4'>No recent imports</td></tr>"
body = f"""<!doctype html><html><head><meta charset='utf-8'><meta name='viewport' content='width=device-width, initial-scale=1'><meta http-equiv='refresh' content='10'><title>{esc(title)}</title><style>body{{font-family:system-ui,sans-serif;margin:0;background:#111;color:#eee}}header{{display:flex;align-items:center;gap:1rem;background:#3b3b3b;padding:.7rem 1.2rem;border-bottom:1px solid #111;flex-wrap:wrap}}header h1{{margin:0;font-size:1.5rem}}.pill{{background:#222;border:1px solid #555;padding:.35rem .7rem}}main{{padding:1rem}}.grid{{display:grid;grid-template-columns:repeat(auto-fit,minmax(260px,1fr));gap:1rem}}.card{{background:#1d1d1d;border:1px solid #333;border-radius:12px;padding:1rem;margin-bottom:1rem;min-width:0;overflow:hidden}}input,button{{padding:.55rem;border:1px solid #555;background:#2b2b2b;color:#eee}}button{{cursor:pointer}}.filename{{overflow-wrap:anywhere;word-break:break-word}}.tiny{{font-size:.78rem;color:#999;line-height:1.25;overflow-wrap:anywhere}}.tiny span{{color:#bbb}}.ok{{color:#60d394}}.warn{{color:#ffd166}}progress{{width:100%;height:1.2rem;accent-color:#7fd37f}}table{{width:100%;border-collapse:collapse;background:#2b2b2b}}th,td{{padding:.65rem;border-bottom:1px solid #111;text-align:left;vertical-align:middle}}th{{background:#444;color:#ddd}}tr:nth-child(even){{background:#333}}tr.active{{background:#3f4a3f}}tr.processing{{background:#4a4232}}pre{{white-space:pre-wrap;overflow-wrap:anywhere;max-height:28rem;overflow:auto}}.muted{{color:#aaa}}a{{color:#8ecae6}}</style></head><body><header><h1>📥 Importarr</h1><span class='pill'>{esc(s['queue']['files'])} videos ready</span><span class='pill'>{esc(s['queue']['processingFiles'])} unpacking</span><span class='pill'>ready {gib(s['queue']['bytes'])}</span><span class='pill'>1h {esc(s['processed']['last1h'])} · 24h {esc(s['processed']['last24h'])} · total {esc(s['processed']['total'])}</span></header><main><div class='grid'><section class='card'><h2>Status</h2><p class='{('warn' if warnings else 'ok')}'>{esc(s['health']['state'])}</p><p>Service: {esc(s['service']['activeState'])}/{esc(s['service']['subState'])}</p><p>Runtime: {secs(s['service']['runtimeSeconds'])}</p><p>Memory current: {gib(s['service']['memoryCurrentBytes'])}</p><p>Memory peak: {gib(s['service']['memoryPeakBytes'])}</p><p class='muted'>Scheduled automatically every 15 minutes. `_UNPACK_` folders are shown as unpacking, not ready.</p></section><section class='card'><h2>Current file</h2>{progress_html}</section><section class='card'><h2>Processed</h2><p>Last 1h: {esc(s['processed']['last1h'])} imported</p><p>Last 24h: {esc(s['processed']['last24h'])} imported</p><p>Total: {esc(s['processed']['total'])} imported</p></section><section class='card'><h2>Warnings</h2>{warning_html}</section></div><section class='card'><h2>Add manual batch</h2><form onsubmit="event.preventDefault();fetch('/api/manual-batches',{{method:'POST',headers:{{'content-type':'application/json'}},body:JSON.stringify({{path:this.path.value}})}}).then(()=>location.reload())"><input name='path' placeholder='folder under manual downloads' size='60'><button>Add folder once</button></form></section><section class='card'><h2>Jobs</h2><table><thead><tr><th></th><th>Name</th><th>State</th><th>Progress / Size</th><th>Runtime</th></tr></thead><tbody>{current_row}{queue_rows}</tbody></table></section><section class='card'><h2>History</h2><table><thead><tr><th></th><th>Name</th><th>Type</th><th colspan='2'>Time</th></tr></thead><tbody>{history_rows}</tbody></table></section><section class='card'><h2>Recent log</h2><pre>{log_text}</pre></section><p><a href='/api/status'>/api/status</a> · <a href='/api/queue'>/api/queue</a> · <a href='/api/logs?limit=100'>/api/logs</a></p></main></body></html>"""
return body.encode()
class Handler(BaseHTTPRequestHandler):
def send(self, code: int, content_type: str, data: bytes) -> None:
self.send_response(code)
self.send_header("Content-Type", content_type)
self.send_header("Cache-Control", "no-store")
self.end_headers()
self.wfile.write(data)
def do_GET(self) -> None:
parsed = urllib.parse.urlparse(self.path)
if parsed.path == "/":
self.send(200, "text/html; charset=utf-8", page())
elif parsed.path == "/api/status":
self.send(200, "application/json", json.dumps(status()).encode())
elif parsed.path == "/api/queue":
self.send(200, "application/json", json.dumps(queue_status()).encode())
elif parsed.path == "/api/logs":
params = urllib.parse.parse_qs(parsed.query)
limit = int(params.get("limit", ["100"])[0])
self.send(200, "application/json", json.dumps(read_logs(limit)).encode())
elif parsed.path == "/health":
self.send(200, "application/json", json.dumps(fast_health()).encode())
else:
self.send(404, "text/plain", b"not found")
def do_POST(self) -> None:
parsed = urllib.parse.urlparse(self.path)
if parsed.path != "/api/manual-batches":
self.send(404, "text/plain", b"not found")
return
length = int(self.headers.get("content-length") or 0)
try:
payload = json.loads(self.rfile.read(length) or b"{}")
except json.JSONDecodeError:
self.send(400, "application/json", json.dumps({"ok": False, "error": "invalid json"}).encode())
return
ok, message = add_manual_batch(str(payload.get("path") or ""))
self.send(200 if ok else 400, "application/json", json.dumps({"ok": ok, "result": message}).encode())
def log_message(self, fmt: str, *args: object) -> None:
return
def main() -> None:
host = os.getenv("IMPORTARR_BIND_HOST", "0.0.0.0")
port = int(os.getenv("IMPORTARR_BIND_PORT", "8095"))
ThreadingHTTPServer((host, port), Handler).serve_forever()
if __name__ == "__main__":
main()
-75
View File
@@ -1,75 +0,0 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>📥 - idle · Importarr {{ status.build.version }}</title>
<link rel="stylesheet" href="/static/importarr.css">
</head>
<body>
<header class="topbar">
<div><h1>Importarr</h1><p>Manual SABnzbd imports, safely gated by SAB completion.</p></div>
<div class="build"><strong>{{ status.build.version }}</strong><span>{{ status.build.git_sha[:12] }} · {{ status.build.build_date }}</span></div>
</header>
<main>
<section class="cards">
<article><strong>{{ status.imported_total }}</strong><span>Imported total</span></article>
<article><strong>{{ status.failed_total }}</strong><span>Failed total</span></article>
<article><strong>{{ status.manual_batches }}</strong><span>Manual batches</span></article>
<article><strong>{{ status.category }}</strong><span>SAB category</span></article>
</section>
<section class="panel">
<h2>Service info</h2>
<dl class="info">
<dt>Name</dt><dd>{{ status.build.name }}</dd>
<dt>Version</dt><dd>{{ status.build.version }}</dd>
<dt>Build date</dt><dd>{{ status.build.build_date }}</dd>
<dt>Git SHA</dt><dd>{{ status.build.git_sha }}</dd>
<dt>Started</dt><dd>{{ status.build.started_at }}</dd>
<dt>Python</dt><dd>{{ status.build.python }}</dd>
<dt>SAB URL</dt><dd>{{ status.sab_url }}</dd>
<dt>Download root</dt><dd>{{ status.download_root }}</dd>
<dt>Movies root</dt><dd>{{ status.movies_root }}</dd>
<dt>TV root</dt><dd>{{ status.tv_root }}</dd>
<dt>Write auth</dt><dd>{{ 'enabled' if status.auth_enabled else 'disabled' }}</dd>
<dt>Queue mode</dt><dd id="queue-mode">{{ status.control.queue_mode }}</dd>
<dt>Current job</dt><dd id="current-job">{{ status.current or 'idle' }}</dd>
</dl>
</section>
<section class="panel">
<h2>Queue controls</h2>
<p>Pause and stop prevent new jobs from being added to the queue. They do not interrupt an import already in progress; use cancel current job for that.</p>
<div class="controls">
<button type="button" data-control="start">Start</button>
<button type="button" data-control="pause">Pause</button>
<button type="button" data-control="stop">Stop</button>
<button type="button" data-control="cancel-current" class="danger">Cancel current job</button>
</div>
</section>
<section>
<h2>Manual batches</h2>
<form id="batch-form" class="inline-form">
<input name="path" placeholder="folder under download root">
<input id="batch-picker" type="file" webkitdirectory directory multiple hidden>
<button type="button" id="browse-batch">Browse…</button>
<button>Add batch</button>
</form>
<table><thead><tr><th>ID</th><th>Status</th><th>Path</th></tr></thead><tbody>
{% for batch in batches %}<tr><td>{{ batch.id }}</td><td>{{ batch.status }}</td><td>{{ batch.path }}</td></tr>{% endfor %}
</tbody></table>
</section>
<section>
<h2>Jobs</h2><button id="force-run" type="button">Force run now</button><div id="jobs">Loading…</div>
</section>
</main>
<script>
async function refresh(){ const r=await fetch('/api/jobs'); const d=await r.json(); document.title=d.jobs.length?`📥 ${d.jobs.length} jobs - Importarr`:'📥 - idle · Importarr'; document.getElementById('jobs').innerHTML='<table><tr><th>Name</th><th>State</th><th>Context</th></tr>'+d.jobs.map(j=>`<tr><td>${j.name||''}</td><td><span class="state">${j.state}</span></td><td><small>${j.relative_path||j.storage||j.reason||''}</small></td></tr>`).join('')+'</table>'; if(d.control){ document.getElementById('queue-mode').textContent=d.control.queue_mode; document.getElementById('current-job').textContent=d.control.current||'idle'; } }
document.querySelectorAll('[data-control]').forEach(button=>button.addEventListener('click', async()=>{ await fetch(`/api/control/${button.dataset.control}`,{method:'POST'}); await refresh(); }));
document.getElementById('browse-batch').addEventListener('click',()=>document.getElementById('batch-picker').click());
document.getElementById('batch-picker').addEventListener('change',e=>{ const f=e.target.files[0]; if(!f)return; const top=(f.webkitRelativePath||'').split('/')[0]; if(top) document.querySelector('#batch-form [name="path"]').value=top; });
document.getElementById('batch-form').addEventListener('submit', async e=>{ e.preventDefault(); await fetch('/api/manual-batches',{method:'POST',headers:{'content-type':'application/json'},body:JSON.stringify({path:e.target.path.value})}); location.reload(); });
document.getElementById('force-run').addEventListener('click', async()=>{ await fetch('/api/import/run-now',{method:'POST',headers:{'content-type':'application/json'},body:JSON.stringify({force:true})}); await refresh(); });
refresh(); setInterval(refresh, 10000);
</script>
</body>
</html>
-2
View File
@@ -12,7 +12,6 @@ license = "MIT"
dependencies = [ dependencies = [
"fastapi>=0.111", "fastapi>=0.111",
"httpx>=0.27", "httpx>=0.27",
"jinja2>=3.1",
"pydantic>=2.7", "pydantic>=2.7",
"uvicorn[standard]>=0.30", "uvicorn[standard]>=0.30",
] ]
@@ -22,7 +21,6 @@ test = ["pytest>=8.2", "pytest-asyncio>=0.23"]
[project.scripts] [project.scripts]
importarr = "importarr.main:run" importarr = "importarr.main:run"
importarr-status = "importarr.status_ui:main"
manual-media-import = "importarr.worker:main" manual-media-import = "importarr.worker:main"
[tool.pytest.ini_options] [tool.pytest.ini_options]
+1 -1
View File
@@ -48,7 +48,7 @@ def test_manual_queue_items_are_persisted_and_imported(tmp_path, monkeypatch):
queued = main.state.list_queue_items() queued = main.state.list_queue_items()
assert len(queued) == 1 assert len(queued) == 1
assert queued[0]["source_type"] == "manual" assert queued[0]["source_type"] == "manual"
assert queued[0]["state"] == "manual_batch" assert queued[0]["state"] == "ready"
assert main._import_manual_batches(main.Importer(movies, tv)) == 1 assert main._import_manual_batches(main.Importer(movies, tv)) == 1
assert main.state.list_queue_items() == [] assert main.state.list_queue_items() == []
+468
View File
@@ -1,3 +1,6 @@
import asyncio
import threading
from importarr.config import Settings from importarr.config import Settings
from importarr.state import State from importarr.state import State
@@ -42,6 +45,125 @@ def test_start_reenables_manual_queue_sync(tmp_path, monkeypatch):
assert len(main.state.list_queue_items()) == 1 assert len(main.state.list_queue_items()) == 1
def test_default_control_commands_target_manual_import_service(monkeypatch):
monkeypatch.delenv("IMPORTARR_START_COMMAND", raising=False)
monkeypatch.delenv("IMPORTARR_STOP_COMMAND", raising=False)
monkeypatch.delenv("IMPORTARR_RESTART_COMMAND", raising=False)
settings = Settings.from_env()
assert settings.start_command == ["systemctl", "start", "manual-media-import.service"]
assert settings.stop_command == ["systemctl", "stop", "manual-media-import.service"]
assert settings.restart_command == ["systemctl", "restart", "manual-media-import.service"]
def test_start_control_starts_manual_import_service(tmp_path, monkeypatch):
main, _download, _movies, _tv = configure_main(tmp_path, monkeypatch)
main.settings.start_command = ["systemctl", "start", "manual-media-import.service"]
calls = []
def fake_run(command, **kwargs):
calls.append(command)
return main.subprocess.CompletedProcess(command, 0, stdout="started", stderr="")
monkeypatch.setattr(main.subprocess, "run", fake_run)
result = main.start_queue()
assert result["control"]["queue_mode"] == "running"
assert result["command_result"]["command"] == ["systemctl", "start", "manual-media-import.service"]
assert calls == [["systemctl", "start", "manual-media-import.service"]]
def test_stop_control_stops_manual_import_service(tmp_path, monkeypatch):
main, _download, _movies, _tv = configure_main(tmp_path, monkeypatch)
main.settings.stop_command = ["systemctl", "stop", "manual-media-import.service"]
calls = []
def fake_run(command, **kwargs):
calls.append(command)
return main.subprocess.CompletedProcess(command, 0, stdout="stopped", stderr="")
monkeypatch.setattr(main.subprocess, "run", fake_run)
result = main.stop_queue()
assert result["control"]["queue_mode"] == "stopped"
assert result["control"]["cancel_requested"] is True
assert result["command_result"]["command"] == ["systemctl", "stop", "manual-media-import.service"]
assert calls == [["systemctl", "stop", "manual-media-import.service"]]
def test_queue_jobs_include_groups_and_manual_context(tmp_path, monkeypatch):
main, download, _movies, _tv = configure_main(tmp_path, monkeypatch)
batch = download / "Release" / "Season 1"
batch.mkdir(parents=True)
(batch / "Episode.mkv").write_bytes(b"episode")
main.state.add_manual_batch(batch.parent)
main.sync_manual_queue()
jobs = main.queue_jobs()
assert jobs[0]["group"] == "manual_batch"
assert jobs[0]["relative_path"] == "Release/Season 1/Episode.mkv"
assert jobs[0]["can_run_now"] is True
assert jobs[0]["state"] == "ready"
def test_queue_item_retry_ignore_and_remove_actions(tmp_path, monkeypatch):
main, _download, _movies, _tv = configure_main(tmp_path, monkeypatch)
row = main.state.upsert_queue_item(source_type="sab", source_id="job-1", name="Release", state="failed", reason="ImportError")
retried = main.queue_item_action(row["id"], main.QueueItemActionRequest(action="retry"))
assert retried["item"]["state"] == "ready"
assert retried["item"]["reason"] == "retry requested"
ignored = main.queue_item_action(row["id"], main.QueueItemActionRequest(action="ignore"))
assert ignored["item"]["state"] == "skipped"
removed = main.queue_item_action(row["id"], main.QueueItemActionRequest(action="remove"))
assert removed == {"status": "removed", "id": row["id"]}
assert main.state.get_queue_item(row["id"]) is None
def test_queue_item_run_now_imports_only_selected_item(tmp_path, monkeypatch):
main, download, movies, _tv = configure_main(tmp_path, monkeypatch)
batch = download / "Release"
batch.mkdir(parents=True)
selected = batch / "Selected.mkv"
waiting = batch / "Waiting.mkv"
selected.write_bytes(b"selected")
waiting.write_bytes(b"waiting")
selected_row = main.state.upsert_queue_item(source_type="manual", source_id=str(selected), source_path=selected, name=selected.name, state="ready")
main.state.upsert_queue_item(source_type="manual", source_id=str(waiting), source_path=waiting, name=waiting.name, state="ready")
result = main.queue_item_action(selected_row["id"], main.QueueItemActionRequest(action="run-now"))
assert result["status"] == "imported"
assert result["imported"] == 1
assert (movies / "Selected.mkv").read_bytes() == b"selected"
assert waiting.exists()
rows = {row["name"]: row for row in main.state.list_queue_items(active_only=False)}
assert rows["Selected.mkv"]["state"] == "imported"
assert rows["Waiting.mkv"]["state"] == "ready"
def test_current_job_status_includes_progress(tmp_path, monkeypatch):
main, _download, _movies, _tv = configure_main(tmp_path, monkeypatch)
monkeypatch.setattr(main.time, "time", lambda: 110.0)
main.set_current_job("Movie.mkv", bytes_copied=50, total_bytes=200, started_at=100.0)
current = main.control_status()["current"]
assert current["file"] == "Movie.mkv"
assert current["bytes_copied"] == 50
assert current["total_bytes"] == 200
assert current["percent"] == 25
assert current["elapsed_seconds"] == 10
def test_cancel_current_stops_before_next_manual_item(tmp_path, monkeypatch): def test_cancel_current_stops_before_next_manual_item(tmp_path, monkeypatch):
main, download, movies, tv = configure_main(tmp_path, monkeypatch) main, download, movies, tv = configure_main(tmp_path, monkeypatch)
batch = download / "Release" batch = download / "Release"
@@ -78,3 +200,349 @@ def test_cancel_current_stops_active_copy(tmp_path, monkeypatch):
assert source.exists() assert source.exists()
assert not any(movies.glob("*.partial")) assert not any(movies.glob("*.partial"))
assert main.state.list_queue_items(active_only=False)[0]["state"] == "skipped" assert main.state.list_queue_items(active_only=False)[0]["state"] == "skipped"
def test_status_includes_queue_counts(tmp_path, monkeypatch):
main, _download, _movies, _tv = configure_main(tmp_path, monkeypatch)
main.state.upsert_queue_item(source_type="manual", source_id="a", name="A.mkv", state="ready")
main.state.upsert_queue_item(source_type="manual", source_id="b", name="B.mkv", state="failed")
status = main.status()
assert status["queue_total"] == 2
assert status["queue_counts"]["ready"] == 1
assert status["queue_counts"]["failed"] == 1
def test_worker_claimed_failure_retries_item(tmp_path, monkeypatch):
main, download, movies, tv = configure_main(tmp_path, monkeypatch)
batch = download / "Release"
batch.mkdir(parents=True)
source = batch / "A.mkv"
source.write_bytes(b"a")
row = main.state.upsert_queue_item(source_type="manual", source_id=str(source), source_path=source, name=source.name, state="ready")
claimed = main.state.claim_next_queue_item(main.WORKER_ID)
class BrokenImporter:
def import_file(self, *args, **kwargs):
raise RuntimeError("boom")
imported = main._import_queue_item(claimed, BrokenImporter(), from_worker=True)
updated = main.state.get_queue_item(row["id"])
assert imported == 0
assert updated["state"] == "retrying"
assert updated["attempt_count"] == 1
assert updated["next_retry_at"] is not None
def test_startup_releases_stale_claims_from_previous_worker(tmp_path, monkeypatch):
main, _download, _movies, _tv = configure_main(tmp_path, monkeypatch)
row = main.state.upsert_queue_item(source_type="manual", source_id="a", name="A.mkv", state="ready")
main.state.claim_queue_item(row["id"], "old-worker", {"ready"})
main.ensure_worker_running()
updated = main.state.get_queue_item(row["id"])
main.stop_worker()
assert updated["state"] == "retrying"
assert updated["claimed_by"] is None
def test_run_now_conflicts_when_item_is_claimed(tmp_path, monkeypatch):
main, _download, _movies, _tv = configure_main(tmp_path, monkeypatch)
row = main.state.upsert_queue_item(source_type="manual", source_id="a", name="A.mkv", state="ready")
main.state.claim_queue_item(row["id"], "other-worker", {"ready"})
try:
main.queue_item_action(row["id"], main.QueueItemActionRequest(action="run-now"))
except main.HTTPException as exc:
assert exc.status_code == 409
else:
raise AssertionError("expected HTTPException")
def test_bulk_run_now_skips_item_claimed_by_worker(tmp_path, monkeypatch):
main, download, movies, tv = configure_main(tmp_path, monkeypatch)
batch = download / "Release"
batch.mkdir(parents=True)
source = batch / "A.mkv"
source.write_bytes(b"a")
created_batch = main.state.add_manual_batch(batch)
row = main.state.upsert_queue_item(source_type="manual", source_id=str(source), source_path=source, name=source.name, state="ready", batch_id=created_batch["id"])
main.state.claim_queue_item(row["id"], "worker", {"ready"})
assert main._import_manual_batches(main.Importer(movies, tv)) == 0
assert source.exists()
assert main.state.get_queue_item(row["id"])["claimed_by"] == "worker"
def test_retry_loses_atomic_race_with_worker_claim(tmp_path, monkeypatch):
main, _download, _movies, _tv = configure_main(tmp_path, monkeypatch)
row = main.state.upsert_queue_item(source_type="manual", source_id="a", name="A.mkv", state="failed")
original = main.state.transition_queue_item_if_unclaimed
def claim_then_transition(*args, **kwargs):
main.state.claim_queue_item(row["id"], "worker", {"failed"})
return original(*args, **kwargs)
monkeypatch.setattr(main.state, "transition_queue_item_if_unclaimed", claim_then_transition)
try:
main.queue_item_action(row["id"], main.QueueItemActionRequest(action="retry"))
except main.HTTPException as exc:
assert exc.status_code == 409
else:
raise AssertionError("expected HTTPException")
assert main.state.get_queue_item(row["id"])["claimed_by"] == "worker"
def test_ignore_loses_atomic_race_with_worker_claim(tmp_path, monkeypatch):
main, _download, _movies, _tv = configure_main(tmp_path, monkeypatch)
row = main.state.upsert_queue_item(source_type="manual", source_id="a", name="A.mkv", state="ready")
original = main.state.transition_queue_item_if_unclaimed
def claim_then_transition(*args, **kwargs):
main.state.claim_queue_item(row["id"], "worker", {"ready"})
return original(*args, **kwargs)
monkeypatch.setattr(main.state, "transition_queue_item_if_unclaimed", claim_then_transition)
try:
main.queue_item_action(row["id"], main.QueueItemActionRequest(action="ignore"))
except main.HTTPException as exc:
assert exc.status_code == 409
else:
raise AssertionError("expected HTTPException")
assert main.state.get_queue_item(row["id"])["claimed_by"] == "worker"
def test_shutdown_timeout_does_not_release_live_worker_claim(tmp_path, monkeypatch):
main, download, _movies, _tv = configure_main(tmp_path, monkeypatch)
source = download / "A.mkv"
source.parent.mkdir(parents=True)
source.write_bytes(b"a")
row = main.state.upsert_queue_item(source_type="manual", source_id=str(source), source_path=source, name=source.name, state="ready")
started = threading.Event()
finish = threading.Event()
class BlockingImporter:
def import_file(self, *args, **kwargs):
started.set()
finish.wait()
raise RuntimeError("stopped")
def active_worker():
claimed = main.state.claim_queue_item(row["id"], main.WORKER_ID, {"ready"})
main._import_queue_item(claimed, BlockingImporter(), from_worker=True)
worker = threading.Thread(target=active_worker)
monkeypatch.setattr(main, "_worker_thread", worker)
monkeypatch.setattr(main, "WORKER_SHUTDOWN_TIMEOUT_SECONDS", 0.01)
worker.start()
assert started.wait(1)
main.shutdown_queue_worker()
assert worker.is_alive()
assert main.state.get_queue_item(row["id"])["claimed_by"] == main.WORKER_ID
finish.set()
worker.join(1)
assert not worker.is_alive()
assert main.state.get_queue_item(row["id"])["claimed_by"] is None
def test_manual_sync_preserves_claim_after_source_is_unlinked(tmp_path, monkeypatch):
main, download, _movies, _tv = configure_main(tmp_path, monkeypatch)
batch = download / "Release"
batch.mkdir(parents=True)
source = batch / "A.mkv"
source.write_bytes(b"a")
created_batch = main.state.add_manual_batch(batch)
row = main.state.upsert_queue_item(source_type="manual", source_id=str(source), source_path=source, name=source.name, state="ready", batch_id=created_batch["id"])
main.state.claim_queue_item(row["id"], main.WORKER_ID, {"ready"})
source.unlink()
main.sync_manual_queue()
preserved = main.state.get_queue_item(row["id"])
assert preserved["state"] == "importing"
assert preserved["claimed_by"] == main.WORKER_ID
def test_bulk_sab_import_preserves_job_metadata(tmp_path, monkeypatch):
main, download, movies, tv = configure_main(tmp_path, monkeypatch)
release = download / "Release"
release.mkdir(parents=True)
source = release / "Movie.mkv"
source.write_bytes(b"movie")
class FakeSabnzbdClient:
def __init__(self, *args, **kwargs):
pass
async def active_nzo_ids(self):
return set()
async def history(self):
return {"history": {"slots": [{"nzo_id": "SAB-123", "name": "Release", "category": "manual", "status": "Completed", "storage": str(release)}]}}
monkeypatch.setattr(main, "SabnzbdClient", FakeSabnzbdClient)
assert asyncio.run(main._import_ready_sab_jobs(main.Importer(movies, tv))) == 1
row = main.state.list_queue_items(active_only=False)[0]
assert row["job_id"] == "SAB-123"
assert row["sab_category"] == "manual"
def test_remove_conflicts_when_item_is_claimed(tmp_path, monkeypatch):
main, _download, _movies, _tv = configure_main(tmp_path, monkeypatch)
row = main.state.upsert_queue_item(source_type="manual", source_id="a", name="A.mkv", state="ready")
main.state.claim_queue_item(row["id"], "other-worker", {"ready"})
try:
main.queue_item_action(row["id"], main.QueueItemActionRequest(action="remove"))
except main.HTTPException as exc:
assert exc.status_code == 409
else:
raise AssertionError("expected HTTPException")
def test_manual_import_completion_persists_completed_row_and_batch_completion(tmp_path, monkeypatch):
main, download, movies, tv = configure_main(tmp_path, monkeypatch)
batch = download / "Release"
batch.mkdir(parents=True)
source = batch / "Done.mkv"
source.write_bytes(b"done")
created_batch = main.state.add_manual_batch(batch)
imported = main._import_manual_batches(main.Importer(movies, tv))
main.sync_manual_queue()
assert imported == 1
rows = {row["name"]: row for row in main.state.list_queue_items(active_only=False)}
assert rows["Done.mkv"]["state"] == "imported"
assert main.state.get_queue_item(rows["Done.mkv"]["id"])["state"] == "imported"
batch_row = next(batch for batch in main.state.list_manual_batches() if batch["id"] == created_batch["id"])
assert batch_row["status"] == "completed"
def test_worker_failure_stops_retrying_after_limit(tmp_path, monkeypatch):
main, download, _movies, _tv = configure_main(tmp_path, monkeypatch)
batch = download / "Release"
batch.mkdir(parents=True)
source = batch / "A.mkv"
source.write_bytes(b"a")
row = main.state.upsert_queue_item(source_type="manual", source_id=str(source), source_path=source, name=source.name, state="ready")
class BrokenImporter:
def import_file(self, *args, **kwargs):
raise RuntimeError("boom")
for _ in range(main.MAX_RETRY_ATTEMPTS):
claimed = main.state.claim_queue_item(row["id"], main.WORKER_ID, {"ready", "retrying"})
assert claimed is not None
main._import_queue_item(claimed, BrokenImporter(), from_worker=True)
current = main.state.get_queue_item(row["id"])
if current["state"] == "retrying":
main.state.mark_queue_item("manual", current["source_id"], "ready", "retry window elapsed")
updated = main.state.get_queue_item(row["id"])
assert updated["state"] == "failed"
assert updated["attempt_count"] == main.MAX_RETRY_ATTEMPTS
def test_control_update_runs_configured_command(tmp_path, monkeypatch):
main, _download, _movies, _tv = configure_main(tmp_path, monkeypatch)
main.settings.update_command = ["upgrade", "now"]
monkeypatch.setattr(main, "check_update_available", lambda: {"status": "update_available", "current_version": "0.1.0", "latest_version": "0.2.0", "update_available": True, "release_url": None})
calls = []
def fake_run(command, **kwargs):
calls.append((command, kwargs))
return main.subprocess.CompletedProcess(command, 0, stdout="updated", stderr="")
monkeypatch.setattr(main.subprocess, "run", fake_run)
result = main.update_service()
assert result["status"] == "update_available"
assert result["command_result"]["status"] == "ok"
assert result["command_result"]["command"] == ["upgrade", "now"]
assert result["command_result"]["stdout"] == "updated"
assert calls[0][0] == ["upgrade", "now"]
assert calls[0][1].get("shell") is not True
def test_control_update_skips_command_when_current(tmp_path, monkeypatch):
main, _download, _movies, _tv = configure_main(tmp_path, monkeypatch)
main.settings.update_command = ["upgrade", "now"]
monkeypatch.setattr(main, "check_update_available", lambda: {"status": "current", "current_version": "0.2.0", "latest_version": "v0.2.0", "update_available": False, "release_url": None})
def fake_run(command, **kwargs):
raise AssertionError("update command should not run without a newer release")
monkeypatch.setattr(main.subprocess, "run", fake_run)
result = main.update_service()
assert result["status"] == "current"
assert result["command"] == ["upgrade", "now"]
assert result["update_available"] is False
def test_update_check_compares_latest_release(tmp_path, monkeypatch):
main, _download, _movies, _tv = configure_main(tmp_path, monkeypatch)
monkeypatch.setenv("IMPORTARR_VERSION", "0.1.0")
class FakeResponse:
def raise_for_status(self):
return None
def json(self):
return {"tag_name": "v0.2.0", "html_url": "https://example.test/releases/v0.2.0"}
class FakeClient:
def __init__(self, timeout):
self.timeout = timeout
def __enter__(self):
return self
def __exit__(self, exc_type, exc, tb):
return False
def get(self, url, headers):
assert url == main.settings.update_release_url
assert headers["Accept"] == "application/json"
return FakeResponse()
monkeypatch.setattr(main.httpx, "Client", FakeClient)
result = main.check_update_available()
assert result["status"] == "update_available"
assert result["current_version"] == "0.1.0"
assert result["latest_version"] == "v0.2.0"
assert result["update_available"] is True
def test_control_restart_reports_command_failure(tmp_path, monkeypatch):
main, _download, _movies, _tv = configure_main(tmp_path, monkeypatch)
main.settings.restart_command = ["restart"]
def fake_run(command, **kwargs):
return main.subprocess.CompletedProcess(command, 1, stdout="", stderr="failed")
monkeypatch.setattr(main.subprocess, "run", fake_run)
try:
main.restart_service()
except main.HTTPException as exc:
assert exc.status_code == 500
assert exc.detail["status"] == "failed"
assert exc.detail["stderr"] == "failed"
else:
raise AssertionError("expected HTTPException")
+35
View File
@@ -19,9 +19,44 @@ def test_completed_manual_is_ready():
def test_wrong_category_ignored(): def test_wrong_category_ignored():
result = classify_history_item(item(category="*"), set(), "manual", ROOT) result = classify_history_item(item(category="*"), set(), "manual", ROOT)
assert result.ready
assert result.reason == "SAB completed inside Importarr download root"
def test_wrong_category_outside_root_ignored():
result = classify_history_item(item(category="*", storage="/tmp/other/Movie"), set(), "manual", ROOT)
assert result.state == "ignored" assert result.state == "ignored"
def test_sab_storage_root_maps_to_local_download_root():
result = classify_history_item(
item(category="*", storage="/data/downloads/manual/Movie"),
set(),
"manual",
ROOT,
sab_storage_root=Path("/data/downloads/manual"),
)
assert result.ready
assert result.storage == ROOT / "Movie"
def test_radarr_sonarr_storage_roots_are_not_importarr_owned():
for storage in ("/data/downloads/movies/Movie", "/data/downloads/tv/Show"):
result = classify_history_item(
item(category="*", storage=storage),
set(),
"manual",
ROOT,
sab_storage_root=Path("/data/downloads/manual"),
)
assert result.state == "ignored"
def test_sab_cat_field_is_treated_as_category():
result = classify_history_item(item(category=None, cat="manual"), set(), "manual", ROOT)
assert result.ready
def test_queue_item_not_ready(): def test_queue_item_not_ready():
result = classify_history_item(item(), {"1"}, "manual", ROOT) result = classify_history_item(item(), {"1"}, "manual", ROOT)
assert result.state == "processing" assert result.state == "processing"
+79
View File
@@ -0,0 +1,79 @@
from importarr.state import State
def test_delete_queue_items_by_state_can_target_reason(tmp_path):
state = State(tmp_path / "state.db")
state.upsert_queue_item(source_type="sab", source_id="stale", name="Stale", state="ignored", reason="SAB category is not owned by Importarr")
state.upsert_queue_item(source_type="sab", source_id="other", name="Other", state="ignored", reason="other reason")
assert state.delete_queue_items_by_state("sab", "ignored", "SAB category is not owned by Importarr") == 1
rows = state.list_queue_items()
assert len(rows) == 1
assert rows[0]["source_id"] == "other"
def test_upsert_preserves_terminal_state_by_default(tmp_path):
state = State(tmp_path / "state.db")
state.upsert_queue_item(source_type="manual", source_id="a", name="A.mkv", state="failed", reason="boom")
updated = state.upsert_queue_item(source_type="manual", source_id="a", name="A.mkv", state="ready", reason="rescanned")
assert updated["state"] == "failed"
assert updated["reason"] == "boom"
def test_claim_next_queue_item_marks_importing(tmp_path):
state = State(tmp_path / "state.db")
state.upsert_queue_item(source_type="manual", source_id="a", name="A.mkv", state="ready")
row = state.claim_next_queue_item("worker-1")
assert row is not None
assert row["state"] == "importing"
assert row["claimed_by"] == "worker-1"
def test_release_stale_claims_requeues_importing_items(tmp_path):
state = State(tmp_path / "state.db")
state.upsert_queue_item(source_type="manual", source_id="a", name="A.mkv", state="ready")
state.claim_next_queue_item("worker-1")
released = state.release_stale_claims()
row = state.get_queue_item(1)
assert released == 1
assert row["state"] == "retrying"
assert row["claimed_by"] is None
def test_claim_queue_item_requires_unclaimed_allowed_state(tmp_path):
state = State(tmp_path / "state.db")
row = state.upsert_queue_item(source_type="manual", source_id="a", name="A.mkv", state="ready")
claimed = state.claim_queue_item(row["id"], "worker-1", {"ready"})
blocked = state.claim_queue_item(row["id"], "worker-2", {"ready", "importing"})
assert claimed is not None
assert claimed["claimed_by"] == "worker-1"
assert blocked is None
def test_remove_missing_manual_items_keeps_terminal_rows(tmp_path):
state = State(tmp_path / "state.db")
state.upsert_queue_item(source_type="manual", source_id="done", name="Done.mkv", state="imported", batch_id=1)
state.upsert_queue_item(source_type="manual", source_id="pending", name="Pending.mkv", state="ready", batch_id=1)
state.remove_missing_manual_items(1, set())
rows = {row["source_id"]: row for row in state.list_queue_items(active_only=False)}
assert "done" in rows
assert "pending" not in rows
def test_retry_item_respects_next_retry_at(tmp_path):
state = State(tmp_path / "state.db")
state.upsert_queue_item(source_type="manual", source_id="a", name="A.mkv", state="ready")
state.mark_queue_item_result("manual", "a", "retrying", "boom", increment_attempts=True, next_retry_seconds=60)
assert state.claim_next_queue_item("worker-1") is None
+114
View File
@@ -8,6 +8,19 @@ def test_health_contains_build_info(tmp_path, monkeypatch):
assert payload["version"] assert payload["version"]
def test_build_date_is_rendered_in_local_time(monkeypatch):
import importarr.build_info as build_info
monkeypatch.setenv("TZ", "Europe/Copenhagen")
import time
time.tzset()
monkeypatch.setenv("IMPORTARR_BUILD_DATE", "2026-07-29T12:00:00Z")
assert build_info.build_info()["build_date"] == "2026-07-29T14:00:00+02:00"
def test_status_contains_service_configuration(tmp_path, monkeypatch): def test_status_contains_service_configuration(tmp_path, monkeypatch):
monkeypatch.setenv("IMPORTARR_STATE_PATH", str(tmp_path / "state.db")) monkeypatch.setenv("IMPORTARR_STATE_PATH", str(tmp_path / "state.db"))
import importarr.main as main import importarr.main as main
@@ -18,3 +31,104 @@ def test_status_contains_service_configuration(tmp_path, monkeypatch):
assert "movies_root" in payload assert "movies_root" in payload
assert "tv_root" in payload assert "tv_root" in payload
assert "auth_enabled" in payload assert "auth_enabled" in payload
def test_index_serves_react_application(tmp_path, monkeypatch):
monkeypatch.setenv("IMPORTARR_STATE_PATH", str(tmp_path / "state.db"))
import importarr.main as main
from fastapi.testclient import TestClient
response = TestClient(main.app).get("/")
assert response.status_code == 200
assert '<div id="root"></div>' in response.text
assert '/static/assets/app.js' in response.text
assert '/static/assets/app.css' in response.text
def test_frontend_uses_required_stack_and_capabilities():
from pathlib import Path
app = Path("frontend/src/main.jsx").read_text()
package = Path("frontend/package.json").read_text()
assert "lucide-react" in package
assert "@radix-ui/react-dialog" in package
assert "tailwindcss" in package
for endpoint in ("/api/jobs", "/api/settings", "/api/manual-batches", "/api/control/update", "/api/import/run-now"):
assert endpoint in app
def test_frontend_defines_exact_light_and_dark_tokens():
from pathlib import Path
css = Path("frontend/src/globals.css").read_text()
for value in ("#F8FAFC", "#FFFFFF", "#4A43EC", "#7171FF", "#2AD1ED", "#1A1D2E", "#64748B", "#E2E8F0", "#252B42", "#303753", "#23283B", "#42ECF5", "#8B95B7", "#3D4668"):
assert value in css
assert ".dark" in css
def test_settings_endpoint_persists_arr_connection_values(tmp_path, monkeypatch):
monkeypatch.setenv("IMPORTARR_STATE_PATH", str(tmp_path / "state.db"))
import importarr.main as main
from fastapi.testclient import TestClient
response = TestClient(main.app).post(
"/api/settings",
json={
"sab_url": "http://sab:8080",
"sab_api_key": "sab-secret",
"radarr_url": "http://radarr:7878",
"radarr_api_key": "radarr-secret",
"sonarr_url": "http://sonarr:8989",
"sonarr_api_key": "sonarr-secret",
},
)
assert response.status_code == 200
assert response.json() == {
"sab_url": "http://sab:8080",
"sab_api_key_configured": True,
"radarr_url": "http://radarr:7878",
"radarr_api_key_configured": True,
"sonarr_url": "http://sonarr:8989",
"sonarr_api_key_configured": True,
}
assert main.settings.sab_api_key == "sab-secret"
assert main.state.get_app_state("radarr_api_key") == "radarr-secret"
def test_sab_connection_test_reports_success(tmp_path, monkeypatch):
monkeypatch.setenv("IMPORTARR_STATE_PATH", str(tmp_path / "state.db"))
import importarr.main as main
from fastapi.testclient import TestClient
async def fake_queue(self):
return {"queue": {"slots": [{"name": "one"}, {"name": "two"}]}}
monkeypatch.setattr(main.SabnzbdClient, "queue", fake_queue)
response = TestClient(main.app).post(
"/api/settings/test-connection",
json={"service": "sabnzbd", "url": "http://sab:8080", "api_key": "secret"},
)
assert response.status_code == 200
assert response.json() == {"ok": True, "service": "sabnzbd", "message": "Connected to SABnzbd; 2 queued jobs visible."}
def test_connection_test_rejects_unknown_service(tmp_path, monkeypatch):
monkeypatch.setenv("IMPORTARR_STATE_PATH", str(tmp_path / "state.db"))
import importarr.main as main
from fastapi.testclient import TestClient
response = TestClient(main.app).post(
"/api/settings/test-connection",
json={"service": "lidarr", "url": "http://lidarr:8686"},
)
assert response.status_code == 400