Importarr needs a real durable queue model rather than mostly request-triggered import behavior. Radarr/Sonarr/Tdarr all make queued work first-class.
Proposed change
Implement a background queue engine that persists import jobs and transitions them through explicit states.
Scope
Queue table for detected video files and associated SAB/manual-batch metadata.
States such as detected, waiting_for_sab, ready, importing, imported, failed, skipped, and retrying.
Worker loop with polling interval and safe locking.
Idempotent detection so the same file is not imported twice.
API endpoints to list, retry, ignore, and delete queue items.
Acceptance criteria
Jobs survive service restart.
A SAB job is imported only after readiness criteria pass.
Manual batch files appear as queue items and are removed/archived once completed.
Queue state is visible through /api/status and the UI.
## Problem
Importarr needs a real durable queue model rather than mostly request-triggered import behavior. Radarr/Sonarr/Tdarr all make queued work first-class.
## Proposed change
Implement a background queue engine that persists import jobs and transitions them through explicit states.
## Scope
- Queue table for detected video files and associated SAB/manual-batch metadata.
- States such as `detected`, `waiting_for_sab`, `ready`, `importing`, `imported`, `failed`, `skipped`, and `retrying`.
- Worker loop with polling interval and safe locking.
- Idempotent detection so the same file is not imported twice.
- API endpoints to list, retry, ignore, and delete queue items.
## Acceptance criteria
- Jobs survive service restart.
- A SAB job is imported only after readiness criteria pass.
- Manual batch files appear as queue items and are removed/archived once completed.
- Queue state is visible through `/api/status` and the UI.
## Implementation update
- Added a durable SQLite-backed queue worker that claims and processes queued imports in the background.
- Queue items now preserve state across rescans and restarts, track attempts, and support claimed/importing/retrying transitions.
- Exposed queue totals/state counts in `/api/status` and surfaced attempt counts in the UI queue list.
- Kept queue item actions for retry, ignore, remove, and run-now with durable state transitions.
## Commit
- `a088d84` — Implement persistent queue engine Refs #6
## Verification
- `./.venv/bin/python -m pytest -q -W ignore::DeprecationWarning tests/test_state.py tests/test_queue_controls.py`
- Result: 21 passed
daniels
added the ready label 2026-07-30 09:02:14 +02:00
## Review follow-up update
- Fixed stale-claim recovery so startup/shutdown now requeue all importing items instead of only the current worker id.
- Added bounded retries with retry delay metadata and terminal failure after the retry limit.
- Prevented worker/API races by requiring atomic claims for run-now and rejecting ignore/remove/retry while an item is claimed.
- Preserved completed manual queue rows across rescans and only complete batches once no source files and no active items remain.
- Serialized SQLite access with a lock and enabled WAL + busy timeout for safer cross-thread behavior.
## Commit
- `4e3d021` — Fix queue review findings Refs #6
## Verification
- `./.venv/bin/python -m pytest -q -W ignore::DeprecationWarning tests/test_state.py tests/test_queue_controls.py`
- Result: 29 passed
Preserved claimed/importing manual rows during reconciliation so a source unlinked just before result persistence is no longer deleted out from under an active import.
Bounded worker shutdown wait and avoided releasing claims when the worker is still alive after timeout.
Preserved SAB queue metadata (job_id, sab_category) in the bulk run-now/import path.
## Final review follow-up
- Preserved claimed/importing manual rows during reconciliation so a source unlinked just before result persistence is no longer deleted out from under an active import.
- Bounded worker shutdown wait and avoided releasing claims when the worker is still alive after timeout.
- Preserved SAB queue metadata (`job_id`, `sab_category`) in the bulk run-now/import path.
## Commit
- `c5cc090` — Fix remaining queue races Refs #6
## Verification
- `/tmp/opencode/importarr-venv/bin/python -m pytest tests/test_queue_controls.py tests/test_manual_batches.py`
- Result: 31 passed
- Independent review pass: no material findings
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Problem
Importarr needs a real durable queue model rather than mostly request-triggered import behavior. Radarr/Sonarr/Tdarr all make queued work first-class.
Proposed change
Implement a background queue engine that persists import jobs and transitions them through explicit states.
Scope
detected,waiting_for_sab,ready,importing,imported,failed,skipped, andretrying.Acceptance criteria
/api/statusand the UI.Implementation update
/api/statusand surfaced attempt counts in the UI queue list.Commit
a088d84— Implement persistent queue engine Refs #6Verification
./.venv/bin/python -m pytest -q -W ignore::DeprecationWarning tests/test_state.py tests/test_queue_controls.pyReview follow-up update
Commit
4e3d021— Fix queue review findings Refs #6Verification
./.venv/bin/python -m pytest -q -W ignore::DeprecationWarning tests/test_state.py tests/test_queue_controls.pyImplemented the latest concurrency review fixes in commit
7bdab60. Bulk run-now imports now atomically claim manual and SAB queue items; retry/ignore transitions use conditional atomic updates; shutdown waits for the active worker before releasing claims. Added targeted regressions for bulk claim exclusion, retry/ignore claim races, and shutdown ordering. Verification: .venv/bin/python -m pytest -q tests/test_state.py tests/test_queue_controls.py tests/test_manual_batches.py (36 passed; existing FastAPI on_event deprecation warnings only).Final review follow-up
job_id,sab_category) in the bulk run-now/import path.Commit
c5cc090— Fix remaining queue races Refs #6Verification
/tmp/opencode/importarr-venv/bin/python -m pytest tests/test_queue_controls.py tests/test_manual_batches.py