Reference
Architecture
How the codebase is organized — application pages, folder structure, and the request flow behind the scenes.
Application Pages
| Page | URL | Description |
|---|---|---|
| Landing | / | Public homepage with feature overview |
| Login | /pages/login.html | Sign in with email and password |
| Dashboard | /pages/dashboard.html | Stats overview and recent activity |
| Documents | /pages/documents.html | Upload, browse, filter, approve documents |
| AI Search | /pages/search.html | Semantic search across all documents |
| Q&A Chat | /pages/chat.html | Ask questions, get cited AI answers |
| Analytics | /pages/analytics.html | Usage trends and knowledge gaps |
| Reapprovals | /pages/reapprovals.html | Admin/staff review queue for rejected-document reapproval requests |
| Activity Log | /pages/activity-log.html | Admin/staff audit-log feed with filters and CSV export |
| Settings | /pages/settings.html | Profile, theme, user and org management |
| Getting started | /pages/onboarding.html | First-steps checklist (Help menu); steps tick themselves from the account's activity |
| Support | /pages/support.html | Help/contact information |
| Terms | /pages/terms.html | Terms & Conditions |
Project Folder Structure
SKMS/ <- your project root
│
├── run.py <- start the server: python run.py
├── setup_db.py <- one script: creates the DB (if missing),
│ schema, and demo accounts — safe to re-run
├── start.bat <- Windows: Quick Start / Full Install
│ launcher (Docker, Ollama, DB, app)
├── start.sh <- macOS/Linux: same launcher as start.bat
├── pytest.ini <- test runner configuration
├── alembic.ini <- database migration configuration
├── docker-compose.yml <- starts MinIO + Redis + ONLYOFFICE + NudeNet
│ (+ optionally Postgres/the API itself)
├── Dockerfile <- builds the API into a Docker container (uv)
├── .dockerignore <- keeps .venv/.env/etc. out of the image
├── .env <- your local settings
│
├── app/ <- all Python / FastAPI source code
│ ├── main.py <- FastAPI app: routers, static files,
│ │ health check, startup (MinIO + Ollama)
│ ├── config.py <- reads all settings from .env
│ ├── database.py <- async SQLAlchemy engine and session
│ │
│ ├── models/ <- database table definitions (SQLAlchemy ORM)
│ │ ├── organization.py <- organizations table (CIT, CAS, COE...)
│ │ ├── user.py <- users table + UserRole enum
│ │ ├── document.py <- documents + versions, activity, access
│ │ ├── reapproval.py <- reapproval requests for rejected docs
│ │ ├── folder.py <- folder tree (nested folders)
│ │ ├── tag.py <- document tags
│ │ ├── chat.py <- chat sessions, messages, citations
│ │ ├── analytics.py <- search logs, analytics snapshots
│ │ ├── audit.py <- audit trail
│ │ ├── activity.py <- per-document/user activity feed
│ │ ├── notification.py <- in-app notifications
│ │ └── branding.py <- runtime logo/title/favicon config
│ │
│ ├── schemas/ <- Pydantic request/response data shapes
│ ├── routers/ <- API endpoint handlers
│ │ ├── auth.py <- POST /api/auth/login|logout|refresh
│ │ ├── documents.py <- documents CRUD, approve/reject, cancel,
│ │ │ related docs, request-reapproval
│ │ ├── reapprovals.py <- admin/staff reapproval review queue
│ │ ├── folders.py <- /api/folders/*
│ │ ├── search.py <- GET /api/search, /api/search/suggest
│ │ ├── chat.py <- /api/chat/sessions/*
│ │ ├── analytics.py <- /api/analytics/*
│ │ ├── users.py <- /api/users/*
│ │ ├── orgs.py <- /api/orgs/*
│ │ ├── organizations.py <- org CRUD (Settings → org management)
│ │ ├── admin.py <- /api/admin/activity-log, CSV export
│ │ └── branding.py <- GET /api/branding (public, unauthenticated)
│ │
│ ├── services/ <- business logic layer
│ │ ├── ai_service.py <- Ollama: embeddings, summaries, RAG chat,
│ │ │ citation grounding, chat titles,
│ │ │ sensitive-content screening (text, vision,
│ │ │ NudeNet via nudenet_service over HTTP)
│ │ ├── moderation_service.py <- chat-message moderation (keyword + LLM)
│ │ ├── storage_service.py <- MinIO: upload, download, validate, delete
│ │ ├── onlyoffice_service.py <- ONLYOFFICE editor config/callbacks
│ │ ├── search_service.py <- pgvector cosine similarity search
│ │ ├── analytics_service.py <- dashboard SQL aggregation queries
│ │ ├── document_processor.py <- background task: extract → embed →
│ │ │ summarize → screen (cancellable)
│ │ ├── document_service.py <- shared document CRUD/query helpers
│ │ ├── auth_service.py <- login, lockout, token refresh logic
│ │ ├── notification_service.py <- creates/dispatches in-app notifications
│ │ ├── navigation_service.py <- sidebar/nav-group data for the frontend
│ │ └── chat_cancel.py <- cancels an in-flight chat generation
│ │
│ ├── tasks/ <- scheduled/background jobs
│ │ ├── process_document.py <- entry point run by document_processor.py
│ │ ├── analytics_snapshot.py <- periodic analytics rollup
│ │ └── scheduler.py <- in-process job scheduler
│ │
│ ├── utils/
│ │ └── file_utils.py <- shared file/path helpers
│ │
│ └── core/ <- shared infrastructure
│ ├── security.py <- JWT creation/decoding, bcrypt hashing
│ ├── dependencies.py <- get_current_user, require_admin,
│ │ require_staff_or_admin
│ ├── permissions.py <- role/visibility access-check helpers
│ ├── exceptions.py <- NotFoundError, ForbiddenError
│ └── audit.py <- writes action records to audit_logs table
│
├── static/ <- HTML, CSS, JavaScript (served by FastAPI)
│ ├── pages/ <- all HTML pages (see table above)
│ ├── js/
│ │ ├── app.js <- global helpers: icon(), Modal (confirm/
│ │ │ prompt/alert as proper in-app forms)
│ │ ├── api.js <- apiFetch() wrapper, base URL, auth header
│ │ ├── auth.js <- session guard, token storage, silent refresh
│ │ ├── layout.js <- sidebar, logout, page header
│ │ ├── docPreview.js <- shared document preview modal
│ │ ├── imageZoom.js <- zoom/pan image viewer used by previews
│ │ └── utils.js <- debounce, truncate, showToast, formatDate
│ ├── css/
│ └── images/
│
├── nudenet_service/ <- separate NudeNet container (AGPL-isolated),
│ called over HTTP; never imported by app/
├── migrations/ <- Alembic async database migrations
├── tests/ <- test suite
├── pyproject.toml <- dependencies (managed by uv)
└── uv.lock <- exact resolved versions for every platform
How a Document Becomes Searchable
- The browser uploads the file to
POST /api/documents; it's stored in MinIO and a row is created withprocessing_status = pending - A background task (
document_processor.py) runs: extract text (including OCR for scanned PDFs/images via the vision model) → generate an embedding vianomic-embed-text→ generate a summary viaqwen2.5:3b→ screen for sensitive content - Each stage is checkpointed so an in-progress upload can be cancelled cleanly at any point
- Image uploads get a second, dedicated screen on top of the text-based one: the NudeNet classifier (running in its own
skms_nudenetcontainer, reached over HTTP atNUDENET_SERVICE_URL) and the vision model each independently check for nudity, and the vision model also checks for graphic violence/gore — a hit from either flags the document - If the content screen flags it, the document is auto-rejected with a reason; if a screening layer errors instead of completing, the document is left in
pendingfor manual review rather than auto-approved; otherwise it's live and searchable immediately - Search converts a query into the same embedding space and ranks documents by cosine similarity in pgvector, combined with keyword matches on title/author/tags
How a Chat Answer Gets Its Citations
- The message is moderated (keyword filter, then an LLM classifier), then embedded and run through the same hybrid search as the Search page — but with logging off, so chat prompts never appear in the Dashboard's search analytics or Knowledge Gaps
- The top matches are handed to
qwen2.5:3bas numbered[Document N]blocks and the answer streams to the browser token by token; it's saved to the database the moment it finishes, before any citation work starts - Citations are chosen after the answer exists, in three passes: (1) any document the answer names outright ("Based on Document 2…") is cited directly; (2) otherwise, the new terms the answer introduced — names, numbers, rare words that weren't in the question — are looked up verbatim in each candidate's text, and a strong match is cited without asking the model; (3) only if neither is conclusive is the model asked which documents it used, with a weak lexical match as the tiebreak if it says none (
ai_service.py:select_used_citations(),_lexical_grounding()) - On a brand-new conversation the model finally writes a short title from the first exchange, pushed to the history rail as a
titleevent - Every one of these calls uses the same
num_ctx(CHAT_NUM_CTX) and a 30-minutekeep_alive— Ollama keys its loaded runner on (model, context size), so a mismatch anywhere in the chain would reload the model from disk mid-message
How Editing and Version History Work
- Office/PDF files open in an embedded ONLYOFFICE editor; the Document Server fetches the file through
GET /documents/{id}/office-download(a proxied, token-scoped URL rather than a raw MinIO presigned link) and saves back throughPOST /documents/{id}/office-callback - Every save snapshots the previous file as a
document_versionsrow, uploads the new bytes, bumpsdocuments.version, stores thechanges.zipand history metadata ONLYOFFICE returns, and re-runs the extract → embed → summarize pipeline so search and chat reflect the edit - The editor's own File → Version History panel is fed from those rows by
GET /documents/{id}/office-history(the list) and/office-history/{n}(a signed payload pointing the Document Server at that version's bytes and, when available, the previous version plus its changes archive for diff highlighting) - Restoring a version swaps the stored file back if the bytes differ (or the text snapshot for text-only edits), records the pre-restore state as another version, and re-processes — so a restore is itself undoable