XQ commited on
Commit
3f19c23
·
1 Parent(s): 14ab2b8

Update health check and cloud deployment

Browse files
.env.example CHANGED
@@ -71,6 +71,16 @@ EVALUATOR_LLM_MODEL=llama-3.3-70b-versatile
71
  # GROQ_MODEL=qwen/qwen3-32b
72
  # LOCAL_EMBEDDING_MODEL=paraphrase-multilingual-MiniLM-L12-v2
73
 
 
 
 
 
 
 
 
 
 
 
74
  # *****************************************************************************
75
  # EXAMPLE 2e — CLOUD MODE (Anthropic LLM + local embeddings)
76
  # *****************************************************************************
 
71
  # GROQ_MODEL=qwen/qwen3-32b
72
  # LOCAL_EMBEDDING_MODEL=paraphrase-multilingual-MiniLM-L12-v2
73
 
74
+ # *****************************************************************************
75
+ # EXAMPLE 2d — CLOUD MODE (AWS Bedrock)
76
+ # *****************************************************************************
77
+ # LLM_PROVIDER=bedrock
78
+ # EMBEDDING_PROVIDER=bedrock
79
+ # AWS_REGION=eu-west-1
80
+ # AWS_BEDROCK_MODEL=anthropic.claude-sonnet-4-20250514-v1:0
81
+ # AWS_BEDROCK_EMBEDDING_MODEL=amazon.titan-embed-text-v2:0
82
+ # Note: Uses default AWS credential chain (env vars, ~/.aws/credentials, or IAM role)
83
+
84
  # *****************************************************************************
85
  # EXAMPLE 2e — CLOUD MODE (Anthropic LLM + local embeddings)
86
  # *****************************************************************************
.github/README.md CHANGED
@@ -15,8 +15,8 @@ A RAG application that lets users ask questions about documents in any language
15
  | Agent flows | Plan-and-Execute with six tools, ReAct sub-agent and conversation memory |
16
  | Evaluation | RAGAS metrics (faithfulness, answer relevancy, context precision) |
17
  | Traceability | Each answer includes source references with chunk ID and page number, plus structured logging |
18
- | Provider abstraction | Factory pattern that allows swapping between Ollama, OpenAI, Azure OpenAI, Anthropic and Google GenAI without touching business code |
19
- | Deployment | Docker Compose for local setup, Hugging Face Spaces for the public demo |
20
 
21
  ### How it works
22
 
@@ -80,10 +80,29 @@ The test set is auto-generated and biased toward single-document factual questio
80
 
81
  ### Provider support
82
 
83
- LLM and embedding backends are configured through environment variables. Supported providers are Ollama, OpenAI, Azure OpenAI, Anthropic, Google GenAI and Groq. The default setup (Ollama and HuggingFace) runs entirely locally without any API keys.
84
 
85
  See `.env.example` for per-provider configuration.
86
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
87
  ### Try it live
88
 
89
  The demo lives at [xq-dokumentassistent.hf.space](https://xq-dokumentassistent.hf.space).
@@ -164,20 +183,27 @@ src/
164
  reranker.py # cross-encoder
165
  api/
166
  main.py
167
- routes.py # /query, /ingest, /health
168
  agent/
169
  intent_classifier.py
170
  router.py # pipeline mode (AGENT_MODE=pipeline)
171
  tools.py # six retrieval tools and ToolResultStore
172
  plan_and_execute.py # Plan-and-Execute agent (AGENT_MODE=react)
173
  memory.py # conversation memory for multi-turn
 
174
  evaluation/
175
  evaluator.py # RAGAS metrics
176
  ui/
177
  app.py # Streamlit frontend
178
  scripts/
179
  ingest.py
 
180
  e2e_test.py
181
  tests/
182
  docs/ # example PDFs or texts (KU AI public documents)
 
 
 
 
 
183
  ```
 
15
  | Agent flows | Plan-and-Execute with six tools, ReAct sub-agent and conversation memory |
16
  | Evaluation | RAGAS metrics (faithfulness, answer relevancy, context precision) |
17
  | Traceability | Each answer includes source references with chunk ID and page number, plus structured logging |
18
+ | Provider abstraction | Factory pattern that allows swapping between Ollama, OpenAI, Azure OpenAI, AWS Bedrock, Anthropic and Google GenAI without touching business code |
19
+ | Deployment | Docker Compose (local), Azure Container Apps, AWS ECS Fargate, Hugging Face Spaces (demo) |
20
 
21
  ### How it works
22
 
 
80
 
81
  ### Provider support
82
 
83
+ LLM and embedding backends are configured through environment variables. Supported providers are Ollama, OpenAI, Azure OpenAI, AWS Bedrock, Anthropic, Google GenAI and Groq. The default setup (Ollama and HuggingFace) runs entirely locally without any API keys.
84
 
85
  See `.env.example` for per-provider configuration.
86
 
87
+ ### Cloud deployment
88
+
89
+ The application is cloud-agnostic by design. Business code depends only on LangChain abstract interfaces; the concrete provider is selected at deploy time via environment variables.
90
+
91
+ | Layer | Azure | AWS | Local |
92
+ |---|---|---|---|
93
+ | LLM / Embeddings | Azure OpenAI | Bedrock (Claude, Titan) | Ollama + HuggingFace |
94
+ | Container registry | ACR | ECR | - |
95
+ | Runtime | Container Apps | ECS Fargate | docker-compose |
96
+ | CI/CD | GitHub Actions | GitHub Actions | - |
97
+
98
+ GitHub Actions workflows are included for both clouds:
99
+
100
+ - `ci.yml` runs lint, type check, and tests on every push and PR
101
+ - `deploy-azure.yml` builds, pushes to ACR, and deploys to Azure Container Apps
102
+ - `deploy-aws.yml` builds, pushes to ECR, and deploys to ECS Fargate
103
+
104
+ Health probes (`/health/live` for liveness, `/health/ready` for readiness) are used by container orchestrators to manage rolling deployments.
105
+
106
  ### Try it live
107
 
108
  The demo lives at [xq-dokumentassistent.hf.space](https://xq-dokumentassistent.hf.space).
 
183
  reranker.py # cross-encoder
184
  api/
185
  main.py
186
+ routes.py # /query, /ingest, /health/live, /health/ready
187
  agent/
188
  intent_classifier.py
189
  router.py # pipeline mode (AGENT_MODE=pipeline)
190
  tools.py # six retrieval tools and ToolResultStore
191
  plan_and_execute.py # Plan-and-Execute agent (AGENT_MODE=react)
192
  memory.py # conversation memory for multi-turn
193
+ session_store.py # SQLite-backed per-session memory persistence
194
  evaluation/
195
  evaluator.py # RAGAS metrics
196
  ui/
197
  app.py # Streamlit frontend
198
  scripts/
199
  ingest.py
200
+ evaluate.py # RAGAS evaluation CLI
201
  e2e_test.py
202
  tests/
203
  docs/ # example PDFs or texts (KU AI public documents)
204
+ .github/
205
+ workflows/
206
+ ci.yml # lint + test on push/PR
207
+ deploy-azure.yml # build, push ACR, deploy Container Apps
208
+ deploy-aws.yml # build, push ECR, deploy ECS Fargate
209
  ```
.github/workflows/ci.yml ADDED
@@ -0,0 +1,49 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ name: CI
2
+
3
+ on:
4
+ push:
5
+ branches: [main]
6
+ pull_request:
7
+ branches: [main]
8
+
9
+ jobs:
10
+ lint-and-test:
11
+ runs-on: ubuntu-latest
12
+
13
+ steps:
14
+ - uses: actions/checkout@v4
15
+
16
+ - uses: actions/setup-python@v5
17
+ with:
18
+ python-version: "3.11"
19
+ cache: pip
20
+
21
+ - name: Install dependencies
22
+ run: pip install -r requirements.txt
23
+
24
+ - name: Lint with ruff
25
+ run: |
26
+ pip install ruff
27
+ ruff check src/ tests/
28
+
29
+ - name: Type check with mypy
30
+ run: |
31
+ pip install mypy
32
+ mypy src/ --ignore-missing-imports --no-error-summary || true
33
+
34
+ - name: Run tests
35
+ run: pytest tests/ -v --ignore=tests/evaluation/
36
+ env:
37
+ LLM_PROVIDER: ollama
38
+ EMBEDDING_PROVIDER: local
39
+
40
+ build-image:
41
+ runs-on: ubuntu-latest
42
+ needs: lint-and-test
43
+ if: github.ref == 'refs/heads/main'
44
+
45
+ steps:
46
+ - uses: actions/checkout@v4
47
+
48
+ - name: Build Docker image
49
+ run: docker build -f Dockerfile.compose -t doc-assistant:${{ github.sha }} .
.github/workflows/deploy-aws.yml ADDED
@@ -0,0 +1,71 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ name: Deploy to AWS
2
+
3
+ on:
4
+ # Uncomment to auto-deploy after CI passes on main:
5
+ # workflow_run:
6
+ # workflows: [CI]
7
+ # types: [completed]
8
+ # branches: [main]
9
+ workflow_dispatch:
10
+
11
+ env:
12
+ AWS_REGION: ${{ vars.AWS_REGION || 'eu-west-1' }}
13
+ ECR_REPOSITORY: doc-assistant
14
+ ECS_CLUSTER: ${{ vars.ECS_CLUSTER }}
15
+ ECS_SERVICE: doc-assistant
16
+ TASK_DEFINITION_FAMILY: doc-assistant
17
+
18
+ jobs:
19
+ deploy:
20
+ runs-on: ubuntu-latest
21
+ if: ${{ github.event.workflow_run.conclusion == 'success' || github.event_name == 'workflow_dispatch' }}
22
+
23
+ permissions:
24
+ id-token: write
25
+ contents: read
26
+
27
+ steps:
28
+ - uses: actions/checkout@v4
29
+
30
+ - name: Configure AWS credentials
31
+ uses: aws-actions/configure-aws-credentials@v4
32
+ with:
33
+ role-to-assume: ${{ secrets.AWS_ROLE_ARN }}
34
+ aws-region: ${{ env.AWS_REGION }}
35
+
36
+ - name: Login to ECR
37
+ id: ecr-login
38
+ uses: aws-actions/amazon-ecr-login@v2
39
+
40
+ - name: Build and push image to ECR
41
+ id: build
42
+ env:
43
+ ECR_REGISTRY: ${{ steps.ecr-login.outputs.registry }}
44
+ run: |
45
+ IMAGE_TAG=${{ env.ECR_REGISTRY }}/${{ env.ECR_REPOSITORY }}:${{ github.sha }}
46
+ docker build -f Dockerfile.compose -t $IMAGE_TAG .
47
+ docker push $IMAGE_TAG
48
+ echo "image=$IMAGE_TAG" >> $GITHUB_OUTPUT
49
+
50
+ - name: Download current task definition
51
+ run: |
52
+ aws ecs describe-task-definition \
53
+ --task-definition ${{ env.TASK_DEFINITION_FAMILY }} \
54
+ --query taskDefinition \
55
+ > task-definition.json
56
+
57
+ - name: Update task definition with new image
58
+ id: task-def
59
+ uses: aws-actions/amazon-ecs-render-task-definition@v1
60
+ with:
61
+ task-definition: task-definition.json
62
+ container-name: api
63
+ image: ${{ steps.build.outputs.image }}
64
+
65
+ - name: Deploy to ECS
66
+ uses: aws-actions/amazon-ecs-deploy-task-definition@v2
67
+ with:
68
+ task-definition: ${{ steps.task-def.outputs.task-definition }}
69
+ service: ${{ env.ECS_SERVICE }}
70
+ cluster: ${{ env.ECS_CLUSTER }}
71
+ wait-for-service-stability: true
.github/workflows/deploy-azure.yml ADDED
@@ -0,0 +1,61 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ name: Deploy to Azure
2
+
3
+ on:
4
+ # Uncomment to auto-deploy after CI passes on main:
5
+ # workflow_run:
6
+ # workflows: [CI]
7
+ # types: [completed]
8
+ # branches: [main]
9
+ workflow_dispatch:
10
+
11
+ env:
12
+ AZURE_RESOURCE_GROUP: ${{ vars.AZURE_RESOURCE_GROUP }}
13
+ ACR_NAME: ${{ vars.ACR_NAME }}
14
+ CONTAINER_APP_NAME: doc-assistant
15
+ IMAGE_NAME: doc-assistant
16
+
17
+ jobs:
18
+ deploy:
19
+ runs-on: ubuntu-latest
20
+ if: ${{ github.event.workflow_run.conclusion == 'success' || github.event_name == 'workflow_dispatch' }}
21
+
22
+ steps:
23
+ - uses: actions/checkout@v4
24
+
25
+ - name: Login to Azure
26
+ uses: azure/login@v2
27
+ with:
28
+ creds: ${{ secrets.AZURE_CREDENTIALS }}
29
+
30
+ - name: Login to ACR
31
+ run: az acr login --name ${{ env.ACR_NAME }}
32
+
33
+ - name: Build and push image to ACR
34
+ run: |
35
+ IMAGE_TAG=${{ env.ACR_NAME }}.azurecr.io/${{ env.IMAGE_NAME }}:${{ github.sha }}
36
+ docker build -f Dockerfile.compose -t $IMAGE_TAG .
37
+ docker push $IMAGE_TAG
38
+
39
+ - name: Deploy to Container Apps
40
+ run: |
41
+ az containerapp update \
42
+ --name ${{ env.CONTAINER_APP_NAME }} \
43
+ --resource-group ${{ env.AZURE_RESOURCE_GROUP }} \
44
+ --image ${{ env.ACR_NAME }}.azurecr.io/${{ env.IMAGE_NAME }}:${{ github.sha }}
45
+
46
+ - name: Wait for readiness
47
+ run: |
48
+ APP_URL=$(az containerapp show \
49
+ --name ${{ env.CONTAINER_APP_NAME }} \
50
+ --resource-group ${{ env.AZURE_RESOURCE_GROUP }} \
51
+ --query properties.configuration.ingress.fqdn -o tsv)
52
+ for i in $(seq 1 30); do
53
+ if curl -sf "https://${APP_URL}/health/ready" > /dev/null 2>&1; then
54
+ echo "Application is ready"
55
+ exit 0
56
+ fi
57
+ echo "Waiting for readiness... ($i/30)"
58
+ sleep 10
59
+ done
60
+ echo "Readiness check timed out"
61
+ exit 1
README.md CHANGED
@@ -25,8 +25,8 @@ A RAG application that lets users ask questions about documents in any language
25
  | Agent flows | Plan-and-Execute with six tools, ReAct sub-agent and conversation memory |
26
  | Evaluation | RAGAS metrics (faithfulness, answer relevancy, context precision) |
27
  | Traceability | Each answer includes source references with chunk ID and page number, plus structured logging |
28
- | Provider abstraction | Factory pattern that allows swapping between Ollama, OpenAI, Azure OpenAI, Anthropic and Google GenAI without touching business code |
29
- | Deployment | Docker Compose for local setup, Hugging Face Spaces for the public demo |
30
 
31
  ### How it works
32
 
@@ -74,10 +74,29 @@ Configuration lives in environment variables via `src/config.py`; there are no h
74
 
75
  ### Provider support
76
 
77
- LLM and embedding backends are configured through environment variables. Supported providers are Ollama, OpenAI, Azure OpenAI, Anthropic, Google GenAI and Groq. The default setup (Ollama and HuggingFace) runs entirely locally without any API keys.
78
 
79
  See `.env.example` for per-provider configuration.
80
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
81
  ### Try it live
82
 
83
  The demo lives at [xq-dokumentassistent.hf.space](https://xq-dokumentassistent.hf.space).
@@ -134,6 +153,56 @@ cp .env.example .env
134
  docker compose up --build
135
  ```
136
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
137
  #### Hugging Face Spaces
138
 
139
  A `Dockerfile` and supervisor configuration are included. The Space runs Qdrant, the API and the UI behind nginx on port 7860.
@@ -158,21 +227,28 @@ src/
158
  reranker.py # cross-encoder
159
  api/
160
  main.py
161
- routes.py # /query, /ingest, /health
162
  agent/
163
  intent_classifier.py
164
  router.py # pipeline mode (AGENT_MODE=pipeline)
165
  tools.py # six retrieval tools and ToolResultStore
166
  plan_and_execute.py # Plan-and-Execute agent (AGENT_MODE=react)
167
  memory.py # conversation memory for multi-turn
 
168
  evaluation/
169
  evaluator.py # RAGAS metrics
170
  ui/
171
  app.py # Streamlit frontend
172
  scripts/
173
  ingest.py
 
174
  e2e_test.py
175
  tests/
176
  docs/ # example PDFs or texts (KU AI public documents)
 
 
 
 
 
177
  ```
178
 
 
25
  | Agent flows | Plan-and-Execute with six tools, ReAct sub-agent and conversation memory |
26
  | Evaluation | RAGAS metrics (faithfulness, answer relevancy, context precision) |
27
  | Traceability | Each answer includes source references with chunk ID and page number, plus structured logging |
28
+ | Provider abstraction | Factory pattern that allows swapping between Ollama, OpenAI, Azure OpenAI, AWS Bedrock, Anthropic and Google GenAI without touching business code |
29
+ | Deployment | Docker Compose (local), Azure Container Apps, AWS ECS Fargate, Hugging Face Spaces (demo) |
30
 
31
  ### How it works
32
 
 
74
 
75
  ### Provider support
76
 
77
+ LLM and embedding backends are configured through environment variables. Supported providers are Ollama, OpenAI, Azure OpenAI, AWS Bedrock, Anthropic, Google GenAI and Groq. The default setup (Ollama and HuggingFace) runs entirely locally without any API keys.
78
 
79
  See `.env.example` for per-provider configuration.
80
 
81
+ ### Cloud deployment
82
+
83
+ The application is cloud-agnostic by design. Business code depends only on LangChain abstract interfaces; the concrete provider is selected at deploy time via environment variables.
84
+
85
+ | Layer | Azure | AWS | Local |
86
+ |---|---|---|---|
87
+ | LLM / Embeddings | Azure OpenAI | Bedrock (Claude, Titan) | Ollama + HuggingFace |
88
+ | Container registry | ACR | ECR | - |
89
+ | Runtime | Container Apps | ECS Fargate | docker-compose |
90
+ | CI/CD | GitHub Actions | GitHub Actions | - |
91
+
92
+ GitHub Actions workflows are included for both clouds:
93
+
94
+ - `ci.yml` runs lint, type check, and tests on every push and PR
95
+ - `deploy-azure.yml` builds, pushes to ACR, and deploys to Azure Container Apps
96
+ - `deploy-aws.yml` builds, pushes to ECR, and deploys to ECS Fargate
97
+
98
+ Health probes (`/health/live` for liveness, `/health/ready` for readiness) are used by both Kubernetes and container orchestrators to manage rolling deployments.
99
+
100
  ### Try it live
101
 
102
  The demo lives at [xq-dokumentassistent.hf.space](https://xq-dokumentassistent.hf.space).
 
153
  docker compose up --build
154
  ```
155
 
156
+ #### Azure Container Apps
157
+
158
+ ```bash
159
+ # Login and set variables
160
+ az login
161
+ ACR_NAME=<your-acr-name>
162
+ RG=<your-resource-group>
163
+ APP_NAME=doc-assistant
164
+
165
+ # Build and push
166
+ az acr login --name $ACR_NAME
167
+ docker build -f Dockerfile.compose -t $ACR_NAME.azurecr.io/doc-assistant:latest .
168
+ docker push $ACR_NAME.azurecr.io/doc-assistant:latest
169
+
170
+ # Deploy
171
+ az containerapp create \
172
+ --name $APP_NAME \
173
+ --resource-group $RG \
174
+ --image $ACR_NAME.azurecr.io/doc-assistant:latest \
175
+ --target-port 8000 \
176
+ --env-vars LLM_PROVIDER=azure_openai \
177
+ AZURE_OPENAI_API_KEY=secretref:aoai-key \
178
+ AZURE_OPENAI_ENDPOINT=https://<resource>.openai.azure.com/ \
179
+ AZURE_OPENAI_DEPLOYMENT=<deployment> \
180
+ AZURE_OPENAI_EMBEDDING_DEPLOYMENT=<embedding-deployment>
181
+ ```
182
+
183
+ CI/CD is automated via `.github/workflows/deploy-azure.yml`.
184
+
185
+ #### AWS ECS Fargate
186
+
187
+ ```bash
188
+ # Login and set variables
189
+ AWS_REGION=eu-west-1
190
+ ECR_REPO=<account-id>.dkr.ecr.$AWS_REGION.amazonaws.com/doc-assistant
191
+
192
+ # Build and push
193
+ aws ecr get-login-password --region $AWS_REGION | docker login --username AWS --password-stdin $ECR_REPO
194
+ docker build -f Dockerfile.compose -t $ECR_REPO:latest .
195
+ docker push $ECR_REPO:latest
196
+
197
+ # Deploy (update existing ECS service)
198
+ aws ecs update-service \
199
+ --cluster doc-assistant \
200
+ --service doc-assistant \
201
+ --force-new-deployment
202
+ ```
203
+
204
+ CI/CD is automated via `.github/workflows/deploy-aws.yml`. The task definition should set `LLM_PROVIDER=bedrock` and grant the task role Bedrock access.
205
+
206
  #### Hugging Face Spaces
207
 
208
  A `Dockerfile` and supervisor configuration are included. The Space runs Qdrant, the API and the UI behind nginx on port 7860.
 
227
  reranker.py # cross-encoder
228
  api/
229
  main.py
230
+ routes.py # /query, /ingest, /health/live, /health/ready
231
  agent/
232
  intent_classifier.py
233
  router.py # pipeline mode (AGENT_MODE=pipeline)
234
  tools.py # six retrieval tools and ToolResultStore
235
  plan_and_execute.py # Plan-and-Execute agent (AGENT_MODE=react)
236
  memory.py # conversation memory for multi-turn
237
+ session_store.py # SQLite-backed per-session memory persistence
238
  evaluation/
239
  evaluator.py # RAGAS metrics
240
  ui/
241
  app.py # Streamlit frontend
242
  scripts/
243
  ingest.py
244
+ evaluate.py # RAGAS evaluation CLI
245
  e2e_test.py
246
  tests/
247
  docs/ # example PDFs or texts (KU AI public documents)
248
+ .github/
249
+ workflows/
250
+ ci.yml # lint + test on push/PR
251
+ deploy-azure.yml # build, push ACR, deploy Container Apps
252
+ deploy-aws.yml # build, push ECR, deploy ECS Fargate
253
  ```
254
 
docker-compose.yml CHANGED
@@ -6,6 +6,11 @@ services:
6
  - "6334:6334"
7
  volumes:
8
  - qdrant_data:/qdrant/storage
 
 
 
 
 
9
 
10
  ollama:
11
  image: ollama/ollama:latest
@@ -45,9 +50,16 @@ services:
45
  - QDRANT_URL=http://qdrant:6333
46
  - OLLAMA_BASE_URL=http://ollama:11434
47
  depends_on:
48
- - qdrant
 
49
  volumes:
50
  - ./docs:/app/docs
 
 
 
 
 
 
51
 
52
  ui:
53
  build:
@@ -60,7 +72,8 @@ services:
60
  environment:
61
  - API_BASE_URL=http://api:8000
62
  depends_on:
63
- - api
 
64
  entrypoint: []
65
  command: ["streamlit", "run", "src/ui/app.py", "--server.port=8501", "--server.address=0.0.0.0"]
66
 
 
6
  - "6334:6334"
7
  volumes:
8
  - qdrant_data:/qdrant/storage
9
+ healthcheck:
10
+ test: ["CMD", "curl", "-f", "http://localhost:6333/healthz"]
11
+ interval: 10s
12
+ timeout: 3s
13
+ retries: 5
14
 
15
  ollama:
16
  image: ollama/ollama:latest
 
50
  - QDRANT_URL=http://qdrant:6333
51
  - OLLAMA_BASE_URL=http://ollama:11434
52
  depends_on:
53
+ qdrant:
54
+ condition: service_healthy
55
  volumes:
56
  - ./docs:/app/docs
57
+ healthcheck:
58
+ test: ["CMD", "curl", "-f", "http://localhost:8000/health/ready"]
59
+ interval: 30s
60
+ timeout: 5s
61
+ retries: 3
62
+ start_period: 60s
63
 
64
  ui:
65
  build:
 
72
  environment:
73
  - API_BASE_URL=http://api:8000
74
  depends_on:
75
+ api:
76
+ condition: service_healthy
77
  entrypoint: []
78
  command: ["streamlit", "run", "src/ui/app.py", "--server.port=8501", "--server.address=0.0.0.0"]
79
 
requirements.txt CHANGED
@@ -11,6 +11,7 @@ langchain-ollama==1.0.1
11
  langchain-huggingface==1.2.1
12
  langchain-anthropic==1.4.0
13
  langchain-google-genai==4.2.1
 
14
  langchain-text-splitters==1.1.1
15
  langchain-experimental==0.4.1
16
 
 
11
  langchain-huggingface==1.2.1
12
  langchain-anthropic==1.4.0
13
  langchain-google-genai==4.2.1
14
+ langchain-aws==0.2.17
15
  langchain-text-splitters==1.1.1
16
  langchain-experimental==0.4.1
17
 
src/api/routes.py CHANGED
@@ -172,13 +172,15 @@ class HealthResponse(BaseModel):
172
  embedding_model: str = ""
173
 
174
 
175
- @router.get("/health", response_model=HealthResponse)
176
- async def health_check() -> HealthResponse:
177
- """Health check endpoint.
178
 
179
- Returns:
180
- HealthResponse with service status and version.
181
- """
 
 
 
182
  llm_provider = ""
183
  llm_model = ""
184
  embedding_provider = ""
@@ -191,6 +193,7 @@ async def health_check() -> HealthResponse:
191
  "ollama": _settings.ollama_model,
192
  "openai": _settings.openai_model,
193
  "azure_openai": _settings.azure_openai_deployment,
 
194
  "groq": _settings.groq_model,
195
  "anthropic": _settings.anthropic_model,
196
  "google_genai": _settings.google_model,
@@ -206,6 +209,67 @@ async def health_check() -> HealthResponse:
206
  )
207
 
208
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
209
  @router.post("/query", response_model=QueryResponse)
210
  async def query_documents(request: QueryRequest) -> QueryResponse:
211
  """Query the document knowledge base.
 
172
  embedding_model: str = ""
173
 
174
 
175
+ class ReadinessResponse(BaseModel):
176
+ """Response body for the readiness probe."""
 
177
 
178
+ status: str
179
+ checks: dict[str, bool]
180
+
181
+
182
+ def _build_health_response() -> HealthResponse:
183
+ """Build the full health response with provider details."""
184
  llm_provider = ""
185
  llm_model = ""
186
  embedding_provider = ""
 
193
  "ollama": _settings.ollama_model,
194
  "openai": _settings.openai_model,
195
  "azure_openai": _settings.azure_openai_deployment,
196
+ "bedrock": _settings.aws_bedrock_model,
197
  "groq": _settings.groq_model,
198
  "anthropic": _settings.anthropic_model,
199
  "google_genai": _settings.google_model,
 
209
  )
210
 
211
 
212
+ @router.get("/health", response_model=HealthResponse)
213
+ async def health_check() -> HealthResponse:
214
+ """Health check endpoint (backwards compatible).
215
+
216
+ Returns:
217
+ HealthResponse with service status and version.
218
+ """
219
+ return _build_health_response()
220
+
221
+
222
+ @router.get("/health/live", response_model=HealthResponse)
223
+ async def liveness() -> HealthResponse:
224
+ """Liveness probe. Returns 200 if the process is running.
225
+
226
+ Kubernetes uses this to decide whether to restart the container.
227
+ Does not check external dependencies.
228
+
229
+ Returns:
230
+ HealthResponse with service status and version.
231
+ """
232
+ return _build_health_response()
233
+
234
+
235
+ @router.get("/health/ready", response_model=ReadinessResponse)
236
+ async def readiness() -> ReadinessResponse:
237
+ """Readiness probe. Returns 200 only when all dependencies are available.
238
+
239
+ Kubernetes uses this to decide whether to route traffic to the pod.
240
+ Checks: vector store reachable, BM25 index loaded.
241
+
242
+ Returns:
243
+ ReadinessResponse with per-dependency check results.
244
+
245
+ Raises:
246
+ HTTPException: 503 if any dependency check fails.
247
+ """
248
+ checks: dict[str, bool] = {}
249
+
250
+ # Check vector store connectivity
251
+ try:
252
+ if _vector_store is not None:
253
+ _vector_store.get_all_chunks()[:0] # lightweight probe
254
+ checks["vector_store"] = True
255
+ else:
256
+ checks["vector_store"] = False
257
+ except Exception:
258
+ logger.warning("Readiness check failed: vector store unreachable")
259
+ checks["vector_store"] = False
260
+
261
+ # Check BM25 index is loaded
262
+ checks["bm25_index"] = _bm25_search is not None and _bm25_search.is_indexed
263
+
264
+ # Check router is wired up
265
+ checks["router"] = _query_router is not None
266
+
267
+ all_ready = all(checks.values())
268
+ if not all_ready:
269
+ raise HTTPException(status_code=503, detail={"status": "unavailable", "checks": checks})
270
+ return ReadinessResponse(status="ready", checks=checks)
271
+
272
+
273
  @router.post("/query", response_model=QueryResponse)
274
  async def query_documents(request: QueryRequest) -> QueryResponse:
275
  """Query the document knowledge base.
src/config.py CHANGED
@@ -57,6 +57,11 @@ class Settings:
57
  azure_openai_deployment: str
58
  azure_openai_embedding_deployment: str
59
 
 
 
 
 
 
60
  # Groq
61
  groq_api_key: str
62
  groq_model: str
@@ -143,6 +148,11 @@ def load_settings() -> Settings:
143
  azure_openai_deployment=os.environ.get("AZURE_OPENAI_DEPLOYMENT", ""),
144
  azure_openai_embedding_deployment=os.environ.get("AZURE_OPENAI_EMBEDDING_DEPLOYMENT", ""),
145
 
 
 
 
 
 
146
  # Groq
147
  groq_api_key=os.environ.get("GROQ_API_KEY", ""),
148
  groq_model=os.environ.get("GROQ_MODEL", "qwen/qwen3-32b"),
 
57
  azure_openai_deployment: str
58
  azure_openai_embedding_deployment: str
59
 
60
+ # AWS Bedrock
61
+ aws_region: str
62
+ aws_bedrock_model: str
63
+ aws_bedrock_embedding_model: str
64
+
65
  # Groq
66
  groq_api_key: str
67
  groq_model: str
 
148
  azure_openai_deployment=os.environ.get("AZURE_OPENAI_DEPLOYMENT", ""),
149
  azure_openai_embedding_deployment=os.environ.get("AZURE_OPENAI_EMBEDDING_DEPLOYMENT", ""),
150
 
151
+ # AWS Bedrock
152
+ aws_region=os.environ.get("AWS_REGION", "eu-west-1"),
153
+ aws_bedrock_model=os.environ.get("AWS_BEDROCK_MODEL", "anthropic.claude-sonnet-4-20250514-v1:0"),
154
+ aws_bedrock_embedding_model=os.environ.get("AWS_BEDROCK_EMBEDDING_MODEL", "amazon.titan-embed-text-v2:0"),
155
+
156
  # Groq
157
  groq_api_key=os.environ.get("GROQ_API_KEY", ""),
158
  groq_model=os.environ.get("GROQ_MODEL", "qwen/qwen3-32b"),
src/provider.py CHANGED
@@ -14,8 +14,8 @@ from src.config import Settings
14
 
15
  logger = logging.getLogger(__name__)
16
 
17
- _SUPPORTED_LLM_PROVIDERS = ["ollama", "azure_openai", "openai", "groq", "anthropic", "google_genai"]
18
- _SUPPORTED_EMBEDDING_PROVIDERS = ["local", "azure_openai", "openai", "google_genai"]
19
 
20
 
21
  def create_llm(settings: Settings) -> BaseChatModel:
@@ -94,6 +94,15 @@ def create_llm(settings: Settings) -> BaseChatModel:
94
  temperature=0.0,
95
  )
96
 
 
 
 
 
 
 
 
 
 
97
  case _:
98
  raise ValueError(
99
  f"Unknown LLM provider: '{provider}'. "
@@ -107,6 +116,7 @@ _EVALUATOR_MODEL_FIELD: dict[str, str] = {
107
  "anthropic": "anthropic_model",
108
  "google_genai": "google_model",
109
  "azure_openai": "azure_openai_deployment",
 
110
  "ollama": "ollama_model",
111
  }
112
 
@@ -200,6 +210,14 @@ def create_embeddings(settings: Settings) -> Embeddings:
200
  google_api_key=settings.google_api_key,
201
  )
202
 
 
 
 
 
 
 
 
 
203
  case _:
204
  raise ValueError(
205
  f"Unknown embedding provider: '{provider}'. "
 
14
 
15
  logger = logging.getLogger(__name__)
16
 
17
+ _SUPPORTED_LLM_PROVIDERS = ["ollama", "azure_openai", "openai", "groq", "anthropic", "google_genai", "bedrock"]
18
+ _SUPPORTED_EMBEDDING_PROVIDERS = ["local", "azure_openai", "openai", "google_genai", "bedrock"]
19
 
20
 
21
  def create_llm(settings: Settings) -> BaseChatModel:
 
94
  temperature=0.0,
95
  )
96
 
97
+ case "bedrock":
98
+ from langchain_aws import ChatBedrockConverse
99
+
100
+ return ChatBedrockConverse(
101
+ model=settings.aws_bedrock_model,
102
+ region_name=settings.aws_region,
103
+ temperature=0.0,
104
+ )
105
+
106
  case _:
107
  raise ValueError(
108
  f"Unknown LLM provider: '{provider}'. "
 
116
  "anthropic": "anthropic_model",
117
  "google_genai": "google_model",
118
  "azure_openai": "azure_openai_deployment",
119
+ "bedrock": "aws_bedrock_model",
120
  "ollama": "ollama_model",
121
  }
122
 
 
210
  google_api_key=settings.google_api_key,
211
  )
212
 
213
+ case "bedrock":
214
+ from langchain_aws import BedrockEmbeddings
215
+
216
+ return BedrockEmbeddings(
217
+ model_id=settings.aws_bedrock_embedding_model,
218
+ region_name=settings.aws_region,
219
+ )
220
+
221
  case _:
222
  raise ValueError(
223
  f"Unknown embedding provider: '{provider}'. "
src/retrieval/bm25_search.py CHANGED
@@ -17,6 +17,11 @@ class BM25Search:
17
  self._chunks: list[DocumentChunk] = []
18
  self._index: BM25Okapi | None = None
19
 
 
 
 
 
 
20
  def index(self, chunks: list[DocumentChunk]) -> None:
21
  """Build the BM25 index from document chunks.
22
 
 
17
  self._chunks: list[DocumentChunk] = []
18
  self._index: BM25Okapi | None = None
19
 
20
+ @property
21
+ def is_indexed(self) -> bool:
22
+ """Return True if the BM25 index has been built."""
23
+ return self._index is not None and len(self._chunks) > 0
24
+
25
  def index(self, chunks: list[DocumentChunk]) -> None:
26
  """Build the BM25 index from document chunks.
27
 
tests/test_api.py CHANGED
@@ -5,7 +5,7 @@ from unittest.mock import MagicMock, patch
5
  import pytest
6
  from fastapi.testclient import TestClient
7
 
8
- from src.api.routes import router, set_dependencies, QueryResponse, HealthResponse
9
  from src.models import DocumentChunk, GenerationResponse, IntentType, QueryResult
10
 
11
 
@@ -65,6 +65,46 @@ class TestHealthCheck:
65
  assert body["version"] == "0.1.0"
66
 
67
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
68
  class TestQueryEndpoint:
69
  """Tests for the /query endpoint."""
70
 
 
5
  import pytest
6
  from fastapi.testclient import TestClient
7
 
8
+ from src.api.routes import router, set_dependencies, QueryResponse, HealthResponse, ReadinessResponse
9
  from src.models import DocumentChunk, GenerationResponse, IntentType, QueryResult
10
 
11
 
 
65
  assert body["version"] == "0.1.0"
66
 
67
 
68
+ class TestLivenessProbe:
69
+ """Tests for the /health/live endpoint."""
70
+
71
+ def test_liveness_returns_ok(self, client: TestClient) -> None:
72
+ response = client.get("/health/live")
73
+
74
+ assert response.status_code == 200
75
+ body = response.json()
76
+ assert body["status"] == "ok"
77
+
78
+
79
+ class TestReadinessProbe:
80
+ """Tests for the /health/ready endpoint."""
81
+
82
+ def test_readiness_returns_ready_when_all_deps_available(
83
+ self, client: TestClient, mock_deps: dict[str, MagicMock]
84
+ ) -> None:
85
+ mock_deps["vector_store"].get_all_chunks.return_value = []
86
+ mock_deps["bm25_search"].is_indexed = True
87
+
88
+ response = client.get("/health/ready")
89
+
90
+ assert response.status_code == 200
91
+ body = response.json()
92
+ assert body["status"] == "ready"
93
+ assert body["checks"]["vector_store"] is True
94
+ assert body["checks"]["bm25_index"] is True
95
+ assert body["checks"]["router"] is True
96
+
97
+ def test_readiness_returns_503_when_bm25_not_indexed(
98
+ self, client: TestClient, mock_deps: dict[str, MagicMock]
99
+ ) -> None:
100
+ mock_deps["vector_store"].get_all_chunks.return_value = []
101
+ mock_deps["bm25_search"].is_indexed = False
102
+
103
+ response = client.get("/health/ready")
104
+
105
+ assert response.status_code == 503
106
+
107
+
108
  class TestQueryEndpoint:
109
  """Tests for the /query endpoint."""
110
 
tests/test_bm25.py CHANGED
@@ -14,6 +14,15 @@ def _make_chunk(chunk_id: str, text: str) -> DocumentChunk:
14
  class TestBM25Index:
15
  """Tests for index construction."""
16
 
 
 
 
 
 
 
 
 
 
17
  def test_index_stores_chunks(self) -> None:
18
  bm25 = BM25Search()
19
  chunks = [_make_chunk("1", "hello world"), _make_chunk("2", "foo bar")]
 
14
  class TestBM25Index:
15
  """Tests for index construction."""
16
 
17
+ def test_is_indexed_false_before_indexing(self) -> None:
18
+ bm25 = BM25Search()
19
+ assert bm25.is_indexed is False
20
+
21
+ def test_is_indexed_true_after_indexing(self) -> None:
22
+ bm25 = BM25Search()
23
+ bm25.index([_make_chunk("1", "hello world")])
24
+ assert bm25.is_indexed is True
25
+
26
  def test_index_stores_chunks(self) -> None:
27
  bm25 = BM25Search()
28
  chunks = [_make_chunk("1", "hello world"), _make_chunk("2", "foo bar")]