Postgres persistance + job queue + api with sse #9
No files matched your search
@@ -15,8 +15,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
cmake \
|
||||
pkg-config \
|
||||
clang \
|
||||
valkey-server \
|
||||
valkey-tools \
|
||||
postgresql \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
RUN groupadd --gid ${USER_GID:-1000} $USERNAME \
|
||||
|
||||
@@ -22,3 +22,5 @@ METRICS_BIND_ADDR=
|
||||
# Sandboxed tool execution
|
||||
# DOCKER_HOST=
|
||||
SANDBOX_MAX_ITERATIONS=8
|
||||
|
||||
DATABASE_URL=
|
||||
@@ -2,6 +2,14 @@ when:
|
||||
event:
|
||||
- push
|
||||
|
||||
services:
|
||||
- name: postgres
|
||||
image: postgres:18
|
||||
environment:
|
||||
POSTGRES_USER: postgres
|
||||
POSTGRES_PASSWORD: postgres
|
||||
POSTGRES_DB: steep
|
||||
|
||||
steps:
|
||||
- name: fmt
|
||||
image: rust:1.99
|
||||
@@ -15,10 +23,17 @@ steps:
|
||||
- rustup component add clippy
|
||||
- cargo clippy --workspace --all-targets --all-features -- -D warnings
|
||||
|
||||
- name: wait-for-postgres
|
||||
image: postgres:18
|
||||
commands:
|
||||
- until pg_isready -h postgres -U postgres; do sleep 1; done
|
||||
|
|
||||
|
||||
- name: test
|
||||
image: rust:1.99
|
||||
environment:
|
||||
DATABASE_URL: postgresql://postgres:postgres@postgres:5432/steep
|
||||
commands:
|
||||
- cargo test --workspace --all-targets
|
||||
- cargo test --workspace --all-targets -- --include-ignored
|
||||
|
||||
- name: container-build
|
||||
image: quay.io/buildah/stable
|
||||
|
||||
@@ -28,9 +28,9 @@ hex = "0.4"
|
||||
bytes = "1.1"
|
||||
metrics = "0.24"
|
||||
metrics-exporter-prometheus = { version = "0.18", default-features = false, features = ["http-listener"] }
|
||||
apalis = { version = "0.7", features = ["catch-panic", "limit"] }
|
||||
apalis-redis = "0.7"
|
||||
redis = { version = "0.32", features = ["tokio-comp", "connection-manager"] }
|
||||
toasty = { version = "0.11", features = ["postgresql", "serde", "jiff"] }
|
||||
toasty-cli = "0.11"
|
||||
jiff = "0.2"
|
||||
|
||||
[profile.release]
|
||||
debug = 1
|
||||
@@ -10,10 +10,7 @@ RUN cargo build --release --package steep-server
|
||||
|
||||
FROM debian:trixie-slim
|
||||
|
||||
# git clones the pull request. ca-certificates is what every HTTPS call needs
|
||||
# (Gitea, OpenRouter, git). The shared libraries are the non-base ones the binary
|
||||
# links against, as reported by `ldd target/release/steep-server`; libc, libm
|
||||
# and libgcc_s come from the base image.
|
||||
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
ca-certificates \
|
||||
git \
|
||||
@@ -26,6 +23,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
WORKDIR /app
|
||||
COPY --from=builder /app/target/release/steep-server ./steep-server
|
||||
|
||||
# Exec form, so the binary is PID 1 and receives the SIGTERM it handles to shut
|
||||
# down gracefully.
|
||||
CMD ["./steep-server"]
|
||||
COPY crates/steep-server/toasty ./toasty
|
||||
ENV MIGRATIONS_DIR=/app/toasty
|
||||
|
||||
CMD ["sh", "-c", "./steep-server migration apply && exec ./steep-server"]
|
||||
@@ -38,7 +38,7 @@ Steep reads its configuration from environment variables (a `.env` file is suppo
|
||||
| `GITEA_URL` | Base URL of your Gitea instance |
|
||||
| `GITEA_TOKEN` | Gitea API token |
|
||||
| `GITEA_TIMEOUT` | Gitea API request timeout in seconds |
|
||||
| `REDIS_URL` | Redis connection URL used for the durable review queue (e.g. `redis://127.0.0.1:6379`) |
|
||||
| `DATABASE_URL` | PostgreSQL connection URL used for the durable review queue (e.g. `postgresql://user:pass@localhost:5432/steep`) |
|
||||
| `METRICS_BIND_ADDR` | *(optional)* Bind address for the Prometheus metrics endpoint (e.g. `0.0.0.0:9100`). If unset, the metrics exporter is disabled. |
|
||||
| `SENTRY_DSN` | *(optional)* Sentry DSN for error tracking |
|
||||
| `RUST_LOG` | *(optional)* Log level, defaults to `info` |
|
||||
@@ -102,6 +102,26 @@ hooks have run, the container is disconnected from the network for the rest of
|
||||
the review. Every container command is bounded by a timeout, and the container,
|
||||
network and image are removed when the review ends (including on failure).
|
||||
|
||||
## Migrations
|
||||
|
||||
The database schema is versioned with [Toasty](https://github.com/tokio-rs/toasty) migrations, stored in `crates/steep-server/toasty/` (commit them).
|
||||
|
||||
After changing a model in `crates/steep-server/src/schemas/`, generate a migration:
|
||||
|
||||
```sh
|
||||
cargo run -p steep-server -- migration generate --name describe_change
|
||||
```
|
||||
|
||||
Apply the pending migrations **before starting the service** — the server does not migrate at boot:
|
||||
|
||||
```sh
|
||||
cargo run -p steep-server -- migration apply
|
||||
```
|
||||
|
||||
The `Containerfile` does this at startup: its entrypoint runs `steep-server migration apply` before exec'ing the server. The CLI reads the migration files from `MIGRATIONS_DIR` (defaults to the crate's `toasty/` directory; the image sets it to `/app/toasty`).
|
||||
|
||||
Other subcommands: `migration snapshot`, `migration drop`, `migration reset`. They read `DATABASE_URL`.
|
||||
|
||||
## Development
|
||||
|
||||
The easiest way to get started is with the provided [Dev Container](https://containers.dev/) (VS Code or Zed with the dev container extension).
|
||||
@@ -129,7 +149,7 @@ Set `METRICS_BIND_ADDR` (e.g. `0.0.0.0:9100`) to enable it. The metrics are then
|
||||
|---|---|---|
|
||||
| `steep_webhooks_received_total` | counter | Total webhooks received (label: `event_type`) |
|
||||
| `steep_webhooks_duplicate_total` | counter | Webhooks rejected as duplicates (label: `event_type`) |
|
||||
| `steep_webhooks_enqueue_failed_total` | counter | Webhooks dropped because enqueueing to Redis failed (label: `event_type`) |
|
||||
| `steep_webhooks_enqueue_failed_total` | counter | Webhooks dropped because enqueueing to PostgreSQL failed (label: `event_type`) |
|
||||
| `steep_bot_tasks_active` | gauge | Bot tasks currently in progress |
|
||||
| `steep_bot_tasks_skipped_total` | counter | Reviews skipped because they were already done (label: `event_type`) |
|
||||
| `steep_bot_tasks_completed_total` | counter | Bot tasks completed successfully (label: `event_type`) |
|
||||
|
||||
@@ -26,6 +26,6 @@ metrics = { workspace = true }
|
||||
metrics-exporter-prometheus = { workspace = true }
|
||||
devcontainer-rs = { path = "../devcontainer-rs" }
|
||||
tempfile = "3"
|
||||
apalis = { workspace = true }
|
||||
apalis-redis = { workspace = true }
|
||||
redis = { workspace = true }
|
||||
toasty = { workspace = true }
|
||||
toasty-cli = { workspace = true }
|
||||
jiff = { workspace = true }
|
||||
@@ -14,10 +14,7 @@ use tracing::{info, instrument};
|
||||
|
||||
use tokio_util::sync::CancellationToken;
|
||||
|
||||
use apalis::prelude::Storage;
|
||||
|
||||
use crate::consts::{GITEA_EVENT_TYPE_HEADER_NAME, GITEA_SIG_HEADER_NAME, MAX_WEBHOOK_BODY_SIZE};
|
||||
use crate::dedup::{Dedup, WEBHOOK_TTL};
|
||||
use crate::errors::AppError;
|
||||
use crate::gitea::WebhookType;
|
||||
use crate::metrics;
|
||||
@@ -64,20 +61,19 @@ async fn webhook(
|
||||
metrics::webhook_received(event_type);
|
||||
|
||||
let event_id = wb.event_id();
|
||||
let webhook_key = Dedup::webhook_key(event_id);
|
||||
if !app_state
|
||||
.dedup
|
||||
.claim(&webhook_key, WEBHOOK_TTL)
|
||||
.await
|
||||
.map_err(anyhow::Error::from)?
|
||||
{
|
||||
metrics::webhook_duplicate(event_type);
|
||||
return Err(AppError::AlreadyProcessedErr);
|
||||
match app_state.dedup.is_known(event_id).await {
|
||||
Ok(true) => {
|
||||
metrics::webhook_duplicate(event_type);
|
||||
return Err(AppError::AlreadyProcessedErr);
|
||||
}
|
||||
Ok(false) => {}
|
||||
Err(err) => {
|
||||
metrics::webhook_enqueue_failed(event_type);
|
||||
return Err(AppError::EnqueueErr(anyhow::Error::from(err)));
|
||||
}
|
||||
}
|
||||
|
||||
let mut storage = app_state.storage.clone();
|
||||
if let Err(err) = storage.push(wb).await {
|
||||
app_state.dedup.release(&webhook_key).await.ok();
|
||||
if let Err(err) = app_state.queue.push(wb).await {
|
||||
metrics::webhook_enqueue_failed(event_type);
|
||||
return Err(AppError::EnqueueErr(anyhow::Error::from(err)));
|
||||
}
|
||||
|
||||
@@ -1,15 +1,21 @@
|
||||
use std::sync::Arc;
|
||||
|
||||
use futures_util::StreamExt;
|
||||
use serde::{Deserialize, Deserializer};
|
||||
use tokio::sync::Semaphore;
|
||||
use tokio_util::sync::CancellationToken;
|
||||
use tracing::{error, info, instrument};
|
||||
|
||||
use crate::{
|
||||
dedup::{Dedup, REVIEW_TTL},
|
||||
bot_actions::review::ReviewOutcome,
|
||||
dedup::{Dedup, ReviewRecord},
|
||||
gitea::{GiteaAPI, WebhookType},
|
||||
job_queue::JobQueue,
|
||||
metrics,
|
||||
open_router::OpenRouterClient,
|
||||
sandbox::SandboxConfig,
|
||||
schemas::Job,
|
||||
};
|
||||
use apalis::prelude::{Data, Error, Monitor, WorkerBuilder, WorkerBuilderExt, WorkerFactoryFn};
|
||||
use apalis_redis::RedisStorage;
|
||||
use serde::{Deserialize, Deserializer};
|
||||
use tokio_util::sync::CancellationToken;
|
||||
use tracing::{error, info, instrument};
|
||||
|
||||
#[derive(Deserialize, Debug)]
|
||||
pub struct ReviewResult {
|
||||
@@ -127,6 +133,7 @@ pub struct Bot {
|
||||
open_router_model: String,
|
||||
sandbox: SandboxConfig,
|
||||
dedup: Dedup,
|
||||
queue: JobQueue,
|
||||
}
|
||||
|
||||
impl Bot {
|
||||
@@ -137,6 +144,7 @@ impl Bot {
|
||||
open_router_model: String,
|
||||
sandbox: SandboxConfig,
|
||||
dedup: Dedup,
|
||||
queue: JobQueue,
|
||||
) -> Self {
|
||||
Self {
|
||||
gitea_api,
|
||||
@@ -145,40 +153,119 @@ impl Bot {
|
||||
open_router_model,
|
||||
sandbox,
|
||||
dedup,
|
||||
queue,
|
||||
}
|
||||
}
|
||||
|
||||
/// Consomme la file Redis jusqu'à l'annulation de `shutdown`.
|
||||
/// Consomme la file jusqu'à l'annulation de `shutdown`.
|
||||
///
|
||||
/// Les jobs en cours sont laissés se terminer, et ceux qui n'ont pas encore
|
||||
/// été acquittés restent dans Redis : ils seront repris au prochain démarrage.
|
||||
pub async fn start(
|
||||
&self,
|
||||
storage: RedisStorage<WebhookType>,
|
||||
shutdown: CancellationToken,
|
||||
) -> anyhow::Result<()> {
|
||||
/// Un sémaphore borne la concurrence ; les jobs en cours sont laissés se
|
||||
/// terminer. Les jobs non terminés restent en base et seront repris au
|
||||
/// prochain démarrage.
|
||||
pub async fn start(&self, shutdown: CancellationToken) -> anyhow::Result<()> {
|
||||
info!("Bot started");
|
||||
|
||||
let worker = WorkerBuilder::new("steep-review")
|
||||
.concurrency(self.max_concurrent)
|
||||
.data(self.clone())
|
||||
.backend(storage)
|
||||
.build_fn(handle_review);
|
||||
let semaphore = Arc::new(Semaphore::new(self.max_concurrent));
|
||||
let mut tasks = tokio::task::JoinSet::new();
|
||||
let mut pending = std::pin::pin!(self.queue.pending());
|
||||
|
||||
Monitor::new()
|
||||
.register(worker)
|
||||
.run_with_signal(async move {
|
||||
shutdown.cancelled().await;
|
||||
Ok(())
|
||||
})
|
||||
.await?;
|
||||
loop {
|
||||
let job = tokio::select! {
|
||||
biased;
|
||||
_ = shutdown.cancelled() => break,
|
||||
job = pending.next() => match job {
|
||||
Some(job) => job,
|
||||
None => break,
|
||||
},
|
||||
};
|
||||
|
||||
while let Some(res) = tasks.try_join_next() {
|
||||
if let Err(err) = res {
|
||||
error!("Task panicked: {err}");
|
||||
}
|
||||
}
|
||||
|
||||
let permit = semaphore.clone().acquire_owned().await?;
|
||||
|
qpismont marked this conversation as resolved
Outdated
Herald
commented
[bug] **[bug]** `BOT_MAX_CONCURRENT=0` est accepté par le parsing de configuration, mais `acquire_owned()` attendra alors indéfiniment un permis. Comme cette attente n'est pas incluse dans un `select!` avec l'annulation, l'arrêt peut rester bloqué jusqu'au délai forcé de 60 secondes. Il faut rejeter zéro au chargement de configuration ou rendre l'attente annulable.
Herald
commented
[bug] L’attente d’un permis n’écoute pas **[bug]** L’attente d’un permis n’écoute pas `shutdown`. Si tous les slots sont occupés au moment de l’arrêt, le bot attend qu’un slot se libère puis lance quand même le job en attente, et peut continuer ainsi à vider la file. Cela contredit le comportement annoncé où seuls les jobs déjà en cours terminent ; rendez aussi cette attente interruptible.
|
||||
|
||||
if let Err(err) = self.queue.mark_processing(job.id).await {
|
||||
|
qpismont marked this conversation as resolved
Outdated
Herald
commented
[bug] En cas d'échec de **[bug]** En cas d'échec de `mark_processing`, le code journalise l'erreur mais lance quand même le traitement. Le job reste alors `Pending` et peut être repris par le poller pendant que la review est déjà en cours, ce qui crée des traitements et commentaires en double. Il faut abandonner ce job (ou le remettre explicitement dans un état contrôlé) lorsque la réservation échoue.
Herald
commented
[bug] Si la mise à jour du statut échoue, le job est quand même traité. Il reste alors **[bug]** Si la mise à jour du statut échoue, le job est quand même traité. Il reste alors `Pending` et sera récupéré au prochain sondage, ce qui peut lancer plusieurs reviews concurrentes pour le même événement. Ne lancez pas le traitement si cette transition échoue, ou garantissez une prise en charge atomique.
|
||||
error!(%err, "failed to mark job as processing");
|
||||
}
|
||||
|
||||
let bot = self.clone();
|
||||
metrics::increment_task_active();
|
||||
|
qpismont marked this conversation as resolved
Herald
commented
[bug] En cas d’échec de **[bug]** En cas d’échec de `mark_processing`, le job est tout de même traité. Il reste alors Pending en base et pourra être repris après redémarrage — ou par une autre instance — pendant que cette exécution peut déjà publier la review. N’exécutez pas le job si sa transition d’état échoue.
|
||||
|
||||
tasks.spawn(async move {
|
||||
bot.process(job).await;
|
||||
drop(permit);
|
||||
metrics::decrement_task_active();
|
||||
});
|
||||
}
|
||||
|
||||
tasks.join_all().await;
|
||||
|
||||
info!("Bot shutting down complete");
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn process(&self, job: Job) {
|
||||
let event_type = job.payload.event_type_str();
|
||||
|
||||
match self.dedup.is_reviewed(job.event_id).await {
|
||||
Ok(true) => {
|
||||
metrics::task_skipped(event_type);
|
||||
info!("Review already done, skipping");
|
||||
|
||||
if let Err(err) = self.queue.remove(job.id).await {
|
||||
error!(%err, "failed to remove job");
|
||||
}
|
||||
return;
|
||||
}
|
||||
Ok(false) => {}
|
||||
Err(err) => error!(%err, "Dedup lookup failed, reviewing anyway"),
|
||||
}
|
||||
|
||||
let payload = job.payload.0.clone();
|
||||
let started = std::time::Instant::now();
|
||||
let result = self.exec(payload).await;
|
||||
let duration_ms = started.elapsed().as_millis() as u64;
|
||||
|
||||
let (ok, outcome) = match result {
|
||||
Ok(outcome) => {
|
||||
metrics::task_completed(event_type);
|
||||
info!("Task completed");
|
||||
(true, outcome)
|
||||
}
|
||||
Err(err) => {
|
||||
metrics::task_failed(event_type);
|
||||
error!(%err, "Task error");
|
||||
sentry_anyhow::capture_anyhow(&err);
|
||||
(false, ReviewOutcome::empty())
|
||||
}
|
||||
};
|
||||
|
||||
let record = ReviewRecord {
|
||||
event_id: job.event_id,
|
||||
repo: job.repo.clone(),
|
||||
pr: job.pr,
|
||||
ok,
|
||||
model: self.open_router_model.clone(),
|
||||
markdown: outcome.markdown,
|
||||
cost: outcome.cost,
|
||||
comments: outcome.comments as u64,
|
||||
duration_ms,
|
||||
};
|
||||
|
||||
if let Err(err) = self.dedup.mark_reviewed(record).await {
|
||||
|
qpismont marked this conversation as resolved
Herald
commented
[bug] Même quand **[bug]** Même quand `exec` échoue, le code enregistre une `Review` avec `ok = false`, puis supprime le job. Un webhook de retry sera ensuite considéré comme déjà traité par `is_reviewed` et ignoré : une panne temporaire de Gitea, du sandbox ou d'OpenRouter devient donc un échec définitif sans possibilité de retry. Les jobs échoués devraient rester en file (avec backoff/retry) ou être distingués des reviews effectivement publiées.
|
||||
error!(%err, "failed to record review");
|
||||
}
|
||||
|
||||
if let Err(err) = self.queue.remove(job.id).await {
|
||||
|
qpismont marked this conversation as resolved
Herald
commented
[bug] Si **[bug]** Si `mark_reviewed` échoue, l’erreur est seulement journalisée, puis le job est supprimé. Une panne temporaire de la base peut donc faire perdre définitivement l’événement sans enregistrer son résultat. Ne supprimez le job qu’après l’enregistrement réussi, ou conservez-le pour reprise.
|
||||
error!(%err, "failed to remove job");
|
||||
}
|
||||
}
|
||||
|
||||
#[instrument(skip(self, webhook), fields(repo, pr))]
|
||||
pub async fn exec(&self, webhook: WebhookType) -> anyhow::Result<()> {
|
||||
pub async fn exec(&self, webhook: WebhookType) -> anyhow::Result<ReviewOutcome> {
|
||||
match &webhook {
|
||||
WebhookType::Review(p) => {
|
||||
tracing::Span::current().record("repo", &p.repository.full_name);
|
||||
@@ -203,40 +290,3 @@ impl Bot {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async fn handle_review(webhook: WebhookType, bot: Data<Bot>) -> Result<(), Error> {
|
||||
let event_type = webhook.event_type_str();
|
||||
let review_key = Dedup::review_key(webhook.event_id());
|
||||
|
||||
match bot.dedup.is_marked(&review_key).await {
|
||||
Ok(true) => {
|
||||
metrics::task_skipped(event_type);
|
||||
info!("Review already done, skipping");
|
||||
return Ok(());
|
||||
}
|
||||
Ok(false) => {}
|
||||
Err(err) => error!(%err, "Dedup lookup failed, reviewing anyway"),
|
||||
}
|
||||
|
||||
metrics::increment_task_active();
|
||||
let result = bot.exec(webhook).await;
|
||||
metrics::decrement_task_active();
|
||||
|
||||
match result {
|
||||
Ok(()) => {
|
||||
metrics::task_completed(event_type);
|
||||
info!("Task completed");
|
||||
|
||||
if let Err(err) = bot.dedup.mark(&review_key, REVIEW_TTL).await {
|
||||
error!(%err, "Failed to mark review as done");
|
||||
}
|
||||
}
|
||||
Err(err) => {
|
||||
metrics::task_failed(event_type);
|
||||
error!(%err, "Task error");
|
||||
sentry_anyhow::capture_anyhow(&err);
|
||||
}
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
@@ -12,6 +12,22 @@ use crate::{
|
||||
text::excerpt,
|
||||
};
|
||||
|
||||
pub struct ReviewOutcome {
|
||||
pub markdown: String,
|
||||
pub cost: Option<f64>,
|
||||
pub comments: usize,
|
||||
}
|
||||
|
||||
impl ReviewOutcome {
|
||||
pub fn empty() -> Self {
|
||||
Self {
|
||||
markdown: String::new(),
|
||||
cost: None,
|
||||
comments: 0,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[instrument(skip(gitea_api, open_router_client, sandbox_config, tools, review_payload))]
|
||||
pub async fn exec_review(
|
||||
gitea_api: &GiteaAPI,
|
||||
@@ -20,7 +36,7 @@ pub async fn exec_review(
|
||||
sandbox_config: &SandboxConfig,
|
||||
tools: Vec<Tool>,
|
||||
review_payload: ReviewPayload,
|
||||
) -> anyhow::Result<()> {
|
||||
) -> anyhow::Result<ReviewOutcome> {
|
||||
tracing::info!(
|
||||
repo = %review_payload.repository.full_name,
|
||||
pr = review_payload.pull_request.number,
|
||||
@@ -36,7 +52,7 @@ pub async fn exec_review(
|
||||
)
|
||||
.await?;
|
||||
|
||||
let bot_result: Result<ReviewResult, anyhow::Error> = async {
|
||||
let bot_result: Result<(ReviewResult, String), anyhow::Error> = async {
|
||||
let full_name = &review_payload.repository.full_name;
|
||||
let index = review_payload.pull_request.number;
|
||||
|
||||
@@ -91,15 +107,21 @@ pub async fn exec_review(
|
||||
)
|
||||
.await?;
|
||||
|
||||
Ok(review_result)
|
||||
Ok((review_result, final_review_markdown))
|
||||
}
|
||||
.await;
|
||||
|
||||
match bot_result {
|
||||
Ok(_) => {
|
||||
Ok((review_result, markdown)) => {
|
||||
gitea_api
|
||||
.delete_comment(&review_payload.repository.full_name, new_comment.id)
|
||||
.await
|
||||
.await?;
|
||||
|
||||
Ok(ReviewOutcome {
|
||||
markdown,
|
||||
cost: review_result.cost,
|
||||
comments: review_result.reviews.len(),
|
||||
})
|
||||
}
|
||||
Err(e) => {
|
||||
gitea_api
|
||||
@@ -108,17 +130,13 @@ pub async fn exec_review(
|
||||
&review_payload.repository.full_name,
|
||||
new_comment.id,
|
||||
)
|
||||
.await
|
||||
.await?;
|
||||
|
||||
Err(e)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Runs the review inside a sandbox container, letting the model explore the
|
||||
/// repository with tools before answering.
|
||||
///
|
||||
/// The answer of the model is parsed by [`ReviewResult::from_str`], which the
|
||||
/// agent loop enforces: an answer that is not a review is sent back to the model
|
||||
/// for correction.
|
||||
async fn run_sandboxed_review(
|
||||
gitea_api: &GiteaAPI,
|
||||
open_router_client: &OpenRouterClient,
|
||||
|
||||
@@ -0,0 +1,43 @@
|
||||
use toasty::Db;
|
||||
|
||||
use crate::schemas::{Job, JobStatus};
|
||||
|
||||
pub async fn open(url: &str) -> toasty::Result<Db> {
|
||||
Db::builder()
|
||||
.models(toasty::models!(crate::*))
|
||||
.connect(url)
|
||||
.await
|
||||
}
|
||||
|
||||
pub async fn connect(url: &str) -> toasty::Result<Db> {
|
||||
let mut db = open(url).await?;
|
||||
|
||||
Job::update_by_status(JobStatus::Processing)
|
||||
|
qpismont marked this conversation as resolved
Herald
commented
[bug] Au démarrage, tous les jobs **[bug]** Au démarrage, tous les jobs `Processing` sont remis à `Pending` sans notion de propriétaire ou de lease. Si deux instances tournent simultanément, le démarrage de l'une peut réenfiler les jobs activement traités par l'autre, en plus de la course déjà présente dans le poller. La récupération après crash doit être limitée aux jobs dont le lease a expiré, et le traitement doit être atomiquement réservé.
Herald
commented
[bug] Au démarrage, chaque instance remet à **[bug]** Au démarrage, chaque instance remet à `Pending` tous les jobs `Processing`, y compris ceux qu’une autre instance est encore en train de traiter. Un redémarrage ou déploiement pendant une review peut donc la faire exécuter en double. La récupération doit distinguer les jobs réellement abandonnés de ceux détenus par des workers actifs.
|
||||
.status(JobStatus::Pending)
|
||||
.exec(&mut db)
|
||||
.await?;
|
||||
|
||||
Ok(db)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
pub(crate) async fn test_connect() -> Db {
|
||||
static DB: tokio::sync::OnceCell<Db> = tokio::sync::OnceCell::const_new();
|
||||
|
||||
DB.get_or_init(|| async {
|
||||
let url = std::env::var("DATABASE_URL").expect("DATABASE_URL must be set");
|
||||
|
||||
let migrations = std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("toasty");
|
||||
let config = toasty_cli::Config::new()
|
||||
.migration(toasty_cli::MigrationConfig::new().path(migrations));
|
||||
|
||||
toasty_cli::ToastyCli::with_config(open(&url).await.expect("open pool"), config)
|
||||
.parse_from(["toasty", "migration", "apply"])
|
||||
.await
|
||||
.expect("apply migrations");
|
||||
|
||||
connect(&url).await.expect("connect to postgres")
|
||||
})
|
||||
.await
|
||||
.clone()
|
||||
}
|
||||
@@ -1,66 +1,71 @@
|
||||
use std::time::Duration;
|
||||
use toasty::Db;
|
||||
|
||||
use redis::aio::ConnectionManager;
|
||||
use crate::schemas::{Job, Review};
|
||||
|
||||
pub const WEBHOOK_TTL: Duration = Duration::from_secs(24 * 60 * 60);
|
||||
pub const REVIEW_TTL: Duration = Duration::from_secs(7 * 24 * 60 * 60);
|
||||
pub struct ReviewRecord {
|
||||
pub event_id: u64,
|
||||
pub repo: String,
|
||||
pub pr: u64,
|
||||
pub ok: bool,
|
||||
pub model: String,
|
||||
pub markdown: String,
|
||||
pub cost: Option<f64>,
|
||||
pub comments: u64,
|
||||
pub duration_ms: u64,
|
||||
}
|
||||
|
||||
#[derive(Clone)]
|
||||
pub struct Dedup {
|
||||
conn: ConnectionManager,
|
||||
db: Db,
|
||||
}
|
||||
|
||||
impl Dedup {
|
||||
pub fn new(conn: ConnectionManager) -> Self {
|
||||
Self { conn }
|
||||
pub fn new(db: Db) -> Self {
|
||||
Self { db }
|
||||
}
|
||||
|
||||
pub fn webhook_key(event_id: u64) -> String {
|
||||
format!("steep:webhook:{event_id}")
|
||||
pub async fn is_known(&self, event_id: u64) -> toasty::Result<bool> {
|
||||
let mut db = self.db.clone();
|
||||
|
||||
if Job::filter_by_event_id(event_id)
|
||||
.first()
|
||||
.exec(&mut db)
|
||||
.await?
|
||||
.is_some()
|
||||
{
|
||||
return Ok(true);
|
||||
}
|
||||
|
||||
Ok(Review::filter_by_event_id(event_id)
|
||||
.first()
|
||||
.exec(&mut db)
|
||||
.await?
|
||||
.is_some())
|
||||
}
|
||||
|
||||
pub fn review_key(event_id: u64) -> String {
|
||||
format!("steep:review:{event_id}")
|
||||
pub async fn is_reviewed(&self, event_id: u64) -> toasty::Result<bool> {
|
||||
let mut db = self.db.clone();
|
||||
|
||||
Ok(Review::filter_by_event_id(event_id)
|
||||
.first()
|
||||
.exec(&mut db)
|
||||
.await?
|
||||
.is_some())
|
||||
}
|
||||
|
||||
pub async fn claim(&self, key: &str, ttl: Duration) -> redis::RedisResult<bool> {
|
||||
let set: Option<String> = redis::cmd("SET")
|
||||
.arg(key)
|
||||
.arg(1)
|
||||
.arg("NX")
|
||||
.arg("EX")
|
||||
.arg(ttl.as_secs())
|
||||
.query_async(&mut self.conn.clone())
|
||||
.await?;
|
||||
pub async fn mark_reviewed(&self, record: ReviewRecord) -> toasty::Result<()> {
|
||||
let mut db = self.db.clone();
|
||||
|
||||
Ok(set.is_some())
|
||||
}
|
||||
|
||||
pub async fn mark(&self, key: &str, ttl: Duration) -> redis::RedisResult<()> {
|
||||
let _: String = redis::cmd("SET")
|
||||
.arg(key)
|
||||
.arg(1)
|
||||
.arg("EX")
|
||||
.arg(ttl.as_secs())
|
||||
.query_async(&mut self.conn.clone())
|
||||
.await?;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub async fn is_marked(&self, key: &str) -> redis::RedisResult<bool> {
|
||||
let exists: i64 = redis::cmd("EXISTS")
|
||||
.arg(key)
|
||||
.query_async(&mut self.conn.clone())
|
||||
.await?;
|
||||
|
||||
Ok(exists > 0)
|
||||
}
|
||||
|
||||
pub async fn release(&self, key: &str) -> redis::RedisResult<()> {
|
||||
let _: i64 = redis::cmd("DEL")
|
||||
.arg(key)
|
||||
.query_async(&mut self.conn.clone())
|
||||
Review::upsert_by_event_id(record.event_id)
|
||||
.repo(record.repo)
|
||||
.pr(record.pr)
|
||||
.ok(record.ok)
|
||||
.model(record.model)
|
||||
.markdown(record.markdown)
|
||||
.cost(record.cost)
|
||||
.comments(record.comments)
|
||||
.duration_ms(record.duration_ms)
|
||||
.exec(&mut db)
|
||||
.await?;
|
||||
|
||||
Ok(())
|
||||
@@ -70,51 +75,42 @@ impl Dedup {
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
|
qpismont marked this conversation as resolved
Herald
commented
[performance] **[performance]** `Review::all()` charge toutes les reviews et leurs champs, notamment le Markdown, à chaque rafraîchissement du dashboard. Cette requête et la réponse JSON grossissent sans limite avec l’historique. Ajoutez une pagination ou une limite et ne chargez que les champs nécessaires à la liste.
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn webhook_key_is_namespaced() {
|
||||
assert_eq!(Dedup::webhook_key(42), "steep:webhook:42");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn review_key_is_namespaced() {
|
||||
assert_eq!(Dedup::review_key(42), "steep:review:42");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_two_keys_for_the_same_event_differ() {
|
||||
assert_ne!(Dedup::webhook_key(7), Dedup::review_key(7));
|
||||
}
|
||||
|
||||
async fn redis_dedup() -> Dedup {
|
||||
let url = std::env::var("REDIS_URL").expect("REDIS_URL must be set");
|
||||
Dedup::new(apalis_redis::connect(url).await.expect("connect to redis"))
|
||||
}
|
||||
use crate::schemas::Review;
|
||||
|
||||
#[tokio::test]
|
||||
#[ignore = "requires a running Redis (REDIS_URL)"]
|
||||
async fn claim_then_claim_is_a_duplicate() {
|
||||
let dedup = redis_dedup().await;
|
||||
let key = "steep:test:claim";
|
||||
dedup.release(key).await.unwrap();
|
||||
#[ignore = "requires DATABASE_URL"]
|
||||
async fn a_recorded_review_is_known_and_reviewed() {
|
||||
let db = crate::db::test_connect().await;
|
||||
let dedup = Dedup::new(db.clone());
|
||||
|
||||
assert!(dedup.claim(key, Duration::from_secs(30)).await.unwrap());
|
||||
assert!(!dedup.claim(key, Duration::from_secs(30)).await.unwrap());
|
||||
let event_id = 999_000_101;
|
||||
Review::delete_by_event_id(&mut db.clone(), event_id)
|
||||
.await
|
||||
.ok();
|
||||
|
||||
dedup.release(key).await.unwrap();
|
||||
}
|
||||
assert!(!dedup.is_known(event_id).await.unwrap());
|
||||
assert!(!dedup.is_reviewed(event_id).await.unwrap());
|
||||
|
||||
#[tokio::test]
|
||||
#[ignore = "requires a running Redis (REDIS_URL)"]
|
||||
async fn mark_then_is_marked() {
|
||||
let dedup = redis_dedup().await;
|
||||
let key = "steep:test:mark";
|
||||
dedup.release(key).await.unwrap();
|
||||
dedup
|
||||
.mark_reviewed(ReviewRecord {
|
||||
event_id,
|
||||
repo: "owner/repo".to_string(),
|
||||
pr: 1,
|
||||
ok: true,
|
||||
model: "test/model".to_string(),
|
||||
markdown: "looks good".to_string(),
|
||||
cost: Some(0.01),
|
||||
comments: 2,
|
||||
duration_ms: 5,
|
||||
})
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
assert!(!dedup.is_marked(key).await.unwrap());
|
||||
dedup.mark(key, Duration::from_secs(30)).await.unwrap();
|
||||
assert!(dedup.is_marked(key).await.unwrap());
|
||||
assert!(dedup.is_reviewed(event_id).await.unwrap());
|
||||
assert!(dedup.is_known(event_id).await.unwrap());
|
||||
|
||||
dedup.release(key).await.unwrap();
|
||||
Review::delete_by_event_id(&mut db.clone(), event_id)
|
||||
.await
|
||||
.unwrap();
|
||||
}
|
||||
}
|
||||
@@ -11,7 +11,7 @@ pub struct EnvConfig {
|
||||
pub gitea_url: String,
|
||||
pub gitea_token: String,
|
||||
pub gitea_timeout: u64,
|
||||
pub redis_url: String,
|
||||
pub database_url: String,
|
||||
pub metrics_bind_addr: Option<String>,
|
||||
pub sandbox_max_iterations: usize,
|
||||
}
|
||||
@@ -26,7 +26,7 @@ pub fn load_config() -> anyhow::Result<EnvConfig> {
|
||||
let gitea_url = try_get_env("GITEA_URL")?;
|
||||
let gitea_token = try_get_env("GITEA_TOKEN")?;
|
||||
let gitea_timeout = try_get_env("GITEA_TIMEOUT")?.parse()?;
|
||||
let redis_url = try_get_env("REDIS_URL")?;
|
||||
let database_url = try_get_env("DATABASE_URL")?;
|
||||
let metrics_bind_addr = std::env::var("METRICS_BIND_ADDR").ok();
|
||||
let sandbox_max_iterations = std::env::var("SANDBOX_MAX_ITERATIONS")
|
||||
.ok()
|
||||
@@ -43,7 +43,7 @@ pub fn load_config() -> anyhow::Result<EnvConfig> {
|
||||
gitea_url,
|
||||
gitea_token,
|
||||
gitea_timeout,
|
||||
redis_url,
|
||||
database_url,
|
||||
metrics_bind_addr,
|
||||
sandbox_max_iterations,
|
||||
})
|
||||
|
||||
@@ -281,7 +281,7 @@ async fn read_capped(response: reqwest::Response) -> anyhow::Result<String> {
|
||||
Ok(String::from_utf8_lossy(&buf).into_owned())
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize, Deserialize)]
|
||||
#[derive(Debug, Serialize, Deserialize, Clone)]
|
||||
pub enum WebhookType {
|
||||
Review(ReviewPayload),
|
||||
}
|
||||
@@ -298,9 +298,21 @@ impl WebhookType {
|
||||
WebhookType::Review(_) => "review",
|
||||
}
|
||||
}
|
||||
|
||||
pub fn repository_full_name(&self) -> &str {
|
||||
match self {
|
||||
WebhookType::Review(payload) => &payload.repository.full_name,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn pull_request_number(&self) -> u64 {
|
||||
match self {
|
||||
WebhookType::Review(payload) => payload.pull_request.number,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Deserialize, Serialize, Debug)]
|
||||
#[derive(Deserialize, Serialize, Debug, Clone)]
|
||||
pub struct ReviewPayload {
|
||||
pub action: String,
|
||||
pub pull_request: PullRequest,
|
||||
@@ -308,13 +320,13 @@ pub struct ReviewPayload {
|
||||
pub comment: Comment,
|
||||
}
|
||||
|
||||
#[derive(Deserialize, Serialize, Debug)]
|
||||
#[derive(Deserialize, Serialize, Debug, Clone)]
|
||||
pub struct PullRequest {
|
||||
pub number: u64,
|
||||
pub title: String,
|
||||
}
|
||||
|
||||
#[derive(Deserialize, Serialize, Debug)]
|
||||
#[derive(Deserialize, Serialize, Debug, Clone)]
|
||||
pub struct Comment {
|
||||
pub id: u64,
|
||||
pub body: String,
|
||||
@@ -325,7 +337,7 @@ pub struct User {
|
||||
pub login: String,
|
||||
}
|
||||
|
||||
#[derive(Deserialize, Serialize, Debug)]
|
||||
#[derive(Deserialize, Serialize, Debug, Clone)]
|
||||
pub struct Repository {
|
||||
pub full_name: String,
|
||||
pub clone_url: String,
|
||||
|
||||
@@ -0,0 +1,196 @@
|
||||
use std::{collections::VecDeque, time::Duration};
|
||||
|
||||
use futures_util::stream::{self, Stream};
|
||||
use toasty::Db;
|
||||
use tracing::error;
|
||||
|
||||
use crate::{
|
||||
gitea::WebhookType,
|
||||
schemas::{Job, JobStatus},
|
||||
};
|
||||
|
||||
/// Délai entre deux interrogations de la table quand la file est vide.
|
||||
const POLL_INTERVAL: Duration = Duration::from_secs(1);
|
||||
|
||||
/// File des reviews à faire.
|
||||
#[derive(Clone)]
|
||||
pub struct JobQueue {
|
||||
db: Db,
|
||||
}
|
||||
|
||||
impl JobQueue {
|
||||
pub fn new(db: Db) -> Self {
|
||||
Self { db }
|
||||
}
|
||||
|
||||
/// Enfile une nouvelle review.
|
||||
///
|
||||
/// L'unicité de `event_id` fait échouer l'insertion d'un doublon.
|
||||
pub async fn push(&self, webhook: WebhookType) -> toasty::Result<()> {
|
||||
let event_id = webhook.event_id();
|
||||
let repo = webhook.repository_full_name().to_string();
|
||||
let pr = webhook.pull_request_number();
|
||||
|
||||
let mut db = self.db.clone();
|
||||
|
||||
toasty::create!(Job {
|
||||
event_id,
|
||||
repo,
|
||||
pr,
|
||||
payload: webhook,
|
||||
status: JobStatus::Pending,
|
||||
})
|
||||
.exec(&mut db)
|
||||
.await?;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Flux des jobs en attente.
|
||||
///
|
||||
/// C'est un `Stream` (et non un `Iterator`) : l'accès à la base est
|
||||
/// asynchrone. Le flux interroge la table en boucle et met en tampon les
|
||||
/// jobs trouvés.
|
||||
pub fn pending(&self) -> impl Stream<Item = Job> {
|
||||
let db = self.db.clone();
|
||||
|
||||
stream::unfold(VecDeque::new(), move |mut buffer| {
|
||||
let db = db.clone();
|
||||
|
||||
async move {
|
||||
|
qpismont marked this conversation as resolved
Herald
commented
[bug] La sélection des jobs Pending et leur passage à Processing sont deux opérations distinctes, sans réservation atomique. Avec plusieurs instances, chacune peut récupérer le même job et lancer la même review avant que l’autre ne le marque Processing. Il faut réclamer les jobs de façon atomique. **[bug]** La sélection des jobs Pending et leur passage à Processing sont deux opérations distinctes, sans réservation atomique. Avec plusieurs instances, chacune peut récupérer le même job et lancer la même review avant que l’autre ne le marque Processing. Il faut réclamer les jobs de façon atomique.
|
||||
let mut db = db;
|
||||
|
||||
loop {
|
||||
if let Some(job) = buffer.pop_front() {
|
||||
return Some((job, buffer));
|
||||
}
|
||||
|
||||
match Job::filter_by_status(JobStatus::Pending)
|
||||
|
qpismont marked this conversation as resolved
Outdated
Herald
commented
[bug] La lecture des jobs **[bug]** La lecture des jobs `Pending` et leur passage à `Processing` ne sont pas atomiques. Avec plusieurs instances du serveur (ou plusieurs consommateurs), elles peuvent toutes récupérer le même job avant l'appel à `mark_processing`, puis exécuter plusieurs fois la review et publier des commentaires dupliqués. Il faut effectuer une réservation conditionnelle en base (`UPDATE ... WHERE status = 'pending' ... RETURNING`) plutôt que seulement filtrer puis mettre à jour.
Herald
commented
[performance] Cette requête récupère tous les jobs en attente sans limite, puis les accumule dans le **[performance]** Cette requête récupère tous les jobs en attente sans limite, puis les accumule dans le `VecDeque` à la ligne suivante. Une file PostgreSQL qui reçoit plus de jobs qu'elle n'en traite peut donc faire croître la mémoire du processus sans borne pratique. Il faudrait paginer ou limiter le nombre de jobs réservés par poll.
Herald
commented
[performance] Cette requête récupère tous les jobs en attente, puis les met en mémoire dans le buffer. Si la file grossit ou si le service redémarre avec un grand retard à rattraper, chaque worker peut charger un volume non borné de payloads JSON. Limitez la taille des lots et ne chargez que les jobs nécessaires à la capacité disponible. **[performance]** Cette requête récupère tous les jobs en attente, puis les met en mémoire dans le buffer. Si la file grossit ou si le service redémarre avec un grand retard à rattraper, chaque worker peut charger un volume non borné de payloads JSON. Limitez la taille des lots et ne chargez que les jobs nécessaires à la capacité disponible.
Herald
commented
[bug] La lecture des jobs en attente n’en réserve aucun : deux instances peuvent récupérer le même job avant que l’une d’elles n’appelle **[bug]** La lecture des jobs en attente n’en réserve aucun : deux instances peuvent récupérer le même job avant que l’une d’elles n’appelle `mark_processing`, puis lancer deux reviews et poster des résultats en double. Il faut réclamer les jobs de façon atomique, par exemple avec une mise à jour conditionnelle ou un verrouillage `SKIP LOCKED`.
|
||||
.exec(&mut db)
|
||||
.await
|
||||
{
|
||||
Ok(jobs) if jobs.is_empty() => {
|
||||
tokio::time::sleep(POLL_INTERVAL).await;
|
||||
}
|
||||
Ok(jobs) => buffer.extend(jobs),
|
||||
Err(err) => {
|
||||
error!(%err, "failed to poll pending jobs");
|
||||
tokio::time::sleep(POLL_INTERVAL).await;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
/// Marque un job comme en cours de traitement.
|
||||
pub async fn mark_processing(&self, id: u64) -> toasty::Result<()> {
|
||||
let mut db = self.db.clone();
|
||||
|
||||
Job::update_by_id(id)
|
||||
.status(JobStatus::Processing)
|
||||
.exec(&mut db)
|
||||
.await?;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Retire un job de la file, une fois la review enregistrée.
|
||||
pub async fn remove(&self, id: u64) -> toasty::Result<()> {
|
||||
let mut db = self.db.clone();
|
||||
|
||||
Job::delete_by_id(&mut db, id).await?;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use std::time::Duration;
|
||||
|
||||
use futures_util::StreamExt;
|
||||
|
||||
use super::*;
|
||||
use crate::gitea::{Comment, PullRequest, Repository, ReviewPayload};
|
||||
|
||||
fn webhook(event_id: u64) -> WebhookType {
|
||||
WebhookType::Review(ReviewPayload {
|
||||
action: "created".to_string(),
|
||||
pull_request: PullRequest {
|
||||
number: 1,
|
||||
title: "Test".to_string(),
|
||||
},
|
||||
repository: Repository {
|
||||
full_name: "owner/repo".to_string(),
|
||||
clone_url: "https://example.com/owner/repo.git".to_string(),
|
||||
},
|
||||
comment: Comment {
|
||||
id: event_id,
|
||||
body: "@bot review".to_string(),
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
async fn next_job(queue: &JobQueue, event_id: u64) -> Job {
|
||||
let mut pending = std::pin::pin!(queue.pending());
|
||||
|
||||
tokio::time::timeout(Duration::from_secs(10), async {
|
||||
loop {
|
||||
if let Some(job) = pending.next().await
|
||||
&& job.event_id == event_id
|
||||
{
|
||||
break job;
|
||||
}
|
||||
}
|
||||
})
|
||||
.await
|
||||
.expect("timed out waiting for the job")
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
#[ignore = "requires DATABASE_URL"]
|
||||
async fn a_pushed_job_is_pending_then_removed() {
|
||||
let db = crate::db::test_connect().await;
|
||||
let queue = JobQueue::new(db.clone());
|
||||
|
||||
let event_id = 999_000_201;
|
||||
Job::delete_by_event_id(&mut db.clone(), event_id)
|
||||
.await
|
||||
.ok();
|
||||
|
||||
queue.push(webhook(event_id)).await.unwrap();
|
||||
|
||||
let job = next_job(&queue, event_id).await;
|
||||
queue.mark_processing(job.id).await.unwrap();
|
||||
queue.remove(job.id).await.unwrap();
|
||||
|
||||
assert!(
|
||||
Job::filter_by_event_id(event_id)
|
||||
.first()
|
||||
.exec(&mut db.clone())
|
||||
.await
|
||||
.unwrap()
|
||||
.is_none()
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
#[ignore = "requires DATABASE_URL"]
|
||||
async fn a_duplicate_event_id_is_rejected() {
|
||||
let db = crate::db::test_connect().await;
|
||||
let queue = JobQueue::new(db.clone());
|
||||
|
||||
let event_id = 999_000_202;
|
||||
Job::delete_by_event_id(&mut db.clone(), event_id)
|
||||
.await
|
||||
.ok();
|
||||
|
||||
queue.push(webhook(event_id)).await.unwrap();
|
||||
assert!(queue.push(webhook(event_id)).await.is_err());
|
||||
|
||||
Job::delete_by_event_id(&mut db.clone(), event_id)
|
||||
.await
|
||||
.unwrap();
|
||||
}
|
||||
}
|
||||
@@ -1,12 +1,8 @@
|
||||
use std::time::Duration;
|
||||
|
||||
use crate::{
|
||||
bot::Bot,
|
||||
dedup::Dedup,
|
||||
gitea::{GiteaAPI, WebhookType},
|
||||
open_router::OpenRouterClient,
|
||||
sandbox::SandboxConfig,
|
||||
state::AppState,
|
||||
bot::Bot, dedup::Dedup, gitea::GiteaAPI, job_queue::JobQueue, open_router::OpenRouterClient,
|
||||
sandbox::SandboxConfig, state::AppState,
|
||||
};
|
||||
|
||||
use dotenvy::dotenv;
|
||||
@@ -19,13 +15,16 @@ mod api;
|
||||
mod bot;
|
||||
mod bot_actions;
|
||||
mod consts;
|
||||
mod db;
|
||||
mod dedup;
|
||||
mod env;
|
||||
mod errors;
|
||||
mod gitea;
|
||||
mod job_queue;
|
||||
mod metrics;
|
||||
mod open_router;
|
||||
mod sandbox;
|
||||
mod schemas;
|
||||
mod state;
|
||||
mod text;
|
||||
|
||||
@@ -38,6 +37,11 @@ const SHUTDOWN_TIMEOUT: Duration = Duration::from_secs(60);
|
||||
fn main() -> anyhow::Result<()> {
|
||||
dotenv().ok();
|
||||
|
||||
// Sous-commande de migration : `steep-server migration <generate|apply|...>`.
|
||||
if std::env::args().nth(1).as_deref() == Some("migration") {
|
||||
return tokio::runtime::Runtime::new()?.block_on(run_migrations());
|
||||
}
|
||||
|
||||
tracing_subscriber::registry()
|
||||
.with(fmt::layer())
|
||||
.with(
|
||||
@@ -63,6 +67,24 @@ fn main() -> anyhow::Result<()> {
|
||||
tokio::runtime::Runtime::new()?.block_on(run())
|
||||
}
|
||||
|
||||
async fn run_migrations() -> anyhow::Result<()> {
|
||||
let database_url = env::try_get_env("DATABASE_URL")?;
|
||||
let db = db::open(&database_url).await?;
|
||||
|
||||
let migrations = match std::env::var("MIGRATIONS_DIR") {
|
||||
Ok(dir) => std::path::PathBuf::from(dir),
|
||||
Err(_) => std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("toasty"),
|
||||
};
|
||||
let config =
|
||||
toasty_cli::Config::new().migration(toasty_cli::MigrationConfig::new().path(migrations));
|
||||
|
||||
toasty_cli::ToastyCli::with_config(db, config)
|
||||
.parse_and_run()
|
||||
.await?;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn run() -> anyhow::Result<()> {
|
||||
let config = env::load_config()?;
|
||||
|
||||
@@ -101,10 +123,9 @@ async fn run() -> anyhow::Result<()> {
|
||||
);
|
||||
}
|
||||
|
||||
let redis_conn = apalis_redis::connect(config.redis_url.clone()).await?;
|
||||
let storage: apalis_redis::RedisStorage<WebhookType> =
|
||||
apalis_redis::RedisStorage::new(redis_conn.clone());
|
||||
let dedup = Dedup::new(redis_conn);
|
||||
let db = db::connect(&config.database_url).await?;
|
||||
let dedup = Dedup::new(db.clone());
|
||||
let queue = JobQueue::new(db);
|
||||
|
||||
let bot = Bot::new(
|
||||
gitea_api,
|
||||
@@ -113,12 +134,13 @@ async fn run() -> anyhow::Result<()> {
|
||||
config.open_router_model.clone(),
|
||||
sandbox,
|
||||
dedup.clone(),
|
||||
queue.clone(),
|
||||
);
|
||||
|
||||
let app_state = AppState {
|
||||
storage: storage.clone(),
|
||||
bot_name: gitea_user.login.clone(),
|
||||
queue,
|
||||
dedup,
|
||||
bot_name: gitea_user.login.clone(),
|
||||
config,
|
||||
};
|
||||
|
||||
@@ -143,7 +165,7 @@ async fn run() -> anyhow::Result<()> {
|
||||
tokio::select! {
|
||||
result = async {
|
||||
tokio::try_join!(
|
||||
bot.start(storage, shutdown.clone()),
|
||||
bot.start(shutdown.clone()),
|
||||
api::start(app_state, shutdown.clone()),
|
||||
signal
|
||||
)
|
||||
|
||||
@@ -54,7 +54,7 @@ pub fn describe() {
|
||||
describe_counter!(
|
||||
"steep_webhooks_enqueue_failed_total",
|
||||
Unit::Count,
|
||||
"Webhooks dropped because enqueueing to Redis failed"
|
||||
"Webhooks dropped because enqueueing to PostgreSQL failed"
|
||||
);
|
||||
describe_gauge!(
|
||||
"steep_bot_tasks_active",
|
||||
|
||||
@@ -0,0 +1,31 @@
|
||||
use jiff::Timestamp;
|
||||
|
||||
use crate::gitea::WebhookType;
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, toasty::Embed)]
|
||||
pub enum JobStatus {
|
||||
Pending,
|
||||
Processing,
|
||||
}
|
||||
|
||||
#[derive(Debug, toasty::Model)]
|
||||
pub struct Job {
|
||||
#[key]
|
||||
#[auto]
|
||||
pub id: u64,
|
||||
|
||||
#[unique]
|
||||
pub event_id: u64,
|
||||
|
||||
pub repo: String,
|
||||
pub pr: u64,
|
||||
|
||||
#[column(type = jsonb)]
|
||||
pub payload: toasty::Json<WebhookType>,
|
||||
|
||||
#[index]
|
||||
pub status: JobStatus,
|
||||
|
||||
#[auto]
|
||||
pub created_at: Timestamp,
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
pub mod job;
|
||||
pub mod review;
|
||||
|
||||
pub use job::{Job, JobStatus};
|
||||
pub use review::Review;
|
||||
@@ -0,0 +1,35 @@
|
||||
use jiff::Timestamp;
|
||||
|
||||
#[derive(Debug, toasty::Model)]
|
||||
pub struct Review {
|
||||
#[key]
|
||||
#[auto]
|
||||
pub id: u64,
|
||||
|
||||
#[unique]
|
||||
pub event_id: u64,
|
||||
|
||||
pub repo: String,
|
||||
pub pr: u64,
|
||||
|
||||
/// Vrai si la review a abouti.
|
||||
pub ok: bool,
|
||||
|
||||
/// Modèle utilisé.
|
||||
pub model: String,
|
||||
|
||||
/// Markdown posté sur la pull request (vide en cas d'échec).
|
||||
pub markdown: String,
|
||||
|
||||
/// Coût OpenRouter en USD, si connu.
|
||||
pub cost: Option<f64>,
|
||||
|
||||
/// Nombre de commentaires produits.
|
||||
pub comments: u64,
|
||||
|
||||
/// Durée du traitement en millisecondes.
|
||||
pub duration_ms: u64,
|
||||
|
||||
#[default(jiff::Timestamp::now())]
|
||||
pub completed_at: Timestamp,
|
||||
}
|
||||
@@ -1,11 +1,9 @@
|
||||
use apalis_redis::RedisStorage;
|
||||
|
||||
use crate::{dedup::Dedup, env::EnvConfig, gitea::WebhookType};
|
||||
use crate::{dedup::Dedup, env::EnvConfig, job_queue::JobQueue};
|
||||
|
||||
#[derive(Clone)]
|
||||
pub struct AppState {
|
||||
pub storage: RedisStorage<WebhookType>,
|
||||
pub bot_name: String,
|
||||
pub queue: JobQueue,
|
||||
pub dedup: Dedup,
|
||||
pub bot_name: String,
|
||||
pub config: EnvConfig,
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
version = 1
|
||||
|
||||
[[migrations]]
|
||||
id = 3569579561409902287
|
||||
name = "0000_init.sql"
|
||||
snapshot_name = "0000_snapshot.toml"
|
||||
@@ -0,0 +1,28 @@
|
||||
CREATE TYPE "job_status" AS ENUM ('pending', 'processing');
|
||||
CREATE TABLE "jobs" (
|
||||
"id" BIGINT NOT NULL GENERATED BY DEFAULT AS IDENTITY,
|
||||
"event_id" BIGINT NOT NULL,
|
||||
"repo" TEXT NOT NULL,
|
||||
"pr" BIGINT NOT NULL,
|
||||
"payload" JSONB NOT NULL,
|
||||
"status" "job_status" NOT NULL,
|
||||
"created_at" TIMESTAMPTZ(6) NOT NULL,
|
||||
PRIMARY KEY ("id")
|
||||
);
|
||||
CREATE UNIQUE INDEX "index_jobs_by_event_id" ON "jobs" ("event_id");
|
||||
CREATE INDEX "index_jobs_by_status" ON "jobs" ("status");
|
||||
CREATE TABLE "reviews" (
|
||||
"id" BIGINT NOT NULL GENERATED BY DEFAULT AS IDENTITY,
|
||||
"event_id" BIGINT NOT NULL,
|
||||
"repo" TEXT NOT NULL,
|
||||
"pr" BIGINT NOT NULL,
|
||||
"ok" BOOLEAN NOT NULL,
|
||||
"model" TEXT NOT NULL,
|
||||
"markdown" TEXT NOT NULL,
|
||||
"cost" DOUBLE PRECISION,
|
||||
"comments" BIGINT NOT NULL,
|
||||
"duration_ms" BIGINT NOT NULL,
|
||||
"completed_at" TIMESTAMPTZ(6) NOT NULL,
|
||||
PRIMARY KEY ("id")
|
||||
);
|
||||
CREATE UNIQUE INDEX "index_reviews_by_event_id" ON "reviews" ("event_id");
|
||||
@@ -0,0 +1,160 @@
|
||||
version = 1
|
||||
|
||||
[schema]
|
||||
|
||||
[[schema.tables]]
|
||||
id = 0
|
||||
name = "jobs"
|
||||
primary_key = { columns = [{ table = 0, index = 0 }], index = { table = 0, index = 0 } }
|
||||
|
||||
[[schema.tables.columns]]
|
||||
id = { table = 0, index = 0 }
|
||||
name = "id"
|
||||
ty = "U64"
|
||||
storage_ty = { UnsignedInteger = 8 }
|
||||
auto_increment = true
|
||||
|
||||
[[schema.tables.columns]]
|
||||
id = { table = 0, index = 1 }
|
||||
name = "event_id"
|
||||
ty = "U64"
|
||||
storage_ty = { UnsignedInteger = 8 }
|
||||
|
||||
[[schema.tables.columns]]
|
||||
id = { table = 0, index = 2 }
|
||||
name = "repo"
|
||||
ty = "String"
|
||||
storage_ty = "Text"
|
||||
|
||||
[[schema.tables.columns]]
|
||||
id = { table = 0, index = 3 }
|
||||
name = "pr"
|
||||
ty = "U64"
|
||||
storage_ty = { UnsignedInteger = 8 }
|
||||
|
||||
[[schema.tables.columns]]
|
||||
id = { table = 0, index = 4 }
|
||||
name = "payload"
|
||||
ty = "String"
|
||||
storage_ty = "Jsonb"
|
||||
|
||||
[[schema.tables.columns]]
|
||||
id = { table = 0, index = 5 }
|
||||
name = "status"
|
||||
ty = "String"
|
||||
storage_ty = { Enum = { name = "job_status", variants = [{ name = "pending" }, { name = "processing" }] } }
|
||||
|
||||
[[schema.tables.columns]]
|
||||
id = { table = 0, index = 6 }
|
||||
name = "created_at"
|
||||
ty = "Timestamp"
|
||||
storage_ty = { Timestamp = 6 }
|
||||
|
||||
[[schema.tables.indices]]
|
||||
id = { table = 0, index = 0 }
|
||||
name = "index_jobs_by_id"
|
||||
on = 0
|
||||
columns = [{ column = { table = 0, index = 0 }, op = "Eq", scope = "Partition" }]
|
||||
unique = true
|
||||
primary_key = true
|
||||
|
||||
[[schema.tables.indices]]
|
||||
id = { table = 0, index = 1 }
|
||||
name = "index_jobs_by_event_id"
|
||||
on = 0
|
||||
columns = [{ column = { table = 0, index = 1 }, op = "Eq", scope = "Partition" }]
|
||||
unique = true
|
||||
|
||||
[[schema.tables.indices]]
|
||||
id = { table = 0, index = 2 }
|
||||
name = "index_jobs_by_status"
|
||||
on = 0
|
||||
columns = [{ column = { table = 0, index = 5 }, op = "Eq", scope = "Partition" }]
|
||||
|
||||
[[schema.tables]]
|
||||
id = 1
|
||||
name = "reviews"
|
||||
primary_key = { columns = [{ table = 1, index = 0 }], index = { table = 1, index = 0 } }
|
||||
|
||||
[[schema.tables.columns]]
|
||||
id = { table = 1, index = 0 }
|
||||
name = "id"
|
||||
ty = "U64"
|
||||
storage_ty = { UnsignedInteger = 8 }
|
||||
auto_increment = true
|
||||
|
||||
[[schema.tables.columns]]
|
||||
id = { table = 1, index = 1 }
|
||||
name = "event_id"
|
||||
ty = "U64"
|
||||
storage_ty = { UnsignedInteger = 8 }
|
||||
|
||||
[[schema.tables.columns]]
|
||||
id = { table = 1, index = 2 }
|
||||
name = "repo"
|
||||
ty = "String"
|
||||
storage_ty = "Text"
|
||||
|
||||
[[schema.tables.columns]]
|
||||
id = { table = 1, index = 3 }
|
||||
name = "pr"
|
||||
ty = "U64"
|
||||
storage_ty = { UnsignedInteger = 8 }
|
||||
|
||||
[[schema.tables.columns]]
|
||||
id = { table = 1, index = 4 }
|
||||
name = "ok"
|
||||
ty = "Bool"
|
||||
storage_ty = "Boolean"
|
||||
|
||||
[[schema.tables.columns]]
|
||||
id = { table = 1, index = 5 }
|
||||
name = "model"
|
||||
ty = "String"
|
||||
storage_ty = "Text"
|
||||
|
||||
[[schema.tables.columns]]
|
||||
id = { table = 1, index = 6 }
|
||||
name = "markdown"
|
||||
ty = "String"
|
||||
storage_ty = "Text"
|
||||
|
||||
[[schema.tables.columns]]
|
||||
id = { table = 1, index = 7 }
|
||||
name = "cost"
|
||||
ty = "F64"
|
||||
storage_ty = { Float = 8 }
|
||||
nullable = true
|
||||
|
||||
[[schema.tables.columns]]
|
||||
id = { table = 1, index = 8 }
|
||||
name = "comments"
|
||||
ty = "U64"
|
||||
storage_ty = { UnsignedInteger = 8 }
|
||||
|
||||
[[schema.tables.columns]]
|
||||
id = { table = 1, index = 9 }
|
||||
name = "duration_ms"
|
||||
ty = "U64"
|
||||
storage_ty = { UnsignedInteger = 8 }
|
||||
|
||||
[[schema.tables.columns]]
|
||||
id = { table = 1, index = 10 }
|
||||
name = "completed_at"
|
||||
ty = "Timestamp"
|
||||
storage_ty = { Timestamp = 6 }
|
||||
|
||||
[[schema.tables.indices]]
|
||||
id = { table = 1, index = 0 }
|
||||
name = "index_reviews_by_id"
|
||||
on = 1
|
||||
columns = [{ column = { table = 1, index = 0 }, op = "Eq", scope = "Partition" }]
|
||||
unique = true
|
||||
primary_key = true
|
||||
|
||||
[[schema.tables.indices]]
|
||||
id = { table = 1, index = 1 }
|
||||
name = "index_reviews_by_event_id"
|
||||
on = 1
|
||||
columns = [{ column = { table = 1, index = 1 }, op = "Eq", scope = "Partition" }]
|
||||
unique = true
|
||||
[performance]
cargo install cargo-deny --lockedrecompile l'outil à chaque pipeline, ce qui ajoute plusieurs minutes de CI. Il vaut mieux utiliser une image préconstruite (ghcr.io/embarkstudios/cargo-deny) ou un binaire précompilé. Par ailleurs les étapes CI utilisentrust:1.99alors que leContainerfilebuild avecrust:1.98-trixie: les versions de toolchain divergent.