04 - Operate & Measure / AI Application Maintenance

AI Application Maintenance

Ongoing maintenance for AI-touched systems - provider migrations, prompt tuning, dependency upgrades and the quiet engineering work that keeps production systems running as the AI landscape shifts.

Entry packageMaintenance Handover Packfrom EUR 2,6001 system · up to 2 weeks from scopingWhat is included
In short
01

Model deprecations without a scramble

Provider migrations planned and executed with an eval run guarding the output.

02

Prompts tuned on a cadence

Quarterly re-eval, regression check and minor tuning as your inputs drift.

03

A quarterly report you can budget from

What changed, what to expect, what to budget for next quarter, in writing.

23Minds provides ongoing maintenance for AI-touched applications.

Show the full summary

AI systems need maintenance that traditional applications do not: provider migrations when a model is deprecated, prompt tuning as inputs evolve, eval re-baselining, dependency upgrades for the LLM SDK layer, cost monitoring as spend drifts. Retainer model fits teams with continuous AI features; fixed engagements fit specific events like a provider migration. Provider notice before a model retirement differs: at least 60 days at Anthropic for publicly released models, at least six months at OpenAI for generally available ones (checked 25 September 2026).

Why 23Minds

What makes this service defensible.

  • Handover pack opens the retainer

    State audit, dependency and model inventory, migration plan, runbooks and an opening report in two weeks.

  • Eval-protected migrations

    No provider or model switch without a before-and-after score.

  • Retainer or fixed engagement

    Continuous AI features get monthly hours; a single migration gets a fixed price.

  • ISO 9001:2015 quality system

    Certified by DNV, no. 10000439701-MSC-RvA-POL; maintenance runs as a documented procedure.

How we work

The process, phase by phase.

  1. 1

    1. Handover

    Read the system. Map dependencies, prompts, eval setup, observability and on-call procedures. Document gaps.

    Week 1-2
    A folder passed along a yellow arrow into an open box: the system taken over, with its dependencies, prompts and on-call procedures mapped and the gaps listed.
  2. 2

    2. Stabilization (if needed)

    Address critical maintenance debt before steady-state retainer: missing evals, ungoverned prompts, stale SDKs.

    Week 2-4 (one-time)
    A jagged line settling into a flat yellow line: the critical maintenance debt paid down before steady-state work begins.
  3. 3

    3. Steady-state retainer

    Monthly maintenance hours, quarterly check-ins, ad-hoc support for provider events or incidents.

    Ongoing
    A calendar with a steady yellow heartbeat line running across it: monthly maintenance, quarterly check-ins and support when a provider changes something.
  4. 4

    4. Quarterly review

    Written report on what we touched, what is drifting, what to budget next quarter.

    Quarterly
    A ring split into four equal segments, one of them yellow: the quarterly written report on what changed, what is drifting and what to budget next.
How to start

Pick the way in that fits your budget and timing.

Every practice has a free first step, an entry package with a written deliverable and its price listed here, and a full project or retainer quoted after a first call. Once the scope is agreed, the price is fixed, with no hidden costs.

  1. 1
    Step 1 · Free

    Intro call or self-assessment

    60 minutes · 1 engineer · no obligation

    Free
    Show details
    • Intro call with an engineer, or the online self-assessment
    • A written next step within one business day
    • No obligation and no sales deck
    Talk to an engineer
  2. Recommended
    2
    Step 2 · Entry package

    Maintenance Handover Pack

    1 system · up to 2 weeks from scoping

    from EUR 2,600 net

    The price is fixed once the scope is agreed on the intro call, no hidden costs; extras only from the Not included list.

    Show details
    • State audit of one system
    • Dependency and model inventory
    • Provider migration plan
    • Runbooks and an opening report
    • Not included: Ongoing maintenance hours, fixing issues found in the audit, 24/7 on-call, SLA.
    Ask for this package
  3. 3
    Step 3 · Project or retainer

    Full scope, quoted after a first call

    Full scope · phased delivery · quoted per phase

    from EUR 2,800 net per month, quoted after a first call
    Show details
    • Maintenance retainer
    • Provider migrations and quarterly prompt tune-up
    • Written quarterly report
    Start the conversation
Sounds familiar?

The situations that bring teams to this service.

  • Provider deprecates a model; the team has no migration plan and no time to write one
  • Prompt that worked at launch produces worse outputs six months later - drift
  • LLM SDK version is stuck at launch-day version; security patches piling up
Show the other problems (1)
  • Cost trends are rising slowly enough that nobody noticed until the quarterly review
Who it's for
  • Engineering teams with AI features in production who need part-time senior support
  • CTOs whose team built the AI feature and moved on; nobody owns it now
  • Product teams whose AI providers deprecate models faster than the team can react
  • Mid-stage companies extending or maintaining AI features without growing the team
Failure modes

Where this goes wrong, and what we do about it.

  1. 1. The model your feature stands on gets retiredWhat stays with you: regression set with results for old and new model side by side, and the decision recorded per difference
    What happens

    Providers deprecate on notice, and the notice is measured in weeks. A feature built on a specific model works until the day it does not, and the migration then happens under time pressure with no way to tell whether quality moved, because there was never a baseline.

    What we do about it

    We keep a regression set for the behaviours that matter, run it against the replacement before the switch, and report the differences rather than a pass or fail. Where a difference is acceptable we record why.

  2. 2. Prompts drift with no history and no testWhat stays with you: versioned prompts with review, version stamped on responses, and behaviour tests that fail for the right reason
    What happens

    A prompt gets edited in place because it is text, not code. There is no diff anyone reviews, no version attached to the output, and no test that fails. Six edits later the behaviour has moved and nobody can say which change did it or return to what worked.

    What we do about it

    We move prompts under version control with the same review as code, attach the version to every response, and add cases that fail when the behaviour changes rather than when the text changes.

  3. 3. The unglamorous work has no ownerWhat stays with you: maintenance schedule with named owners and the automation that removed the repetitive part
    What happens

    Dependency updates, provider migrations, expiring keys and quota changes are invisible until they are an incident. Invisible work does not get assigned, so it waits for the person who notices, and that person is usually whoever is on call when it breaks.

    What we do about it

    We name the recurring work, put it on a cadence with an owner, and automate the parts that can be automated so the rest is small enough to actually happen.

What you get

Deliverables, not deliverable-shaped slides.

  • Monthly maintenance hours covering the issues above (retainer)
  • Provider migration plans + execution when models deprecate (eval-protected)
  • Quarterly prompt tune-up: re-eval, regression check, minor tuning
  • Dependency upgrade cadence: SDKs, observability libraries, eval frameworks
  • Quarterly written report: what changed, what to expect, what to budget for next quarter
FAQ

Questions procurement teams ask.

Both work. Retainer (10-40 hours/month) fits teams with continuous AI features in production. Fixed engagements (provider migration, eval re-baselining, prompt re-tune) fit teams with a specific event.
Retainer clients get ad-hoc incident support - we join the bridge, help debug, write the post-mortem. SLAs are agreed in the retainer contract.
We handle AI-specific operations (prompt drift, model migrations, eval pipelines, LLM cost monitoring). Generic SRE work (Kubernetes, infrastructure-as-code, traditional alerting) is in scope only when it intersects with the AI features.
Standard maintenance event. We re-baseline eval against the new model, run regression tests, migrate prompts where the new model needs different patterns, measure before/after on quality, latency and cost. Provider rules set the calendar: Anthropic announces the retirement of a publicly released model at least 60 days ahead, OpenAI gives at least six months for generally available models and at least three for specialised variants, and preview models can get much shorter notice. In practice, Anthropic retired nine model versions on six dates between October 2025 and August 2026, with 60 to 189 days' notice. Answer quality can also change without a retirement: GPT-4's accuracy at identifying prime numbers fell from 84% in March 2023 to 51% in June 2023 (Chen, Zaharia and Zou).
Yes. Handover phase covers reading and documenting the system. We may need to address maintenance debt before steady-state retainer becomes possible.
Guides

Guides that lead to this service.

Talk to an engineer.

Describe where you are with AI application maintenance. A reply within one business day.

Talk to an engineer
Szczecin - ul. Wawrzyniaka 6WWarszawaZielona GóraKraków