AI Integration: How to Add AI to an Existing Software System

The problem, described by its symptom

The symptom does not look like an AI problem at first. A content pipeline runs on a schedule, produces articles, updates a spreadsheet, rebuilds a hub page, writes a sitemap and regenerates llms.txt. Everything completes. The cron job exits with no error. And yet the output is wrong in ways nobody notices for days: a topic appears in the spreadsheet as published but the article is not on the site, or the sitemap lists a URL that returns a 404, or the hub page links to an article that was never written.

The reason is that an AI call is a network call with a non-deterministic result. It can time out. It can return an empty string. It can return a perfectly valid answer that stops mid-sentence because the token limit was reached. If the code treats the model like a local function that always returns, every downstream step runs on data that does not exist. The spreadsheet says done. The site says otherwise.

How someone notices: usually through a third party. A search console report shows a submitted URL that 404s. A reader reports a broken link from the hub page. Or someone opens the spreadsheet and counts the rows against the published pages by hand, and the numbers do not match. The pipeline reported success the whole time.

How to diagnose it

Start by separating the model call from everything else. In the publishing system we maintain, the engine already had a clear shape: read a row from the spreadsheet, fill an HTML template, write the file. The diagnosis question is whether the model call is the only new moving part, or whether it has leaked into the rest of the pipeline.

Concrete steps that worked:

• Run the pipeline with the dry-run flag (--conferir in this case) so it generates and prints but writes nothing. If the output is already wrong in dry run, the bug is in generation, not in file writing.
• Read the run log line by line. The log records the topic chosen, the OG image generated, the llms.txt update with its total count, the hub rebuild, the published URL, the spreadsheet update and any checklist warnings. Gaps in that sequence tell you which step silently skipped.
• Compare the spreadsheet row count against the sitemap entry count. A mismatch means at least one row was marked done without a published URL.
• Check the model response object for finish_reason. If it is length, the answer was truncated and the code likely wrote a partial article.
• Grep the engine file for the model call and confirm it appears exactly once. If it appears in more than one place, you have more than one failure path to reason about.

In this pipeline the sitemap bug became visible only because the log listed the published URL for each article and the sitemap writer ran after it. Comparing the two lists showed a URL in the log that never reached the sitemap. Without the log, the only signal was the finished cron job.

What the data showed

The evidence from this pipeline is not a metric. It is a set of observed failure modes and the controls that were added because each one actually happened.

• The model call needed a 90-second timeout. Without it, a slow response held the whole run open.
• It needed 3 attempts with 5 seconds between them. Transient failures are common enough that a single attempt loses work.
• It needed a separate retry path for finish_reason == length. A truncated answer is not an API error, so a naive retry loop never catches it.
• When all attempts fail, the topic is skipped, logged, and reported on Telegram. The cron runs again the next day. Nobody is woken at night.
• Before any change to the prompt, the team runs --conferir, which generates and prints but writes nothing.
• The engine file is backed up with a date suffix before each change. There were four backups in September alone.
• There are two prompts, not one. The Portuguese prompt uses a commercial structure. The English prompt uses a technical one: symptom, diagnosis, data, fix, verification. Language is a dictionary key, not a second program.
• Cost control: one model call per article, temperature 0.7, max 6,000 tokens, no agent loops.

The pattern across all of these is the same. Each control exists because its absence caused a real, observed problem. None of them were designed upfront.

The fix, concrete and implementable

The fix was not a rewrite. The existing engine already read a spreadsheet and wrote HTML from a template. The model call was inserted as one function that receives (topic, evidence, style rules) and returns text. Everything else stayed exactly as it was.

The function signature is the important part. It takes plain data in and returns plain text out. It does not touch the filesystem, the spreadsheet or the sitemap. That separation is what makes the retry logic small enough to reason about.

def generate(topic, evidence, style_rules, language):
    prompt = PROMPTS[language](topic, evidence, style_rules)
    for attempt in range(3):
        try:
            resp = client.create(
                prompt=prompt,
                temperature=0.7,
                max_tokens=6000,
                timeout=90,
            )
            text = resp.text.strip()
            if resp.finish_reason == "length":
                raise TruncatedAnswer()
            if not text:
                raise EmptyAnswer()
            return text
        except (Timeout, TruncatedAnswer, EmptyAnswer, APIError):
            if attempt < 2:
                time.sleep(5)
    return None

When generate returns None, the caller skips the topic, logs it and reports on Telegram. The cron runs again the next day. Nothing is written to the spreadsheet for that topic, so the next run picks it up.

Language handling is a dictionary, not a branch in the pipeline:

PROMPTS = {
    "pt": build_prompt_commercial,
    "en": build_prompt_technical,
}

The Portuguese prompt uses a commercial structure. The English prompt uses a technical one: symptom, diagnosis, data, fix, verification. Both return a string. The rest of the engine does not know which language produced the text.

Two operational controls make this safe to change:

• --conferir generates and prints but writes nothing. It is run before any change to a prompt, so a bad prompt produces console output instead of a published page.
• The engine file is backed up with a date suffix before each change. Four backups in September alone. Restoring is a file copy, not a git archaeology session.

Logging is the last piece and the one that pays for itself. Every run writes: topic chosen, OG image generated, llms.txt updated with the total count, hub rebuilt, URL published, spreadsheet updated, checklist warnings. That sequence is what made the sitemap bug visible, because the published URL appeared in the log and never reached the sitemap.

Cost control is deliberately boring: one model call per article, temperature 0.7, max 6,000 tokens, no agent loops. One call means one failure path, one timeout, one retry policy. If you want to read more about how this kind of automation is scoped, there is a page on automation.

How to verify the fix worked

Measure before and after on the same pipeline, with the same inputs.

• Count of topics skipped per run. Before the controls, a failed topic either blocked the run or was marked done without a page. After, it is skipped and logged, so the number is explicit and should equal the number of Telegram reports.
• Count of truncated articles. Before, a finish_reason == length answer was written as a partial article. After, it triggers a retry and, if it still fails, a skip. The number of partial articles on disk should be zero.
• Spreadsheet rows versus sitemap entries. These two counts should match after every run. A mismatch means a step between them failed.
• Log completeness. Every run should have all seven entries: topic, OG image, llms.txt count, hub rebuild, URL, spreadsheet update, checklist warnings. A missing entry localises the failure.
• Dry-run output before each prompt change. If --conferir output looks wrong, nothing was written.

Run the pipeline in dry-run mode first, then compare the log sequence against the checklist above. If every entry is present and the spreadsheet and sitemap counts match, the run is clean.

Frequently asked questions