Local LLM Pipeline Context Drift Issue in Multi-Step Agentic Work

Practical Findings from Two Months of LLM Pipeline Testing

A developer recently shared results from running a multi-step job search automation pipeline for two months. The pipeline involved research, CV drafting, and cover letter generation. Testing was conducted using Llama-3.3-70b-versatile on both Groq's free tier and local Ollama during evening runs over several weeks.

Where Local Models Lost Ground

While local models won on privacy, cost, and not worrying about quotas per session, they faced significant issues in agentic workflows:

• Context Drift in Multi-Step Pipelines: Local models would successfully complete step 2 but forget what was established in step 1 by the time they reached step 4. The developer observed this across 5 to 6 node pipelines where maintaining coherent context was crucial.
• Comparison with Cloud Models: Claude on Groq's free tier didn't exhibit this context drift problem nearly as much, suggesting better performance in maintaining context across sequential tasks.

Hidden Free Tier Pitfall

The developer highlighted another practical issue: free tier models get retired quietly without warning. You can set up a pipeline with a specific model, walk away for a few weeks, and return to find half your configuration broken with wrong outputs.

The developer noted this wasn't a benchmark post but actual experience, and they're genuinely open to being wrong about the context drift part while asking what's actually working for multi-step agentic work currently.