TL;DR
I have built AI systems that handle 700,000+ monthly customer interactions at Wizz Air and improved compliance document accuracy from 72% to 96.2% at Sanofi. I have also reviewed a lot of AI engineering resumes and sat in on hiring processes at companies trying to staff up. The hiring mistakes are consistent and expensive.
This is not a post about how to write a job description or how to structure an offer letter. It is about the specific mistakes founders and technical leads make when evaluating AI engineers, and what to screen for instead.
What skills actually matter in 2026
The AI engineering landscape shifted dramatically between 2022 and 2026. In 2022, "AI engineer" meant someone who trained models: PyTorch, fine-tuning, GPU clusters, custom architectures. That person still exists and is still valuable, but they are not what most product companies need.
Most product companies need someone who can take foundation models from OpenAI, Anthropic, or Google and integrate them reliably into a production system. That requires a completely different skill set.
Production reliability
Can the engineer handle the non-deterministic nature of LLM output? Do they know how to build validation layers, retry logic, and fallback paths? A developer who has only built demos will ship an integration that works 95% of the time in testing and then silently fails in production in ways that are invisible until a user complains.
Ask them directly: how do you handle LLM output that does not match your expected schema? What does your retry strategy look like? What happens to the user if the AI call fails entirely? If they do not have concrete answers, they have not shipped production AI.
Retrieval architecture
RAG is the dominant pattern for knowledge-grounded AI features. An AI engineer who cannot articulate the difference between dense retrieval and hybrid retrieval, who cannot explain why chunking strategy affects retrieval quality, and who has never built an evaluation suite for a retrieval pipeline is not qualified to build your AI knowledge system.
The Sanofi compliance system I rebuilt went from 72% to 96.2% accuracy. That improvement came almost entirely from fixing the retrieval layer, not from changing the model. The original engineer who built the broken version had impressive ML credentials and had never thought once about recall@k.
Evaluation and monitoring
How does the engineer know if the AI feature is working? If the answer is "we test it manually before shipping," you have a problem. Production AI systems need automated evaluation pipelines with ground-truth test sets, quality metrics tracked over time, and alerting when those metrics degrade.
Model providers update their models. Your prompts that worked last month may produce subtly worse outputs today. An engineer who does not instrument this will not notice the regression until users have been getting bad answers for weeks.
Cost management
At 700,000+ monthly interactions, token cost is a real line item. Engineers who have never operated at scale often write integrations that send five times more tokens than necessary, use the wrong model tier for the task, or cache nothing. Ask them how they manage LLM API costs in production. Ask them about the token budget for a specific feature they built. Blank stares here are expensive.
What sounds impressive but is not
Several things appear frequently on AI engineering resumes that tell you almost nothing useful.
Fine-tuning experience
Fine-tuning a model is technically interesting and is sometimes the right solution, but it is rarely what product companies need in 2026. The costs have come down significantly, the tooling has improved, and it has become a weekend project. Someone whose resume leads with fine-tuning experience without production deployment context is a researcher, not a product engineer.
Hugging Face leaderboard mentions
Benchmark performance on academic datasets has almost no correlation with performance on your specific production workload. Engineers who lead with benchmark comparisons are usually selling you on model selection, not on their ability to build reliable systems around models.
GitHub repos with 500 stars
A popular demo repository is not production experience. Look at the code. Is there error handling? Is there output validation? Is there any evaluation tooling? Demos are built to show that something is possible, not to run reliably at scale. Treat demo repos as table stakes, not as evidence of production capability.
Portfolio red flags
These patterns in a candidate's portfolio should prompt deeper questions.
- Every project is a chatbot. Chatbots are the hello world of LLM integration. If the entire portfolio is chatbots with no RAG systems, no structured output processing, no multi-step agent work, the engineer has not been challenged yet.
- No mention of failures or debugging. Real production AI work involves debugging hallucinations, diagnosing retrieval quality drops, and handling model updates that break existing prompts. Engineers who describe only successes either have not shipped at scale or are not being honest.
- All projects are solo demos with no users. Zero production traffic means zero exposure to the problems that actually matter: rate limits, latency spikes, token cost at scale, model drift, and the long tail of input edge cases that break your prompt.
- The tech stack is always the same. GPT-4 plus LangChain plus Pinecone for every project suggests someone who learned one stack and has never had to reason about model trade-offs, retrieval alternatives, or cost optimization.
Questions to ask in interviews
Technical screens for AI engineers are often too focused on machine learning theory that does not apply to product development. These questions are more useful.
"Walk me through the last time your AI feature gave wrong answers in production."
This is the most important question. How did they detect it? How did they diagnose whether it was a model issue, a prompt issue, or a retrieval issue? How did they fix it without breaking what was working? An engineer who has never encountered this problem has not shipped production AI. An engineer who cannot articulate a structured debugging approach will repeat the same cycle indefinitely.
"How do you version and deploy prompt changes?"
Prompts are application logic. They need version control, testing, and a deployment process. Engineers who change prompts by editing strings in application code and deploying have never managed a mature AI system. You want to hear about prompt configuration management, A/B testing, and rollback capability.
"What would you do if retrieval quality for our knowledge base dropped by 15% after we added 500 new documents?"
This tests retrieval diagnosis skills without requiring domain knowledge. The answer should involve checking whether the new documents changed the chunk size distribution, whether the embedding model handles the new domain well, whether the index needs retuning, and how they would measure the quality drop in the first place. If they reach for "retrain the model" immediately, they are solving the wrong problem.
"Show me how you would evaluate whether this feature is working."
Give them a concrete hypothetical: an AI that answers customer support questions about your product. What metrics would they track? How would they build the evaluation dataset? What would trigger an alert? Engineers who default to user feedback as the primary signal are operating reactively. You want someone who builds leading indicators: task completion rate, confidence scoring, escalation rate, correction rate.
What to pay
Salary ranges for AI engineers in 2026 have stabilized somewhat after the spike of 2023 to 2024, but they remain elevated relative to general software engineering.
Full-time AI engineer with 3+ years of production experience: $160,000 to $260,000 total compensation at a US-based company. Add 15 to 20% for San Francisco or New York. Senior AI engineers at well-funded companies can reach $300,000+ with equity.
Contract and fractional rates: $180 to $350 per hour for genuinely senior production experience. Anyone charging $400 or more per hour should be able to point to specific production systems at scale, named clients, and measurable outcomes. If the portfolio is demos and blog posts, the rate does not match the value.
Offshore rates in Eastern Europe and South Asia have stayed lower: $60 to $120 per hour for solid mid-level work. The quality ceiling is real; post-deployment debugging is expensive when the engineer who built it is eight time zones away and has moved to the next project.
The honest shortcut
If you are an early-stage founder who needs production AI work done, hiring a full-time AI engineer before you have validated product-market fit is usually the wrong move. The ramp time is three to four months before they are fully effective in your codebase. The salary is a fixed cost you carry regardless of whether the AI features land.
A senior fractional AI engineer who has shipped production systems across multiple industries can compress three months of ramp into three weeks, deliver the initial architecture, and leave you with a codebase that a junior engineer can maintain. That is the economic case for fractional over full-time at the early stage.
When you do need to hire full-time, use the questions and red flags above. The difference between a good hire and a bad one is not which certifications they hold or how many GitHub stars they have. It is whether they have actually debugged a broken AI system in production and built the evaluation infrastructure to catch the next one before users do.
