Tech • IA • Crypto
The prompting playbook
6/10TL;DR
Effective prompt engineering relies on structured design, evaluation-driven iteration, and combining prompts with tools or agent workflows rather than relying on instructions alone.
KEY POINTS
Prompting remains a core engineering skill
Prompting is one of the earliest and most enduring skills in building systems with large language models. Even as models improve, performance still depends heavily on how instructions, context, and constraints are expressed. Poorly maintained prompts can degrade over time, especially when multiple contributors add patches without clear ownership.
Evaluation frameworks are essential
Reliable prompt improvement requires systematic evaluation. Test suites should include control cases, known edge cases, and boundary scenarios where the model must refuse or escalate. These evaluations help distinguish between fixable prompting issues and limitations in model capability when migrating to new systems.
Common failure in legacy prompts
Prompts often become bloated with mixed concerns such as policy, tone, and outdated fixes. In one case involving a telecom support assistant for Meridian Mobile, accumulated instructions led to degraded performance after a model change. Lack of structure and redundant constraints made it harder for the model to interpret priorities correctly.
Structured prompts improve performance
Reorganizing prompts into clearly defined sections—such as role, policy, tone, and reasoning—can significantly improve outcomes without changing core logic. Simple formatting techniques, including tagged sections, help models better distinguish between instruction types and reduce ambiguity.
Overfitting to defensive rules can backfire
Instructions designed to prevent past errors may become counterproductive with newer models. For example, a rule warning against giving incorrect plan details caused the system to withhold accurate information already available in user data. Modern models may follow such constraints too strictly, leading to underperformance.
Models may withhold correct information
While hallucination is a known issue, models can also fail by being overly cautious. In the telecom example, the system avoided answering a valid question about hotspot data despite having the correct value (5 GB) in context. This highlights the need to balance safety rules with completeness.
Instructions do not create capabilities
Simply telling a model to “calculate correctly” does not improve its ability to perform math. Introducing tools—such as a dedicated proration calculator—proved far more effective. Tool integration allows models to delegate precise tasks instead of relying on unreliable internal reasoning.
Balanced incentives guide better decisions
Models optimize based on given objectives. When instructed to avoid costly escalations (e.g., $8 per case), the system avoided handing off even when necessary. Adding the trade-off—such as the cost of refunds and loss of customer trust—led to better decisions, including correct escalation behavior.
Output consistency benefits from constraints
Defining output formats and enforcing them through mechanisms like stop sequences improves reliability. While less critical for conversational responses, structured outputs such as JSON benefit significantly from explicit formatting rules or schema enforcement.
Agent-based workflows outperform single prompts
For complex tasks like staff scheduling, splitting work into multiple steps—generation, evaluation, and repair—proved more effective than a single prompt. This approach reduced errors, token usage, and latency compared to both smaller and larger standalone models.
Model choice and architecture must align
Larger models with advanced reasoning capabilities reduced errors but increased latency and cost. In contrast, smaller models combined with better prompts or agentic loops achieved comparable or better performance more efficiently, demonstrating the importance of system design beyond model selection.
Agent loops enable flexibility
Multi-step systems allow dynamic constraints to be introduced at runtime, such as scheduling preferences or interpersonal considerations. This avoids constant backend changes and makes systems more adaptable to real-world variability.
CONCLUSION
Robust AI systems depend on disciplined prompt design, rigorous evaluation, and thoughtful integration of tools and workflows, rather than relying on increasingly complex instructions alone.