Skill

Review

Audit score 70

building-with-llms

refoundai/lenny-skills

Help users build effective AI applications using practical techniques from product leaders and AI practitioners.

What is building-with-llms?

This skill guides users through building with LLMs by helping diagnose problems, apply relevant prompting and architecture techniques, and establish evaluation practices. Use it when someone is writing prompts, designing AI features, implementing RAG, creating agents, running evals, or improving AI output quality.

Diagnose whether issues are prompt-related, context-related, or model-selection related
Apply specific prompting patterns like few-shot examples, decomposition, and self-criticism
Design robust architectures using context engineering, RAG, model layering, and specialized models
Establish evaluation practices with binary scoring and LLM-as-judge validation
Identify and challenge common mistakes like relying on vibes, skipping evals, or using wrong models

How to install building-with-llms

npx skills add https://github.com/refoundai/lenny-skills --skill building-with-llms

Claude Code

Cursor

Windsurf

Cline

How to use building-with-llms

1.Understand the user's use case by asking what they're building and the core user problem
2.Diagnose the root cause—is it a prompt issue, lack of context, or wrong model choice?
3.Apply relevant techniques: few-shot examples for style, decomposition for complexity, self-criticism for accuracy
4.Implement architecture patterns like context engineering and model layering for robustness
5.Establish evals with binary Pass/Fail scoring and validate any LLM-as-judge approaches
6.Challenge common mistakes like relying on vibes, skipping evaluation, or giving up after one failure

Use cases

Good for

Improving chatbot or agent accuracy through better prompting and context engineering
Implementing RAG systems by focusing on data preparation quality over vector database choice
Building code assistants or content generation tools with appropriate latency constraints
Establishing evaluation frameworks to measure model reliability before shipping to users
Debugging AI feature failures by retrying, cross-pollinating between models, or adding context

Who it's for

Product managers building AI features
Engineers implementing LLM-based systems
AI practitioners designing prompts and evaluation frameworks
Teams deploying agents or RAG systems
Anyone iterating on AI application quality

building-with-llms FAQ

Should I focus on prompt engineering or architecture?

Context engineering beats prompt engineering. If a model makes bad decisions, it's usually lack of context. Feed better data via RAG or MCP rather than just tweaking prompts.

How do I know if my AI output is good enough?

You need evals, not vibes. A 60% reliable model needs different UX than 95% or 99.5%. Use binary Pass/Fail scoring and validate with human experts if using LLM-as-judge.

What should I do when my prompt fails?

Retry the exact same prompt—models are stochastic and success rates are much higher on retry. If that doesn't work, try cross-pollinating by running the same request on a different model.

Should I use one large model or multiple specialized models?

Use a society of models with different roles. Claude Sonnet for coding, other models for critiquing. Specialized models for specialized tasks outperform one general model.

How do I improve RAG system quality?

Focus on data preparation, not vector database choice. Rewrite source data into Q&A format and add annotations for context humans take for granted.

Full instructions (SKILL.md)

Source of truth, from refoundai/lenny-skills.

name: building-with-llms description: Help users build effective AI applications. Use when someone is building with LLMs, writing prompts, designing AI features, implementing RAG, creating agents, running evals, or trying to improve AI output quality.

Building with LLMs

Help the user build effective AI applications using practical techniques from 60 product leaders and AI practitioners.

How to Help

When the user asks for help building with LLMs:

Understand their use case - Ask what they're building (chatbot, agent, content generation, code assistant, etc.)
Diagnose the problem - Help identify if issues are prompt-related, context-related, or model-selection related
Apply relevant techniques - Share specific prompting patterns, architecture approaches, or evaluation methods
Challenge common mistakes - Push back on over-reliance on vibes, skipping evals, or using the wrong model for the task

Core Principles

Prompting

Few-shot examples beat descriptions Sander Schulhoff: "If there's one technique I'd recommend, it's few-shot prompting—giving examples of what you want. Instead of describing your writing style, paste a few previous emails and say 'write like this.'"

Provide your point of view Wes Kao: "Sharing my POV makes output way better. Don't just ask 'What would you say?' Tell it: 'I want to say no, but I'd like to preserve the relationship. Here's what I'd ideally do...'"

Use decomposition for complex tasks Sander Schulhoff: "Ask 'What subproblems need solving first?' Get the list, solve each one, then synthesize. Don't ask the model to solve everything at once."

Self-criticism improves output Sander Schulhoff: "Ask the LLM to check and critique its own response, then improve it. Models can catch their own errors when prompted to look."

Roles help style, not accuracy Sander Schulhoff: "Roles like 'Act as a professor' don't help accuracy tasks. But they're great for controlling tone and style in creative work."

Put context at the beginning Sander Schulhoff: "Place long context at the start of your prompt. It gets cached (cheaper), and the model won't forget its task when processing."

Architecture

Context engineering > prompt engineering Bret Taylor: "If a model makes a bad decision, it's usually lack of context. Fix it at the root—feed better data via MCP or RAG."

RAG quality = data prep quality Chip Huyen: "The biggest gains come from data preparation, not vector database choice. Rewrite source data into Q&A format. Add annotations for context humans take for granted."

Layer models for robustness Bret Taylor: "Having AI supervise AI is effective. Layer cognitive steps—one model generates, another reviews. This moves you from 90% to 99% accuracy."

Use specialized models for specialized tasks Amjad Masad: "We use Claude Sonnet for coding, other models for critiquing. A 'society of models' with different roles outperforms one general model."

200ms is the latency threshold Ryan J. Salva (GitHub Copilot): "The sweet spot for real-time suggestions is ~200ms. Slower feels like an interruption. Design your architecture around this constraint."

Evaluation

Evals are mandatory, not optional Kevin Weil (OpenAI): "Writing evals is becoming a core product skill. A 60% reliable model needs different UX than 95% or 99.5%. You can't design without knowing your accuracy."

Binary scores > Likert scales Hamel Husain: "Force Pass/Fail, not 1-5 scores. Scales produce meaningless averages like '3.7'. Binary forces real decisions."

Start with vibes, evolve to evals Howie Liu: "For novel products, start with open-ended vibes testing. Only move to formal evals once use cases converge."

Validate your LLM judge Hamel Husain: "If using LLM-as-judge, you must eval the eval. Measure agreement with human experts. Iterate until it aligns."

Building & Iteration

Retry failures—models are stochastic Benjamin Mann (Anthropic): "If it fails, try the exact same prompt again. Success rates are much higher on retry than on banging on a broken approach."

Be ambitious in your asks Benjamin Mann: "The difference between effective and ineffective Claude Code users: ambitious requests. Ask for the big change, not incremental tweaks."

Cross-pollinate between models Guillermo Rauch: "When stuck after 100+ iterations, copy the code to a different model (e.g., from v0 to ChatGPT o1). Fresh perspective unblocks you."

Compounding engineering Dan Shipper: "For every unit of work, make the next unit easier. Save prompts that work. Build a library. Your team's AI effectiveness compounds."

Working with AI Tools

Learn to read and debug, not memorize syntax Amjad Masad: "The ROI on coding doubles every 6 months because AI amplifies it. Focus on reading code and debugging—syntax is handled."

Use chat mode to understand Anton Osika: "Use 'chat mode' to ask the AI to explain its logic. 'Why did you do this? What am I missing?' Treat it as a tutor."

Vibe coding is a real skill Elena Verna: "I put vibe coding on my resume. Build functional prototypes with natural language before handing to engineering."

Questions to Help Users

"What are you building and what's the core user problem?"
"What does the model get wrong most often?"
"Are you measuring success systematically or going on vibes?"
"What context does the model have access to?"
"Have you tried few-shot examples?"
"What happens when you retry failed prompts?"

Common Mistakes to Flag

Vibes forever - Eventually you need real evals, not just "it feels good"
Prompt-only thinking - Often the fix is better context, not better prompts
One model for everything - Different models excel at different tasks
Giving up after one failure - Stochastic systems need retries
Skipping the human review - AI output needs human validation, especially early on