ai-product-strategy
refoundai/lenny-skills
Strategic guidance for building AI products using frameworks from 94 product leaders.
What is ai-product-strategy?
Help users make informed decisions about AI product strategy, including where to apply AI, build vs. buy decisions, model selection, and human-AI boundaries. Use this when someone is planning an AI product roadmap, integrating AI into existing products, or evaluating AI capabilities for their use case.
- Clarify genuine user problems vs. AI-for-AI's-sake feature creep
- Define human-AI decision boundaries and responsibility allocation
- Guide build vs. buy and model selection architecture decisions
- Design for AI failure modes and non-determinism in production
- Plan feedback loops and evals for continuous system improvement
- Build flexible architectures that evolve with improving model capabilities
How to install ai-product-strategy
npx skills add https://github.com/refoundai/lenny-skills --skill ai-product-strategyHow to use ai-product-strategy
- 1.Understand the user's context: what they're building, what problem they're solving, and where they are in their AI journey
- 2.Help them clarify the specific user problem AI will solve, not just the technology opportunity
- 3.Guide them to define the human-AI boundary: what the AI decides vs. what humans decide
- 4.Help them plan for failure modes and design UX for when AI gets it wrong
- 5.Discuss feedback loops, evals, and observability needed from day one
- 6.Advise on building flexible architectures that can swap models as capabilities improve
Use cases
- Planning an AI product roadmap and prioritizing AI features
- Deciding whether to build custom AI capabilities or buy third-party solutions
- Designing UX that handles AI failures gracefully (e.g., when accuracy drops below acceptable thresholds)
- Setting up evaluation frameworks and observability for AI systems from day one
- Architecting multi-model systems where different models handle different tasks
- Product managers building AI products
- Engineering leaders architecting AI systems
- Founders evaluating where to apply AI in their product
- Technical decision-makers choosing between build vs. buy for AI capabilities
- Teams integrating AI into existing products
ai-product-strategy FAQ
Start by clarifying your specific problem and constraints. Consider build if you need specialized models for unique tasks, proprietary data loops, or tight human-AI boundaries. Buy if the problem is generic and you need speed to market. Most products will use a mix—specialized models for core differentiation, third-party APIs for commodity tasks.
Design for failure from day one. Even at 99% accuracy, that 1% failure rate matters if it harms users. Plan UX that gracefully degrades, surfaces uncertainty, keeps humans in the loop for high-stakes decisions, and logs failures for improvement. Set up evals and observability to catch failures early.
This is your core PM decision. Ask: What should the AI decide autonomously? What needs human review? Where do humans add irreplaceable value? The boundary depends on your domain, risk tolerance, and user expectations. Revisit it as model capabilities improve.
Build flywheels: log user actions and AI decisions, measure outcomes, use that data to retrain or fine-tune models. Set up feedback loops where users can correct AI mistakes. Establish evals and observability metrics from day one so you can measure improvement and catch regressions.
Future products use multiple models. Use specialized models for different tasks—reasoning models for complex logic, fast models for latency-sensitive tasks, domain-specific models for specialized problems. Use the right tool for each task rather than forcing one model to do everything.
Full instructions (SKILL.md)
Source of truth, from refoundai/lenny-skills.
name: ai-product-strategy description: Help users define AI product strategy. Use when someone is building an AI product, deciding where to apply AI in their product, planning an AI roadmap, evaluating build vs buy for AI capabilities, or figuring out how to integrate AI into existing products.
AI Product Strategy
Help the user make strategic decisions about AI products using frameworks from 94 product leaders and AI practitioners.
How to Help
When the user asks for help with AI product strategy:
- Understand the context - Ask what they're building, what problem they're solving, and where they are in the AI journey
- Clarify the problem - Help distinguish between "AI for AI's sake" and genuine user problems that AI can solve
- Guide architecture decisions - Help them think through build vs buy, model selection, and human-AI boundaries
- Plan for iteration - Emphasize feedback loops, evals, and building for rapid model improvements
Core Principles
Start with the problem, not the AI
Aishwarya Naresh Reganti: "In all the advancements of AI, one slippery slope is to keep thinking about solution complexity and forget the problem you're trying to solve. Start with minimal impact use cases to gain a grip on current capabilities."
Define the human-AI boundary
Adriel Frederick: "When working on algorithmic products, your job is figuring out what the algorithm should be responsible for, what people are responsible for, and the framework for making decisions." This boundary is the core PM decision.
AI is magical duct tape
Alex Komoroske: "LLMs are magical duct tape—distilled intuition of society. They make writing 'good enough' software significantly cheaper but increase marginal inference costs." Understand the new cost structure.
Build for the slope, not the snapshot
Asha Sharma: "You have to build for the slope instead of the snapshot of where you are." AI capabilities change fast—build flexible architectures that can swap models as they improve.
Design for squishiness
Alex Komoroske: "Even at 99% accuracy, if it punches the user in the face 1% of the time, that's not a viable product. Design assuming the AI will be squishy and not fully accurate."
Flywheels beat first-mover advantage
Aishwarya Naresh Reganti: "It's not about being first to have an agent. It's about building the right flywheels to improve over time." Log human actions to create data loops for system improvement.
Society of models, not single models
Amjad Masad: "Future products will be made of many different models—it's quite a heavy engineering project." Use specialized models for different tasks (reasoning vs speed vs coding).
Use the right tool for each task
Albert Cheng: "We run chess engines for evaluations. LLMs translate that into natural language. Use the right technology for the right task." Don't use LLMs where deterministic algorithms excel.
Humans are the bottleneck
Alexander Embiricos: "The current limiting factor is human typing speed and multitasking on prompts. Build systems that are 'default useful' without constant prompting."
Account for non-determinism
Aishwarya Naresh Reganti: "Most people ignore the non-determinism. You don't know how users will behave with natural language, and you don't know how the LLM will respond." Build for variability.
Agents need autonomy + complexity + natural interaction
Aparna Chennapragada: "Effective agents have (1) increasing autonomy to handle higher-order tasks, (2) ability to handle complex multi-step workflows, and (3) natural, often asynchronous interaction."
Rebuild your intuitions
Aishwarya Naresh Reganti: "Leaders have to get hands-on—not implementing, but rebuilding intuitions. Be comfortable that your intuitions might not be right." Block time daily to stay current.
Questions to Help Users
- "What specific user problem are you solving with AI?"
- "What should the AI decide vs. what should humans decide?"
- "How will you handle the 5% of cases where the AI fails?"
- "What feedback loops will improve the system over time?"
- "Are you building for today's model capabilities or anticipating improvements?"
- "Have you set up evals and observability?"
Common Mistakes to Flag
- AI for AI's sake - Adding AI features without clear user problems
- Single-model thinking - Not considering specialized models for different tasks
- Ignoring the failures - Not designing UX for when AI gets it wrong
- Static architecture - Building systems that can't evolve with model improvements
- Skipping evals - Not establishing measurement and observability from day one
- Over-automation - Removing humans from loops where they add value
Deep Dive
For all 179 insights from 94 guests, see references/guest-insights.md
Related Skills
- Building with LLMs
- AI Evals
- Evaluating New Technology
- Platform Strategy
Related skills
More from refoundai/lenny-skills and the wider catalog.
analyzing-user-feedback
Extract actionable insights from customer feedback using techniques from 56 product leaders.
behavioral-product-design
Apply behavioral science principles to product design for habit formation, retention, and reduced friction.
brand-storytelling
Craft compelling brand narratives that make your story memorable and repeatable.
building-a-promotion-case
Build a compelling promotion case by framing it as solving company problems, not personal advancement.
building-sales-team
Build and scale a high-performing sales organization using proven frameworks from 14 product leaders.
building-team-culture
Build strong team culture by articulating values, creating psychological safety, and designing for decentralized decision-making.