Recent experience is solid but lacks the depth and duration needed for this role. Would prefer someone with at least 3 years of direct experience in this area. The position requires leading AI strategy and calls for a more tenured ML/AI background.
A lot of relevant AI/ML experience on paper, including RAG, fine-tuning, inference services, and MLOps, but not convinced by the underlying production software engineering depth. The background is more Data Science/ML to AI Engineer, whereas we have been consistently favoring strong production engi…
Likely to be viewed as a strong production ML engineer who has recently transitioned into GenAI, rather than the senior, product-focused Applied AI engineer the team is seeking.
Tweets, press, and people that show why this team is worth your time.
Kastle is hiring its Applied AI Engineer. This person is actually going to grow into the Head of AI. The hire owns Kastle's entire AI system — which models Kastle uses, how it evaluates them, how the voice agents self-improve from every customer interaction and human feedback loop.
ML researchers, pure ML engineers, and PhD-track profiles have been the dominant failure archetype. The right archetype: applied AI engineer with production shipping track record, evaluation-mindset, model-picking intuition, and LLM training / fine-tuning fluency. Systems design and coding chops are required. Depth in fundamentals over depth in research.
What You'll Own
- AI Model Integration: fine-tune and deploy LLMs for real-time voice-based borrower interactions
- Prompt Engineering: design and iterate on prompt strategies to optimize AI performance and compliance
- Evaluate LLMs and AI Agents: build high-quality evals using Kastle's proprietary datasets to benchmark AI agent performance and run experiments
- Custom AI Solutions: train and implement domain-specific AI models for consumer lending
- Scalability & Compliance: ensure AI solutions meet regulatory standards (FDCPA, RESPA, TILA) and scale efficiently
- Data Pipelines & APIs: build robust AI-driven workflows that integrate seamlessly with loan servicing platforms
- Continuous Optimization: monitor performance metrics, continuously improve agent quality and borrower experience
Requirements
- 4-8 years of production applied AI / ML engineering experience
- Deployed real production AI systems that real users depend on — not notebooks, not research demos
- Strong Python + deep learning framework fluency (TensorFlow, PyTorch)
- LLM orchestration framework experience (LangChain, LlamaIndex, DSPy, Instructor, or comparable)
- Fine-tuning + eval mindset — has personally fine-tuned models against proprietary datasets, built eval harnesses, run experiments, iterated to metric targets
- Real production shipping track record
- Product mindset — translates business requirements into AI solutions
- Strong communication
- SF-based or willing to relocate — 5 days/week Visa: H-1B transfer OK, TN OK (no OPT; O-1 too slow)
Profile (Megan's priorities, decoded from the intake) Tier 1 — Applied AI Engineers at AI Agent Companies Sierra, Decagon, Glean, Cresta, Observe.AI, Rox, Parloa, Regal.io — direct competitive shape OpenAI Applied / Solutions, Anthropic Applied Voice AI companies: Retell, Vapi, Deepgram, Cartesia, ElevenLabs, PolyAI, Bland, Air.ai, HappyRobot — direct match for real-time voice AI production shipping Perplexity Applied, Cursor Applied
Tier 2 — Applied AI Engineers with Real Evaluation / Fine-Tuning Track Record Scale AI Applied, Snorkel AI, Mercor applied Adept, Cohere Applied, AI21 Applied Weights & Biases (production users), Braintrust, Comet ML, Arize AI (applied side, not research) Together AI, Fireworks AI, Modal, Replicate — applied AI shipping Instructor, Guardrails AI, LangSmith production users
Tier 3 — Ex-YC Founders / Founding Engineers Who Shipped LLM Products YC W22-W26 batches — founders of LLM / AI-agent products that grew Founding engineers at vertical AI companies that reached Series A
Tier 4 — Compliance-Heavy Vertical AI Applied Engineers Legal AI: Harvey, Eve, Spellbook, Ironclad, Klarity Healthcare AI: Perspectives Health, Sully, Heidi, Written, Ease, Abridge, Ambience Financial AI: Ocrolus, Blend, Truework, Palantir Foundry FS Compliance AI: Kharon, Norm AI, Klarity
Tier 5 — Applied ML at Fast-Moving Tech Companies with Production LLM Systems Meta AI applied (not research), Google AI applied (not research) — with production shipping evidence Amazon Alexa applied AI Microsoft AI applied
Strong Bonuses
- Prior startup or founder experience
- Fine-tuning production LLMs on proprietary datasets
- Eval harness architecture experience
- RLHF / RLAIF pipelines shipped in production
- Voice AI, telephony, real-time media applied experience
- Compliance-heavy production AI (FDCPA / RESPA / TILA / HIPAA / SOC 2 / PCI)
- Applied AI blog posts, talks, OSS
Location and Visa
- H-1B transfer OK, TN OK
- OPT is a hard no
- O-1 acceptable but pipeline too slow to prioritize