Senior Software Engineer - AI/ML and Prompt Engineering

Software Engineer III - AI/ML, Prompt Engineer

You are a strategic thinker passionate about building prompt-driven, production-grade AI that measurably improves risk and control outcomes across enterprise workflows. You have found the right team.

As a Prompt Engineering & Applied ML Associate at JPMorgan Chase as a part of our Risk & Controls AI team, you will spend each day translating business intent into grounded, auditable decision support by combining prompt engineering with applied machine learning

 

Job responsibilities 

• Design production-grade prompts for complex enterprise workflows; test, iterate, and optimize based on outcomes.
• Apply familiarity with ML models, including classification, NLP, and transformer-based architectures. Implement LLM integration patterns such as retrieval-augmented generation (RAG), chain-of-thought prompting, and response validation.
• Define and execute prompt/model evaluation criteria (accuracy, consistency, hallucination rate, policy adherence).
• Design and run offline and online experiments to improve prompts and tune model performance.
• Implement guardrails, safety filters, and fallback strategies in production AI/ML workflows. Use ML frameworks such as PyTorch, TensorFlow, scikit-learn, and/or Hugging Face Transformers.
• Build multi-step agent workflows using LangChain or similar orchestration frameworks.
• Leverage modern databases—including vector stores (e.g., Pinecone, pgvector), graph databases, and relational/NoSQL systems—for retrieval and persistence.
• Optimize embeddings usage, tokenization strategies, and context-window management. Integrate model serving infrastructure, API-based model providers, and model routing strategies.
• Prepare data, engineer features, and curate datasets for ML training and evaluation. Monitor AI/ML-assisted production workflows with observability and drift detection practices. Translate complex business requirements into structured prompt, model, and system designs.
• Develop Python-based orchestration, data transformation, and automation scripts. Collaborate effectively with software engineers, product owners, and control stakeholders.
• Communicate clearly in writing and verbally to both technical and business audiences.

Required qualifications, capabilities, and skills

• BS/BA degree in Computer Science, Engineering, Machine Learning, Data Science, Statistics, or equivalent experience
• 5+ years of hands-on experience in machine learning, applied AI, or prompt engineering in production environments
• 5+ years of experience with Python and ML/AI tooling (model development, evaluation, and deployment)
• 3+ years of experience with LLM application development, prompt engineering, or NLP systems
• 1+ years of experience with experimentation frameworks, A/B testing, and production monitoring of AI features

Preferred qualifications, capabilities, and skills

• Experience with responsible AI practices and model governance
• Experience in financial services, technology risk, or controls-oriented environments
• Relevant ML/AI certifications (e.g., AWS ML Specialty, Google Professional ML Engineer)
• Understanding of enterprise risk, controls, and auditability expectations

Experience Level

Senior Level