Responsibilities
Finding the right customer evidence
A useful answer starts with relevant evidence. You will help us improve how we retrieve feedback for a product question, balancing fast interactive search with deeper analysis in background workflows. This includes evaluating how we prepare and chunk data, choose embeddings, retrieve candidates and rerank results.
Discovering meaningful themes at scale
Customers need to understand recurring and emerging needs across many feedback items. You will help us combine clustering, retrieval and language-model synthesis to identify well-supported themes, preserve differences between customer segments and avoid duplicate or weak conclusions. You will also investigate how to refresh this analysis efficiently as new feedback arrives.
Making suggestions more useful over time
AI can suggest tags, populate fields and connect feedback, insights and product ideas. You will help us use customer acceptance, rejection and edits to evaluate and improve these suggestions. The challenge is to learn from imperfect signals without reinforcing bias or repeatedly presenting suggestions customers have already dismissed.
What you will do
- Turn product problems into testable ML hypotheses, then design experiments that compare approaches against clear baselines.
- Build representative evaluation datasets and workflows, combining measurable checks with human judgement where quality is subjective.
- Develop and improve retrieval, clustering and language-model pipelines, working with product engineers to integrate them into customer workflows.
- Make evidence-based trade-offs between quality, latency, cost and operational complexity. Use simpler approaches when they meet the need, and evaluate fine-tuning where it offers a measurable benefit.
- Evaluate performance across customer segments and dataset sizes, including sparse and imbalanced data, rather than relying only on aggregate metrics.
- Help take capabilities from experimentation into production, with regression checks, monitoring, controlled rollouts and clear ways to diagnose failures.
- Work with product, design and engineering colleagues to interpret customer feedback and translate findings into priorities.
- Reuse shared AI infrastructure and collaborate with platform teams, while developing evaluation methods and tooling that can support other Product Collection use cases.
What success would look like
Your initial focus would be agreed with the team. A proposed starting point is finding relevant customer evidence, because it supports the quality of downstream answers and insights.
- Establish a shared baseline: We understand the most important failure patterns and can measure quality, response time and cost on representative examples.
- Improve a priority workflow: We have evidence that a selected approach improves the customer experience, with its limitations and operational trade-offs understood.
- Make improvement repeatable: The team can detect regressions, observe production behaviour and use the results to prioritise further work.
The role offers the opportunity to shape both customer-facing AI capabilities and the evaluation practices behind them, starting with Feedback and identifying useful opportunities for reuse across Product Collection.