Senior Deep Learning Engineer, Accuracy Evaluation
We are seeking senior engineers to pioneer new methodologies for accurately assessing the performance and capabilities of ground-breaking deep learning models, including LLMs, RAG, agents, and vision models. You will collaborate across the organization to bring the latest flagship models from our community and partners to life. This role offers an outstanding opportunity to craft the future of AI at a fast-growing company at the forefront of the AI revolution. Join our team of world-class software engineers and partners to deliver the most advanced models with lightning-fast inference. You'll work on the most powerful, enterprise-grade GPU clusters capable of hundreds of PetaFLOPS and gain early access to unreleased hardware, making a direct impact on NVIDIA's roadmap and the broader AI landscape!
What you’ll be doing:
Design and build decision-grade evaluation environments for NVIDIA's frontier models spanning reasoning, multimodal, long-context, and agentic systems, producing auditable accuracy signals that gate every major model release.
Research and develop novel evaluation methodologies for emerging model families and capability domains (low-precision numerics, multi-turn agentic tasks, code generation) where established benchmarks don't yet exist or don't generalize.
Build and operate the evaluation infrastructure and pipelines including benchmark environments, regression CI systems, and statistical analysis tooling, used by model, product, and applied research teams across NVIDIA.
Partner with model research, training, and customer teams to translate evaluation signals into concrete decisions: release go/no-go, training iteration direction, and competitive positioning against external frontier models.
What we need to see:
BS, MS, or PhD in Computer Science, Machine Learning, Statistics, or a related field
6+ years of hands-on experience with LLMs, designing and running evaluations for large language models or multimodal AI systems, including experience with agentic, multi-turn, or reasoning-heavy settings.
Strong statistical foundations: experimental design, significance testing, regression analysis, and the ability to distinguish signal from noise in benchmark results at scale.
Proven experience building evaluation infrastructure (pipelines, benchmark harnesses, reproducible CI systems) not just consuming existing benchmarks.
Clear, precise communicator who can translate quantitative evaluation results into decisions for researchers, product teams, and senior leadership.
Ways to stand out from the crowd:
Deep familiarity with open-source evaluation frameworks
Experience designing evaluations for agentic systems: tool use, multi-turn reasoning, environment-based benchmarks (SWE-bench, GAIA, WebArena-style), or interactive evaluation settings.
Track record of publishing or contributing to evaluation research, new benchmark design, methodology papers, or reproducibility analyses that shaped how the field measures model capability.
Experience measuring model accuracy under low-precision inference (FP8, INT4, quantization-aware settings) and understanding how calibration and sparsity interact with benchmark results.
Comfort running large-scale workloads on HPC/Slurm clusters, including reproducible experiment management (MLflow, W&B) and compute cost optimization across hundreds of benchmark runs.
NVIDIA is committed to fostering a diverse work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.
Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. For Poland: The base salary range is 375,000 PLN - 650,000 PLN.


Similar roles
Senior AI Engineer (m/f/d)
Artefact·Berlin, Germany
<p><strong>Artefact</strong>, a global AI and data consulting company with rapid growth, is looking for a creative and tech-savvy <strong>Senior AI Engineer (m/f/d)</strong> to join the tech team full-time. Ideally, you are <strong>located in Hamburg, Munich, Berlin</strong> or nearby - but if you're based elsewhere in Germany and a great fit, we would still love to hear from you. Could this…
- On-site
- Full-time
- Blue Card — tied; settle 21–33mo
Senior Data Analyst - Reporting & Insights (m/f/d)
Statista·Hamburg, Germany
At Statista, we’re all about facts and data, for we are the world's leading business data platform. By providing reliable and easy-to-use data as well as various data analytics products and services, we empower people worldwide to make fact-based decisions. Founded in Hamburg in 2007, we have quickly grown into a global company with offices in major cities such as…
- On-site
- Full-time
- Blue Card — tied; settle 21–33mo
Architekt IT [K M]
Fabrity S.A.·Poland
O FIRMIE: Fabrity S.A. to dynamicznie rozwijająca się firma technologiczna, specjalizująca się w inżynierii oprogramowania, generatywnej sztucznej inteligencji oraz przemysłowym Internecie rzeczy (IoT). Tworzymy kompleksowe rozwiązania, które pomagają przedsiębiorstwom automatyzować procesy, zwiększać efektywność operacyjną oraz podejmować kluczowe decyzje dzięki skutecznej analizie danych. Usprawniamy procesy biznesowe, wykorzystując wiodące platformy, takie jak ServiceNow, Microsoft Power Platform oraz nasz autorski produkt low-code, Fastive.…
- Full-time
- PLN 302k–PLN 343k / yr
Programista Developer Power BI
Antal Sp. z o.o.·Poland
Power BI Developer (Azure / Microsoft Fabric) Współpraca B2B przez Antal Model hybrydowy – 1 dzień w tygodniu z biura Długofalowy projekt transformacji danych i rozwiązań analitycznych w środowisku chmurowym Azure Dołącz do zespołu odpowiedzialnego za modernizację platform danych i rozwój zaawansowanych rozwiązań analitycznych. Poszukujemy doświadczonego Power BI Developera, który będzie uczestniczył w migracji środowisk on-premise do chmury oraz tworzeniu…
- Full-time
- PLN 282k–PLN 343k / yr
Mid-Level Program Analyst – Leadership Development
ITDS·Poland
Drive impactful leadership transformation — empower future-ready organizations with innovative development programs! Warszawa-based opportunity with hybrid work model. As a Mid-Level Program Analyst – Leadership Development, you will be working for our client, a prominent leader in the financial services industry. You will contribute to building leadership capability across critical enterprise levels, helping drive business performance through innovative learning initiatives.…
- Full-time
- PLN 139k–PLN 171k / yr
Senior Salesforce Developer Field Service
EPAM Systems·Poznań, Poland
We are seeking a Senior Salesforce Developer to lead the delivery of complex Salesforce Field Service Solutions, providing hands-on development, technical guidance and quality assurance while supporting other developers. Responsibilities Build complex solutions using Salesforce configuration, Screen Flows, record-triggered and scheduled Flows, Apex, batch processing and LWCConfigure Salesforce Field Service scheduling policies, work rules and service objectivesSet up territories, resources…
- Full-time
Account Director
Ntiative·Poland
Account Director | 100% Remote | Employment Contract (UoP) Are you passionate about building long-term client relationships, delivering exceptional B2B marketing campaigns, and creating memorable event experiences? We're looking for an Account Director (or an experienced Senior Account Manager ready for the next step) to join an international, fully remote B2B marketing agency specialising in the technology sector. This is…
- Full-time
Engineering Manager - Machine Learning (AI Products)
Xero·Parnell, New Zealand
The role & impact As Engineering Manager, you'll lead a high-performing, inclusive team of ML and platform engineers, responsible for turning research and models into production-grade ML infrastructure that runs at scale. You'll balance predictable delivery with a relentless focus on system health, taking the lead on MLOps maturity, observability and the operational realities of running AI systems in production,…
- Full-time
- Skilled Migrant (Residence) — PR, points-based, no single employer
