World's Best Scientists 2026 revealed!

D-Index & Metrics

Computer Science

D-Index
63
Citations
34867
World Ranking
2674
National Ranking
157

Overview

Nicolas Heess is affiliated with DeepMind in the United Kingdom and specializes in research at the intersection of computer science and artificial intelligence. Their primary focus is on reinforcement learning and its applications within robotics, particularly in areas involving robotic manipulation, locomotion, and control.

The main fields of study for Heess include:

  • Computer Science

Within these fields, Heess's subfields of expertise cover:

  • Artificial Intelligence
  • Control and Systems Engineering
  • Computer Vision and Pattern Recognition
  • Biomedical Engineering
  • Computational Theory and Mathematics

The major topics addressed in their work consist of:

  • Reinforcement Learning in Robotics
  • Robot Manipulation and Learning
  • Robotic Locomotion and Control
  • Adversarial Robustness in Machine Learning
  • Human Pose and Action Recognition
  • Sports Analytics and Performance
  • Multimodal Machine Learning Applications

Heess has contributed to numerous publications, with frequent venues including:

  • arXiv (Cornell University)
  • Science Robotics
  • Zenodo (CERN European Organization for Nuclear Research)
  • Software Impacts
  • ACM Transactions on Graphics

Among recent papers, the following are notable:

  • Understanding the Impact of Value Selection Heuristics in Scheduling Problems, 2025, arXiv (Cornell University)
  • dm_control: Software and tasks for continuous control, 2020, Software Impacts
  • Distral: Robust Multitask Reinforcement Learning, 2025, Oxford University Research Archive (ORA) (University of Oxford)
  • Learning agile soccer skills for a bipedal robot with deep reinforcement learning, 2024, Science Robotics
  • Catch & Carry, 2020, ACM Transactions on Graphics

Heess's research collaborations often involve recurring coauthors, including:

  • Leonard Hasenclever
  • Martin Riedmiller
  • Markus Wulfmeier
  • Abbas Abdolmaleki
  • Jost Tobias Springenberg

Best Publications

  • Continuous control with deep reinforcement learning

    Timothy P. Lillicrap;Jonathan J. Hunt;Alexander Pritzel;Nicolas Heess

  • Recurrent Models of Visual Attention

    Volodymyr Mnih;Nicolas Heess;Alex Graves;koray kavukcuoglu

  • Relational inductive biases, deep learning, and graph networks

    Peter W. Battaglia;Jessica B. Hamrick;Victor Bapst;Alvaro Sanchez-Gonzalez

  • Deterministic Policy Gradient Algorithms

    David Silver;Guy Lever;Nicolas Heess;Thomas Degris

  • Emergence of Locomotion Behaviours in Rich Environments

    Nicolas Heess;Dhruva Tb;Srinivasan Sriram;Jay Lemmon

  • FeUdal Networks for Hierarchical Reinforcement Learning

    Alexander Sasha Vezhnevets;Simon Osindero;Tom Schaul;Nicolas Heess

  • Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards

    Matej Vecerík;Todd Hester;Jonathan Scholz;Fumin Wang

  • A Generalist Agent

    Unknown

  • Learning continuous control policies by stochastic value gradients

    Nicolas Heess;Greg Wayne;David Silver;Timothy Lillicrap

  • Sample Efficient Actor-Critic with Experience Replay.

    Ziyu Wang;Victor Bapst;Nicolas Heess;Volodymyr Mnih

  • Sim-to-Real Robot Learning from Pixels with Progressive Nets

    Andrei A. Rusu;Matej Vecerík;Thomas Rothörl;Nicolas Heess

  • Attend, infer, repeat: fast scene understanding with generative models

    S. M. Ali Eslami;Nicolas Heess;Theophane Weber;Yuval Tassa

  • Distributed Distributional Deterministic Policy Gradients

    Gabriel Barth-Maron;Matthew W. Hoffman;David Budden;Will Dabney

  • Graph Networks as Learnable Physics Engines for Inference and Control

    Alvaro Sanchez-Gonzalez;Nicolas Heess;Jost Tobias Springenberg;Josh Merel

  • Distral: robust multitask reinforcement learning

    Yee Whye Teh;Victor Bapst;Wojciech Marian Czarnecki;John Quan

  • Unsupervised Learning of 3D Structure from Images

    Danilo Jimenez Rezende;S. M. Ali Eslami;Shakir Mohamed;Peter W. Battaglia

  • Learning by Playing - Solving Sparse Reward Tasks from Scratch

    Martin A. Riedmiller;Roland Hafner;Thomas Lampe;Michael Neunert

  • Gradient estimation using stochastic computation graphs

    John Schulman;Nicolas Heess;Theophane Weber;Pieter Abbeel

  • Reinforcement and Imitation Learning for Diverse Visuomotor Skills

    Yuke Zhu;Ziyu Wang;Josh Merel;Andrei A. Rusu

  • Learning generative texture models with extended Fields-of-Experts

    Nicolas Heess;Christopher K. I. Williams;Geoffrey E. Hinton

  • Imagination-Augmented Agents for Deep Reinforcement Learning

    Sébastien Racanière;Theophane Weber;David P. Reichert;Lars Buesing

  • Maximum a Posteriori Policy Optimisation

    Abbas Abdolmaleki;Jost Tobias Springenberg;Yuval Tassa;Rémi Munos

  • Learning an Embedding Space for Transferable Robot Skills

    Karol Hausman;Jost Tobias Springenberg;Ziyu Wang;Nicolas Heess

  • dm_control: Software and Tasks for Continuous Control

    Yuval Tassa;Saran Tunyasuvunakool;Alistair Muldal;Yotam Doron

Frequent Co-Authors

Martin Riedmiller
Martin Riedmiller DeepMind (United Kingdom)
Jost Tobias Springenberg
Jost Tobias Springenberg University of Freiburg
Yuval Tassa
Yuval Tassa Google (United States)
Razvan Pascanu
Razvan Pascanu DeepMind (United Kingdom)
Yee Whye Teh
Yee Whye Teh University of Oxford
Timothy P. Lillicrap
Timothy P. Lillicrap University College London
David Silver
David Silver DeepMind (United Kingdom)
Raia Hadsell
Raia Hadsell DeepMind (United Kingdom)
Rémi Munos
Rémi Munos French Institute for Research in Computer Science and Automation - INRIA
Nando de Freitas
Nando de Freitas DeepMind (United Kingdom)

External Links

If you think any of the details on this page are incorrect, let us know.

Report an issue

We appreciate your kind effort to assist us to improve this page, it would be helpful providing us with as much detail as possible in the text box below:

Related Online Degrees & Career Pathways

As you consider studying Computer Science in the USA, a range of related online degrees and career pathways can broaden your options. Many online schools that accept low gpa are available, making it easier for students to pursue a tech-driven education even if their academic journey has not always been perfect. This flexibility allows more people to access accredited programs and kickstart their careers.

If you are eager to earn your degree faster, exploring a computer science accelerated program can help you enter the workforce quickly. These intensive programs are ideal for motivated students ready to dive into in-demand fields.

Computer Science can also open doors to interdisciplinary careers. For example, graduates with interests in sustainability can benefit from paired or dual degrees in environmental sciences. There are many high-paying jobs with environmental science degree, especially in roles that combine tech with environmental stewardship.

Affordability is a key factor for many students. Consider programs like the environmental engineering degree online, which offer budget-friendly tuition rates and the potential for impactful, technology-driven careers in sustainability.

Best Scientists Citing Nicolas Heess

Trending Scientists