State, Action and Reward Space

This page was last edited on 11 November 2025

Space is a fundamental concept in understanding Reinforcement Learning. We will encounter the term space in defining the environment, in code, in the selection and implementation of algorithms, and in the interpretation of results.

From a strictly mathematical point of view, a space is a set of n elements:

• It can be discrete (a finite set of elements): {0, 1, 2, 3}
• It can be continuous (an interval or an infinite set): [0.0, 1.0]
• It can be n-dimensional (a vector in space): (position, velocity, temperature)

In a Reinforcement Learning application, a space defines the range of possible values for certain essential elements, such as:

Type of space	Description
State Space	The set of all possible states of the environment. Examples: position, speed, temperature, etc.
Action Space	The set of all possible actions the agent can take.
Observation Space	In POMDP or partially observable RL, it is what the agent actually perceives.
Reward Space	The range of possible reward values.

ANALOGIES

1. In a video game:
   • State Space: all the places where the character can be (the game map).
   • Action Space: all available commands (jump, left, right, attack).
   • Observation Space: what you see on the screen, which is just part of the whole map.
   • Reward Space: the score you can receive when you accomplish something in the game
2. Lego box:
   • State Space: all possible combinations of pieces on the table (position, color, size).
   • Action Space: you can place a piece, move it, remove it, etc.
   • Observation Space: you only see part of the table (if you’re looking through a camera).
   • Reward Space: how close the result is to the desired final model.

The relationship between space and learning

Spaces define:

• How complex the problem is (small space = easier learning).
• What kind of algorithms you can use (Q-Learning doesn’t work with continuous actions).
• How you map reality into an RL model.

Why is it essential to understand spaces?

• It helps you correctly define the RL environment.
• It influences your choice of algorithm (e.g., classic algorithms vs. algorithms for continuous actions).
• It allows you to efficiently map the real world into an RL model.
• It helps you properly structure neural networks.

In conclusion, space is not just a technical detail — it is the foundation on which the agent learns and explores the world.

---

References:

• “Welcome to Spinning Up in Deep RL!” – Open AI.
• “An Introduction to Reinforcement Learning” by Deepanshu Tiwari

---

Gradients << Previous | Next >> Normalization