cart pole:state space - chunhualiao/public-docs GitHub Wiki

cart pole

4 Dimensions

The value of state_size in cart-pole/cart-pole.py is 4.

This is because the CartPole-v1 environment from Gym defines the state space with 4 dimensions, representing:

  • Cart Position: The horizontal position of the cart on the track.
  • Cart Velocity: The velocity of the cart.
  • Pole Angle: The angle of the pole with respect to the vertical axis.
  • Pole Angular Velocity: The angular velocity of the pole. In the code, state_size is set using the following lines:
    env = gym.make('CartPole-v1')
    state_size = env.observation_space.shape[0]

gym.make('CartPole-v1') creates an instance of the CartPole-v1 environment. env.observation_space accesses the observation space of the environment, which describes the format of the state. For CartPole-v1, it's a continuous space (Box) with a shape of (4,). env.observation_space.shape[0] retrieves the first element of the shape tuple, which is 4. This gives the number of dimensions in the state space, hence state_size becomes 4. Therefore, state_size = 4 because the state of the CartPole environment is defined by these four parameters.

Value Ranges

The value ranges for the four dimensions in the state space of CartPole-v1 are as follows:

  • Cart Position: Ranges from -4.8 to 4.8 meters.
  • Cart Velocity: Ranges from -∞ to ∞ (theoretically unbounded, but practically limited by the environment's physics and episode termination conditions).
  • Pole Angle: Ranges from approximately -24 degrees to 24 degrees (or -0.418 to 0.418 radians). This is the angle the pole makes with the vertical axis.
  • Pole Angular Velocity: Ranges from -∞ to ∞ (also theoretically unbounded, practically limited). It's important to note that while Cart Velocity and Pole Angular Velocity are theoretically unbounded, in practice, the environment is designed to terminate episodes when these values become excessively large, preventing instability in the simulation.

These ranges are inherent to the CartPole-v1 environment definition within the gym library. They define the boundaries within which the simulation operates and are crucial for understanding the state space when developing reinforcement learning agents for this environment.

Initialization

The value ranges for the CartPole-v1 environment are not configured directly within the cart-pole/cart-pole.py code you provided. These ranges are actually defined within the gym library itself, specifically in the environment definition for CartPole-v1.

The cart-pole/cart-pole.py script utilizes the pre-built CartPole-v1 environment from gym using the line:

env = gym.make('CartPole-v1')

This line imports and instantiates the CartPole-v1 environment as it is defined in the gym library. The environment's properties, including the state space ranges, are encapsulated within gym's internal implementation.

To find the exact configuration, you would need to inspect the source code of the gym library, specifically the definition of the CartPoleEnv class. This is typically found within the gym/envs/classic_control/cartpole.py file in the gym library's source code.

In summary, the ranges are pre-defined as part of the CartPole-v1 environment in the gym library, and your cart-pole/cart-pole.py code uses these default ranges by instantiating the environment from gym. You would need to look at the gym library's source code to see the precise configuration.