Skip to content

Obs relative delta controller dict

Air Force Research Laboratory (AFRL) Autonomous Capabilities Team (ACT3) Reinforcement Learning (RL) Core.

This is a US Government Work not subject to copyright protection in the US.

The use, dissemination or disclosure of data in this file is subject to limitation or restriction. See accompanying README and LICENSE for details.

AvailablePlatforms

RelativeObsDeltaActionDict (BaseDictWrapperGlue, RelativeObsDeltaAction) ¤

RelativeObsDeltaActionDict is a glue class that wraps another glue class. It treats the actions passed to it as a delta from a linked observation E.G. if the wrapped action space has has roll as one of the controls, then a delta action of 0.2 would move the absolute roll position 0.2 higher than it is as measured by the linked roll sensor.

Source code in corl/glues/controller_wrappers/obs_relative_delta_controller_dict.py 
class RelativeObsDeltaActionDict(BaseDictWrapperGlue, RelativeObsDeltaAction):  # type: ignore[misc]
    """
    RelativeObsDeltaActionDict is a glue class that wraps another glue class.
    It treats the actions passed to it as a delta from a linked observation
    E.G. if the wrapped action space has has roll as one of the controls, then a delta action of
    0.2 would move the absolute roll position 0.2 higher than it is as measured by the linked roll sensor.
    """

    def __init__(self, **kwargs) -> None:  # pylint: disable=super-init-not-called
        self.config: RelativeObsDeltaActionDictValidator  # type: ignore[assignment]
        BaseDictWrapperGlue.__init__(**kwargs)
        self._logger = logging.getLogger(RelativeObsDeltaActionDict.__name__)

        controller_keys = self.glues().keys()
        if 'controller' not in controller_keys:
            raise KeyError('Missing key: controller')
        if 'sensor' not in controller_keys:
            raise KeyError('Missing key: sensor')

        self.controller: ControllerGlue = typing.cast(ControllerGlue, self.glues()["controller"])
        if not isinstance(self.controller, ControllerGlue):
            raise RuntimeError(
                "Error: RelativeObsDeltaActionDict expects the glue wrapped on the 'controller' key to be a ControllerGlue, "
                f"got {self.controller}"
            )
        self.relative_obs_glue: ObserveSensor = typing.cast(ObserveSensor, self.glues()["sensor"])
        if not isinstance(self.relative_obs_glue, ObserveSensor):
            raise RuntimeError(
                "Error: RelativeObsDeltaActionDict expects the glue wrapped on the 'sensor' key to be a ObserveSensor, "
                f"got {self.relative_obs_glue}"
            )

        # verify that the config setup is not going to get the user into a situation where they are
        # only accessing one part of the obs but applying that obs as the base position for multiple actions
        if self.config.obs_index and len(list(self.controller.action_space().values())[0].low) != 1:
            raise RuntimeError(
                f"ERROR: your glue {self.get_unique_name()} has an action space length of more than 1, "
                "but you specified though obs_index to access only 1 component of the obs "
                "from the wrapped observe Sensor, to fix this error in your config for this glue define 'obs_index': null"
            )

        self.step_size = EnvSpaceUtil.convert_config_param_to_space(
            action_space=self.controller.action_space(), parameter=self.config.step_size
        )

        self._is_wrap = self.config.is_wrap

        self.saved_action_deltas = OrderedDict()
        for space_name, space in self.action_space().items():
            if self.config.initial_value is not None:
                self.saved_action_deltas[space_name] = np.asarray([self.config.initial_value], dtype=np.float32)
            else:
                self.saved_action_deltas[space_name] = space.low

    @property
    def get_validator(self) -> typing.Type[RelativeObsDeltaActionDictValidator]:  # type: ignore[override]
        return RelativeObsDeltaActionDictValidator

get_validator: Type[corl.glues.controller_wrappers.obs_relative_delta_controller_dict.RelativeObsDeltaActionDictValidator] property readonly ¤

returns the validator for this class

Returns:

Type Description
Type[corl.glues.controller_wrappers.obs_relative_delta_controller_dict.RelativeObsDeltaActionDictValidator]

BaseAgentGlueValidator -- A pydantic validator to be used to validate kwargs

RelativeObsDeltaActionDictValidator (BaseDictWrapperGlueValidator) pydantic-model ¤

A dict that contains a floating point scalar for each action in the action space,

    by which the corresponding delta action is scaled prior to converting the action
    to the wrapped space.
    e.g. A throttle DeltaAction.apply_action([0.2]) with step_size=[.05] would move the
    absolute throttle position to 0.01 higher than it was at the end of the last step.

Source code in corl/glues/controller_wrappers/obs_relative_delta_controller_dict.py 
class RelativeObsDeltaActionDictValidator(BaseDictWrapperGlueValidator):
    """
    step_size:      A dict that contains a floating point scalar for each action in the action space,
                    by which the corresponding delta action is scaled prior to converting the action
                    to the wrapped space.
                    e.g. A throttle DeltaAction.apply_action([0.2]) with step_size=[.05] would move the
                    absolute throttle position to 0.01 higher than it was at the end of the last step.
    """
    step_size: float = 1.0
    obs_index: typing.Optional[int] = 0
    is_wrap: bool = False
    initial_value: typing.Optional[float] = None

    @validator("step_size")
    @classmethod
    def check_step_scale(cls, v):
        """
        verifies range of step scale values
        """
        if v >= 1.0 or v < 0:
            raise ValueError("RelativeObsDeltaActionValidator got step size of more that 1.0 or less than 0")
        return v

check_step_scale(v) classmethod ¤

verifies range of step scale values

Source code in corl/glues/controller_wrappers/obs_relative_delta_controller_dict.py 
@validator("step_size")
@classmethod
def check_step_scale(cls, v):
    """
    verifies range of step scale values
    """
    if v >= 1.0 or v < 0:
        raise ValueError("RelativeObsDeltaActionValidator got step size of more that 1.0 or less than 0")
    return v