Diverse frame rate

Agents need not operate at the same frame rate. The BaseAgent allows a user to specify the rate at which an agent should be processed by the environment (which also afects the rate at which actions are generated by a policy).

The BaseAgentParser provides a field frame_rate which will define the rate (in Hertz) that the agent will be processed (e.g. a {frame_rate: 0.5} means that the agent will be processed every other time step and have a period == 2). By default, agent frame rates are rounded to the nearest 20th of a second.

The rate the simulation is run at, is the lcm of all the agents in the system (see multi_agent_env.py for the most up-to-date implementation):

agent_periods = [1 / agent.frame_rate for agent in agents]
sim_period = get_sim_period(agent_periods)

def get_sim_period(agent_periods: typing.List[float], max_agent_rate_hz: int = 20):
    def compute_lcm(values: typing.List[fractions.Fraction]) -> float:
        assert len(values) > 0
        lcm = values[0].denominator
        for v in values:
            lcm = lcm // math.gcd(lcm, v.denominator) * v.denominator
        return 1.0 / lcm

    agent_periods = [fractions.Fraction(p).limit_denominator(max_agent_rate_hz) for p in agent_periods]
    sim_period = compute_lcm(agent_periods)
    return sim_period

>>> get_sim_period(agent_periods=[.3, .5, .4])  # [Fraction(3, 10), Fraction(1, 2), Fraction(2, 5)]
0.1
>>> get_sim_period(agent_periods=[.33, .5, .4])  # [Fraction(1, 3), Fraction(1, 2), Fraction(2, 5)]
0.03333333333333333
>>> get_sim_period(agent_periods=[3, 2, 1]) # [Fraction(3, 1), Fraction(2, 1), Fraction(1, 1)]
1.0