Create New Environments

Environment Structure

In Ecole, it is possible to customize the reward or observation returned by the environment. The RewardFunction and ObservatioFunction are taking some responsability away from the environment. We call what is left, i.e. an environment without rewards or observations, the environment Dynamics. In other words, the dynamics define the bare bone transitions of the Markov Decision Process.

Dynamics have an interface similar to environments, but with different input parameters and return types. In fact environments are wrapper around dynamics that drive the following orchestration:

• Environments store the state as a Model;

• Forward the Model to the Dynamics to start a new episode or transition to receive an action set;

• Forward the Model to the RewardFunction and ObservationFunction to recieve an observation and reward;

• Return everything to the user.

One susbtancial difference between the environment and the dynamics is the seeding behaviour. Given that this is not an easy topic, it is discussed in Seeding.

Creating Dynamics

Reset and Step

Creating dynamics is very similar to creating reward and observation functions. It can be done from scratch or by inheriting an existing one. The following examples shows how we can inherit BranchingDynamics to deactivate cutting plane and presolve.

Note

For directly changing SCIP parameters, directly pass them to the environment construtor.

Given that there is a large number of parameters to change, we want to use one of SCIP default mode by calling SCIPsetPresolving and SCIPsetSeparating through PyScipOpt (SCIP doc).

We will do so by overriding reset_dynamics(), which get called by reset(). The similar method step_dynamics(), which is called by step() does not need to be changed in this example so we do not override it.

import ecole
from pyscipopt.scip import PY_SCIP_PARAMSETTING


class SimpleBranchingDynamics(ecole.dynamics.BranchingDynamics):

    def reset_dynamics(self, model):
        # Share memory with Ecole model
        pyscipopt_model = model.as_pyscipopt()

        pyscipopt_model.setPresolve(PY_SCIP_PARAMSETTING.OFF)
        pyscipopt_model.setSeparating(PY_SCIP_PARAMSETTING.OFF)

        # Let the parent class get the model to the root node and return
        # the done flag / action_set
        return super().reset_dynamics(model)

With the SimpleBranchingDynamics, we have defined what we want the solver to do. Now, to use it as a full environent that can manage observations and rewards, we wrap it with the Environment.

class SimpleBranching(ecole.environment.Environment):
    __Dynamics__ = SimpleBranchingDynamics

SimpleBranching is a fully featured environment as any other in Ecole.

Passing parameters

We can make the previous example more flexible by deciding what we want to disable. To do so, we will take parameters in the constructor

class SimpleBranchingDynamics(ecole.dynamics.BranchingDynamics):

    def __init__(self, disable_presolve=True, disable_cuts=True, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.disable_presolve = disable_presolve
        self.disable_cuts = disable_cuts

    def reset_dynamics(self, model):
        # Share memory with Ecole model
        pyscipopt_model = model.as_pyscipopt()

        if self.disable_presolve:
            pyscipopt_model.setPresolve(PY_SCIP_PARAMSETTING.OFF)
        if self.disable_cuts:
            pyscipopt_model.setSeparating(PY_SCIP_PARAMSETTING.OFF)

        # Let the parent class get the model to the root node and return
        # the done flag / action_set
        return super().reset_dynamics(model)


class SimpleBranching(ecole.environment.Environment):
    __Dynamics__ = SimpleBranchingDynamics

The constructor arguments are forwarded from the __init__() constructor:

env = SimpleBranching(observation_function=None, disable_cuts=False)

Similarily, extra arguments given to the environemnt reset() and step() are forwarded to the associated Dynamics methods.