Create New Environments¶
In Ecole, it is possible to customize the reward or
observation returned by the environment.
taking some responsability away from the environment.
We call what is left, i.e. an environment without rewards or observations, the environment
In other words, the dynamics define the bare bone transitions of the Markov Decision Process.
Dynamics have an interface similar to environments, but with different input parameters and return types. In fact environments are wrapper around dynamics that drive the following orchestration:
Environments store the state as a
Return everything to the user.
One susbtancial difference between the environment and the dynamics is the seeding behaviour. Given that this is not an easy topic, it is discussed in Seeding.
Reset and Step¶
Creating dynamics is very similar to
creating reward and observation functions.
It can be done from scratch or by inheriting an existing one.
The following examples shows how we can inherit
deactivate cutting plane and presolve.
For directly changing SCIP parameters, directly pass them to the environment construtor.
Given that there is a large number of parameters to change, we want to use one of SCIP default mode
SCIPsetSeparating through PyScipOpt
We will do so by overriding
get called by
The similar method
step_dynamics(), which is called
step() does not need to be changed in this
example so we do not override it.
import ecole from pyscipopt.scip import PY_SCIP_PARAMSETTING class SimpleBranchingDynamics(ecole.dynamics.BranchingDynamics): def reset_dynamics(self, model): # Share memory with Ecole model pyscipopt_model = model.as_pyscipopt() pyscipopt_model.setPresolve(PY_SCIP_PARAMSETTING.OFF) pyscipopt_model.setSeparating(PY_SCIP_PARAMSETTING.OFF) # Let the parent class get the model to the root node and return # the done flag / action_set return super().reset_dynamics(model)
SimpleBranchingDynamics, we have defined what we want the solver to do.
Now, to use it as a full environent that can manage observations and rewards, we wrap it with the
class SimpleBranching(ecole.environment.Environment): __Dynamics__ = SimpleBranchingDynamics
SimpleBranching is a fully featured environment as any other in Ecole.
We can make the previous example more flexible by deciding what we want to disable. To do so, we will take parameters in the constructor
class SimpleBranchingDynamics(ecole.dynamics.BranchingDynamics): def __init__(self, disable_presolve=True, disable_cuts=True, *args, **kwargs): super().__init__(*args, **kwargs) self.disable_presolve = disable_presolve self.disable_cuts = disable_cuts def reset_dynamics(self, model): # Share memory with Ecole model pyscipopt_model = model.as_pyscipopt() if self.disable_presolve: pyscipopt_model.setPresolve(PY_SCIP_PARAMSETTING.OFF) if self.disable_cuts: pyscipopt_model.setSeparating(PY_SCIP_PARAMSETTING.OFF) # Let the parent class get the model to the root node and return # the done flag / action_set return super().reset_dynamics(model) class SimpleBranching(ecole.environment.Environment): __Dynamics__ = SimpleBranchingDynamics
The constructor arguments are forwarded from the
env = SimpleBranching(observation_function=None, disable_cuts=False)
Similarily, extra arguments given to the environemnt
step() are forwarded to the associated