The protocol expected to define a valid reward function is given below.
Class repsonsible for extracting rewards.
Reward functions are objects given to the
EnvironmentComposerto extract the reward used for learning.
obtain_reward(model: ecole.scip.Model, done: bool) → float¶
Extract reward for arriving on given state.
Extract the reward for arriving on the state given by
model. A reward is typically computed by transitioning from a state
S1to a state
S2. For perfomance reasons, intermediate states are not kept. The reward function is reponsible for keeping track of relevant information from previous states. This can safely be done in this method as it will only be called once per state i.e., this method is not a getter and can have side effects.
model – The SCIP model defining the current state of the solver.
done – A flag indicating wether the state is terminal (as decided by the environment).
The return is passed to the user by the environment.
The list of reward functions relevant to users is given below.
LP Iteration difference.
The reward is defined as the number of iterations spent in solving the Linear Programs associated with the problem since the previous state.
obtain_reward(self: ecole.reward.LpIterations, model: ecole.scip.Model, done: bool = False) → float¶
Update the internal LP iteration count and return the difference.
The difference in LP iterations is computed in between calls.
Number of nodes difference.
The reward is defined as the total number of nodes processed since the previous state.
obtain_reward(self: ecole.reward.NNodes, model: ecole.scip.Model, done: bool = False) → float¶
Update the internal node count and return the difference.
The difference in number of nodes is computed in between calls.
The following reward functions are used internally by Ecole.
Proxy class for doing arithmetic on reward functions.
An object of this class is returned by reward function operators to forward calls to the reward function parameters of the operator.
obtain_reward(self: ecole.reward.Arithmetic, model: object, done: bool = False) → float¶
Obtain the reward of result of the operator.
obtain_rewardon all reward function parameters that were used to create this object and compute the operation on the results.