ScipyTrustConstrV1#
- class zfit.minimize.ScipyTrustConstrV1(tol=None, init_trust_radius=None, gradient=None, hessian=None, verbosity=None, maxiter=None, criterion=None, strategy=None, name='SciPy trust-constr V1')[source]#
Bases:
ScipyBaseMinimizerV1
Trust-region based local minimizer.
This implenemtation wraps the minimizers in SciPy optimize.
- Parameters
tol (
Optional
[float
]) – Termination value for the convergence/stopping criterion of the algorithm in order to determine if the minimum has been found. Defaults to 1e-3.init_trust_radius (
Optional
[int
]) – Initial trust-region radius.gradient (
Union
[str
,Callable
,None
]) –Define the method to use for the gradient computation that the minimizer should use. This can be the gradient provided by the loss itself or method from the minimizer. In general, using the zfit provided automatic gradient is more precise and needs less computation time for the evaluation compared to a numerical method, but it may not always be possible. In this case, zfit switches to a generic, numerical gradient which in general performs worse than if the minimizer has its own numerical gradient. The following are possible choices:
If set to
False
or'zfit'
(orNone
; default), the gradient of the loss (usually the automatic gradient) will be used; the minimizer won’t use an internal algorithm.True
tells the minimizer to use its default internal gradient estimation. This can be specified more clearly using the arguments'2-point'
and'3-point'
, which specify the numerical algorithm the minimizer should use in order to estimate the gradient.hessian (
Union
[None
,Callable
,str
,HessianUpdateStrategy
]) –Define the method to use for the hessian computation that the minimizer should use. This can be the hessian provided by the loss itself or method from the minimizer.
While the exact gradient can speed up the convergence and is often beneficial, this ain’t true for the computation of the (inverse) Hessian matrix. Due to the \(n^2\) number of entries (compared to \(n\) in the gradient) from the \(n\) parameters, this can grow quite large and become computationally expensive.
Therefore, many algorithms use an approximated (inverse) Hessian matrix making use of the gradient updates instead of calculating the exact matrix. This turns out to be precise enough and usually considerably speeds up the convergence.
The following are possible choices:
If set to
False
or'zfit'
, the hessian defined in the loss (usually using automatic differentiation) will be used; the minimizer won’t use an internal algorithm.AHessianUpdateStrategy
that holds an approximation of the hessian. For exampleBFGS
(which performs usually best) orSR1
(sometimes unstable updates).True
(orNone
; default) tells the minimizer to use its default internal hessian approximation. Arguments'2-point'
and'3-point'
specify which numerical algorithm the minimizer should use in order to estimate the hessian. This is only possible if the gradient is provided by zfit and not an internal numerical method is already used to determine it.Verbosity of the minimizer. Has to be between 0 and 10. The verbosity has the meaning:
a value of 0 means quiet and no output
above 0 up to 5, information that is good to know but without flooding the user, corresponding to a “INFO” level.
A value above 5 starts printing out considerably more and is used more for debugging purposes.
Setting the verbosity to 10 will print out every evaluation of the loss function and gradient.
Some minimizers offer additional output which is also distributed as above but may duplicate certain printed values.
maxiter (
Union
[int
,str
,None
]) – Approximate number of iterations. This corresponds to roughly the maximum number of evaluations of thevalue
, ‘gradient`` orhessian
.criterion (
Optional
[ConvergenceCriterion
]) – Criterion of the minimum. This is an estimated measure for the distance to the minimum and can include the relative or absolute changes of the parameters, function value, gradients and more. If the value of the criterion is smaller thanloss.errordef * tol
, the algorithm stopps and it is assumed that the minimum has been found.strategy (
Optional
[ZfitStrategy
]) – A class of typeZfitStrategy
that takes no input arguments in the init. Determines the behavior of the minimizer in certain situations, most notably when encountering NaNs. It can also implement a callback function.name (
str
) – Human-readable name of the minimizer.
- create_criterion(loss=None, params=None)#
Create a criterion instance for the given loss and parameters.
- Parameters
loss (
Optional
[ZfitLoss
]) – Loss that is used for the criterion. Can be None if called inside_minimize
params (
Optional
[Iterable
[TypeVar
(ParameterType
, bound=Dict
[str
, zfit.core.interfaces.ZfitParameter])]]) – Parameters that will be associated with the loss in this order. Can be None if called within_minimize
.
- Return type
ConvergenceCriterion
- Returns
ConvergenceCriterion to check if the function converged.
- create_evaluator(loss=None, params=None, numpy_converter=None, strategy=None)#
Make a loss evaluator using the strategy and more from the minimizer.
Convenience factory for the loss evaluator. This wraps the loss to return a numpy array, to catch NaNs, stop on maxiter and evaluate the gradient and hessian without the need to specify the order every time.
- Parameters
loss (
Optional
[ZfitLoss
]) – Loss to be wrapped. Can be None if called inside_minimize
params (
Optional
[Iterable
[TypeVar
(ParameterType
, bound=Dict
[str
, zfit.core.interfaces.ZfitParameter])]]) – Parameters that will be associated with the loss in this order. Can be None if called within_minimize
.strategy (
Optional
[ZfitStrategy
]) – Instance of a Strategy that will be used during the evaluation.
- Returns
The evaluator that wraps the Loss ant Strategy with the current parameters.
- Return type
LossEval
- minimize(loss, params=None, init=None)#
Fully minimize the
loss
with respect toparams
, optionally using information frominit
.The minimizer changes the parameter values in order to minimize the loss function until the convergence criterion value is less than the tolerance. This is a stateless function that can take a
FitResult
in order to initialize the minimization.- Parameters
loss (
ZfitLoss
|Callable
) – Loss to be minimized until convergence is reached. Usually aZfitLoss
.attribute (- If this is a simple callable that takes an array as argument and an attribute errordef. The) –
can be set to any arbitrary function like
def loss(x): return - x ** 2 loss.errordef = 0.5 # as an example minimizer.minimize(loss, [2, 5])
If not TensorFlow is used inside the function, make sure to set
zfit.run.set_graph_mode(False)
andzfit.run.set_autograd_mode(False)
.method (- A FitResult can be provided as the only argument to the) – parameters to be minimized are taken from it. This allows to easily chain minimization algorithms.
the (in which case the loss as well as) – parameters to be minimized are taken from it. This allows to easily chain minimization algorithms.
params (
Optional
[Iterable
[ZfitParameter
]]) –The parameters with respect to which to minimize the
loss
. IfNone
, the parameters will be taken from theloss
.In order to fix the parameter values to a specific value (and thereby make them indepented of their current value), a dictionary mapping a parameter to a value can be given.
If
loss
is a callable,params
can also be (instead ofParameters
):an array of initial values
for more control, a
dict
with the keys:value
(required): array-like initial values.name
: list of unique names of the parameters.lower
: array-like lower limits of the parameters,upper
: array-like upper limits of the parameters,step_size
: array-like initial step size of the parameters (approximately the expected uncertainty)
This will create internally a single parameter for each value that can be accessed in the
FitResult
via params. Repeated calls can therefore (in the current implement) cause a memory increase. The recommended way is to re-use parameters (just taken from theFitResult
attributeparams
).init (
Optional
[ZfitResult
]) –A result of a previous minimization that provides auxiliary information such as the starting point for the parameters, the approximation of the covariance and more. Which information is used can depend on the specific minimizer implementation.
In general, the assumption is that the loss provided is similar enough to the one provided in
init
.What is assumed to be close:
the parameters at the minimum of loss will be close to the parameter values at the minimum of init.
Covariance matrix, or in general the shape, of init to the loss at its minimum.
What is explicitly _not_ assumed to be the same:
absolute value of the loss function. If init has a function value at minimum x of fmin, it is not assumed that
loss
will have the same/similar value at x.parameters that are used in the minimization may differ in order or which are fixed.
- Return type
- Returns
The fit result containing all information about the minimization.
Examples
Using the ability to restart a minimization with a previous result allows to use a more global search algorithm with a high tolerance and an additional local minimization to polish the found minimum.
result_approx = minimizer_global.minimize(loss, params) result = minimizer_local.minimize(result_approx)
For a simple usage with a callable only, the parameters can be given as an array of initial values.
def func(x): return np.log(np.sum(x ** 2)) func.errordef = 0.5 params = [1.1, 3.5, 8.35] # initial values result = minimizer.minimize(func, param)