
Convex Relaxation Regression: BlackBox Optimization of Smooth Functions by Learning Their Convex Envelopes
Finding efficient and provable methods to solve nonconvex optimization ...
read it

Nonconvex Optimization for Machine Learning
A vast majority of machine learning algorithms train their models and pe...
read it

General risk measures for robust machine learning
A wide array of machine learning problems are formulated as the minimiza...
read it

Gradient Flows in Dataset Space
The current practice in machine learning is traditionally modelcentric,...
read it

Convex Optimization Over RiskNeutral Probabilities
We consider a collection of derivatives that depend on the price of an u...
read it

Nonsmoothness in Machine Learning: specific structure, proximal identification, and applications
Nonsmoothness is often a curse for optimization; but it is sometimes a b...
read it

Safeguarded Learned Convex Optimization
Many applications require repeatedly solving a certain type of optimizat...
read it
Optimization in Machine Learning: A Distribution Space Approach
We present the viewpoint that optimization problems encountered in machine learning can often be interpreted as minimizing a convex functional over a function space, but with a nonconvex constraint set introduced by model parameterization. This observation allows us to repose such problems via a suitable relaxation as convex optimization problems in the space of distributions over the training parameters. We derive some simple relationships between the distributionspace problem and the original problem, e.g. a distributionspace solution is at least as good as a solution in the original space. Moreover, we develop a numerical algorithm based on mixture distributions to perform approximate optimization directly in distribution space. Consistency of this approximation is established and the numerical efficacy of the proposed algorithm is illustrated on simple examples. In both theory and practice, this formulation provides an alternative approach to largescale optimization in machine learning.
READ FULL TEXT
Comments
There are no comments yet.