Degrees of freedom for piecewise Lipschitz estimators

Frederik Riis Mikkelsen; Niels Richard Hansen

doi:10.1214/17-AIHP822

Degrees of freedom for piecewise Lipschitz estimators

Frederik Riis Mikkelsen, Niels Richard Hansen

Institut for Matematiske Fag

3 Citationer (Scopus)

1 Downloads (Pure)

Abstract

A representation of the degrees of freedom akin to Stein’s lemma is given for a class of estimators of a mean value
parameter in Rn. Contrary to previous results our representation holds for a range of discontinues estimators. It shows that even
though the discontinuities form a Lebesgue null set, they cannot be ignored when computing degrees of freedom. Estimators
with discontinuities arise naturally in regression if data driven variable selection is used. Two such examples, namely best subset
selection and lasso-OLS, are considered in detail in this paper. For lasso-OLS the general representation leads to an estimate
of the degrees of freedom based on the lasso solution path, which in turn can be used for estimating the risk of lasso-OLS.
A similar estimate is proposed for best subset selection. The usefulness of the risk estimates for selecting the number of variables
is demonstrated via simulations with a particular focus on lasso-OLS.

Originalsprog	Engelsk
Tidsskrift	Annales de l'Institut Henri Poincaré, Probabilités et Statistiques
Vol/bind	54
Udgave nummer	2
Sider (fra-til)	819-841
Antal sider	23
ISSN	0246-0203
DOI	https://doi.org/10.1214/17-AIHP822
Status	Udgivet - 1 maj 2018

Adgang til dokumentet

10.1214/17-AIHP822

euclid.aihp.1524643230-2Forlagets udgivne version, 973 KBLicens: Ikke-specificeret

http://arxiv.org/pdf/1601.03524Licens: Andet

Citationsformater

@article{60609f59635646babc4a4f193d2f4d1b,

title = "Degrees of freedom for piecewise Lipschitz estimators",

abstract = "A representation of the degrees of freedom akin to Stein{\textquoteright}s lemma is given for a class of estimators of a mean valueparameter in Rn. Contrary to previous results our representation holds for a range of discontinues estimators. It shows that eventhough the discontinuities form a Lebesgue null set, they cannot be ignored when computing degrees of freedom. Estimatorswith discontinuities arise naturally in regression if data driven variable selection is used. Two such examples, namely best subsetselection and lasso-OLS, are considered in detail in this paper. For lasso-OLS the general representation leads to an estimateof the degrees of freedom based on the lasso solution path, which in turn can be used for estimating the risk of lasso-OLS.A similar estimate is proposed for best subset selection. The usefulness of the risk estimates for selecting the number of variablesis demonstrated via simulations with a particular focus on lasso-OLS.",

author = "Mikkelsen, {Frederik Riis} and Hansen, {Niels Richard}",

year = "2018",

month = may,

day = "1",

doi = "10.1214/17-AIHP822",

language = "English",

volume = "54",

pages = "819--841",

journal = "Annales de l'institut Henri Poincare (B) Probability and Statistics",

issn = "0246-0203",

publisher = "Institute Henri Poincar{\'e}",

number = "2",

}

TY - JOUR

T1 - Degrees of freedom for piecewise Lipschitz estimators

AU - Mikkelsen, Frederik Riis

AU - Hansen, Niels Richard

PY - 2018/5/1

Y1 - 2018/5/1

N2 - A representation of the degrees of freedom akin to Stein’s lemma is given for a class of estimators of a mean valueparameter in Rn. Contrary to previous results our representation holds for a range of discontinues estimators. It shows that eventhough the discontinuities form a Lebesgue null set, they cannot be ignored when computing degrees of freedom. Estimatorswith discontinuities arise naturally in regression if data driven variable selection is used. Two such examples, namely best subsetselection and lasso-OLS, are considered in detail in this paper. For lasso-OLS the general representation leads to an estimateof the degrees of freedom based on the lasso solution path, which in turn can be used for estimating the risk of lasso-OLS.A similar estimate is proposed for best subset selection. The usefulness of the risk estimates for selecting the number of variablesis demonstrated via simulations with a particular focus on lasso-OLS.

AB - A representation of the degrees of freedom akin to Stein’s lemma is given for a class of estimators of a mean valueparameter in Rn. Contrary to previous results our representation holds for a range of discontinues estimators. It shows that eventhough the discontinuities form a Lebesgue null set, they cannot be ignored when computing degrees of freedom. Estimatorswith discontinuities arise naturally in regression if data driven variable selection is used. Two such examples, namely best subsetselection and lasso-OLS, are considered in detail in this paper. For lasso-OLS the general representation leads to an estimateof the degrees of freedom based on the lasso solution path, which in turn can be used for estimating the risk of lasso-OLS.A similar estimate is proposed for best subset selection. The usefulness of the risk estimates for selecting the number of variablesis demonstrated via simulations with a particular focus on lasso-OLS.

U2 - 10.1214/17-AIHP822

DO - 10.1214/17-AIHP822

M3 - Journal article

SN - 0246-0203

VL - 54

SP - 819

EP - 841

JO - Annales de l'institut Henri Poincare (B) Probability and Statistics

JF - Annales de l'institut Henri Poincare (B) Probability and Statistics

IS - 2

ER -

Degrees of freedom for piecewise Lipschitz estimators

Abstract

Adgang til dokumentet

Fingeraftryk

Citationsformater