fbpx
Wikipedia

Kaplan–Meier estimator

The Kaplan–Meier estimator,[1][2] also known as the product limit estimator, is a non-parametric statistic used to estimate the survival function from lifetime data. In medical research, it is often used to measure the fraction of patients living for a certain amount of time after treatment. In other fields, Kaplan–Meier estimators may be used to measure the length of time people remain unemployed after a job loss,[3] the time-to-failure of machine parts, or how long fleshy fruits remain on plants before they are removed by frugivores. The estimator is named after Edward L. Kaplan and Paul Meier, who each submitted similar manuscripts to the Journal of the American Statistical Association.[4] The journal editor, John Tukey, convinced them to combine their work into one paper, which has been cited more than 65,000 times since its publication in 1958.[5][6]

An example of a Kaplan–Meier plot for two conditions associated with patient survival.

The estimator of the survival function (the probability that life is longer than ) is given by:

with a time when at least one event happened, di the number of events (e.g., deaths) that happened at time , and the individuals known to have survived (have not yet had an event or been censored) up to time .

Basic concepts edit

A plot of the Kaplan–Meier estimator is a series of declining horizontal steps which, with a large enough sample size, approaches the true survival function for that population. The value of the survival function between successive distinct sampled observations ("clicks") is assumed to be constant.

An important advantage of the Kaplan–Meier curve is that the method can take into account some types of censored data, particularly right-censoring, which occurs if a patient withdraws from a study, is lost to follow-up, or is alive without event occurrence at last follow-up. On the plot, small vertical tick-marks state individual patients whose survival times have been right-censored. When no truncation or censoring occurs, the Kaplan–Meier curve is the complement of the empirical distribution function.

In medical statistics, a typical application might involve grouping patients into categories, for instance, those with Gene A profile and those with Gene B profile. In the graph, patients with Gene B die much quicker than those with Gene A. After two years, about 80% of the Gene A patients survive, but less than half of patients with Gene B.

To generate a Kaplan–Meier estimator, at least two pieces of data are required for each patient (or each subject): the status at last observation (event occurrence or right-censored), and the time to event (or time to censoring). If the survival functions between two or more groups are to be compared, then a third piece of data is required: the group assignment of each subject.[7]

Problem definition edit

Let   be a random variable, which we think of as the time that elapses between the start of the possible exposure period,  , and the time that an event of interest takes place,  . As indicated above, the goal is to estimate the survival function   underlying  . Recall that this function is defined as

 , where   is the time.

Let   be independent, identically distributed random variables, whose common distribution is that of  :   is the random time when some event   happened. The data available for estimating   is not  , but the list of pairs   where for  ,   is a fixed, deterministic integer, the censoring time of event   and  . In particular, the information available about the timing of event   is whether the event happened before the fixed time   and if so, then the actual time of the event is also available. The challenge is to estimate   given this data.

Derivation of the Kaplan–Meier estimator edit

Here, we show two derivations of the Kaplan–Meier estimator. Both are based on rewriting the survival function in terms of what is sometimes called hazard, or mortality rates. However, before doing this it is worthwhile to consider a naive estimator.

A naive estimator edit

To understand the power of the Kaplan–Meier estimator, it is worthwhile to first describe a naive estimator of the survival function.

Fix   and let  . A basic argument shows that the following proposition holds:

Proposition 1: If the censoring time   of event   exceeds   ( ), then   if and only if  .

Let   be such that  . It follows from the above proposition that

 

Let   and consider only those  , i.e. the events for which the outcome was not censored before time  . Let   be the number of elements in  . Note that the set   is not random and so neither is  . Furthermore,   is a sequence of independent, identically distributed Bernoulli random variables with common parameter  . Assuming that  , this suggests to estimate   using

 

where the second equality follows because   implies  , while the last equality is simply a change of notation.

The quality of this estimate is governed by the size of  . This can be problematic when   is small, which happens, by definition, when a lot of the events are censored. A particularly unpleasant property of this estimator, that suggests that perhaps it is not the "best" estimator, is that it ignores all the observations whose censoring time precedes  . Intuitively, these observations still contain information about  : For example, when for many events with  ,   also holds, we can infer that events often happen early, which implies that   is large, which, through   means that   must be small. However, this information is ignored by this naive estimator. The question is then whether there exists an estimator that makes a better use of all the data. This is what the Kaplan–Meier estimator accomplishes. Note that the naive estimator cannot be improved when censoring does not take place; so whether an improvement is possible critically hinges upon whether censoring is in place.

The plug-in approach edit

By elementary calculations,

 

where the second to last equality used that   is integer valued and for the last line we introduced

 

By a recursive expansion of the equality  , we get

 

Note that here  .

The Kaplan–Meier estimator can be seen as a "plug-in estimator" where each   is estimated based on the data and the estimator of   is obtained as a product of these estimates.

It remains to specify how   is to be estimated. By Proposition 1, for any   such that  ,   and   both hold. Hence, for any   such that  ,

 

By a similar reasoning that lead to the construction of the naive estimator above, we arrive at the estimator

 

(think of estimating the numerator and denominator separately in the definition of the "hazard rate"  ). The Kaplan–Meier estimator is then given by

 

The form of the estimator stated at the beginning of the article can be obtained by some further algebra. For this, write   where, using the actuarial science terminology,   is the number of known deaths at time  , while   is the number of those persons who are alive (and not being censored) at time  .

Note that if  ,  . This implies that we can leave out from the product defining   all those terms where  . Then, letting   be the times   when  ,   and  , we arrive at the form of the Kaplan–Meier estimator given at the beginning of the article:

 

As opposed to the naive estimator, this estimator can be seen to use the available information more effectively: In the special case mentioned beforehand, when there are many early events recorded, the estimator will multiply many terms with a value below one and will thus take into account that the survival probability cannot be large.

Derivation as a maximum likelihood estimator edit

Kaplan–Meier estimator can be derived from maximum likelihood estimation of the discrete hazard function.[8][self-published source?] More specifically given   as the number of events and   the total individuals at risk at time  , discrete hazard rate   can be defined as the probability of an individual with an event at time  . Then survival rate can be defined as:

 

and the likelihood function for the hazard function up to time   is:

 

therefore the log likelihood will be:

 

finding the maximum of log likelihood with respect to   yields:

 

where hat is used to denote maximum likelihood estimation. Given this result, we can write:

 

More generally (for continuous as well as discrete survival distributions), the Kaplan-Meier estimator may be interpreted as a nonparametric maximum likelihood estimator.[9]

Benefits and limitations edit

The Kaplan–Meier estimator is one of the most frequently used methods of survival analysis. The estimate may be useful to examine recovery rates, the probability of death, and the effectiveness of treatment. It is limited in its ability to estimate survival adjusted for covariates; parametric survival models and the Cox proportional hazards model may be useful to estimate covariate-adjusted survival.

The Kaplan-Meier estimator is directly related to the Nelson-Aalen estimator and both maximize the empirical likelihood.[10]

Statistical considerations edit

The Kaplan–Meier estimator is a statistic, and several estimators are used to approximate its variance. One of the most common estimators is Greenwood's formula:[11]

 

where   is the number of cases and   is the total number of observations, for  .

For a 'sketch' of the mathematical derivation of the equation above, click on "show" to reveal

Greenwood's formula is derived[12][self-published source?] by noting that probability of getting   failures out of   cases follows a binomial distribution with failure probability  . As a result for maximum likelihood hazard rate   we have   and  . To avoid dealing with multiplicative probabilities we compute variance of logarithm of   and will use the delta method to convert it back to the original variance:

 

using martingale central limit theorem, it can be shown that the variance of the sum in the following equation is equal to the sum of variances:[12]

 

as a result we can write:

 

using the delta method once more:

 

as desired.


In some cases, one may wish to compare different Kaplan–Meier curves. This can be done by the log rank test, and the Cox proportional hazards test.

Other statistics that may be of use with this estimator are pointwise confidence intervals,[13] the Hall-Wellner band[14] and the equal-precision band.[15]

Software edit

  • Mathematica: the built-in function SurvivalModelFit creates survival models.[16]
  • SAS: The Kaplan–Meier estimator is implemented in the proc lifetest procedure.[17]
  • R: the Kaplan–Meier estimator is available as part of the survival package.[18][19][20]
  • Stata: the command sts returns the Kaplan–Meier estimator.[21][22]
  • Python: the lifelines and scikit-survival packages each include the Kaplan–Meier estimator.[23][24]
  • MATLAB: the ecdf function with the 'function','survivor' arguments can calculate or plot the Kaplan–Meier estimator.[25]
  • StatsDirect: The Kaplan–Meier estimator is implemented in the Survival Analysis menu.[26]
  • SPSS: The Kaplan–Meier estimator is implemented in the Analyze > Survival > Kaplan-Meier... menu.[27]
  • Julia: the Survival.jl package includes the Kaplan–Meier estimator.[28]
  • Epi Info: Kaplan–Meier estimator survival curves and results for the log rank test are obtained with the KMSURVIVAL command.[29]

See also edit

References edit

  1. ^ Kaplan, E. L.; Meier, P. (1958). "Nonparametric estimation from incomplete observations". J. Amer. Statist. Assoc. 53 (282): 457–481. doi:10.2307/2281868. JSTOR 2281868.
  2. ^ Kaplan, E.L. in a retrospective on the seminal paper in "This week's citation classic". Current Contents 24, 14 (1983). Available from UPenn as PDF.
  3. ^ Meyer, Bruce D. (1990). "Unemployment Insurance and Unemployment Spells" (PDF). Econometrica. 58 (4): 757–782. doi:10.2307/2938349. JSTOR 2938349. S2CID 154632727.
  4. ^ Stalpers, Lukas J A; Kaplan, Edward L (May 4, 2018). "Edward L. Kaplan and the Kaplan-Meier Survival Curve". BSHM Bulletin: Journal of the British Society for the History of Mathematics. 33 (2): 109–135. doi:10.1080/17498430.2018.1450055. S2CID 125941631.
  5. ^ Kaplan, E. L.; Meier, Paul (1958). "Nonparametric Estimation from Incomplete Observations". Journal of the American Statistical Association. 53 (282): 457–481. doi:10.1080/01621459.1958.10501452. Retrieved February 27, 2023.
  6. ^ . Chicago Tribune. August 18, 2011.
  7. ^ Rich, Jason T.; Neely, J. Gail; Paniello, Randal C.; Voelker, Courtney C. J.; Nussenbaum, Brian; Wang, Eric W. (September 2010). "A practical guide to understanding Kaplan-Meier curves". Otolaryngology–Head and Neck Surgery. 143 (3): 331–336. doi:10.1016/j.otohns.2010.05.007. PMC 3932959. PMID 20723767.
  8. ^ "STAT331 Unit 3" (PDF). Retrieved May 12, 2023.
  9. ^ Andersen, Per Kragh; Borgan, Ornulf; Gill, Richard D.; Keiding, Niels (1993). Statistical models based on counting processes. New York: Springer-Verlag. ISBN 0-387-97872-0.
  10. ^ Zhou, M. (2015). Empirical Likelihood Method in Survival Analysis (1st ed.). Chapman and Hall/CRC. https://doi.org/10.1201/b18598, https://books.google.com/books?id=9-b5CQAAQBAJ&dq=Does+the+Nelson%E2%80%93Aalen+estimator+construct+an+empirical+likelihood%3F&pg=PA7
  11. ^ Greenwood, Major (1926). A report on the natural duration of cancer. Issue 33 of Reports on public health and medical subjects. HMSO. OCLC 14713088.
  12. ^ a b "The Greenwood and Exponential Greenwood Confidence Intervals in Survival Analysis" (PDF). Retrieved May 12, 2023.
  13. ^ Fay, Michael P.; Brittain, Erica H.; Proschan, Michael A. (September 1, 2013). "Pointwise confidence intervals for a survival distribution with small samples or heavy censoring". Biostatistics. 14 (4): 723–736. doi:10.1093/biostatistics/kxt016. PMC 3769999. PMID 23632624.
  14. ^ Hall, W. J.; Wellner, Jon A. (1980). "Confidence bands for a survival curve from censored data". Biometrika. 67 (1): 133–143. doi:10.1093/biomet/67.1.133.
  15. ^ Nair, Vijayan N. (August 1984). "Confidence Bands for Survival Functions With Censored Data: A Comparative Study". Technometrics. 26 (3): 265–275. doi:10.1080/00401706.1984.10487964.
  16. ^ "Survival Analysis – Mathematica SurvivalModelFit". wolfram.com. Retrieved August 14, 2017.
  17. ^ "SAS/STAT(R) 14.1 User's Guide". support.sas.com. Retrieved May 12, 2023.
  18. ^ Therneau, Terry M. (August 9, 2022). "survival: Survival Analysis". The Comprehensive R Archive Network. Retrieved November 30, 2022.
  19. ^ Willekens, Frans (2014). "Statistical Packages for Multistate Life History Analysis". Multistate Analysis of Life Histories with R. Use R!. Springer. pp. 135–153. doi:10.1007/978-3-319-08383-4_6. ISBN 978-3-319-08383-4.
  20. ^ Chen, Ding-Geng; Peace, Karl E. (2014). Clinical Trial Data Analysis Using R. CRC Press. pp. 99–108. ISBN 9781439840214.
  21. ^ "sts — Generate, graph, list, and test the survivor and cumulative hazard functions" (PDF). Stata Manual.
  22. ^ Cleves, Mario (2008). An Introduction to Survival Analysis Using Stata (Second ed.). College Station: Stata Press. pp. 93–107. ISBN 978-1-59718-041-2.
  23. ^ "lifelines — lifelines 0.27.7 documentation". lifelines.readthedocs.io. Retrieved May 12, 2023.
  24. ^ "sksurv.nonparametric.kaplan_meier_estimator — scikit-survival 0.20.0". scikit-survival.readthedocs.io. Retrieved May 12, 2023.
  25. ^ "Empirical cumulative distribution function – MATLAB ecdf". mathworks.com. Retrieved June 16, 2016.
  26. ^ "Kaplan-Meier Survival Estimates". statsdirect.co.uk. Retrieved May 12, 2023.
  27. ^ "Kaplan-Meier method in SPSS Statistics | Laerd Statistics".
  28. ^ "Kaplan-Meier · Survival.jl".
  29. ^ "Epi Info™ User Guide - Command Reference - Analysis Commands: KMSURVIVAL". Retrieved October 30, 2023.

Further reading edit

  • Aalen, Odd; Borgan, Ornulf; Gjessing, Hakon (2008). Survival and Event History Analysis: A Process Point of View. Springer. pp. 90–104. ISBN 978-0-387-68560-1.
  • Greene, William H. (2012). "Nonparametric and Semiparametric Approaches". Econometric Analysis (Seventh ed.). Prentice-Hall. pp. 909–912. ISBN 978-0-273-75356-8.
  • Jones, Andrew M.; Rice, Nigel; D'Uva, Teresa Bago; Balia, Silvia (2013). "Duration Data". Applied Health Economics. London: Routledge. pp. 139–181. ISBN 978-0-415-67682-3.
  • Singer, Judith B.; Willett, John B. (2003). Applied Longitudinal Data Analysis: Modeling Change and Event Occurrence. New York: Oxford University Press. pp. 483–487. ISBN 0-19-515296-4.

External links edit

  • Dunn, Steve (2002). "Survival Curves: Accrual and The Kaplan–Meier Estimate". Cancer Guide. Statistics.
  • Three evolving Kaplan–Meier curves on YouTube

kaplan, meier, estimator, also, known, product, limit, estimator, parametric, statistic, used, estimate, survival, function, from, lifetime, data, medical, research, often, used, measure, fraction, patients, living, certain, amount, time, after, treatment, oth. The Kaplan Meier estimator 1 2 also known as the product limit estimator is a non parametric statistic used to estimate the survival function from lifetime data In medical research it is often used to measure the fraction of patients living for a certain amount of time after treatment In other fields Kaplan Meier estimators may be used to measure the length of time people remain unemployed after a job loss 3 the time to failure of machine parts or how long fleshy fruits remain on plants before they are removed by frugivores The estimator is named after Edward L Kaplan and Paul Meier who each submitted similar manuscripts to the Journal of the American Statistical Association 4 The journal editor John Tukey convinced them to combine their work into one paper which has been cited more than 65 000 times since its publication in 1958 5 6 An example of a Kaplan Meier plot for two conditions associated with patient survival The estimator of the survival function S t displaystyle S t the probability that life is longer than t displaystyle t is given by S t i ti t 1 dini displaystyle widehat S t prod limits i t i leq t left 1 frac d i n i right with ti displaystyle t i a time when at least one event happened di the number of events e g deaths that happened at time ti displaystyle t i and ni displaystyle n i the individuals known to have survived have not yet had an event or been censored up to time ti displaystyle t i Contents 1 Basic concepts 2 Problem definition 3 Derivation of the Kaplan Meier estimator 3 1 A naive estimator 3 2 The plug in approach 3 3 Derivation as a maximum likelihood estimator 4 Benefits and limitations 5 Statistical considerations 6 Software 7 See also 8 References 9 Further reading 10 External linksBasic concepts editA plot of the Kaplan Meier estimator is a series of declining horizontal steps which with a large enough sample size approaches the true survival function for that population The value of the survival function between successive distinct sampled observations clicks is assumed to be constant An important advantage of the Kaplan Meier curve is that the method can take into account some types of censored data particularly right censoring which occurs if a patient withdraws from a study is lost to follow up or is alive without event occurrence at last follow up On the plot small vertical tick marks state individual patients whose survival times have been right censored When no truncation or censoring occurs the Kaplan Meier curve is the complement of the empirical distribution function In medical statistics a typical application might involve grouping patients into categories for instance those with Gene A profile and those with Gene B profile In the graph patients with Gene B die much quicker than those with Gene A After two years about 80 of the Gene A patients survive but less than half of patients with Gene B To generate a Kaplan Meier estimator at least two pieces of data are required for each patient or each subject the status at last observation event occurrence or right censored and the time to event or time to censoring If the survival functions between two or more groups are to be compared then a third piece of data is required the group assignment of each subject 7 Problem definition editLet t 0 displaystyle tau geq 0 nbsp be a random variable which we think of as the time that elapses between the start of the possible exposure period t0 displaystyle t 0 nbsp and the time that an event of interest takes place t1 displaystyle t 1 nbsp As indicated above the goal is to estimate the survival function S displaystyle S nbsp underlying t displaystyle tau nbsp Recall that this function is defined as S t Prob t gt t displaystyle S t operatorname Prob tau gt t nbsp where t 0 1 displaystyle t 0 1 dots nbsp is the time Let t1 tn 0 displaystyle tau 1 dots tau n geq 0 nbsp be independent identically distributed random variables whose common distribution is that of t displaystyle tau nbsp tj displaystyle tau j nbsp is the random time when some event j displaystyle j nbsp happened The data available for estimating S displaystyle S nbsp is not tj j 1 n displaystyle tau j j 1 dots n nbsp but the list of pairs t j cj j 1 n displaystyle tilde tau j c j j 1 dots n nbsp where for j n 1 2 n displaystyle j in n 1 2 dots n nbsp cj 0 displaystyle c j geq 0 nbsp is a fixed deterministic integer the censoring time of event j displaystyle j nbsp and t j min tj cj displaystyle tilde tau j min tau j c j nbsp In particular the information available about the timing of event j displaystyle j nbsp is whether the event happened before the fixed time cj displaystyle c j nbsp and if so then the actual time of the event is also available The challenge is to estimate S t displaystyle S t nbsp given this data Derivation of the Kaplan Meier estimator editHere we show two derivations of the Kaplan Meier estimator Both are based on rewriting the survival function in terms of what is sometimes called hazard or mortality rates However before doing this it is worthwhile to consider a naive estimator A naive estimator edit To understand the power of the Kaplan Meier estimator it is worthwhile to first describe a naive estimator of the survival function Fix k n 1 n displaystyle k in n 1 dots n nbsp and let t gt 0 displaystyle t gt 0 nbsp A basic argument shows that the following proposition holds Proposition 1 If the censoring time ck displaystyle c k nbsp of event k displaystyle k nbsp exceeds t displaystyle t nbsp ck t displaystyle c k geq t nbsp then t k t displaystyle tilde tau k geq t nbsp if and only if tk t displaystyle tau k geq t nbsp Let k displaystyle k nbsp be such that ck t displaystyle c k geq t nbsp It follows from the above proposition that Prob tk t Prob t k t displaystyle operatorname Prob tau k geq t operatorname Prob tilde tau k geq t nbsp Let Xk I t k t displaystyle X k mathbb I tilde tau k geq t nbsp and consider only those k C t k ck t displaystyle k in C t k c k geq t nbsp i e the events for which the outcome was not censored before time t displaystyle t nbsp Let m t C t displaystyle m t C t nbsp be the number of elements in C t displaystyle C t nbsp Note that the set C t displaystyle C t nbsp is not random and so neither is m t displaystyle m t nbsp Furthermore Xk k C t displaystyle X k k in C t nbsp is a sequence of independent identically distributed Bernoulli random variables with common parameter S t Prob t t displaystyle S t operatorname Prob tau geq t nbsp Assuming that m t gt 0 displaystyle m t gt 0 nbsp this suggests to estimate S t displaystyle S t nbsp using S naive t 1m t k ck tXk 1 k n t k t 1 k n ck t 1 k n t k t m t displaystyle hat S text naive t frac 1 m t sum k c k geq t X k frac 1 leq k leq n tilde tau k geq t 1 leq k leq n c k geq t frac 1 leq k leq n tilde tau k geq t m t nbsp where the second equality follows because t k t displaystyle tilde tau k geq t nbsp implies ck t displaystyle c k geq t nbsp while the last equality is simply a change of notation The quality of this estimate is governed by the size of m t displaystyle m t nbsp This can be problematic when m t displaystyle m t nbsp is small which happens by definition when a lot of the events are censored A particularly unpleasant property of this estimator that suggests that perhaps it is not the best estimator is that it ignores all the observations whose censoring time precedes t displaystyle t nbsp Intuitively these observations still contain information about S t displaystyle S t nbsp For example when for many events with ck lt t displaystyle c k lt t nbsp tk lt ck displaystyle tau k lt c k nbsp also holds we can infer that events often happen early which implies that Prob t t displaystyle operatorname Prob tau leq t nbsp is large which through S t 1 Prob t t displaystyle S t 1 operatorname Prob tau leq t nbsp means that S t displaystyle S t nbsp must be small However this information is ignored by this naive estimator The question is then whether there exists an estimator that makes a better use of all the data This is what the Kaplan Meier estimator accomplishes Note that the naive estimator cannot be improved when censoring does not take place so whether an improvement is possible critically hinges upon whether censoring is in place The plug in approach edit By elementary calculations S t Prob t gt t t gt t 1 Prob t gt t 1 1 Prob t t t gt t 1 Prob t gt t 1 1 Prob t t t t Prob t gt t 1 q t S t 1 displaystyle begin aligned S t amp operatorname Prob tau gt t mid tau gt t 1 operatorname Prob tau gt t 1 4pt amp 1 operatorname Prob tau leq t mid tau gt t 1 operatorname Prob tau gt t 1 4pt amp 1 operatorname Prob tau t mid tau geq t operatorname Prob tau gt t 1 4pt amp q t S t 1 end aligned nbsp where the second to last equality used that t displaystyle tau nbsp is integer valued and for the last line we introduced q t 1 Prob t t t t displaystyle q t 1 operatorname Prob tau t mid tau geq t nbsp By a recursive expansion of the equality S t q t S t 1 displaystyle S t q t S t 1 nbsp we get S t q t q t 1 q 0 displaystyle S t q t q t 1 cdots q 0 nbsp Note that here q 0 1 Prob t 0 t gt 1 1 Prob t 0 displaystyle q 0 1 operatorname Prob tau 0 mid tau gt 1 1 operatorname Prob tau 0 nbsp The Kaplan Meier estimator can be seen as a plug in estimator where each q s displaystyle q s nbsp is estimated based on the data and the estimator of S t displaystyle S t nbsp is obtained as a product of these estimates It remains to specify how q s 1 Prob t s t s displaystyle q s 1 operatorname Prob tau s mid tau geq s nbsp is to be estimated By Proposition 1 for any k n displaystyle k in n nbsp such that ck s displaystyle c k geq s nbsp Prob t s Prob t k s displaystyle operatorname Prob tau s operatorname Prob tilde tau k s nbsp and Prob t s Prob t k s displaystyle operatorname Prob tau geq s operatorname Prob tilde tau k geq s nbsp both hold Hence for any k n displaystyle k in n nbsp such that ck s displaystyle c k geq s nbsp Prob t s t s Prob t k s Prob t k s displaystyle operatorname Prob tau s tau geq s operatorname Prob tilde tau k s operatorname Prob tilde tau k geq s nbsp By a similar reasoning that lead to the construction of the naive estimator above we arrive at the estimator q s 1 1 k n ck s t k s 1 k n ck s t k s 1 1 k n t k s 1 k n t k s displaystyle hat q s 1 frac 1 leq k leq n c k geq s tilde tau k s 1 leq k leq n c k geq s tilde tau k geq s 1 frac 1 leq k leq n tilde tau k s 1 leq k leq n tilde tau k geq s nbsp think of estimating the numerator and denominator separately in the definition of the hazard rate Prob t s t s displaystyle operatorname Prob tau s tau geq s nbsp The Kaplan Meier estimator is then given by S t s 0tq s displaystyle hat S t prod s 0 t hat q s nbsp The form of the estimator stated at the beginning of the article can be obtained by some further algebra For this write q s 1 d s n s displaystyle hat q s 1 d s n s nbsp where using the actuarial science terminology d s 1 k n tk s displaystyle d s 1 leq k leq n tau k s nbsp is the number of known deaths at time s displaystyle s nbsp while n s 1 k n t k s displaystyle n s 1 leq k leq n tilde tau k geq s nbsp is the number of those persons who are alive and not being censored at time s 1 displaystyle s 1 nbsp Note that if d s 0 displaystyle d s 0 nbsp q s 1 displaystyle hat q s 1 nbsp This implies that we can leave out from the product defining S t displaystyle hat S t nbsp all those terms where d s 0 displaystyle d s 0 nbsp Then letting 0 t1 lt t2 lt lt tm displaystyle 0 leq t 1 lt t 2 lt dots lt t m nbsp be the times s displaystyle s nbsp when d s gt 0 displaystyle d s gt 0 nbsp di d ti displaystyle d i d t i nbsp and ni n ti displaystyle n i n t i nbsp we arrive at the form of the Kaplan Meier estimator given at the beginning of the article S t i ti t 1 dini displaystyle hat S t prod i t i leq t left 1 frac d i n i right nbsp As opposed to the naive estimator this estimator can be seen to use the available information more effectively In the special case mentioned beforehand when there are many early events recorded the estimator will multiply many terms with a value below one and will thus take into account that the survival probability cannot be large Derivation as a maximum likelihood estimator edit Kaplan Meier estimator can be derived from maximum likelihood estimation of the discrete hazard function 8 self published source More specifically given di displaystyle d i nbsp as the number of events and ni displaystyle n i nbsp the total individuals at risk at time ti displaystyle t i nbsp discrete hazard rate hi displaystyle h i nbsp can be defined as the probability of an individual with an event at time ti displaystyle t i nbsp Then survival rate can be defined as S t i ti t 1 hi displaystyle S t prod limits i t i leq t 1 h i nbsp and the likelihood function for the hazard function up to time ti displaystyle t i nbsp is L hj j i dj j i nj j i j 1ihjdj 1 hj nj dj njdj displaystyle mathcal L h j j leq i mid d j j leq i n j j leq i prod j 1 i h j d j 1 h j n j d j n j choose d j nbsp therefore the log likelihood will be log L j 1i djlog hj nj dj log 1 hj log njdj displaystyle log mathcal L sum j 1 i left d j log h j n j d j log 1 h j log n j choose d j right nbsp finding the maximum of log likelihood with respect to hi displaystyle h i nbsp yields log L hi dih i ni di1 h i 0 h i dini displaystyle frac partial log mathcal L partial h i frac d i widehat h i frac n i d i 1 widehat h i 0 Rightarrow widehat h i frac d i n i nbsp where hat is used to denote maximum likelihood estimation Given this result we can write S t i ti t 1 h i i ti t 1 dini displaystyle widehat S t prod limits i t i leq t left 1 widehat h i right prod limits i t i leq t left 1 frac d i n i right nbsp More generally for continuous as well as discrete survival distributions the Kaplan Meier estimator may be interpreted as a nonparametric maximum likelihood estimator 9 Benefits and limitations editThe Kaplan Meier estimator is one of the most frequently used methods of survival analysis The estimate may be useful to examine recovery rates the probability of death and the effectiveness of treatment It is limited in its ability to estimate survival adjusted for covariates parametric survival models and the Cox proportional hazards model may be useful to estimate covariate adjusted survival The Kaplan Meier estimator is directly related to the Nelson Aalen estimator and both maximize the empirical likelihood 10 Statistical considerations editThe Kaplan Meier estimator is a statistic and several estimators are used to approximate its variance One of the most common estimators is Greenwood s formula 11 Var S t S t 2 i ti tdini ni di displaystyle widehat operatorname Var left widehat S t right widehat S t 2 sum i t i leq t frac d i n i n i d i nbsp where di displaystyle d i nbsp is the number of cases and ni displaystyle n i nbsp is the total number of observations for ti lt t displaystyle t i lt t nbsp For a sketch of the mathematical derivation of the equation above click on show to revealGreenwood s formula is derived 12 self published source by noting that probability of getting di displaystyle d i nbsp failures out of ni displaystyle n i nbsp cases follows a binomial distribution with failure probability hi displaystyle h i nbsp As a result for maximum likelihood hazard rate h i di ni displaystyle widehat h i d i n i nbsp we have E h i hi displaystyle E left widehat h i right h i nbsp and Var h i hi 1 hi ni displaystyle operatorname Var left widehat h i right h i 1 h i n i nbsp To avoid dealing with multiplicative probabilities we compute variance of logarithm of S t displaystyle widehat S t nbsp and will use the delta method to convert it back to the original variance Var log S t 1S t 2Var S t Var S t S t 2Var log S t displaystyle begin aligned operatorname Var left log widehat S t right amp sim frac 1 widehat S t 2 operatorname Var left widehat S t right Rightarrow operatorname Var left widehat S t right amp sim widehat S t 2 operatorname Var left log widehat S t right end aligned nbsp using martingale central limit theorem it can be shown that the variance of the sum in the following equation is equal to the sum of variances 12 log S t i ti tlog 1 h i displaystyle log widehat S t sum limits i t i leq t log left 1 widehat h i right nbsp as a result we can write Var S t S t 2Var i ti tlog 1 h i S t 2 i ti tVar log 1 h i displaystyle begin aligned operatorname Var widehat S t amp sim widehat S t 2 operatorname Var left sum i t i leq t log left 1 widehat h i right right amp sim widehat S t 2 sum limits i t i leq t operatorname Var left log left 1 widehat h i right right end aligned nbsp using the delta method once more Var S t S t 2 i ti t log 1 h i h i 2Var h i S t 2 i ti t 11 h i 2h i 1 h i ni S t 2 i ti th ini 1 h i S t 2 i ti tdini ni di displaystyle begin aligned operatorname Var widehat S t amp sim widehat S t 2 sum i t i leq t left frac partial log left 1 widehat h i right partial widehat h i right 2 operatorname Var left widehat h i right amp widehat S t 2 sum i t i leq t left frac 1 1 widehat h i right 2 frac widehat h i left 1 widehat h i right n i amp widehat S t 2 sum i t i leq t frac widehat h i n i left 1 widehat h i right amp widehat S t 2 sum i t i leq t frac d i n i n i d i end aligned nbsp as desired In some cases one may wish to compare different Kaplan Meier curves This can be done by the log rank test and the Cox proportional hazards test Other statistics that may be of use with this estimator are pointwise confidence intervals 13 the Hall Wellner band 14 and the equal precision band 15 Software editMathematica the built in function SurvivalModelFit creates survival models 16 SAS The Kaplan Meier estimator is implemented in the proc lifetest procedure 17 R the Kaplan Meier estimator is available as part of the survival package 18 19 20 Stata the command sts returns the Kaplan Meier estimator 21 22 Python the lifelines and scikit survival packages each include the Kaplan Meier estimator 23 24 MATLAB the ecdf function with the function survivor arguments can calculate or plot the Kaplan Meier estimator 25 StatsDirect The Kaplan Meier estimator is implemented in the Survival Analysis menu 26 SPSS The Kaplan Meier estimator is implemented in the Analyze gt Survival gt Kaplan Meier menu 27 Julia the Survival jl package includes the Kaplan Meier estimator 28 Epi Info Kaplan Meier estimator survival curves and results for the log rank test are obtained with the KMSURVIVAL command 29 See also editSurvival Analysis Frequency of exceedance Median lethal dose Nelson Aalen estimatorReferences edit Kaplan E L Meier P 1958 Nonparametric estimation from incomplete observations J Amer Statist Assoc 53 282 457 481 doi 10 2307 2281868 JSTOR 2281868 Kaplan E L in a retrospective on the seminal paper in This week s citation classic Current Contents 24 14 1983 Available from UPenn as PDF Meyer Bruce D 1990 Unemployment Insurance and Unemployment Spells PDF Econometrica 58 4 757 782 doi 10 2307 2938349 JSTOR 2938349 S2CID 154632727 Stalpers Lukas J A Kaplan Edward L May 4 2018 Edward L Kaplan and the Kaplan Meier Survival Curve BSHM Bulletin Journal of the British Society for the History of Mathematics 33 2 109 135 doi 10 1080 17498430 2018 1450055 S2CID 125941631 Kaplan E L Meier Paul 1958 Nonparametric Estimation from Incomplete Observations Journal of the American Statistical Association 53 282 457 481 doi 10 1080 01621459 1958 10501452 Retrieved February 27 2023 Paul Meier 1924 2011 Chicago Tribune August 18 2011 Rich Jason T Neely J Gail Paniello Randal C Voelker Courtney C J Nussenbaum Brian Wang Eric W September 2010 A practical guide to understanding Kaplan Meier curves Otolaryngology Head and Neck Surgery 143 3 331 336 doi 10 1016 j otohns 2010 05 007 PMC 3932959 PMID 20723767 STAT331 Unit 3 PDF Retrieved May 12 2023 Andersen Per Kragh Borgan Ornulf Gill Richard D Keiding Niels 1993 Statistical models based on counting processes New York Springer Verlag ISBN 0 387 97872 0 Zhou M 2015 Empirical Likelihood Method in Survival Analysis 1st ed Chapman and Hall CRC https doi org 10 1201 b18598 https books google com books id 9 b5CQAAQBAJ amp dq Does the Nelson E2 80 93Aalen estimator construct an empirical likelihood 3F amp pg PA7 Greenwood Major 1926 A report on the natural duration of cancer Issue 33 of Reports on public health and medical subjects HMSO OCLC 14713088 a b The Greenwood and Exponential Greenwood Confidence Intervals in Survival Analysis PDF Retrieved May 12 2023 Fay Michael P Brittain Erica H Proschan Michael A September 1 2013 Pointwise confidence intervals for a survival distribution with small samples or heavy censoring Biostatistics 14 4 723 736 doi 10 1093 biostatistics kxt016 PMC 3769999 PMID 23632624 Hall W J Wellner Jon A 1980 Confidence bands for a survival curve from censored data Biometrika 67 1 133 143 doi 10 1093 biomet 67 1 133 Nair Vijayan N August 1984 Confidence Bands for Survival Functions With Censored Data A Comparative Study Technometrics 26 3 265 275 doi 10 1080 00401706 1984 10487964 Survival Analysis Mathematica SurvivalModelFit wolfram com Retrieved August 14 2017 SAS STAT R 14 1 User s Guide support sas com Retrieved May 12 2023 Therneau Terry M August 9 2022 survival Survival Analysis The Comprehensive R Archive Network Retrieved November 30 2022 Willekens Frans 2014 Statistical Packages for Multistate Life History Analysis Multistate Analysis of Life Histories with R Use R Springer pp 135 153 doi 10 1007 978 3 319 08383 4 6 ISBN 978 3 319 08383 4 Chen Ding Geng Peace Karl E 2014 Clinical Trial Data Analysis Using R CRC Press pp 99 108 ISBN 9781439840214 sts Generate graph list and test the survivor and cumulative hazard functions PDF Stata Manual Cleves Mario 2008 An Introduction to Survival Analysis Using Stata Second ed College Station Stata Press pp 93 107 ISBN 978 1 59718 041 2 lifelines lifelines 0 27 7 documentation lifelines readthedocs io Retrieved May 12 2023 sksurv nonparametric kaplan meier estimator scikit survival 0 20 0 scikit survival readthedocs io Retrieved May 12 2023 Empirical cumulative distribution function MATLAB ecdf mathworks com Retrieved June 16 2016 Kaplan Meier Survival Estimates statsdirect co uk Retrieved May 12 2023 Kaplan Meier method in SPSS Statistics Laerd Statistics Kaplan Meier Survival jl Epi Info User Guide Command Reference Analysis Commands KMSURVIVAL Retrieved October 30 2023 Further reading editAalen Odd Borgan Ornulf Gjessing Hakon 2008 Survival and Event History Analysis A Process Point of View Springer pp 90 104 ISBN 978 0 387 68560 1 Greene William H 2012 Nonparametric and Semiparametric Approaches Econometric Analysis Seventh ed Prentice Hall pp 909 912 ISBN 978 0 273 75356 8 Jones Andrew M Rice Nigel D Uva Teresa Bago Balia Silvia 2013 Duration Data Applied Health Economics London Routledge pp 139 181 ISBN 978 0 415 67682 3 Singer Judith B Willett John B 2003 Applied Longitudinal Data Analysis Modeling Change and Event Occurrence New York Oxford University Press pp 483 487 ISBN 0 19 515296 4 External links editDunn Steve 2002 Survival Curves Accrual and The Kaplan Meier Estimate Cancer Guide Statistics Three evolving Kaplan Meier curves on YouTube Retrieved from https en wikipedia org w index php title Kaplan Meier estimator amp oldid 1217054019, wikipedia, wiki, book, books, library,

article

, read, download, free, free download, mp3, video, mp4, 3gp, jpg, jpeg, gif, png, picture, music, song, movie, book, game, games.