fbpx
Wikipedia

Cross-industry standard process for data mining

January 01, 1970

The Cross-industry standard process for data mining, known as CRISP-DM,[1] is an open standard process model that describes common approaches used by data mining experts. It is the most widely-used analytics model.[2]

In 2015, IBM released a new methodology called Analytics Solutions Unified Method for Data Mining/Predictive Analytics[3][4] (also known as ASUM-DM), which refines and extends CRISP-DM.

History edit

CRISP-DM was conceived in 1996 and became a European Union project under the ESPRIT funding initiative in 1997. The project was led by five companies: Integral Solutions Ltd (ISL), Teradata, Daimler AG, NCR Corporation, and OHRA, an insurance company.

This core consortium brought different experiences to the project. ISL, was later acquired and merged into SPSS. The computer giant NCR Corporation produced the Teradata data warehouse and its own data mining software. Daimler-Benz had a significant data mining team. OHRA was starting to explore the potential use of data mining.

The first version of the methodology was presented at the 4th CRISP-DM SIG Workshop in Brussels in March 1999,[5] and published as a step-by-step data mining guide later that year.[6]

Between 2006 and 2008, a CRISP-DM 2.0 SIG was formed, and there were discussions about updating the CRISP-DM process model.[7] The current status of these efforts is not known. However, the original crisp-dm.org website cited in the reviews,[8][9] and the CRISP-DM 2.0 SIG website are both no longer active.[7]

While many non-IBM data mining practitioners use CRISP-DM,[10][11][12] IBM is the primary corporation that currently uses the CRISP-DM process model. It makes some of the old CRISP-DM documents available for download and it has incorporated it into its SPSS Modeler product.[6]

Based on current research, CRISP-DM is the most widely used form of data-mining model because of its various advantages which solved the existing problems in the data mining industries. Some of the drawbacks of this model is that it does not perform project management activities. The success of CRISP-DM is largely attributable to the fact that it is industry, tool, and application neutral.[13]

Major phases edit

 
Process diagram showing the relationship between the different phases of CRISP-DM

CRISP-DM breaks the process of data mining into six major phases:[14]

  • Business Understanding
  • Data Understanding
  • Data Preparation
  • Modeling
  • Evaluation
  • Deployment

The sequence of the phases is not strict and moving back and forth between different phases is usually required. The arrows in the process diagram indicate the most important and frequent dependencies between phases. The outer circle in the diagram symbolizes the cyclic nature of data mining itself. A data mining process continues after a solution has been deployed. The lessons learned during the process can trigger new, often more focused business questions, and subsequent data mining processes will benefit from the experiences of previous ones.

Polls and Alternative Process Frameworks edit

Polls conducted at the same website (KDNuggets) in 2002, 2004, 2007, and 2014 show that it was the leading methodology used by industry data miners who decided to respond to the survey.[10][11][12][15] The only other data mining approach named in these polls was SEMMA. However, SAS Institute clearly states that SEMMA is not a data mining methodology, but rather a "logical organization of the functional toolset of SAS Enterprise Miner." A review and critique of data mining process models in 2009 called the CRISP-DM the "de facto standard for developing data mining and knowledge discovery projects."[16] Other reviews of CRISP-DM and data mining process models include Kurgan and Musilek's 2006 review,[8] and Azevedo and Santos' 2008 comparison of CRISP-DM and SEMMA.[9] Efforts to update the methodology started in 2006, but have, as of June 2015, not led to a new version, and the "Special Interest Group" (SIG) responsible along with the website has long disappeared (see History of CRISP-DM).

In 2024, Harvard Business Review published an updated framework, bizML, that is designed for greater relevance to business personnel and to be specific for machine learning projects in particular, rather than for analytics, data science, or data mining projects in general.[17]

References edit

  1. ^ Shearer C., The CRISP-DM model: the new blueprint for data mining, J Data Warehousing (2000); 5:13—22.
  2. ^ What IT Needs To Know About The Data Mining Process Published by Forbes, 29 July 2015, retrieved June 24, 2018
  3. ^ Have you seen ASUM-DM?, By Jason Haffar, 16 October 2015, SPSS Predictive Analytics, IBM 8 March 2016 at the Wayback Machine
  4. ^ Analytics Solutions Unified Method - Implementations with Agile principles Published by IBM, 1 March 2016, retrieved October 5, 2018
  5. ^ Pete Chapman (1999); The CRISP-DM User Guide.
  6. ^ a b Pete Chapman, Julian Clinton, Randy Kerber, Thomas Khabaza, Thomas Reinartz, Colin Shearer, and Rüdiger Wirth (2000); The CRISP-DM User Guide (entry on semantic scholar, including links to PDFs), (PDF version with high-resolution graphics 12 September 2020 at the Wayback Machine).
  7. ^ a b Colin Shearer (2006); First CRISP-DM 2.0 Workshop Held
  8. ^ a b Lukasz Kurgan and Petr Musilek (2006); A survey of Knowledge Discovery and Data Mining process models. The Knowledge Engineering Review. Volume 21 Issue 1, March 2006, pp 1–24, Cambridge University Press, New York, NY, USA doi: 10.1017/S0269888906000737.
  9. ^ a b Azevedo, A. and Santos, M. F. (2008); KDD, SEMMA and CRISP-DM: a parallel overview. In Proceedings of the IADIS European Conference on Data Mining 2008, pp 182–185.
  10. ^ a b Gregory Piatetsky-Shapiro (2002); KDnuggets Methodology Poll
  11. ^ a b Gregory Piatetsky-Shapiro (2004); KDnuggets Methodology Poll
  12. ^ a b Gregory Piatetsky-Shapiro (2007); KDnuggets Methodology Poll
  13. ^ Mariscal, G., Marban, O., Fernandez, C. (2010). "A Survey of Data Mining and knowledge discovery process Models and methodologies". The Knowledge Engineering Review. 25 (2): 137–166. doi:10.1017/S0269888910000032. S2CID 31359633.{{cite journal}}: CS1 maint: multiple names: authors list (link)
  14. ^ Harper, Gavin; Stephen D. Pickett (August 2006). "Methods for mining HTS data". Drug Discovery Today. 11 (15–16): 694–699. doi:10.1016/j.drudis.2006.06.006. PMID 16846796.
  15. ^ Gregory Piatetsky-Shapiro (2014); KDnuggets Methodology Poll
  16. ^ Martínez-Plumed, Fernando; Contreras-Ochando, Lidia; Ferri, Cèsar; Flach, Peter; Hernández-Orallo, José; Kull, Meelis; Lachiche, Nicolas; Ramírez-Quintana, María José (19 September 2017). "CASP-DM: Context Aware Standard Process for Data Mining". arXiv:1709.09003 [cs.DB].
  17. ^ Eric Siegel (2024); Getting Machine Learning Projects from Idea to Execution

January 01, 1970
cross, industry, standard, process, data, mining, this, article, tone, style, reflect, encyclopedic, tone, used, wikipedia, wikipedia, guide, writing, better, articles, suggestions, july, 2021, learn, when, remove, this, message, known, crisp, open, standard, . This article s tone or style may not reflect the encyclopedic tone used on Wikipedia See Wikipedia s guide to writing better articles for suggestions July 2021 Learn how and when to remove this message The Cross industry standard process for data mining known as CRISP DM 1 is an open standard process model that describes common approaches used by data mining experts It is the most widely used analytics model 2 In 2015 IBM released a new methodology called Analytics Solutions Unified Method for Data Mining Predictive Analytics 3 4 also known as ASUM DM which refines and extends CRISP DM Contents 1 History 2 Major phases 3 Polls and Alternative Process Frameworks 4 ReferencesHistory editCRISP DM was conceived in 1996 and became a European Union project under the ESPRIT funding initiative in 1997 The project was led by five companies Integral Solutions Ltd ISL Teradata Daimler AG NCR Corporation and OHRA an insurance company This core consortium brought different experiences to the project ISL was later acquired and merged into SPSS The computer giant NCR Corporation produced the Teradata data warehouse and its own data mining software Daimler Benz had a significant data mining team OHRA was starting to explore the potential use of data mining The first version of the methodology was presented at the 4th CRISP DM SIG Workshop in Brussels in March 1999 5 and published as a step by step data mining guide later that year 6 Between 2006 and 2008 a CRISP DM 2 0 SIG was formed and there were discussions about updating the CRISP DM process model 7 The current status of these efforts is not known However the original crisp dm org website cited in the reviews 8 9 and the CRISP DM 2 0 SIG website are both no longer active 7 While many non IBM data mining practitioners use CRISP DM 10 11 12 IBM is the primary corporation that currently uses the CRISP DM process model It makes some of the old CRISP DM documents available for download and it has incorporated it into its SPSS Modeler product 6 Based on current research CRISP DM is the most widely used form of data mining model because of its various advantages which solved the existing problems in the data mining industries Some of the drawbacks of this model is that it does not perform project management activities The success of CRISP DM is largely attributable to the fact that it is industry tool and application neutral 13 Major phases edit nbsp Process diagram showing the relationship between the different phases of CRISP DM CRISP DM breaks the process of data mining into six major phases 14 Business Understanding Data Understanding Data Preparation Modeling Evaluation Deployment The sequence of the phases is not strict and moving back and forth between different phases is usually required The arrows in the process diagram indicate the most important and frequent dependencies between phases The outer circle in the diagram symbolizes the cyclic nature of data mining itself A data mining process continues after a solution has been deployed The lessons learned during the process can trigger new often more focused business questions and subsequent data mining processes will benefit from the experiences of previous ones Polls and Alternative Process Frameworks editPolls conducted at the same website KDNuggets in 2002 2004 2007 and 2014 show that it was the leading methodology used by industry data miners who decided to respond to the survey 10 11 12 15 The only other data mining approach named in these polls was SEMMA However SAS Institute clearly states that SEMMA is not a data mining methodology but rather a logical organization of the functional toolset of SAS Enterprise Miner A review and critique of data mining process models in 2009 called the CRISP DM the de facto standard for developing data mining and knowledge discovery projects 16 Other reviews of CRISP DM and data mining process models include Kurgan and Musilek s 2006 review 8 and Azevedo and Santos 2008 comparison of CRISP DM and SEMMA 9 Efforts to update the methodology started in 2006 but have as of June 2015 not led to a new version and the Special Interest Group SIG responsible along with the website has long disappeared see History of CRISP DM In 2024 Harvard Business Review published an updated framework bizML that is designed for greater relevance to business personnel and to be specific for machine learning projects in particular rather than for analytics data science or data mining projects in general 17 References edit Shearer C The CRISP DM model the new blueprint for data mining J Data Warehousing 2000 5 13 22 What IT Needs To Know About The Data Mining Process Published by Forbes 29 July 2015 retrieved June 24 2018 Have you seen ASUM DM By Jason Haffar 16 October 2015 SPSS Predictive Analytics IBM Archived 8 March 2016 at the Wayback Machine Analytics Solutions Unified Method Implementations with Agile principles Published by IBM 1 March 2016 retrieved October 5 2018 Pete Chapman 1999 The CRISP DM User Guide a b Pete Chapman Julian Clinton Randy Kerber Thomas Khabaza Thomas Reinartz Colin Shearer and Rudiger Wirth 2000 The CRISP DM User Guide entry on semantic scholar including links to PDFs PDF version with high resolution graphics Archived 12 September 2020 at the Wayback Machine a b Colin Shearer 2006 First CRISP DM 2 0 Workshop Held a b Lukasz Kurgan and Petr Musilek 2006 A survey of Knowledge Discovery and Data Mining process models The Knowledge Engineering Review Volume 21 Issue 1 March 2006 pp 1 24 Cambridge University Press New York NY USA doi 10 1017 S0269888906000737 a b Azevedo A and Santos M F 2008 KDD SEMMA and CRISP DM a parallel overview In Proceedings of the IADIS European Conference on Data Mining 2008 pp 182 185 a b Gregory Piatetsky Shapiro 2002 KDnuggets Methodology Poll a b Gregory Piatetsky Shapiro 2004 KDnuggets Methodology Poll a b Gregory Piatetsky Shapiro 2007 KDnuggets Methodology Poll Mariscal G Marban O Fernandez C 2010 A Survey of Data Mining and knowledge discovery process Models and methodologies The Knowledge Engineering Review 25 2 137 166 doi 10 1017 S0269888910000032 S2CID 31359633 a href Template Cite journal html title Template Cite journal cite journal a CS1 maint multiple names authors list link Harper Gavin Stephen D Pickett August 2006 Methods for mining HTS data Drug Discovery Today 11 15 16 694 699 doi 10 1016 j drudis 2006 06 006 PMID 16846796 Gregory Piatetsky Shapiro 2014 KDnuggets Methodology Poll Martinez Plumed Fernando Contreras Ochando Lidia Ferri Cesar Flach Peter Hernandez Orallo Jose Kull Meelis Lachiche Nicolas Ramirez Quintana Maria Jose 19 September 2017 CASP DM Context Aware Standard Process for Data Mining arXiv 1709 09003 cs DB Eric Siegel 2024 Getting Machine Learning Projects from Idea to Execution Retrieved from https en wikipedia org w index php title Cross industry standard process for data mining amp oldid 1217762067, wikipedia, wiki, book, books, library,

article

, read, download, free, free download, mp3, video, mp4, 3gp, jpg, jpeg, gif, png, picture, music, song, movie, book, game, games.