A Data Mining & Knowledge Discovery Process Model

Great article with a review of data mining & knowledge discovery process models and methodologies.

Usually this is a process that the data scientists or data modellers don't give the correct importance, but it avoids tons of re-processing and helps to delivery a better solution.

A short intro: "The number of applied in the data mining and knowledge discovery (DM & KD) projects has increased enormously over the past few years (Jaffarian et al., 2008) (Kdnuggets.com, 2007c). As DM & KD development projects became more complex, a number of problems emerged: continuous project planning delays, low productivity and failure to meet user expectations. Neither all the project results are useful (Kdnuggets.com, 2008) (Eisenfeld et al., 2003a) (Eisenfeld et al., 2003b) (Zornes, 2003), nor do all projects end successfully (McMurchy, 2008) (Kdnuggets.com, 2008) (Strand, 2000) (Edelstein & Edelstein, 1997). Today’s failure rate is over 50% (Kdnuggets.com, 2008) (Gartner, 2005) (Gondar, 2005). This situation is in a sense comparable to the circumstances surrounding the software industry in the late 1960s. This was what led to the ’software crisis’ (Naur & Randell, 1969). Software development improved considerably as a result of the new methodologies. This solved some of its earlier problems, and little by little software development grew to be a branch of engineering. This shift has meant that project management and quality assurance problems are being solved. Additionally, it is helping to increase productivity and improve software maintenance. The history of DM & KD is not much different."

Check the full article at http://cdn.intechopen.com/pdfs/5937/InTech-A_data_mining_amp_knowledge_discovery_process_model.pdf