Data Mining

I'm current doing a number of things that revolve around Data Mining and Data Visualisation. I'm interested to hear (and I'm looking for) any tools that are helpful when looking for patterns and useful ways to display data or information.

There are quite a few introductory papers and discussions put out by universities and other institutions. Here are a few I've found or been pointed to:

It seems as though data mining does have some established processes, but can also simply be the application of more than just reporting or aggregation to a data set. The preparation could involve SQL and other reporting methods, but the mining extends to much more sophisticated algorithms. Data mining is still only as good as the data that is used, and the analysis methods utilised to extract information.

The most commonly used description of Data Mining, or Knowledge Discovery and Data Mining (KDD) is as a "non-trivial process of identifying valid, novel, potentially useful, and ultimately understandable patterns in data". This comes from an article by Fayyad, Usama M. and Piatetsky-Shapiro, Gregory and Smyth, Padhraic, 1996. From data mining to knowledge discovery: an overview. In Advances in knowledge discovery and data mining (0-262-56097-6). American Association for Artificial Intelligence, Menlo Park, CA, USA, 1--34 This description requires some expansion to really be useful. It can also be argued that it is too vague. However, it is the standard definition.

The next step is to use some of these methods and tools to data mine! I'll be doing some at QUT over this semester, as well as hopefully some of my own investigation.