October 4, 2006
Data mining is hugely important, but it does have issues with accessibility. The traditional model of data mining goes something like this:
- Data is assembled in a data warehouse from transactional information, with all the effort and expense that requires. Maybe more data is even deliberately gathered. Or maybe the data is in large part acquired, at moderate cost, from third-party providers like credit bureaus.
- The database experts fire up long-running, expensive data extraction processes to select data for analysis. Often, special data warehousing technology is used just for that purpose.
- The statistical experts pound away at the data in their dungeons, torturing it until it reveals its secrets.
- The results are made available to business operating units, both as reports and in the form of executable models.