Quantitative forecasting is an age-old discipline, highly useful across different functions of an organization: from forecasting sales and workforce demand to economic forecasting and inventory planning. Business schools have offered courses with titles such as “Time Series Forecasting”, “Forecasting Time Series Data“, “Business Forecasting“, more specialized courses such as “Demand Planning and Sales Forecasting” or even graduate programs with title “Business and Economic Forecasting“. Simple “Forecasting” is also popular. Such courses are offered at the undergraduate, graduate and even executive education. All these might convey the importance and usefulness of forecasting, but they are far from conveying the coolness of forecasting. … Continue reading Forecasting + Analytics = ?
Some time ago, when I presented the “explain or predict” work, my colleague Avi Gal asked where simulation falls. Simulation is a key method in operations research, as well as in statistics. A related question arose in my mind when thinking of Scott Nestler‘s distinction between descriptive/predictive/prescriptive analytics. Scott defines prescriptive analytics as “what should happen in the future? (optimization, simulation)“. So where does simulation fall? Does it fall in a completely different goal category, or can it be part of the explain/predict/describe framework? My opinion is that simulation, like other data analytics techniques, does not define a goal in … Continue reading Explain or predict: simulation
I attended today’s class in the course Trading Strategies and Systems offered by Prof Vasant Dhar from NYU Stern School of Business. Luckily, Vasant is offering the elective course here at the Indian School of Business, so no need for transatlantic travel. The topic of this class was the use of news in trading. I won’t disclose any trade secrets (you’ll have to attend the class for that), but here’s my point: Trading is a striking example of the distinction between explanation and prediction. Generally, techniques are based on correlations and on “blackbox” predictive models such as neural nets. In particular, text mining and … Continue reading Trading and predictive analytics
Quite a few of my social science colleagues think that predictive modeling is not a kosher tool for theory building. In our 2011 MISQ paper “Predictive Analytics in Information Systems Research” we argue that predictive modeling has a critical role to play not only in theory testing but also in theory building. How does it work? Here’s an interesting example: The new book The Secret Life of Pronouns by the cognitive psychologist Pennebaker is a fascinating read in many ways. The book describes how analysis of written language can be predictive of psychological state. In particular, the author describes an … Continue reading Language and psychological state: explain or predict?
Image from KDnuggets.com While debates over privacy issues related to electronic health records are still ongoing, predictive analytics are beginning to being used with administrative health data (available to health insurance companies, aka, “health provider networks”). One such venue are large data mining contests. Let me describe a few and then get to my point about their contribution to pubic health, medicine and to data mining research. The latest and grandest is the ongoing $3 million prize contest by Hereitage Provider Network, which opened in 2010 and lasts 2 years. The contest’s stated goal is to create “an algorithm that … Continue reading Mining health-related data: How to benefit scientific research
What is the difference between “prediction” and “forecasting”? I heard this being asked quite a few times lately. The Predictive Analytics World conference website has a Predictive Analytics Guide page with the following Q&A: How is predictive analytics different from forecasting? Predictive analytics is something else entirely, going beyond standard forecasting by producing a predictive score for each customer or other organizational element. In contrast, forecasting provides overall aggregate estimates, such as the total number of purchases next quarter. For example, forecasting might estimate the total number of ice cream cones to be purchased in a certain region, while predictive analytics tells you which individual … Continue reading “Predict” or “Forecast”?
Data mining is taught very differently at engineering schools and at business schools. At engineering schools, data mining is taught more technically, deciphering how different algorithms work. In business schools the focus is on how to use algorithms in a business context. Business students with a computer science background can now enjoy both worlds: take a data mining course with a business focus, and supplement it with the free course materials from Stanford Engineering school’s Machine Learning course (including videos of lectures and handouts by Prof Andrew Ng). There are a bunch of other courses with free materials as part of the Stanford … Continue reading Where computer science and business meet
Yesterday I happened to hear talks by two excellent speakers, both on major data mining applications in industry. One common theme was that both speakers gave compelling and easy to grasp examples of what data mining algorithms and statistics can do beyond human intelligence, and how the two relate. The first talk, by IBM’s Global Services Christer Johnson, was given at the 2011 INFORMS Conference on Business Analytics and Operations Research (see video). Christer Johnson described the idea behind Watson, the artificial intelligence computer system developed by IBM that beat two champions of the Jeopardy quiz show. Two main points in the talk … Continue reading The potential of being good
Business Intelligence and Data Mining have become hot buzzwords in the West. Using Google Insights for Search to “see what the world is searching for” (see image below), we can see that the popularity of these two terms seems to have stabilized (if you expand the search to 2007 or earlier, you will see the earlier peak and also that Data Mining was hotter for a while). Click on the image to get to the actual result, with which you can interact directly. There are two very interesting insights from this search result: Looking at the “Regional Interest” for these terms, we see that … Continue reading Analytics: You want to be in Asia
The American Statistical Association’s store used to sell cool T-shirts with the old-time beggar-statistician question “Got Data?” Today it is much easier to find data, thanks to the Internet. Dozens of student teams taking my data mining course have been able to find data from various sources on the Internet for their team projects. Yet, I often receive queries from colleagues in search of data for their students’ projects. This is especially true for short courses, where students don’t have sufficient time to search and gather data (which is highly educational in itself!). One solution that I often offer is … Continue reading Got Data?!