Machine learning - History and Interdisciplinary Relationships

History and Context of Machine Learning What Machine Learning Is: The Formal Definition To truly understand machine learning, it helps to start with a precise definition. Tom Mitchell formalized what it means for a program to "learn." According to his definition, a computer program learns from experience ($E$) with respect to a task ($T$) and a performance measure ($P$) if its performance on task $T$, as measured by $P$, improves with experience $E$. Let's break this down with an example. Consider a spam email classifier: Experience ($E$): The program sees thousands of labeled emails (spam or not spam) Task ($T$): Classify new incoming emails as spam or legitimate Performance measure ($P$): The percentage of emails correctly classified As the program processes more emails and updates its internal rules, its classification accuracy improves. This improvement through experience is what we call learning. This definition is crucial because it moves us beyond vague notions of "smart computers" to something measurable and precise. A Milestone: AlphaGo and Superhuman Performance In 2016, Google's AlphaGo defeated Lee Sedol, one of the world's top players of Go, a game far more complex than chess. This wasn't just a victory—it demonstrated that reinforcement learning (a type of machine learning where programs learn through trial, reward, and punishment) could achieve superhuman performance on tasks requiring intuition and strategic thinking. This milestone showed the world that machine learning had matured from theoretical research to practical, world-changing capability. Machine Learning's Relationship to Other Fields Understanding machine learning requires seeing how it relates to—and differs from—adjacent fields. Let's examine three important relationships. Machine Learning and Artificial Intelligence Machine learning is actually a subset of artificial intelligence. Think of AI as the broad umbrella: any technique that mimics intelligent behavior or automates decision-making falls under AI. Machine learning is one particularly powerful approach within this broader field. The relationship became especially important in the mid-1980s during what's called the connectionist revival. Researchers rediscovered neural networks and implemented backpropagation—an algorithm that allows neural networks to learn from their mistakes by adjusting their internal weights. This breakthrough reinvigorated AI research and laid the foundation for modern deep learning. Machine Learning and Data Mining Machine learning and data mining are related but distinct. Here's the key difference in their goals: Machine learning focuses on prediction. Given known properties or features about something, can we predict an unknown outcome? For example, given a person's age, income, and browsing history, predict whether they'll click an advertisement. Data mining focuses on discovery. What new, previously unknown patterns or structures exist in a large dataset? For example, discovering that customers who buy diapers at certain times also tend to buy beer—a famous (possibly apocryphal) retail pattern. However, the relationship is symbiotic. Data mining often uses machine learning methods—particularly unsupervised learning techniques (which find patterns without predefined labels) and preprocessing steps (which clean and prepare data). The crucial distinction lies in how we evaluate success: machine learning asks "how accurately can we predict known labels?" while data mining asks "what new knowledge can we uncover?" Machine Learning and Statistics This is where it gets subtle, because machine learning and statistics actually have significant overlap. Both fields work with data to make inferences, but their philosophies differ: Traditional statistics draws inferences about an entire population based on a sample. It works from a pre-specified model (you decide on a mathematical form first), then fits data to that model. For instance, a statistician might assume data follows a normal distribution, then estimate the mean and variance. Machine learning seeks generalizable predictive patterns without assuming a particular model structure in advance. Instead, the data itself shapes what model emerges. A machine learning approach might discover that a complex, non-linear relationship best captures the pattern—something a traditional model might miss. Leo Breiman, an influential statistician, formalized this distinction by contrasting: Data models (the statistical approach): Assume a specific mathematical form, then fit it Algorithmic models (the machine learning approach): Let the algorithm find patterns flexibly Today, these fields are increasingly merging into statistical learning—a discipline that combines the rigor and theoretical grounding of statistics with the flexibility and power of machine learning algorithms. This integration produces techniques that are both theoretically sound and practically powerful. The key point: machine learning is more flexible but requires large amounts of data, while traditional statistics is more rigid but can work with smaller samples. Each has strengths the other lacks.