The Most Frequently Used Machine Learning Algorithms are Several Years Old

Today's Retro Reality Data Science

Apr 18, 2024

In the fast-paced world of technology, it's easy to assume that the most cutting-edge algorithms are always the most effective. Yet, finding people who know the origin and history of machine learning algorithms is unusual. Understanding this history is not just a matter of academic interest but a key to unlocking the full potential of data science. There has been a pervasive bias against the data science methodology for several decades, stemming from the perceived lack of sophistication, elegant mathematics, and scientific complexity in data science techniques compared to the more glamorous and cutting-edge approaches in computer science. This bias has led to a need for more common sense and practical business solutions, as the focus has been on pursuing the latest trends and buzzwords.

Linear Regression and Ensemble Methods of Random Forest and Boosting are the most commonly used Machine Learning algorithms. Linear Regression was invented in the 1800s and implemented in software programs in the 1960s. Ensemble Methods were implemented in the 1990s, and processes were improved in the early 2000s. Moreover, single input, single output Time Series Analysis was invented a century ago and implemented in software in the 1960s but became popular in the 1970s among professional statisticians.

Deep Learning, which I teach at universities in California, is a layered version of Machine Learning. Its primary focus is the Neural Network, which began in World War II but was later implemented in the 1950s when John McCarthy coined the term Artificial Intelligence. After several stagnant years of neglect, in the 1980s, Deep Learning Artificial Intelligence began to expand with multi-layered Neural Networks.

Reinforcement Learning, a mostly model-free and assumption-free approach, was invented in the 1950s and implemented into software programs in the 1980s. It combines the academic literature of behavioral psychology with the real-world application of control theory in engineering. Today, its products enrich lives with Recommendation Engines and Personalization Algorithms.

Natural Language Processing started in the 1950s as an original thought and was implemented for machine translation and language translation. It was followed by 1960s chatbots, 1970s language representation, and computational linguistics in the 1980s. The complexity and diversity of Human Language cause many modeling difficulties, leading to the unusual collaboration of computer science, linguistics, and psychology. However, the recent impact of Large Language Models like ChatGPT must be considered, even though they have existed in the open-source community for many years.

The History of Data Science, Machine Learning, and Artificial Intelligence has yet to be written formally in a publication or collaborated in an organization. For several reasons, academics and working professionals have dismissed and ignored the methodologies mentioned in this article. Here is a quick summary of why we are going back in history to go forward with Artificial Intelligence.

The Lack of Computing Resources

In the 1950s and 1960s, a tiny part of the population had access to and could process computer software programs in Data Science. The distribution of computers began to grow in the 1980s and then exponentially in the 1990s, but this was still three or more decades ago. In isolation, the lack of computing resources cannot explain the historical neglect of Machine Learning.

Underfunding of Statistics

Data science, Machine Learning, and Artificial Intelligence are the applications of statistics to business cases. Statistics is in every discipline in the Social Sciences and the Sciences, but Computer Science receives much more funding from universities and businesses several times over. There may be shared resources like a computer lab, but the motivations and objectives are different, such as the value of experimentation in Statistics compared to execution in computer science and engineering. Even more, the mathematics involved in a Data Science Bachelor's degree is less than a Bachelor's in Mathematics.

Methodological Bias

Humans prefer brain-driven, elegant math with a single complex equation over its alternatives. Inputting information into one model with a perfect output is a centuries-old bias. Data-driven iteration through a single model to reduce errors over time or doing the same by propagating a set of models in a network architecture over time is still unusual for many people worldwide. Not to mention, the radical idea of removing the concept of a model in the first place, like Reinforcement Learning, has been done for several decades. Mindset is everything in learning how Data Science, Machine Learning, and Artificial Intelligence can deliver value to any organization, and the number one priority is not elegance, sophistication, or even complexity but usefulness.

Quote by Statistician George Box, in 1976

Check out our Individual AI Accelerator and AI Incubator for Corporate Team Training.

The Global Institute of Data Science

Discussion about this post