Andrey Andreyevich Markov was a child with serious health problems; He walked with the help of crutches until the age of 10. Perhaps for this reason, he spent more time in books than anyone else in his early years. He fell in love with mathematics, so in love that he excelled in it and more or less ignored other subjects in school, leading to him being tagged stubborn and “rebellious”.
His work in his chosen field will ultimately change the way planners, statisticians and programmers look at our world. As a professor of mathematics and statistics at his alma mater, St. Petersburg University, he jointly designed number theory and probability theory in what is now known as Markov processes.
It is an approach to information processing that factors in constant change. In an era long before the dawn of computing, his approach – which is relatively simple when broken down for each application – embraces the idea of steady flow, and uses possible scenarios of change to benefit the system as a whole.
We’ll get to exactly how, but first, consider the far-reaching implications of such a system. It has applications in traffic management, hospital administration, gaming and finance — any field, really, that’s variable by nature.
One area where Markov processes (and its components, Markov chains) have become pivotal in the 21st century is the training of artificial intelligence (AI) programs.
Here’s how it works. Markov chains allow a live system – whether it’s a set of traffic signals or a chatbot conversation – to use its current state to predict which state will follow next and chart its next steps accordingly.
So, to use a simple example, a system of traffic lights can respond to a specific number of cars waiting at a signal by adjusting the duration of the green light to follow. Once the green light is turned on, it starts using only its current data set – the number of cars passing by – to predict its next step.
In the age of artificial intelligence, this may seem like an obvious thing to do, but it helps that frameworks like decision trees, which operate on the “if-then” principle, remain key building blocks in the world of software. . A decision tree is used, for example, by software programs on basic ATM machines. Here, only a limited number of default options can be adjusted. Each signal, the possibility that the user can choose in terms of options, is placed in advance for the machine; Each step is choreographed.
Whereas a decision-tree software program acts akin to a well-rehearsed ballet, a Markov-driven AI model is like a beautifully impromptu jazz performance on a street corner, among strangers.
Rather than requiring a software program to essentially read its cue cards and complete the next step, a Markov process gives the program tools, and thus the ability to recalculate and recalibrate, and “make decisions” as the process progresses.
And so it is that ChatGPT, Bing and other AI chatbots can conduct interactions with the user’s voice, and provide data and creative output (stories, poems, explainers).
Try the ChatGPT prompt “Write me a poem about 2 o’clock in the style of a nursery rhyme” versus “Write me a poem about 2 o’clock in the style of Robert Frost” and here’s a sample of what happens (we’ll leave it up to you to decide which is which).
Sample 1
The night is dark, the air is cold,
But all is quiet and all is still.
The world outside is asleep,
While I’m quietly counting some sheep.
Sample 2
The woods are dark and the stars are bright,
When I’m thinking about the meaning of this lonely night.
The silence is only broken by my deep breath,
And the rustling of leaves in the cool night air.
Putting aside the quality of the term, which like all terms is debatable, ChatGPT could not have delivered the two different styles randomly, quickly and seamlessly without the input and feedback approach of the mid-19th century mathematician Markov.
Millions of random and unexpected data sets such as two poems form the basis of what this AI language model learns; Markov processes form the basis of how that information is used.
How does a Markov process compare to machine learning, another method widely used to “teach” AI programs? With machine learning, in order to classify, examine, investigate, all data must be present. A machine learning program can only adjust within a limited and predetermined range of contexts, and cannot deviate from its set responses based on real-time signals.
One thing that cannot be overlooked, however, when using a Markov process, is the margin of error. Each error leads to a new learning point, making each new scenario encountered a factor for future use. But this makes human involvement a necessity, not just a recommendation.
In a 2013 study on artificial intelligence in medicine by researchers at Indiana University and Centerstone Research Institute in the US, Markov processes were used as the basis for AI-driven decision-making in healthcare. The study used AI to think like a doctor among variables such as cost and complexity, policies and payment methods to develop a computational framework for non-disease-specific scenarios. It was in use to monitor healthcare equipment used by patients, and could predict an upcoming potential failure.
Such a program could largely complement human intervention, the study found.
Ask ChatGPT what the jobs of the future might be, as AI takes over more tasks, and it offers, in #3, without prompting: “Healthcare professionals — the healthcare industry is already using AI and machine learning… There will be a growing need for healthcare professionals who can work alongside them.” (The rest of its top 5 are made up of data scientists, software developers, cybersecurity experts and “as the world becomes more aware of the impact of human activity on the planet”, environmental scientists.)
Partly because of the Markov process, the AI is learning, growing, and “knowing” what it is.
Over time, will the process help AI take over? Not if the quality of the “Robert Frost” verse is any indication.