Fundamental mathematical formulas for machine learning (optimization): argmax Reasoning Backward from the Future

arg_max = F(x) Fundamental Machine Learning Equation

argmax: Reasoning Backward from the Future

The fundamental mathematical principle behind all of machine learning (and optimization) is the following formula:

arg_max_{x∈X} F(x)

It means: among all possible inputs , find the one that maximizes the objective function F(x). The result of this expression is the optimal input x, not the maximum value itself.

This formula represents the idea of reasoning backward from the future to make the best choice now. The space of all possible x’s is often unenumerable, and the function F is usually unknown ahead of time—it’s something we try to learn or approximate. This reflects a “perfect-information” or “goal-directed” perspective.

In contrast, traditional recursive mathematics works in the other direction: using the past to predict the future. For example, the Fibonacci sequence:

Here, we assume F(n-1) and F(n-2) are already known, and we infer F(n). This is the core principle behind dynamic programming. It is a way of reasoning forward from the past.

Therefore, machine learning and optimization are essentially about predicting the future. In fact, the argmax formula can be naturally expressed in the following Python code:

def arg_max(X, F):
    best_x = None
    best_score = float('-inf')
    for x in X:
        score = F(x)
        if score > best_score:
            best_score = score
            best_x = x
    return best_x

However, in real-world applications, we can’t directly implement this because:

• X is an unenumerable set (e.g., all images, all sentences, all action strategies).
• F(x) is a subjective model that needs to be learned or assumed.

We can however, possibly parallelize the search for best_x in above e.g. parallel_for x in X via multithreading.

Machine Learning Algorithms

• Fundamental mathematical formulas for machine learning (optimization): argmax Reasoning Backward from the Future

Math

• Understanding the Sigma Function: Divisors, Multiplicativity, and the Formula
• Toss a Coin 256 Times: The Math Behind an Unbreakable Bitcoin Private Key
• Fundamental Mathematical Formulas for Machine Learning Optimization: Argmax & Reasoning Backward from the Future
• Tetration Operator in Math Simply Explained with Python Algorithms
• ChatGPT-4 uses Math Wolfram Plugin to Solve a Math Brain Teaser Question
• Bash Script to Compute the Math Pi Constant via Monte Carlo Simulation
• Design a Binary Integer Divisble by Three Stream Using Math

–EOF (The Ultimate Computing & Technology Blog) —