AI in Banking. Watch the Math. – Greystone Advisors

Large Language Models (LLM) are the driving capability behind current frontier AI models and are extremely adept at language. This sophistication can create a sense of accuracy that should be approached with caution when attempting to complete detailed mathematical activities.

LLMs have had (and in some fundamental ways still have) difficulties with math, though the picture in 2026 is much more nuanced than it was a few years ago.

Why they struggle

LLMs are next-token predictors trained primarily on text. They excel at statistical patterns in language but are not inherently symbolic reasoners or precise calculators. Key limitations include:

Tokenization and number representation: Byte-pair encoding (BPE) often splits numbers in awkward ways that disrupt place value and arithmetic structure.
Lack of true “number sense” or grounding: Models can fail elementary tasks (e.g., reliable magnitude comparisons like 9.11 vs. 9.9, large-integer addition, or basic operations in scientific notation) even while handling advanced problems. These are sometimes called numeracy failures.
Error accumulation in multi-step reasoning: A single flawed intermediate step can derail the whole solution. They may produce correct final answers via flawed logic, make unwarranted assumptions, or struggle to translate real-world/physical intuition into formal steps.
Contextual formulation: Performance often drops sharply when the mathematical core must be extracted from messy natural-language descriptions rather than clean, abstract problem statements.
Proof rigor and edge cases: Generating fully correct, rigorous proofs (especially without formal tools) remains harder than producing plausible-looking solutions. Recognizing unsolvable or ill-posed problems is also a known weak spot.

These issues stem from the architecture (transformers process tokens in parallel with limited algorithmic fidelity for exact computation) and training data distributions rather than simply “not enough scale.”

How they have evolved

By 2025–2026, top models achieve very high scores on many standard benchmarks:

Near-saturation or high-90s% on GSM8K (grade-school word problems) and the MATH dataset.
Strong performance (often 90%+) on AIME-style contest problems, with some models approaching or hitting perfect scores on recent versions under the right conditions.
Specialized reasoning models (with extended chain-of-thought, process supervision, etc.) handle complex multi-step problems far better than earlier generations.

Frontier systems are even contributing to research-level mathematics—claiming solutions to long-open problems (including high-profile ones) and generating large volumes of new results. This has sparked significant discussion in the math community about verification standards, human understanding of AI-generated proofs, formalization (e.g., into Lean), and the broader role of AI in the field.

Practical reality

With tools (code interpreters, calculators, formal theorem provers), modern LLMs become highly reliable for most practical math, computation, and verification tasks.
Pure text-only mode still shows residual quirks on basic arithmetic, certain edge cases, and fully rigorous novel proofs.
Performance varies by model family, prompting technique (chain-of-thought helps a lot), and problem type. Conceptual explanation, formula derivation, and high-level strategy are usually strengths; pure high-precision calculation without external help is a relative weakness.

Further explanation of model limitations

LLMs are “bad” at math (relative to their strengths in language) primarily because of how they are built and trained, not because of a simple lack of data or scale. Here are the core reasons:

1. They are probabilistic pattern-matchers, not calculators or symbolic engines

Large language models predict the next token based on statistical patterns learned from text. Mathematics demands exact, rule-based computation and logical rigor. A model can generate a fluent-looking solution that follows common patterns in its training data while still making arithmetic or logical errors. Plausibility is rewarded during training; mathematical correctness is not an explicit objective.

2. Tokenization breaks numbers

Most LLMs use byte-pair encoding (BPE) or similar subword tokenizers optimized for natural language frequency, not numerical structure. Numbers are often split into awkward pieces (e.g., “123456” might become multiple tokens that do not cleanly preserve place value). This damages the model’s ability to perform reliable arithmetic, especially with large integers, decimals, or scientific notation. Even simple magnitude comparisons (such as 9.11 vs. 9.9) can fail for this reason.

3. Weak numerical grounding / lack of “number sense”

LLMs do not have an internal representation of numbers as quantities in the way humans (or dedicated numerical systems) do. They treat numerals largely as symbols that co-occur with other symbols. This leads to persistent numeracy failures—elementary operations, comparisons, and precision handling—even in models that perform well on advanced contest or research-level problems. Researchers link this to the classic symbol-grounding problem: the model never truly maps tokens to mathematical values with algorithmic fidelity.

4. Error accumulation in multi-step reasoning

Math problems usually require a chain of precise intermediate steps. A single small mistake (wrong sign, dropped carry, unjustified assumption, misapplied theorem) cascades. While chain-of-thought prompting and process supervision help, pure next-token prediction remains brittle for long, exact sequences.

5. Training data and objective mismatch

Pretraining corpora contain plenty of mathematical text, but much of it is informal, incomplete, or presented in ways that encourage pattern matching rather than executable algorithms. The loss function optimizes for predicting likely text, not for verifying correctness or executing algorithms. As a result, models can memorize solution templates or heuristics that work on familiar distributions while failing on slight variations or basic edge cases.

Additional contributing factors

Positional encodings and embedding geometry are not optimized for the sequential, place-value nature of arithmetic.
Models sometimes produce correct final answers via flawed intermediate reasoning (or vice versa).
They can struggle more when the math must be extracted from messy natural-language descriptions rather than clean abstract statements.

Important nuance (as of 2026)

These limitations are real and well-documented, yet frontier models have become dramatically better at high-level mathematical reasoning through scale, specialized training (including process supervision and implicit chain-of-thought methods), and especially tool use (code interpreters, formal provers such as Lean, external calculators). Many standard benchmarks (GSM8K, MATH, AIME-style contests) are now near saturation for top systems. Research-level problem solving has also advanced rapidly—though verification, human-understandable proofs, and basic numeracy quirks remain active issues.

In conclusion, LLMs are not inherently arithmetic engines. Their architecture and training make exact calculation and rigorous symbolic manipulation unnatural, which is why pure language-model math remains imperfect even as overall capability has soared. Hybrid systems that combine LLMs with external tools largely sidestep the worst of these weaknesses.

Source: Grok, Greystone Advisors.