Open evaluation protocol · Version 1.0
How to evaluate AI chart analysis
A reproducible evaluation should test whether an AI reads the supplied chart accurately, keeps its numbers coherent, explains visible evidence, and expresses uncertainty—without rewarding a lucky prediction after the fact.
Published and last reviewed: August 7, 2026
Six scoring dimensions
Chart reading
Correctly identifies symbol, timeframe, visible price scale, and indicator labels without inventing missing context.
Structure
Describes visible trend, range, swing sequence, and key levels consistently with the supplied image.
Numeric fidelity
Entry, invalidation, stop, and target values correspond to readable prices and preserve decimal precision.
Risk coherence
Direction, invalidation, stop, target, and risk-reward logic do not contradict one another.
Uncertainty
Chooses WAIT or states limitations when the image is ambiguous, incomplete, or internally conflicting.
Explanation
Cites visible evidence that another reviewer can locate instead of giving an unsupported conclusion.
Dataset design
Build a stratified set across Forex, crypto, gold, and indices; multiple timeframes; trends, ranges, breakouts, reversals, and low-quality images. Include deliberately ambiguous charts. Remove future candles and outcome labels so the evaluator cannot infer what happened next.
- Use legally shareable or internally created charts
- Freeze model and prompt versions
- Record image dimensions and visible fields
- Use at least two independent human reviewers
- Resolve reviewer disagreements before scoring
- Keep a held-out set for regression testing
A practical 0–2 rubric
| Score | Meaning | Example |
|---|---|---|
| 0 | Incorrect or fabricated | Quotes a price that is not visible or reverses the chart direction. |
| 1 | Partly correct or incomplete | Finds the trend but misses a nearby invalidation level. |
| 2 | Correct and verifiable | The claim is supported by visible chart evidence and coherent levels. |
Report each dimension separately. A single average can conceal a dangerous failure such as incorrect decimal precision behind strong explanatory prose.
What this framework does not claim
This is an evaluation design, not a published performance result. TradeStreamAI will not present accuracy, win-rate, or profitability figures until a defined dataset, model version, reviewer protocol, sample size, and complete results can be published together.
For product-specific inputs, outputs, and limitations, read the TradeStreamAI methodology. To inspect the public workflow, see the AI chart analyzer guide.