Skip to content

Notealy / Aegis research

Building intelligence for a safer internet.

An open view into the development of Aegis, our specialized AI model for autonomous security intelligence.

Status
Aegis · Active development
Current version
Aegis-1 · Development build
Last update

01Model overview

Meet Aegis.

Aegis is our security-focused AI initiative, designed to advance how vulnerabilities are discovered, understood, validated, and prioritized.

Aegis-1, development build. An open-weight base model with defensive-security instructions. It has not been fine-tuned yet; it is the starting point that future versions are measured against.

  • Vulnerability detection

    Identify relevant security weaknesses in code and configuration from the evidence provided.

    In development

  • Security classification

    Map each identified issue to the correct weakness category, using the MITRE CWE taxonomy.

    In development

  • Contextual reasoning

    Decide whether untrusted input actually reaches a sensitive operation, instead of matching on patterns.

    In development

  • Evidence-based validation

    Support findings with verifiable evidence gathered by security tools, kept separate from model judgment.

    Planned

  • Remediation intelligence

    Explain root causes and propose fixes that can be retested to confirm the issue is resolved.

    Planned

No capability is marked available yet. A capability moves to available only once it ships in a released model and its evaluation has been published here.

02Development progress

Model development, measured.

Every improvement should be measurable, reproducible, and grounded in security research.

Current model version
Aegis-1

Development build

Latest evaluation
November 4, 2026
Evaluated cases
24
Detection
71.0% f1 score
Classification
0.0% cwe accuracy
End-to-end
33.3% task success rate

Values come only from evaluations that have been reviewed and approved for publication, for the current Aegis version. Anything not yet measured is shown as pending, never estimated.

03Benchmark performance

Measuring what matters.

A single accuracy number hides the failures that matter in security. Each benchmark reports the metrics that describe its errors, defined in plain language.

01

Detection

Can the model identify relevant security issues from the provided evidence?

Held-out code samples, each either vulnerable or a secure counterpart. Secure counterparts matter: a model that flags everything would look perfect on recall alone.

Metrics

F1 scoreheadline
A single balance of precision and recall; it is low if either one is low.
Precision
Of the issues the model reported, the share that were real.
Recall
Of the real issues present, the share the model found.
False positive rate
Of the secure cases, the share the model wrongly flagged. Lower is better.

One Aegis measurement so far. A trend chart appears once a second version has been evaluated on the same benchmark; the baseline is shown for reference only.

  • Baseline · Reference model

    · security-reasoning-test-v1 · 24 cases

    F1 score
    62.5%
    Precision
    50.0%10/20
    Recall
    83.3%10/12
    False positive rate
    83.3%10/12

  • Aegis-1 · Development build

    · security-reasoning-test-v1 · 24 cases

    F1 score
    71.0%
    Precision
    57.9%11/19
    Recall
    91.7%11/12
    False positive rate
    66.7%8/12

02

Classification

Can the model correctly categorize the issues it identifies?

The vulnerability is described to the model, so detection is not required; it only has to name the right MITRE CWE category. Identifiers that do not exist in the catalog count as wrong.

Metrics

CWE accuracyheadline
Of the vulnerabilities given to the model, the share assigned a correct CWE category: exact, an accepted alternative, or a more specific child.

One Aegis measurement so far. A trend chart appears once a second version has been evaluated on the same benchmark; the baseline is shown for reference only.

03

End-to-end reasoning

Can the model analyze a security scenario and produce a defensible, evidence-based assessment?

The model performs the full analysis at once: verdict, CWE category, location, and remediation. Scored on the joint result.

Metrics

Task success rateheadline
The share of complete analyses that got everything right: the correct verdict and, for vulnerable code, the correct CWE category.
Recall
Of the real issues present, the share the model found.
False positive rate
Of the secure cases, the share the model wrongly flagged. Lower is better.

One Aegis measurement so far. A trend chart appears once a second version has been evaluated on the same benchmark; the baseline is shown for reference only.

  • Baseline · Reference model

    · security-reasoning-test-v1 · 24 cases

    Task success rate
    25.0%6/24
    Recall
    58.3%7/12
    False positive rate
    50.0%6/12

  • Aegis-1 · Development build

    · security-reasoning-test-v1 · 24 cases

    Task success rate
    33.3%8/24
    Recall
    33.3%4/12
    False positive rate
    8.3%1/12

04Research milestones

The journey so far.

Milestones confirmed by our project records, newest first.

  1. Benchmark

    Report 001 published

    The first evaluation report, with results, error analysis, and limitations. It sets the training targets for the first fine-tuned version.

    Aegis-1 · Development buildReference

  2. Benchmark

    First internal baseline evaluation

    The development build and the baseline were evaluated under identical settings across detection, classification, and end-to-end analysis.

    Aegis-1 · Development build

  3. External benchmark

    External benchmark integration prepared

    Integration with the CyberGym benchmark was set up against a pinned version. No runs have been completed yet.

  4. Evaluation

    Evaluation framework implemented

    A held-out set of vulnerable and secure code samples, a deterministic scorer, and the MITRE CWE catalog as the classification reference.

  5. Baseline

    Baseline model pinned

    An open-weight base model was fixed as the reference point. Every future Aegis version is compared against this exact baseline.

    Aegis-1 · Development build

  6. Research

    Aegis research program established

    A dedicated research codebase for a defensive cybersecurity model, kept separate from the Notealy product.

05Evaluation methodology

How we evaluate Aegis.

Three evaluation modes, from narrow to complete. Each answers one question, and each is scored deterministically against held-out cases.

  1. 01

    Detection

    Can the model identify relevant security issues from the provided evidence?

    Every vulnerable sample is paired with a secure counterpart, so flagging everything scores poorly. We report precision and false positives alongside recall.

  2. 02

    Classification

    Can the model correctly categorize identified issues?

    Categories are checked against the MITRE CWE catalog. A plausible-sounding identifier that does not exist in the catalog is counted as wrong, not as close.

  3. 03

    End-to-end reasoning

    Can the model analyze a security scenario and produce a defensible, evidence-based assessment?

    A full scenario is scored on the final verdict, the evidence it cites, and whether it avoided claims the evidence does not support.

Ground rules

  • Test cases are held out from any data used to train the model.
  • Each new version is compared with a fixed, pinned baseline under identical settings.
  • Results are reported with the dataset version, case count, and date they were measured on.
  • No result is published until it has been reviewed and approved.

Published evaluation reports and benchmark definitions will be linked from each benchmark as they become available.

06Development philosophy

Built for evidence. Not assumptions.

Measurable progress
We evaluate capabilities using documented methodologies and repeatable tests.
Security-first development
We prioritize defensive research, responsible validation, and authorized testing.
Continuous improvement
We iterate through evaluation, analysis, experimentation, and measurable feedback.

Notealy

Security before the threat.

We're building the next generation of security intelligence. Follow the progress of Aegis as our research evolves.

Explore Notealy