top of page

Search


Breaking the Amnesia Cycle in Large Sequence Models
Nested Learning reframes neural models as multi-loop systems updating at different frequencies, revealing that depth stacking hides gradient mechanics and limits continual learning. It interprets optimizers like Momentum and Adam as associative gradient memories and introduces CMS for incremental abstraction. The HOPE module combines self-modification, multi-clock updates, and deep contextual compression, offering a white-box path beyond static backbones for long-context and

Juan Manuel Ortiz de Zarate
Nov 27, 20259 min read


Make Neural Circuits Understandable
The article introduces weight-sparse transformers (models where most weights are zero) to make neural circuits interpretable. These models reveal clear, human-understandable algorithms for language tasks. Sparsity trades off raw capability for clarity, allowing researchers to fully trace mechanisms inside networks and bridge them to dense models for transparency in AI reasoning.

Juan Manuel Ortiz de Zarate
Nov 20, 20259 min read


Compute Among the Stars
Google’s Project Suncatcher envisions moving AI computation into orbit, building constellations of solar-powered satellites equipped with TPUs and laser interlinks. By harnessing the Sun’s constant energy and future low-cost launches, the project proposes a scalable, space-based infrastructure for machine learning. It’s a blueprint for computing beyond Earth—where data centers orbit, powered by sunlight instead of fossil grids.

Juan Manuel Ortiz de Zarate
Nov 11, 20259 min read


AI Can Code, But Can It Engineer?
SWE-Bench Pro marks a turning point in evaluating AI coding agents. Built from complex, real-world software repositories, it reveals that even frontier models like GPT-5 and Claude Opus solve less than 25% of tasks. The benchmark exposes the gap between coding fluency and true engineering ability, redefining how progress toward autonomous software development should be measured.

Juan Manuel Ortiz de Zarate
Nov 5, 202510 min read


The AlphaGo Moment of Neural Architecture Design
ASI-ARCH marks a breakthrough in AI self-innovation: an autonomous system that designs, codes, and validates new neural network architectures without human input. Conducting 1,773 experiments, it discovered 106 state-of-the-art models, revealing a scaling law for scientific discovery. Like AlphaGo’s Move 37, ASI-ARCH exposes principles beyond human intuition, signaling a new era where AI invents AI.

Juan Manuel Ortiz de Zarate
Oct 29, 202510 min read


The Lightning Mind
DeepSeek-V3.2-Exp introduces a new sparse-attention system that lets large language models handle ultra-long contexts efficiently. Using a “lightning indexer” to select only the most relevant tokens, it cuts computation costs while preserving reasoning power. The result is a faster, cheaper, and more cognitively elegant AI that learns what to ignore, bringing machine focus closer to human intelligence.

Juan Manuel Ortiz de Zarate
Oct 22, 20259 min read
bottom of page