Artificial Intelligence and Machine Evolution

Artificial_Intelligence_and_Machine_Evolution 1 Apl. Prof. Dr. Dr. K. Saalbach Geostrategy and Geopolitics Department 1 49069 Osnabrueck Artificial Intelligence and Machine Evolution 02 January 2026 Summary This working paper provides a brief overview on Artificial Intelligence (AI) and Machine Evolution. Machine Evolution can be understood as a situation where machines autonomously create new machines which create further machines and so on. While such an evolution is not yet possible, substantial progress was made in multiple areas which bring machines and AI much closer to full autonomy, self-learning, and adaptation for their self-improvement. These developments accelerate the technological progress, but make huma oversight and interventions increasingly difficult, i.e., instead of being in the loop, humans are moved out of the loop. Advances include the AI-assisted design of computer chips, the AI-assisted design of entire computers, self-learning and improving AI agents in experimental settings as well as automated fabrications. Already in 2024, the machine learning-based AI program AlphaChip generated superhuman chip layouts and created in 2025 the new Ironwood TPU as one of the fastest marketed chips. China is advancing as well with the chip-designing AI QiMeng that already designed chips comparable with commercial chips. The chip design research is advancing rapidly. The first three-dimensional monolithic 3D chip to accelerate AI was presented by researchers of the Stanford University in December 2025. The Shanghai-based company CHIPX released a photonic quantum chip that for some mathematical operations was 1000 times faster than conventional chips. Quilter's physics-driven reinforcement learning AI system designed in the Project Speedrun a working 843-component Linux computer in only 38 hours and autonomously completed 98% of placement, routing, and physics. New ways of AI training and learning were proposed in 2025; the networking and information exchange between AI agents in the first step and the open-ended evolution of AI agents in the second step by the so-called Darwin Gödel Machine (DGM). Researchers from Google DeepMind proposed that AI systems learn from each other and share their experience permanently with unlimited development above humans, but networking and knowledge exchange between AI system may amplify the risk for system errors and malfunction of the AI agent ecosystem by exchange of inaccurate data. In 2025, the Darwin Gödel Machine (DGM) was developed, a self-improving Coding AI Agent system that iteratively modifies its own code and empirically validates each change using coding benchmarks which was effective in experiments. The spread of robots may open the way into fully automated fabrications. The human control of AI can be done by observation of output and action, but can be complemented by monitoring of the chain of thoughts, i.e., the internal AI protocols that show their reasoning (thinking) steps. The ability to observe the chain of thought of an AI is called monitorability. There are concerns that novel AI architectures may reduce the monitorability despite findings of hidden misalignment (scheming) where frontier AI models hide thoughts e.g., to get rewards or to avoid reinforcement learning. Various researchers express serious concerns about a potentially uncontrolled evolution e.g., by self-learning. The key concern is that humans are currently not really prepared for this situation and may not be able to provide sufficient oversight, i.e., the situation may get out of human control and for this reason, a strict governance with regulations and supervision should be established. Artificial_Intelligence_and_Machine_Evolution 2 Content 1 Machine Evolution ................................................................................................................................ 3 1.1 Introduction ................................................................................................................................... 3 1.2 AI-assisted Chip Design .................................................................................................................. 3 1.3 AI-assisted Computer Design ........................................................................................................ 4 1.4 Self-Learning AI .............................................................................................................................. 4 1.5 Fully Automated Fabrication ......................................................................................................... 6 1.6 Monitoring and Oversight of AI ..................................................................................................... 6 2 Summary ............................................................................................................................................... 7 3 References ............................................................................................................................................ 8 Artificial_Intelligence_and_Machine_Evolution 3 1 Machine Evolution 1.1 Introduction This working paper provides a brief overview on Artificial Intelligence and Machine Evolution. Machine Evolution can be understood as a situation where machines autonomously create new machines which create further machines and so on. While such an evolution is not yet possible, substantial progress was made in multiple areas which bring machines and AI much closer to full autonomy, self-learning, and adaptation for their self-improvement. These developments accelerate the technological progress, but make huma oversight and interventions increasingly difficult, i.e., instead of being in the loop, humans are moved out of the loop. Advances include the AI-assisted design of computer chips, the AI-assisted design of entire computers, self-learning and improving AI agents in experimental settings as well as automated fabrications. The key challenge for the human developers is to ensure oversight and monitoring of activities to avoid system errors, hallucinations, and contamination of AI ecosystems with inaccurate data. AI models combine learning algorithms with up to billions of parameters and large training data sets, which results in opaque black-box systems, this is known as explainability, interpretability or predictability issue 1 . How does an AI ‚know‘ an object? Sometimes, an AI is very good in identifying objects or animals, but nobody can explain which parameters were used, e.g., structure, color, size, patterns and so on. This makes AI systems unpredictable to some extent as nobody can say under which conditions an error may occur. This problem could be amplified in autonomous systems. 1.2 AI-assisted Chip Design Already today, the design of computer chips is dependent on computer support. Due to the extreme complexity, the design cannot be done anymore by humans alone, it requires design software (called electronic design automation, or EDA, software). Already in 2024, the machine learning-based program AlphaChip generated superhuman chip layouts, e.g., resulting in a 6.2% size reduction of Trillium Tensor Processing Unit (TPU) 2 compared to the previous chip generation. This also reduces the design time from weeks or even months to hours. AlphaChip is improving chip floor-planning by stepwise improvement of circuit placements 3 . In late 2025, the new Ironwood TPU became operational which means that AI-assisted chip design now produced one of the fastest marketed chips 4 . This chip will be offered as a 256-chip cluster and a 9,216-chip cluster 5 .Google is designing chips “for AI with AI” 6 , while the other companies either produce chips or AIs which makes AI-assisted design to a strategic market advantage. In parallel, China is advancing in the AI race as well, e.g., by DeepSeek, Manus, Kimi K2, Qwen, and the chip-designing AI QiMeng 7 . QiMeng consists of three layers, a domain-specialized Large Processor Chip Model (LPCM) for novel architectures as bottom-layer, a Hardware Design Agent and a Software Design Agent for automated hardware and software design in the middle-layer, and various processor chip design applications in the top-layer 8 . While the 1 Arrieta et al. 2020, p.83, Chaudry/Klein 2023 2 Goldie et al. 2024 3 Goldie/Mirhoseini 2024 4 Marini 2025 5 Spencer 2025 6 In December 2025, Google Gemini version 3.0 was considered as one of the leading frontier AI models, such as DeepSeek version 3.2 and ChatGPT version 5.2. 7 Zhang, R. et al. 2025 8 Zhang R. et al. 2025 Artificial_Intelligence_and_Machine_Evolution 4 developers of QiMeng see room for further improvement, QiMeng was already able to design the QiMeng-Computer Processing Unit (CPU)-v1 comparable to Intel 486 from the1990ies, while the second version QiMeng-CPU-v2 was comparable to Arm Cortex A53 from the 2010ies, i.e., chip designs comparable with commercial chips 9 . The problem that the AI is an opaque black-box system is also present in chip design and makes a root cause analysis in case of problems difficult 10 . The chip design research is advancing rapidly. The first three-dimensional 3D chip to accelerate AI was presented by researchers of the Stanford University in December 2025. It is a real 3D chip, i.e., not simply consisting of layers of 2D chips, but a monolithic design that can fully utilize the third dimension resulting in a twelve-fold improvement on real AI workloads and an improved energy efficiency 11 . The Shanghai-based company CHIPX released a photonic quantum chip that can accelerate AI, the coming 6G network, and quantum computing development. For some mathematical operations, the chip was 1000 times faster than conventional chips 12 . The chip is already produced with about 12,000 wafers a year, each wafer with around 350 chips 13 . In the long run, photonic chips, i.e., using light instead of electrons for information transfer, could replace conventional chips as they are faster and more energy-efficient. China already has the largest global market share in the photonic industry which originally was led by the United States and Europe 14 . 1.3 AI-assisted Computer Design The company Quilter presented the first computer that was architected, placed, routed, and validated by a physics-driven AI which was trained for reasoning on physics, i.e., is different from conventional large language models 15 . Quilter's physics-driven reinforcement learning AI system designed in the Project Speedrun a working 843-component Linux computer around a NXP i.MX 8M Mini chip in 38 hours instead of the 11 weeks that human engineers would have needed for this task and the computer could be successfully booted on the first try 16 . The AI of Quilter autonomously completed 98% of placement, routing, and physics, i.e., the engineer acted mainly as supervisor. The outcome of the project was that AI is now capable of designing computers that operate in the real word environments 17 . If the AI takes over, chips and new computers will be designed by computers, i.e., the humans will not know anymore how the AI and the computer are working which makes surveillance and control quite challenging. The creation of computers by other computers which then design again new computers is the start of the machine evolution. 1.4 Self-Learning AI AI learning and robot learning increase in similar ways: Robot learning has meanwhile reached stages of 1022 to 1024 FLOPS (floating comma operations) while frontier AI models currently exceeded 1024 FLOPS for their machine learning 18 . The training of AI is dependent from high9 Zhang R. et al. 2025 10 Chen 2025 11 Scheffler 2025 12 Hiverlab 2025, Swayne 2025 13 Swayne 2025 14 EAC 2024 15 Tardif 2025: Parameters used by the AI included signal integrity conditions, trace impedance, jitter and skew thermal propagation, current-carrying capacity, electromagnetic considerations and physical manufacturability. 16 Tardif 2025 17 Tardif 2025 18 Beyerer/Kirchner et al. 2ß25 Artificial_Intelligence_and_Machine_Evolution 5 quality data created by humans, because a training of AI by AI-generated data only leads to a rapid loss of quality, the model collapse. New ways of AI training and learning were proposed in 2025: the networking and information exchange between AI agents in the first step and the open-ended evolution of AI agents in the second step by the so-called Darwin Gödel Machine (DGM) 19 . AI agents are software programs where LLMs with reasoning and discovery ability provide increasing autonomy for achievement of given objectives 20 . The development is rapdily advancing from simple workflows with LLMs to autonomously acting AI Agents and multiagentic systems (MAS) which can run increasingly complex procedures on their own 21 . Researchers from Google DeepMind noted that in scientific areas like mathematics, coding, and science, the knowledge extracted from human data is rapidly approaching a limit and that most high-quality data sources were already utilized. The proposed solution is that AI systems learn from each other and share their experience permanently. 22 With the growing presence of AI agents, they can learn much more from each other than from humans alone. DeepMind expects an Artificial General Intelligence AGI around 2030 and that there will be unlimited development above humans. 23 The next logical step was therefore to allow the communication between AI agents and the interoperability if AI agents and multi-agent systems by new protocols that facilitate communication (such as Glibberlink for communication via phone), interactions (as the new Agent2Agent A2A open protocol of Microsoft 24 ) and for learning by knowledge exchange between AI systems 25 . However, the networking and knowledge exchange between AI system amplifies the risk for system errors and malfunction of the AI agent ecosystem pollution by inaccurate data. Google DeepMind has developed the FACTS Grounding Benchmark which covers the model’s ability to access its internal knowledge accurately, to use search as a tool to retrieve information and synthesize it correctly ability to answer prompts related to input images in a factually correct manner 26 . In a study from December 2025, Gemini 3 Pro led in overall performance, with a FACTS Score of 68.8% which showed a limited accuracy of data handling by the tested models. What will happen, if multiple agents start to exchange inaccurate data? This contributes to the Multi-Agent-Paradox which was shown in a study: adding more agents to a system makes it slower, more expensive, and less accurate 27 . In multi-agentic AI systems, a malicious attacker who gets access to one AI agent may be able to damage further parts if the intruded AI agents provide false input to the other agents, resulting in a model pollution 28 . The question of how AI systems can endlessly improve themselves while getting better at solving relevant problems resulted in the Darwin Gödel Machine (DGM), a self-improving 19 Zhang et al. 2025 20 DSIT 2025 21 Larsen 2024 22 Silver/Sutton 2025 23 Shah et al. 2025. Another aspect is that the AI systems can exist in multiple copies and gain knowledge in parallel which is not possible for human individuals, Hinton 2025 24 Arenas/Brekelmans 2025 25 Rosenbush 2025 26 Deep Mind 2025 27 Zia 2025 28 He et al. 2024 Artificial_Intelligence_and_Machine_Evolution 6 system that iteratively modifies its own code and empirically validates each change using coding benchmarks 29 . The concept of a Gödel machine was proposed in 2007 where a machine should be able to prove a system improvement. In contrast, the Darwin Gödel Machine (DGM) only needs to show empirical evidence that an improvement took place. The DGM is a self-referential selfimprovement of Coding AI Agents 30 . The DGM starts with one coding agent that starts selfmodification with checks against coding benchmarks. Each agent version is archived by the DGM and can then be used for self-modification through open-ended exploration. The process is going on iteratively. In the experiment that for safety reasons was done under strict human supervision in a sandbox, the DGM automatically improved its coding capabilities (e.g., better code editing tools, long-context window management, peer-review mechanisms), increasing performance. The DGM is the practical step to a self-improving AI for endless innovation 31 . 1.5 Fully Automated Fabrication Sam Altman, CEO of the ChatGPT provider OpenAI, expects a wide-spread use of robots in production for 2027 and envisions a completely new development path: “we have to make the first million humanoid robots the old-fashioned way, but then they can operate the entire supply chain—digging and refining minerals, driving trucks, running factories, etc. - to build more robots, which can build more chip fabrication facilities, data centers, etc.” 32 . As a major step towards this vision, Foxconn and Nvidia intend to deploy humanoid robots at a Foxconn plant in Houston to produce Nvidia GB300 AI servers from 2026 on 33 . 1.6 Monitoring and Oversight of AI The human control of AI is dependent from the ability to observe the AI. This can be done by observation of output and actions, but can be complemented by monitoring of the chain of thoughts, i.e., the internal AI protocols that show their reasoning (thinking) steps. The ability of humans to observe the chain of thought of an AI is called monitorability 34 . The leading Tech companies Google, Meta, OpenAI and Anthropic released a joint paper where they expressed major safety concerns about novel AI architectures which reduce the need for ‘loud thinking’ of AI models, i.e., the need to create chain of thoughts with written protocols for important reasoning steps. Without these protocols, it will be very difficult for AI developers to find out what the AI is thinking and how decisions were made which could result in loss of control for humans 35 . A study of OpenAI proposes to investigate change of thoughts by asking follow-up questions to statements which may result in more insight 36 . This is necessary as frontier AI models show hidden misalignment (scheming) where the AI hides its thoughts e.g., to get rewards or to avoid reinforcement learning. 37 Scheming is accompanied by other risks of advanced AI models like deception, situational awareness (when the AI knows that it is in a test situation), power seeking, autonomous replication and agency (goal-directed behavior where the AI tries to push through its own goals) 38 . 29 Zhang, J. et al. 2025. The name Darwin refers to evolution, Gödel to mathematics 30 Zhang, J. et al. 2025 31 Zhang, J. et al. 2025 32 Altman 2025 33 Lee 2025 34 Korbak et al. 2025, Guan 2025 35 Korbak et al. 2025 36 Guan 2025, OpenAI 2025a 37 OpenAI 2025b 38 Grey/Segerie 2025 Artificial_Intelligence_and_Machine_Evolution 7 Various researchers express serious concerns about a potential evolution e.g., by self-learning. The key concern is that humans are currently not really prepared for this situation and may not be able to provide sufficient oversight 39 , i.e., the situation may get out of human control 40 . For this reason, a strict governance with regulations and supervision should be established 41 . If not done, researchers of the Machine Intelligence Research Institute in Berkeley concluded that the most likely outcome will be human extinction 42 . Anthropic’s chief scientist Jared Kaplan views letting AI models train themselves as the most critical risk for an uncontrollable intelligence explosion which could already happen in 2027 43 . 2 Summary This working paper provided a brief overview on Artificial Intelligence and Machine Evolution. Machine Evolution can be understood as a situation where machines autonomously create new machines which create further machines and so on. While such an evolution is not yet possible, substantial progress was made in multiple areas which bring machines and AI much closer to full autonomy, self-learning, and adaptation for improvement. These advances accelerate the technological progress, but make huma oversight and interventions increasingly difficult, i.e., instead of being in the loop, humans are moved out of the loop. Advances include the AI-assisted design of computer chips, the AI-assisted design of entire computers, self-learning and improving AI agents in experimental settings as well as automated fabrications. Already in 2024, the machine learning-based AI program AlphaChip generated superhuman chip layouts and created in 2025 the new Ironwood TPU as one of the fastest marketed chips. China is advancing as well with the chip-designing AI QiMeng that already designed chips comparable with commercial chips. The first three-dimensional monolithic 3D chip to accelerate AI was presented by Researchers of the Stanford University in December 2025. The Shanghai-based company CHIPX released a photonic quantum chip that for some mathematical operations was 1000 times faster than conventional chips. Quilter's physics-driven reinforcement learning AI system designed in the Project Speedrun a working 843-component Linux computer in only 38 hours and autonomously completed 98% of placement, routing, and physics. New ways of AI training and learning were proposed in 2025; the networking and information exchange between AI agents in the first step and the open-ended evolution of AI agents in the second step by the so-called Darwin Gödel Machine (DGM). Researchers from Google DeepMind proposed that AI systems learn from each other and share their experience permanently with unlimited development above humans, but networking and knowledge exchange between AI system may amplify the risk for system errors and malfunction of the AI agent ecosystem pollution by inaccurate data. In 2025, the Darwin Gödel Machine (DGM) was developed, a self-improving Coding AI Agent system that iteratively modifies its own code and empirically validates each change using coding benchmarks which was effective in experiments. The spread of robots may open the way into fully automated fabrications. The human control of AI can be done by observation of output and action, but can be complemented by monitoring of the chain of thoughts, i.e., the internal AI protocols that show their reasoning (thinking) steps. The ability to observe the chain of thought of an AI is called monitorability. There are concerns that novel AI architectures may reduce the monitorability despite findings of hidden misalignment (scheming) where frontier AI models hide thoughts 39 Barnett/Scher 2025, Losey 2025, MIRI 2025 40 Adewumi et al. 2025, Tonsen 2025 41 Adewumi et al. 2025, Barnett/Scher 2025, Losey 2025, MIRI 2025,Tonsen 2025 42 Barnett/Scher 2025, MIRI 2025 43 Landymore 2025 Artificial_Intelligence_and_Machine_Evolution 8 e.g., to get rewards or to avoid reinforcement learning. Various researchers express serious concerns about a potentially uncontrollable evolution e.g., by self-learning. The key concern is that humans are currently not really prepared for this situation and may not be able to provide sufficient oversight, i.e., the situation may get out of human control and for this reason, a strict governance with regulations and supervision should be established. 3 References Adewumi, T. et al. (2025): AI Must not be Fully Autonomous. Machine Learning Group, EISLAB, Sweden arXiv:2507.23330v1 [cs.AI] 31 Jul 2025 Altman, S. (2025): Gentle Singularity. Sam Altman’s Blog 10 June 2025 Arenas, Y., Brekelmans, B. (2025): Empowering multi-agent apps with the open Agent2Agent (A2A) protocol. Microsoft News 07 May 2025 Arrieta, A.B. et al. (2020): Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities, and challenges toward responsible AI. Information Fusion 58(2020), p. 82–111 Beyerer, J., Kirchner, E. et al. (2025): KI in der Robotik. Flexible und anpassbare Systeme durch interaktives Lernen. DOI: https://doi.org/10.48669/pls_2025 Barnett, P., Scher, A. (2025): AI Governance to Avoid Extinction: The Strategic Landscape and Actionable Research Questions Machine Intelligence Research Institute Berkeley, Technical Governance Team May 2025 Chaudhry, H., Klein, L. (2023): Artificial Intelligence and Nuclear Weapons: Problem Analysis and US Policy Recommendations. [email protected] 14th November 2023 Future of Life Institute (FLI) Chen, N. (2025): Integrating AI into Semiconductor Design and Fabrication: Methodologies, Challenges and Future Prospects. ITM Web of Conferences 78, 03007 (2025) https://doi.org/10.1051/itmconf/20257803007 CSEIT 2025 Deep Mind (2025): FACTS Benchmark Suite: Systematically evaluating the factuality of large language models. The FACTS team of Google Deep Mind. December 9, 2025 DSIT (2025): AI Insights - Agentic AI. Paper of the Department of Science, Innovation and Technology and the Government Digital Service of the United Kingdom EAC (2024): EAC International Consulting Executive Report. Political Steering Processes in China in Core Segments of the Photonics Industry. Commissioned by the European Technology Platform Photonics21 Goldie, A. et al. (2024): Addendum: A graph placement methodology for fast chip design doi.org/10.1038/s41586-024-08032-5 Published online 26 September 2024 Goldie, A., Mirhoseini, A. (2024): How AlphaChip transformed computer chip design Science September 26, 2024 Grey, M., Segerie, C.R. (2025): The AI Risk Spectrum From Dangerous Capabilities to Existential Threats. French Center for AI Safety (CeSIA) 20 August 2025 Guan, M.Y. et al. (2025): Monitoring Monitorability. OpenAI December 18, 2025 He, Y. et al. (2024): Security of AI Agents arXiv:2406.08689v3 [cs.CR] 17 Dec 2024 Hinton, G. (2025): KI wird bald deutlich intelligenter als wir sein. Handelsblatt 23-25 Mai 2025, S.26-27 Artificial_Intelligence_and_Machine_Evolution 9 Hiverlab (2025): China’s CHIPX Photonic Chip Production Line Set to Accelerate AI, 6G, and Quantum Computing Development: SCMP https://hiverlab.com/china-chipx-photonicchip-ai-6g-quantum-computing/ June 17, 2025 Khan, S.M., Mann, A., Peterson, D. (2021): The Semiconductor Supply Chain: Assessing National Competitiveness Center for Security and Emerging Technology (CSET) Issue Brief January 2021 Korbak, T. et al. (2025): Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety. arXiv:2507.11473v1 [cs.AI] 15 Jul 2025 Landymore, F. (2025): Anthropic's Chief Scientist Says We're Rapidly Approaching the Moment That Could Doom Us All. Futurism Published Dec 13, 2025 Larsen, B. et al. (2024): Navigating the AI Frontier: A Primer on the Evolution and Impact of AI - World Economic Forum Ai Governance Alliance Safe Systems and Technologies Working Group - World Economic Forum Whitepaper Lee, W.-Y. (2025): Reuters Exclusive: Nvidia, Foxconn in talks to deploy humanoid robots at Houston AI server making plant. June 20, 2025 https://www.reuters.com/world/china/nvidiafoxconn-talks-deploy-humanoid-robots-houston-ai-server-making-plant-2025-06-20/ Losey, R. (2025): Afraid of AI? Learn the Seven Cardinal Dangers and How to Stay Safe – AI Ethics. April 25, 2025 Marini, A. (2025): 3 things to know about Ironwood, our latest TPU. Google Cloud, Nov 25, 2025 MIRI (2025): The Problem - Machine Intelligence Research Institute Berkeley October 2025We Open AI (2025a): Evaluating chain-of-thought monitorability OpenAI Research Publication December 18, 2025 OpenAI (2025b): Detecting and reducing scheming in AI models. Publication Research September 17, 2025 Scheffler, I. (2025): Researchers unveil groundbreaking 3D chip to accelerate AI. Stanford Report December 10, 2025 https://news.stanford.edu/stories/2025/12/monolithic-3d-chipfoundry-breakthrough-ai Shah, R. et al. (2025): An Approach to Technical AGI Safety and Security Google DeepMind Research Paper Swayne, M. (2025): China’s New Photonic Quantum Chip Promises 1,000-Fold Gains for Complex Computing Tasks. The Quantum Insider https://thequantuminsider.com/author/mattswayne/) November 15, 2025 Tardif, A. (2025): Quilter Shows AI Can Now Design Re Hardware With the World’s First Mac Engineered Computer. Published December 10, 2025 By Antoine Tardif, CEO & Founder of Unite.AI Zhang, J. et al. (2025): Darwin Gödel Machine (DGM) – Open-ended Evolution of Selfimproving Agents. arXiv:2505.22954v2 [cs.AI] 26 Sep 2025 Zhang, R. et al. (2025): QiMeng: Fully Automated Hardware and Software Design for Processor Chip. arXiv:2506.05007v1 [cs.AR] 5 Jun 2025 Zia, T. (2025): The Multi-Agent Paradox: Why More Agents Can Lead to Worse Results. Unite.AI December 22, 2025