Engineering Safe Agentic AI: From Model to Grid

By Dr Daniel Kearney, Chief Technology Officer, Firmus

As AI agents take on more consequential work, safety becomes a shared engineering responsibility across the entire ecosystem. Models, applications, compute infrastructure and the physical systems supporting them each shape what an agent can do, and how its actions can be governed. Building trust in this next phase of AI is critical and requires us to design all layers together as a coherent system, with clear boundaries, independent verifiable controls and evidence-based validation.

Our customers – hyperscalers, enterprises and AI natives – bring deep expertise in model evaluation, security protocols and application governance. As their infrastructure partner, our responsibility is to complement that work with controls across the software and hardware we operate. Each layer has a distinct role; safety depends on how well those layers and responsibilities connect.

We are working with NVIDIA on OpenShell Reference Architecture to help govern what AI agents can access and do within the systems we operate. This combines controls on agent permissions and execution with independent monitoring and enforcement, so an agent cannot alter or bypass the safeguards governing its work.

We have demonstrated our commitment to security through proactive engagement in the Open Secure AI Alliance. Our platform teams, spanning hardware and software, have been co-engineering the model-to-grid stack as one unified solution aligned with the NVIDIA DSX AI Factory Platform including DSX Sim and Max LPS. In collaboration with NVIDIA, our teams are examining incremental efficiency improvements and adopting best practices and tooling to address the practical challenges of secure grid integration that meet applicable requirements.

Much of this work happens in the details. A one-per-cent efficiency improvement can look modest on a slide, but for AI systems operating at scale, it can be meaningful. Achieving efficiency gains without compromising performance warrants serious engineering attention. With NVIDIA Max LPS, we’re unlocking stranded power, which requires understanding how a software decision travels through silicon, electrical demand and cooling, and what happens when those systems respond at different speeds.

That is why we design them together.

The HyperCube provides the modular physical foundation, integrating liquid cooling, mechanical and power infrastructure. AI Factory OS provides the software-defined control plane, monitoring key performance metrics and orchestrating workloads in real time. AI FactoryX adds the digital representation through which engineers can model and examine factory behaviour. Their value lies in the connections: relating the workload being scheduled to the infrastructure conditions supporting it.

Agentic AI makes those connections increasingly important. A task can branch into inference, tool calls, code execution and memory access across CPUs, GPUs, memory and storage within milliseconds. Its resource requirements change as it progresses. Workload permissions determine which actions are authorised; infrastructure controls determine how those actions can be accommodated within electrical and thermal limits.

Digitalisation helps make that relationship visible. Time-correlated telemetry connects workload events with physical responses. A calibrated, physics-based digital twin can help engineers optimise operating runbooks before those procedures are applied to equipment.

Integration also requires discipline. An agent authorised to change workload placement must have no authority to rewrite its security policy or bypass equipment protection. Runtime controls, independent hardware enforcement and physical safety interlocks retain distinct responsibilities within the unified system.

Our founder Oliver Curtis puts it simply: “We build from first principles.” It captures Firmus’s engineering discipline: question everything, build carefully, validate each improvement and build on those lessons in the next iteration.

Working with NVIDIA and our customers, we apply that discipline to improving efficiency, addressing grid integration and supporting greater autonomy with rigorous control. By making that work measurable and its methods repeatable, we aim to provide a useful benchmark for how the industry approaches integrated AI infrastructure.