How can we run a private AI platform for classified documents that stays fully under our own control, while keeping the infrastructure busy and cost-effective around the clock?

Government organisations want to apply AI to highly classified documents, for example to classify, summarise and search them. Sending this data to external AI providers is not an option, so the AI has to run on infrastructure that we control ourselves.

The problem is cost. GPU servers are expensive, and a typical private setup is heavily used during office hours but sits mostly idle in the evening, at night and during weekends. That makes private AI hard to justify for individual organisations.

Your challenge is to design a private AI platform that makes the most of every hour of compute. Questions to explore include:

• How can real-time use (people asking questions about documents) and background workloads (processing large archives) share the same hardware without hurting response times?

• Which models do you need, how small can they be, and how do you route each request to the right model?

• Which techniques (batching, quantization, caching, GPU partitioning, scheduling) give the biggest efficiency gains?

• How can multiple organisations share one platform without ever sharing data?

• A reference architecture for a private, on-premises AI platform suitable for classified information.

• A scheduling and workload strategy that combines interactive and batch use on the same infrastructure.

• A small proof of concept or simulation that demonstrates the approach.

• Measurable results: infrastructure utilisation over a full week, cost per 1,000 processed documents and response times under peak load, compared with a naive setup.

Students do not need access to real classified data; public or synthetic documents are fine for testing. Open-source models and tooling are preferred, as the solution must be able to run without any connection to external services. Hardware access may be limited, so a well-founded simulation or a small-scale test on a single GPU or laptop is acceptable. We are happy to provide context on how Dutch government organisations work with documents and archives.