Putting an AI server in your clinic: questions to answer before you buy

The ScribeMD Team ·

Some clinics have decided they want their AI to run in the building: on a machine they own, behind their own firewall, working when the internet isn't. That is a reasonable decision, and it brings questions a cloud subscription never asked. Where does the box go? Who looks after it? What happens when it fails? Which models can it actually run?

This checklist is what we would want answered before a clinic buys any on-premise AI server, ours included. It uses Cortex as the worked example because we know its specifications, and we'll say where those specifications matter.

1. Why on-premise, specifically?

Write down the reason, because it decides which questions matter most. The usual ones:

  • No outside processor. A cloud provider that stores or processes ePHI for you is a business associate even if it only holds encrypted data and has no key (HHS, Guidance on HIPAA and Cloud Computing). Running the model on your own hardware removes the outside party from that step.
  • Working through internet outages. If the scribe is in the building, an outage at your internet provider doesn't stop notes.
  • Control over model versions. A model you host changes only when you change it.
  • Running open models next to the scribe. Some clinics want a general-purpose model for internal questions that never leaves the building.

If none of these apply, a cloud service is simpler. Be honest about that before you buy hardware.

2. Where will it physically sit?

The Security Rule requires facility access controls: policies "to limit physical access to its electronic information systems and the facility or facilities in which they are housed" (45 CFR 164.310(a)(1)). A server holding every recorded visit belongs in a locked room or cabinet, not on the front desk.

Check the practical things too. Cortex is 150 × 150 × 50.5 mm, weighs 1.2 kg and runs from a 240 W external power supply. It needs one Ethernet connection; it has a 10 GbE port. Plan for ventilation, a power outlet on a UPS if you have one, and a network port in the same room.

3. How many clinicians will use it?

On-premise pricing is usually sized to your team rather than billed per visit. Cortex is priced by the number of clinicians who will use it, in US dollars before taxes and shipping:

  • Practice, 1 to 50 clinicians: $9,990
  • Group, 51 to 200 clinicians: $19,900
  • Network, 201 to 1,000 clinicians: $34,000
  • Health system, 1,000+ clinicians: $49,000

Count the clinicians who will record, not the total headcount, and include anyone who does telehealth visits.

4. Which models do you want, and will they fit?

Memory decides which models a box can run. Cortex is built on the NVIDIA GB10 Grace Blackwell Superchip with 128 GB of unified memory shared by the CPU and GPU, and 4 TB of NVMe storage. NVIDIA describes systems with this chip and 128 GB of memory as able to run inference on models up to 200 billion parameters (NVIDIA, DGX Spark).

"Up to" is doing real work there. What fits depends on the model's size, its quantization and the context length you need. The flagship versions of some open models are far larger than any single box. Moonshot AI's Kimi K2, for example, is about 1 TB of weights on its Hugging Face page (Hugging Face, Kimi-K2-Instruct), and DeepSeek-R1 is about 690 GB (Hugging Face, DeepSeek-R1). On a 128 GB machine you run smaller family members, such as 30B and 70B variants, and the ScribeMD scribe next to them.

Ask any vendor for the specific models and versions you will get, and how much memory each uses with the scribe running. Ask what happens to scribe performance when someone else in the clinic is using a large model at the same time.

5. What are the licences?

"Open" models come under different licences, and a model's base model may carry its own. Some have conditions that only apply at very large scale; some require attribution. Keep a record of each model you run, its licence and its base model. We've written a separate guide: what the Qwen, DeepSeek, Kimi and Llama licences say.

6. Who looks after it?

A box you own is a system you maintain. Before you buy, name the person or contractor responsible for:

  • Operating system and software updates, and when they happen.
  • Model updates, and who decides to change a model version.
  • Access control: who can sign in to the box and its admin tools.
  • Backups of anything you need to keep, and where those backups live.
  • Monitoring: who notices if it stops working at 7:45 a.m.

Include the server in your HIPAA risk analysis. NIST's SP 800-66 Revision 2, a cybersecurity resource guide for implementing the HIPAA Security Rule, is a practical companion for that work.

7. What happens when it fails?

One box is one point of failure. The Security Rule requires a contingency plan, including an emergency mode operation plan (45 CFR 164.308(a)(7)). Decide now what clinicians do if the server is down. A simple fallback is a few laptops with ScribeMD Desktop in Local only mode, which runs the scribe on the laptop itself, works offline, and is free for individual doctors. Our downtime guide covers the rest of that plan.

8. How does the clinical workflow change?

It shouldn't, much. With Cortex, every exam room records to the box over the clinic network. The transcript, the note and the ICD-10 and CPT codes are written on the box, then wait in ScribeMD for the clinician to review and sign. Clinicians record as they do today; the AI runs down the hall instead of in a data center.

The clinician still reviews every draft. On-premise changes where the model runs. It doesn't change who is responsible for the note.

9. What does setup involve?

For Cortex: power and one Ethernet cable, then sign in. It arrives with ScribeMD and the open models already installed. Ask any vendor for the equivalent list, and whether someone on their side will help on the day.

The checklist on one page

  1. Reason for on-premise, written down.
  2. A locked, ventilated location with power and network.
  3. Clinician count, including telehealth.
  4. Named models and versions, with memory use alongside the scribe.
  5. A model register with licences and base models.
  6. A named owner for updates, access, backups and monitoring.
  7. The server in your risk analysis.
  8. A fallback for when it is down.
  9. Clinician review and sign-off unchanged.

If Cortex fits, you can place an order request. It isn't a charge: our team confirms the configuration, delivery and invoice with you by email first.