• iglou@programming.dev
    link
    fedilink
    English
    arrow-up
    2
    ·
    edit-2
    2 days ago

    … No. That’s not how it works. AI datacenters are just datacenters with a very heavy focus on GPUs and TPUs. We have plenty of datacenters with GPU and TPU offerings in Europe, just no datacenter that is fully dedicated to it.

    • boonhet@sopuli.xyz
      link
      fedilink
      English
      arrow-up
      1
      ·
      edit-2
      20 hours ago

      The AI data centers have purpose-built racks to let like 10x more power through. A single rack for the newer ones is in the range of 100kW, but there’s talk of planning for even more power density for future generations - I’ve heard of 600 kW racks and potentially 1MW racks. Your average normal data center isn’t equipped with enough cooling to run a significant amount of them, nor are they wired for that kind of power unless built very recently. Just 3 years ago, the GPUs used in data centers were much less power hungry (~400W per chip then vs over 1kW per chip now).

      It also doesn’t make sense to run just a few GPU racks in one data center, a few in another, etc for running frontier models. They benefit too much from low latency, high bandwidth interconnects.

      There are for sure many data centers in Europe that can run modest sized LLMs, no problem. Probably even train them. But we don’t have many data centers that would make it economical to run frontier models. AFAIK there’s one in Norway that OpenAI uses, and Anthropic doesn’t have consumer-facing compute at all in Europe (Not sure if enterprise clients can get a special deal).

      New AI datacenters are being built in the gigawatt scale. It’s a truly disgusting amount of power, my entire country peaks at under 2 GW in even the coldest winters.

      If there was any efficient way to run frontier models in general-purpose data centers, you bet OpenAI and Anthropic would be doing it en masse. Anthropic in particular is starved for compute, that’s why they enacted much stricter usage limits a few months ago. Plus they’d definitely benefit from lower latency, as inference itself takes time and you want the user to start getting output as soon as possible so they’d perceive the service as more responsive.

      So sure, we can run AI workloads in Europe. Hell, I’ve run LLMs on my low-end PC. Mistral uses American cloud companies’ European GPU compute capabilities. But they have much smaller models than GPT 5.whatever they’re up to now or Claude Opus.