Introducing Easy LLM Fine-Tuning on RunPod: Axolotl Made Simple

At RunPod, we're constantly looking for ways to make AI development more accessible. Today, we're excited to announce our newest feature: a pre-configured Axolotl environment for LLM fine-tuning that dramatically simplifies the process of customizing models to your specific needs. Why Fine-Tuning Matters Fine-tuning large language

Introducing Bare Metal: Dedicated GPU Servers with Maximum Control and Savings

AI teams and ML engineers need flexibility, performance, and cost efficiency—and RunPod's new Bare Metal offering delivers exactly that. With Bare Metal, you can now reserve dedicated GPU servers for months or even years, ensuring consistent performance without the hassle of hourly or daily pricing. This means

Deploying Multimodal Models on RunPod

Multimodal AI models integrate various types of data, such as text, images, audio, or video, to allow tasks such as image-text retrieval, video question answering, or speech-to-text. Examples are CLIP, BLIP, and Flamingo, among others, showing what is possible by combining these modes–(but deploying them presents unique challenges including

Open Source Video and LLM: New Model Roundup

Remember when generating decent-looking videos with AI seemed like something only the big tech companies could pull off? Those days are officially over. 2024 brought us a wave of seriously impressive open-source video generation models that anyone can download and start playing with. And here's the kicker -

What Even Is AI? (A Writer & Marketer’s Perspective)

Learn AI With Me: The No-Code Series, Part 1 AI Is Everywhere, But What Is It? If you spend any time online, you’ve probably seen the explosion of AI tools—ChatGPT, MidJourney, DALL·E, Claude, and Gemini. Everyone is talking about AI, but when you ask, "What exactly

Training Flux.1 Dev on the MI300X GPU with Huge Batch Sizes

For this post I'm going to be experimenting with fine-tuning Flux.1 Dev on the world's largest GPU, the AMD MI300X. At 192GB of VRAM per GPU, the MI300X allows training at batch sizes and resolutions that no other GPU currently supports. (While these instructions were

Streamline GPU Cloud Management with RunPod's New REST API

Managing GPU resources has always been a bit of a pain point, with most of the time spent clicking around interfaces with repetitive manual configuration. Our new API lets you control everything through code instead, which is great news for those who'd rather automate repetitive tasks and focus

Runpod Platform

RunPod Achieves SOC 2 Type I Certification: A Major Milestone in Security and Compliance

RunPod has successfully completed its SOC 2 Type I audit, conducted by Sensiba. This marks a significant milestone in our commitment to security, compliance, and trust for our customers and partners.

AccessibleTech

AI, Content, and Courage Over Comfort: Why I Joined RunPod

Alyssa Mazzina Every big move in my career has come down to one thing: choosing courage over comfort. That's why, when I discovered RunPod’s bold, developer-first approach to AI infrastructure, even though I knew next to nothing about AI infrastructure, I knew this was exactly where I

Cloud Computing

Unveiling Enhanced CPU Pods: Docker Runtime and Network Volume Support

We're excited to announce two significant upgrades to our CPU pods that will streamline your development workflow and expand storage options. Our CPU pods now feature Docker runtime (replacing Kata Containers) and support for network volumes—previously exclusive to our GPU pods.

Run Deepseek R1 On Just 480GB of VRAM

Even with the new closed model successes of Grok and Sonnet 3.7, DeepSeek R1 is still considered a heavyweight in the LLM arena as a whole, and remains the uncontested open-source LLM champion (at least until DeepSeek R2 launches, anyway.) We've written before about the concerns of

GitHub Integration Now In GA - Build Images from GitHub Repos Even Faster

RunPod is pleased to announce that our GitHub integration is officially out of beta and ready for production use! This feature enables you to iterate your work more quickly by building packages to deploy on RunPod serverless directly from a GitHub repo, removing all of the friction involved in creating

AI Development

How Online GPUs for Deep Learning Can Supercharge Your AI Models

Online GPUs provide on-demand access to high-performance cloud computing, allowing AI teams to scale resources instantly without heavy infrastructure costs.

AI Development

How to Choose a Cloud GPU for Deep Learning: The Ultimate Guide

Cloud GPUs allow organizations to dynamically scale resources, optimize workflows, and tackle the most demanding AI tasks while effectively managing costs. This guide delves into the benefits of cloud GPUs for deep learning and explores key factors to consider when choosing a provider.

Introduction to Websocket Streaming with RunPod Serverless

In this followup to our 'Hello World' tutorial, we'll create a serverless endpoint that processes base64-encoded files and streams back the results. This will demonstrate how you can work with file input/output over our serverless environment by encoding the file as data within a JSON

Founder Series #1: Origin Story

Let's get more personal and establish a baseline. For everyone that's been enjoying RunPod, thank you for spreading the word. I am Pardeep Singh (aka flash-singh), CTO at RunPod and one of the co-founders along with Zhen Lu. What triggered me to share? Of all things

How To Run a "Hello World" in RunPod Serverless

If you're new to serverless computing and Docker, this guide will walk you through creating your first RunPod serverless endpoint from scratch. We'll build a simple "Hello World" application that demonstrates the basic concepts of serverless deployment on RunPod's platform. You'

Mistrall Small 3 Eschews Synthetic Data - What Does This Mean?

Mistral AI has released Mistral Small 3, which claims to not use synthetic data in their training pipeline. Weighing in at 22B with 32k context, this is a lightweight model that can be run at full weights on an A40, and nearly any GPU spec that we offer while quantized.

The Complete Guide to GPU Requirements for LLM Fine-tuning

When deciding on a GPU spec to train or fine-tune a model, you're likely going to need to hold onto the pod for hours or even days for your training run. Even a difference of a few cents per hour easily adds up, especially if you have a

DeepSeek R1 - What's the Hype?

DeepSeek R1 is a recently released model that has been topping benchmarks in several key areas. Here's some of the leaderboards it's shot to the top of: * LiveBench: Second place, with only GPT4 o1-2024-12-17 surpassing it as of this writing. * Aider: Ditto. * Artificial Analysis: Fifth place,

5090s Are Almost Here: How Do They Shape Up Against the 4090?

Another year has come and another new card generation from NVidia is on the way. 5090s are due to become widely available this January, and RunPod is going to be extremely eager to support them once they do. Along with the new Blackwell architecture, the 5090 is set to launch

How Do I Transfer Data Into My Pod?

We've got a number of questions recently about how to transfer data into pods, and while do we have some transfer methods in our docs we'd like to go into a bit more detail on how to upload files into your pod. In general, for large

What's New for Serverless LLM Usage in RunPod in 2025?

Out of all of the use cases that our serverless architecture has, LLMs are one of the best examples of it. Because so much of LLM use is dependent on the human using it to process, digest, and type a response, you save so much on GPU spend by ensuring

H200 Tensor Core GPUs Now Available on RunPod

We're pleased to announce that H200 is now available on RunPod at a price point of $3.99/hr in Secure Cloud. This GPU spec boosts the available VRAM for NVidia-based applications up to 141GB in a single unit along with increased memory bandwidth. Here are how the

RunPod Sponsors CivitAI's Project Odyssey 2024 Competition

RunPod is proud to sponsor Season 2 of Project Odyssey 2024 from CivitAI, the world's largest AI filmmaking competition. We've written in the past about prominent open source packages like LTX, Mochi, and Hunyuan Video – here's your chance to show off your skills and