July updates: uv support, distributed GPU model training, and how KoBold Metals accelerates mineral discovery with Coiled

July updates: uv support, distributed GPU model training, and how KoBold Metals accelerates mineral discovery with Coiled

Run Python scripts with uv + Coiled

Our blog post on uv, Astral 's Rust‑based Python package and project manager, shows how easy it is to use Coiled with uv to run Python scripts on the cloud. With uv + Coiled, you can:

  • Declare script-specific dependencies directly in your Python file (`uv add --script`)
  • Specify runtime config (container, region, hardware) with inline `# COILED` comments
  • Run your script with `uvx coiled batch run uv run process.py`

📚 Read the blog post.

Coiled Batch: Map over values, .env support, and task coordination

Article content

Over the past month we've released several Coiled Batch enhancements:

  • Easier parallel job submission. Coiled Batch will now iterate over a list of inputs or rows in a file. For example, `coiled batch run --map-over-values "first,second,third" my_script.py` will run three tasks, with `COILED_BATCH_TASK_INPUT` set to “first” for the first task, and so on.
  • Support for .env files. Send environment variables to coiled batch run with `--env-file` and `--secret-env-file`.
  • Coordination among tasks. Each task within a batch job has a number of environment variables set automatically (like task id, cluster id, and IP addresses). This can help with using experiment tracking tools like MLflow or Weights & Biases, where you could tag an MLflow run with `COILED_CLUSTER_ID` to trace which cluster ran a particular training job. These are now explicitly listed in the documentation.

📚 Read the Coiled Batch docs.

Distributed GPU model training with 🤗 Accelerate

Article content

Hugging Face Accelerate simplifies PyTorch model training and inference on distributed hardware. This enables working with large datasets and models that don’t fit into a single machine’s memory. This example shows how to run multi-node distributed training on GPUs with Coiled Batch.

🚀 Try the example.

KoBold Metals Accelerates Mineral Discovery with Coiled

KoBold Metals is reimagining mineral exploration with data science and AI. Focused on sourcing critical battery metals, they use massive geospatial datasets to pinpoint promising exploration targets. As their computational demands grew, they turned to Coiled to scale to the cloud. Today, KoBold uses Coiled for:

  1. Remote sensing analysis across multiple spectral regimes
  2. Hyperspectral processing with hundreds of bands from aircraft surveys
  3. Stochastic inversions that generate subsurface models from surface data

“Coiled has made it possible for us to process huge volumes of data, and that's been really key to getting fast turnarounds on our analysis. The faster we can falsify our hypotheses, the sooner we can decide whether to commit resources to exploration.”

📚 Read the case study.

Handling long‑running workloads beyond AWS Lambda

Article content

Lambda is great for short tasks, but struggles with long-running, compute-heavy jobs. In our new post, we walk through how Coiled handles these limitations and talk through when you can use Coiled versus other AWS tools like Fargate or AWS Batch. 

📚 Read the blog post.


New to Coiled?

It’s easy to get started.

$ pip install coiled
$ coiled quickstart        

To view or add a comment, sign in

More articles by Coiled

Others also viewed

Explore content categories