Digital Independence : Using a local coding agent

This works on a 2021 MacBookPro M1 with 32GB Ram, a typical developer machine, more recent CPUs like M3 or M4 should be more powerful.

For installing llama.cpp and some first steps, see my earlier article.

Install and run Qwen3 Coder LLM

Qwen3 Coder is a powerful open-source AI model for code generation, tool use and agentic tasks. This will use about 16-18 GB of your RAM.

llama-server --port 8080 --n-gpu-layers 99 -fa 1 --jinja -ctk f16 -ctv f16 --kv-unified --threads 4 --temp 0.7 --repeat-penalty 1.0 --reasoning-format none -hf unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:Q4_K_XL        

Install the qwen coding agent

qwen-code is a fork of gemini-cli optimized for using qwen based models running locally, it also works great with other openai api based llm like gpt-oss-120b. But gpt-oss-120b won't fit on a MacBookPro due to RAM limitations.

brew install qwen-code        

Set some environment variables

export OPENAI_BASE_URL=http://localhost:8080/v1
export OPENAI_API_KEY="dummy-api-key"
export OPENAI_MODEL=unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:Q4_K_XL        

Switch to your favorite project and start qwen-code

qwen
# let qwen analyze your project, that may need some time
# only needed once
/init 
# ask some ask questions about your code
Explain this projects software architecture         

Happy coding, you need no internet connection or an expensive AI subscription, it just runs locally on your machine.