almessadi.
Back to Index

Go Concurrency Needs Limits, Not Just Goroutines_

Goroutines are cheap, not free. Worker pools and channels matter because concurrency without backpressure becomes another way to overload a system.

PublishedJune 18, 2024
Reading Time8 min read

The biggest beginner mistake in Go is confusing easy concurrency with free concurrency.

Goroutines are lightweight, which is excellent. They are still work, and they still consume CPU, memory, sockets, and file descriptors indirectly through the tasks they perform.

The Problem With Unbounded Fanout

This is fine at small scale:

for _, file := range files {
    go downloadAndParse(file)
}

It becomes dangerous when the loop size reflects external input or a large backlog.

The usual failure is not "too many goroutines" as an abstract number. It is downstream exhaustion:

  • too many open connections
  • too much I/O pressure
  • too much memory in flight

Why Worker Pools Help

Worker pools create backpressure by limiting how much work can execute concurrently:

func worker(jobs <-chan string, results chan<- error) {
    for file := range jobs {
        results <- downloadAndParse(file)
    }
}

The point is control, not style.

You decide how much work the system should admit at once, based on the real bottleneck.

Channels Are About Coordination

Channels are useful because they make handoff explicit:

  • who produces work
  • who consumes it
  • where the system should block

That is often clearer and safer than letting concurrency fan out implicitly.

The Better Rule

Do not ask "can this be concurrent?"

Ask:

  • what is the bottleneck?
  • how much concurrency can it tolerate?
  • where should backpressure happen?

That is how Go concurrency becomes engineering instead of optimism.

Further Reading