Go Concurrency Patterns Series: Series Overview | Channel Fundamentals →


What are Goroutines?

Goroutines are lightweight threads managed by the Go runtime. They’re one of Go’s most powerful features, allowing you to write concurrent programs that can handle thousands of simultaneous operations with minimal overhead. Think of goroutines as extremely efficient workers that can run independently while sharing the same memory space.

Unlike traditional threads that typically consume 1-2MB of memory each, goroutines start with just 2KB of stack space and grow as needed. This efficiency allows Go programs to spawn millions of goroutines without overwhelming system resources.

The Problem: Expensive Concurrency

Let’s start with a scenario that demonstrates why goroutines are revolutionary. Imagine you’re building a web scraper that needs to fetch data from 10,000 URLs. Using traditional threading approaches:

// Traditional approach - DON'T DO THIS
func traditionalApproach() {
    urls := generateURLs(10000)
    
    for _, url := range urls {
        // This blocks until each request completes
        response := fetchURL(url)
        processResponse(response)
    }
    // This would take hours to complete!
}

This sequential approach would be painfully slow. Even with traditional threads, creating 10,000 OS threads would likely crash your system due to memory constraints.

Enter Goroutines: Lightweight Concurrency

Here’s how goroutines solve this problem elegantly:

package main

import (
    "fmt"
    "net/http"
    "sync"
    "time"
)

func main() {
    urls := []string{
        "https://httpbin.org/delay/1",
        "https://httpbin.org/delay/2", 
        "https://httpbin.org/delay/1",
        "https://httpbin.org/delay/3",
        "https://httpbin.org/delay/1",
    }
    
    start := time.Now()
    
    // Without goroutines - sequential execution
    fmt.Println("Sequential execution:")
    for _, url := range urls {
        fetchURL(url)
    }
    fmt.Printf("Sequential time: %v\n\n", time.Since(start))
    
    // With goroutines - concurrent execution
    start = time.Now()
    fmt.Println("Concurrent execution:")
    
    var wg sync.WaitGroup
    for _, url := range urls {
        wg.Add(1)
        go func(u string) {
            defer wg.Done()
            fetchURL(u)
        }(url)
    }
    wg.Wait()
    
    fmt.Printf("Concurrent time: %v\n", time.Since(start))
}

func fetchURL(url string) {
    start := time.Now()
    resp, err := http.Get(url)
    if err != nil {
        fmt.Printf("Error fetching %s: %v\n", url, err)
        return
    }
    defer resp.Body.Close()
    
    fmt.Printf("Fetched %s in %v\n", url, time.Since(start))
}

Output:

Sequential execution:
Fetched https://httpbin.org/delay/1 in 1.2s
Fetched https://httpbin.org/delay/2 in 2.1s
Fetched https://httpbin.org/delay/1 in 1.1s
Fetched https://httpbin.org/delay/3 in 3.2s
Fetched https://httpbin.org/delay/1 in 1.1s
Sequential time: 8.7s

Concurrent execution:
Fetched https://httpbin.org/delay/1 in 1.1s
Fetched https://httpbin.org/delay/1 in 1.2s
Fetched https://httpbin.org/delay/1 in 1.3s
Fetched https://httpbin.org/delay/2 in 2.1s
Fetched https://httpbin.org/delay/3 in 3.2s
Concurrent time: 3.2s

The concurrent version completes in roughly the time of the slowest request, rather than the sum of all requests!

Goroutine Creation Patterns

1. Basic Goroutine Launch

func basicGoroutine() {
    // Launch a goroutine with the 'go' keyword
    go func() {
        fmt.Println("Hello from goroutine!")
    }()
    
    // Launch named function as goroutine
    go printMessage("Hello World")
    
    // Give goroutines time to execute
    time.Sleep(100 * time.Millisecond)
}

func printMessage(msg string) {
    fmt.Println(msg)
}

2. Goroutines with Parameters

func goroutineWithParams() {
    names := []string{"Alice", "Bob", "Charlie"}
    
    var wg sync.WaitGroup
    for _, name := range names {
        wg.Add(1)
        
        // Capture the loop variable
        go func(n string) {
            defer wg.Done()
            fmt.Printf("Processing %s\n", n)
            time.Sleep(time.Duration(len(n)) * 100 * time.Millisecond)
            fmt.Printf("Finished processing %s\n", n)
        }(name)
    }
    
    wg.Wait()
    fmt.Println("All processing complete")
}

3. Common Pitfall: Variable Capture

func variableCapturePitfall() {
    fmt.Println("❌ WRONG WAY - Variable capture issue:")
    
    var wg sync.WaitGroup
    for i := 0; i < 3; i++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            // This captures the loop variable by reference!
            // All goroutines might see the same value
            fmt.Printf("Wrong: %d\n", i)
        }()
    }
    wg.Wait()
    
    fmt.Println("\n✅ CORRECT WAY - Pass as parameter:")
    for i := 0; i < 3; i++ {
        wg.Add(1)
        go func(val int) {
            defer wg.Done()
            fmt.Printf("Correct: %d\n", val)
        }(i)
    }
    wg.Wait()
}

Real-World Example: Concurrent File Processor

Let’s build a practical example that processes multiple files concurrently:

package main

import (
    "bufio"
    "fmt"
    "os"
    "path/filepath"
    "strings"
    "sync"
    "time"
)

type FileResult struct {
    Filename  string
    LineCount int
    WordCount int
    Duration  time.Duration
    Error     error
}

type FileProcessor struct {
    maxWorkers int
    results    chan FileResult
    wg         sync.WaitGroup
}

func NewFileProcessor(maxWorkers int) *FileProcessor {
    return &FileProcessor{
        maxWorkers: maxWorkers,
        results:    make(chan FileResult, maxWorkers),
    }
}

func (fp *FileProcessor) ProcessFiles(filenames []string) []FileResult {
    // Start result collector
    var results []FileResult
    done := make(chan bool)
    
    go func() {
        for result := range fp.results {
            results = append(results, result)
        }
        done <- true
    }()
    
    // Process files concurrently
    for _, filename := range filenames {
        fp.wg.Add(1)
        go fp.processFile(filename)
    }
    
    // Wait for all processing to complete
    fp.wg.Wait()
    close(fp.results)
    
    // Wait for result collection to finish
    <-done
    
    return results
}

func (fp *FileProcessor) processFile(filename string) {
    defer fp.wg.Done()
    
    start := time.Now()
    result := FileResult{
        Filename: filename,
        Duration: 0,
    }
    
    file, err := os.Open(filename)
    if err != nil {
        result.Error = err
        result.Duration = time.Since(start)
        fp.results <- result
        return
    }
    defer file.Close()
    
    scanner := bufio.NewScanner(file)
    lineCount := 0
    wordCount := 0
    
    for scanner.Scan() {
        lineCount++
        words := strings.Fields(scanner.Text())
        wordCount += len(words)
    }
    
    if err := scanner.Err(); err != nil {
        result.Error = err
    } else {
        result.LineCount = lineCount
        result.WordCount = wordCount
    }
    
    result.Duration = time.Since(start)
    fp.results <- result
}

func main() {
    // Create some sample files for demonstration
    createSampleFiles()
    
    files := []string{
        "sample1.txt",
        "sample2.txt", 
        "sample3.txt",
        "sample4.txt",
    }
    
    fmt.Println("Processing files concurrently...")
    start := time.Now()
    
    processor := NewFileProcessor(4)
    results := processor.ProcessFiles(files)
    
    totalDuration := time.Since(start)
    
    // Display results
    fmt.Printf("\nResults (completed in %v):\n", totalDuration)
    fmt.Println(strings.Repeat("-", 60))
    
    totalLines := 0
    totalWords := 0
    
    for _, result := range results {
        if result.Error != nil {
            fmt.Printf("❌ %s: Error - %v\n", result.Filename, result.Error)
        } else {
            fmt.Printf("✅ %s: %d lines, %d words (processed in %v)\n",
                result.Filename, result.LineCount, result.WordCount, result.Duration)
            totalLines += result.LineCount
            totalWords += result.WordCount
        }
    }
    
    fmt.Println(strings.Repeat("-", 60))
    fmt.Printf("Total: %d lines, %d words across %d files\n", 
        totalLines, totalWords, len(results))
    
    // Cleanup
    cleanupSampleFiles(files)
}

func createSampleFiles() {
    samples := map[string]string{
        "sample1.txt": "Hello World\nThis is a sample file\nWith multiple lines\n",
        "sample2.txt": "Another file\nWith different content\nFor testing purposes\nConcurrent processing\n",
        "sample3.txt": "Short file\nJust two lines\n",
        "sample4.txt": "The longest file\nWith many lines\nTo demonstrate\nConcurrent file processing\nUsing goroutines\nIn Go programming language\n",
    }
    
    for filename, content := range samples {
        os.WriteFile(filename, []byte(content), 0644)
    }
}

func cleanupSampleFiles(files []string) {
    for _, file := range files {
        os.Remove(file)
    }
}

Goroutine Lifecycle Management

1. Preventing Goroutine Leaks

func preventGoroutineLeaks() {
    // ❌ BAD: Goroutine leak
    badExample := func() {
        go func() {
            // This goroutine runs forever!
            for {
                time.Sleep(1 * time.Second)
                fmt.Println("Still running...")
            }
        }()
    }
    
    // ✅ GOOD: Controlled goroutine with cancellation
    goodExample := func() {
        done := make(chan bool)
        
        go func() {
            ticker := time.NewTicker(1 * time.Second)
            defer ticker.Stop()
            
            for {
                select {
                case <-ticker.C:
                    fmt.Println("Working...")
                case <-done:
                    fmt.Println("Shutting down gracefully")
                    return
                }
            }
        }()
        
        // Simulate some work
        time.Sleep(3 * time.Second)
        
        // Signal shutdown
        close(done)
        time.Sleep(100 * time.Millisecond) // Give time to shutdown
    }
    
    fmt.Println("Running good example:")
    goodExample()
}

2. Graceful Shutdown Pattern

type Server struct {
    shutdown chan bool
    wg       sync.WaitGroup
}

func NewServer() *Server {
    return &Server{
        shutdown: make(chan bool),
    }
}

func (s *Server) Start() {
    // Start multiple worker goroutines
    for i := 0; i < 3; i++ {
        s.wg.Add(1)
        go s.worker(i)
    }
    
    fmt.Println("Server started with 3 workers")
}

func (s *Server) worker(id int) {
    defer s.wg.Done()
    
    ticker := time.NewTicker(500 * time.Millisecond)
    defer ticker.Stop()
    
    for {
        select {
        case <-ticker.C:
            fmt.Printf("Worker %d processing...\n", id)
        case <-s.shutdown:
            fmt.Printf("Worker %d shutting down\n", id)
            return
        }
    }
}

func (s *Server) Stop() {
    fmt.Println("Initiating graceful shutdown...")
    close(s.shutdown)
    s.wg.Wait()
    fmt.Println("All workers stopped")
}

func demonstrateGracefulShutdown() {
    server := NewServer()
    server.Start()
    
    // Let it run for a while
    time.Sleep(2 * time.Second)
    
    // Graceful shutdown
    server.Stop()
}

Performance Considerations

Goroutine Overhead

func benchmarkGoroutineOverhead() {
    // Measure goroutine creation overhead
    start := time.Now()
    
    var wg sync.WaitGroup
    numGoroutines := 100000
    
    for i := 0; i < numGoroutines; i++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            // Minimal work
        }()
    }
    
    wg.Wait()
    duration := time.Since(start)
    
    fmt.Printf("Created and executed %d goroutines in %v\n", numGoroutines, duration)
    fmt.Printf("Average time per goroutine: %v\n", duration/time.Duration(numGoroutines))
}

Memory Usage Comparison

func compareMemoryUsage() {
    var m1, m2 runtime.MemStats
    
    // Measure baseline memory
    runtime.GC()
    runtime.ReadMemStats(&m1)
    
    // Create many goroutines
    var wg sync.WaitGroup
    numGoroutines := 10000
    
    for i := 0; i < numGoroutines; i++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            time.Sleep(10 * time.Millisecond)
        }()
    }
    
    // Measure memory with goroutines
    runtime.ReadMemStats(&m2)
    
    wg.Wait()
    
    memoryPerGoroutine := (m2.Alloc - m1.Alloc) / uint64(numGoroutines)
    fmt.Printf("Approximate memory per goroutine: %d bytes\n", memoryPerGoroutine)
}

Best Practices

1. Always Handle Goroutine Completion

// ✅ Use WaitGroup for coordination
func goodCoordination() {
    var wg sync.WaitGroup
    
    for i := 0; i < 5; i++ {
        wg.Add(1)
        go func(id int) {
            defer wg.Done()
            fmt.Printf("Task %d completed\n", id)
        }(i)
    }
    
    wg.Wait() // Wait for all goroutines to complete
}

2. Limit Goroutine Creation

// ✅ Use worker pools for bounded concurrency
func boundedConcurrency() {
    maxWorkers := 10
    jobs := make(chan int, 100)
    
    // Start fixed number of workers
    var wg sync.WaitGroup
    for i := 0; i < maxWorkers; i++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            for job := range jobs {
                processJob(job)
            }
        }()
    }
    
    // Send jobs
    for i := 0; i < 50; i++ {
        jobs <- i
    }
    close(jobs)
    
    wg.Wait()
}

func processJob(id int) {
    fmt.Printf("Processing job %d\n", id)
    time.Sleep(100 * time.Millisecond)
}

3. Proper Error Handling

type Result struct {
    Value int
    Error error
}

func goroutineWithErrorHandling() {
    results := make(chan Result, 5)
    var wg sync.WaitGroup
    
    for i := 0; i < 5; i++ {
        wg.Add(1)
        go func(id int) {
            defer wg.Done()
            
            // Simulate work that might fail
            if id%2 == 0 {
                results <- Result{Value: id * 2, Error: nil}
            } else {
                results <- Result{Value: 0, Error: fmt.Errorf("failed to process %d", id)}
            }
        }(i)
    }
    
    // Close results channel when all goroutines complete
    go func() {
        wg.Wait()
        close(results)
    }()
    
    // Collect results
    for result := range results {
        if result.Error != nil {
            fmt.Printf("Error: %v\n", result.Error)
        } else {
            fmt.Printf("Success: %d\n", result.Value)
        }
    }
}

Common Pitfalls and Solutions

1. Race Conditions

// ❌ Race condition
func raceCondition() {
    counter := 0
    var wg sync.WaitGroup
    
    for i := 0; i < 1000; i++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            counter++ // Race condition!
        }()
    }
    
    wg.Wait()
    fmt.Printf("Counter (with race): %d\n", counter) // Unpredictable result
}

// ✅ Thread-safe version
func threadSafe() {
    var counter int64
    var wg sync.WaitGroup
    
    for i := 0; i < 1000; i++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            atomic.AddInt64(&counter, 1) // Thread-safe
        }()
    }
    
    wg.Wait()
    fmt.Printf("Counter (thread-safe): %d\n", counter) // Always 1000
}

2. Goroutine Leaks

// ❌ Potential goroutine leak
func potentialLeak() {
    ch := make(chan int)
    
    go func() {
        // This goroutine will block forever if no one reads from ch
        ch <- 42
    }()
    
    // If we don't read from ch, the goroutine leaks
}

// ✅ Leak prevention with timeout
func leakPrevention() {
    ch := make(chan int, 1) // Buffered channel prevents blocking
    
    go func() {
        ch <- 42
    }()
    
    select {
    case value := <-ch:
        fmt.Printf("Received: %d\n", value)
    case <-time.After(1 * time.Second):
        fmt.Println("Timeout - no value received")
    }
}

Testing Goroutines

func TestConcurrentFunction(t *testing.T) {
    // Test concurrent execution
    start := time.Now()
    
    var wg sync.WaitGroup
    results := make(chan int, 5)
    
    for i := 0; i < 5; i++ {
        wg.Add(1)
        go func(val int) {
            defer wg.Done()
            time.Sleep(100 * time.Millisecond) // Simulate work
            results <- val * 2
        }(i)
    }
    
    go func() {
        wg.Wait()
        close(results)
    }()
    
    var sum int
    for result := range results {
        sum += result
    }
    
    duration := time.Since(start)
    
    // Verify results
    expected := 0 + 2 + 4 + 6 + 8 // 0*2 + 1*2 + 2*2 + 3*2 + 4*2
    if sum != expected {
        t.Errorf("Expected sum %d, got %d", expected, sum)
    }
    
    // Verify concurrent execution (should be much faster than sequential)
    if duration > 200*time.Millisecond {
        t.Errorf("Execution took too long: %v", duration)
    }
}

Conclusion

Goroutines are the foundation of Go’s concurrency model, providing:

  • Lightweight concurrency: Minimal memory overhead and fast creation
  • Simple syntax: Just add go before a function call
  • Scalability: Handle thousands of concurrent operations efficiently
  • Integration: Work seamlessly with channels and other Go concurrency primitives

Key Takeaways

  1. Always coordinate goroutines using WaitGroups, channels, or context
  2. Prevent goroutine leaks with proper cleanup and cancellation
  3. Handle errors appropriately in concurrent contexts
  4. Limit concurrency when dealing with external resources
  5. Test concurrent code thoroughly, including race condition detection

When to Use Goroutines

  • I/O-bound operations: Network requests, file operations, database queries
  • Independent tasks: Operations that can run in parallel
  • Event handling: Background processing, monitoring, periodic tasks
  • Pipeline processing: Multi-stage data transformation

What’s Next?

Now that you understand goroutines, the next step is learning about channels - Go’s primary mechanism for goroutine communication. Channels allow goroutines to safely share data and coordinate their execution.

In the next post, we’ll explore Channel Fundamentals and learn how to build robust communication patterns between goroutines.


This post is part of the Go Concurrency Patterns series. Each pattern builds upon previous concepts, so I recommend following the suggested learning path for the best experience.