← Login Counter | Series Overview | Ticket Seller →

The Problem: Money Vanishing Into Thin Air

Two people share a bank account with $100. Both check the balance at the same time, see $100, and both withdraw $100. The bank just lost $100. This isn’t a hypothetical—race conditions in financial systems have caused real monetary losses.

The bank account drama illustrates the fundamental challenge of concurrent programming: read-modify-write operations are not atomic. What seems like a simple operation actually involves multiple steps, and when multiple goroutines execute these steps concurrently, chaos ensues.

The Naive Implementation

Let’s start with the broken version:

package main

import (
    "fmt"
    "sync"
)

type BankAccount struct {
    balance int
}

func (a *BankAccount) Withdraw(amount int) bool {
    // Read the balance
    if a.balance >= amount {
        // Simulate some processing time
        // In reality, this could be logging, validation, etc.

        // Modify the balance
        a.balance -= amount
        return true
    }
    return false
}

func (a *BankAccount) Balance() int {
    return a.balance
}

func main() {
    account := &BankAccount{balance: 100}
    var wg sync.WaitGroup

    // Two people trying to withdraw $100 simultaneously
    for i := 0; i < 2; i++ {
        wg.Add(1)
        go func(person int) {
            defer wg.Done()
            if account.Withdraw(100) {
                fmt.Printf("Person %d withdrew $100\n", person)
            } else {
                fmt.Printf("Person %d failed to withdraw\n", person)
            }
        }(i)
    }

    wg.Wait()
    fmt.Printf("Final balance: $%d\n", account.Balance())
}

Output (race condition):

Person 0 withdrew $100
Person 1 withdrew $100
Final balance: $-100

Both goroutines read balance = 100, both see sufficient funds, both withdraw. The bank account goes negative.

The “Easy” Fix: Mutex Lock

The obvious solution is to use a mutex:

type BankAccount struct {
    mu      sync.Mutex
    balance int
}

func (a *BankAccount) Withdraw(amount int) bool {
    a.mu.Lock()
    defer a.mu.Unlock()

    if a.balance >= amount {
        a.balance -= amount
        return true
    }
    return false
}

func (a *BankAccount) Balance() int {
    a.mu.Lock()
    defer a.mu.Unlock()
    return a.balance
}

This works! But the drama is just beginning…

The Transfer Problem: Deadlock Risk

Now let’s try to transfer money between two accounts:

func Transfer(from, to *BankAccount, amount int) error {
    from.mu.Lock()
    defer from.mu.Unlock()

    if from.balance < amount {
        return fmt.Errorf("insufficient funds")
    }

    to.mu.Lock()  // ⚠️ DEADLOCK RISK!
    defer to.mu.Unlock()

    from.balance -= amount
    to.balance += amount
    return nil
}

The Deadlock Scenario:

  • Goroutine A: Transfer(account1, account2, 50)
    • Locks account1
    • Waits for account2 lock
  • Goroutine B: Transfer(account2, account1, 30)
    • Locks account2
    • Waits for account1 lock

Both goroutines wait forever. Classic deadlock.

Deadlock Prevention: Lock Ordering

The solution is to always acquire locks in the same order:

func Transfer(from, to *BankAccount, amount int) error {
    // Determine lock order based on memory address
    first, second := from, to
    if uintptr(unsafe.Pointer(from)) > uintptr(unsafe.Pointer(to)) {
        first, second = to, from
    }

    first.mu.Lock()
    defer first.mu.Unlock()

    second.mu.Lock()
    defer second.mu.Unlock()

    if from.balance < amount {
        return fmt.Errorf("insufficient funds")
    }

    from.balance -= amount
    to.balance += amount
    return nil
}

This ensures locks are always acquired in a consistent order, preventing circular wait conditions.

Interest Calculation While Withdrawing

Now add a goroutine that periodically calculates interest:

func (a *BankAccount) AddInterest(rate float64) {
    a.mu.Lock()
    defer a.mu.Unlock()

    interest := float64(a.balance) * rate
    a.balance += int(interest)
}

// Background goroutine
go func() {
    ticker := time.NewTicker(1 * time.Second)
    defer ticker.Stop()

    for range ticker.C {
        account.AddInterest(0.01) // 1% interest
    }
}()

New problem: If a withdrawal is in progress, interest calculation waits. If interest calculation is running, withdrawals wait. The longer the critical section, the more contention.

Overdraft Protection: The Complexity Cascade

Let’s add overdraft protection with a separate overdraft account:

type BankAccount struct {
    mu              sync.Mutex
    balance         int
    overdraftLimit  int
    overdraftUsed   int
    overdraftAccount *OverdraftAccount
}

func (a *BankAccount) WithdrawWithOverdraft(amount int) error {
    a.mu.Lock()
    defer a.mu.Unlock()

    available := a.balance + (a.overdraftLimit - a.overdraftUsed)
    if available < amount {
        return fmt.Errorf("insufficient funds including overdraft")
    }

    if a.balance >= amount {
        // Normal withdrawal
        a.balance -= amount
    } else {
        // Need overdraft
        shortfall := amount - a.balance
        a.balance = 0

        // ⚠️ Calling external method while holding lock!
        if err := a.overdraftAccount.Debit(shortfall); err != nil {
            return err
        }

        a.overdraftUsed += shortfall
    }

    return nil
}

Problem: Calling overdraftAccount.Debit() while holding the mutex can:

  1. Cause deadlock if Debit() tries to lock the same account
  2. Increase lock contention (longer critical sections)
  3. Make the code harder to reason about

A Better Design: Message Passing

Instead of shared state with locks, use channels for serialization:

type BankAccount struct {
    operations chan func(*accountState)
}

type accountState struct {
    balance int
}

func NewBankAccount(initial int) *BankAccount {
    acc := &BankAccount{
        operations: make(chan func(*accountState)),
    }

    state := &accountState{balance: initial}

    // Single goroutine processes all operations
    go func() {
        for op := range acc.operations {
            op(state)
        }
    }()

    return acc
}

func (a *BankAccount) Withdraw(amount int) bool {
    result := make(chan bool)

    a.operations <- func(state *accountState) {
        if state.balance >= amount {
            state.balance -= amount
            result <- true
        } else {
            result <- false
        }
    }

    return <-result
}

func (a *BankAccount) Balance() int {
    result := make(chan int)

    a.operations <- func(state *accountState) {
        result <- state.balance
    }

    return <-result
}

Benefits:

  • No explicit locks
  • No deadlock risk (single goroutine)
  • Operations are naturally serialized
  • Easier to reason about

Tradeoffs:

  • More goroutines (one per account)
  • Channel overhead
  • Transfer between accounts requires coordination

Implementing Transfer with Channels

func Transfer(from, to *BankAccount, amount int) error {
    result := make(chan error)

    // Send operation to 'from' account
    from.operations <- func(fromState *accountState) {
        if fromState.balance < amount {
            result <- fmt.Errorf("insufficient funds")
            return
        }

        fromState.balance -= amount

        // Send operation to 'to' account
        to.operations <- func(toState *accountState) {
            toState.balance += amount
            result <- nil
        }
    }

    return <-result
}

Performance Considerations

Let’s benchmark the different approaches:

func BenchmarkMutexWithdraw(b *testing.B) {
    account := &BankAccountMutex{balance: 1000000}

    b.RunParallel(func(pb *testing.PB) {
        for pb.Next() {
            account.Withdraw(1)
            account.Deposit(1)
        }
    })
}

func BenchmarkChannelWithdraw(b *testing.B) {
    account := NewBankAccount(1000000)
    defer close(account.operations)

    b.RunParallel(func(pb *testing.PB) {
        for pb.Next() {
            account.Withdraw(1)
            account.Deposit(1)
        }
    })
}

Typical results:

BenchmarkMutexWithdraw-8      3000000    450 ns/op
BenchmarkChannelWithdraw-8    1000000   1200 ns/op

Mutex approach is ~2.5x faster for simple operations, but:

  • Channel approach is easier to extend
  • Channel approach has no deadlock risk
  • For complex operations with multiple accounts, channel overhead becomes less significant

Testing Race Conditions

Use Go’s race detector:

func TestConcurrentWithdrawals(t *testing.T) {
    account := &BankAccount{balance: 1000}
    var wg sync.WaitGroup

    // 100 goroutines each withdrawing $10
    for i := 0; i < 100; i++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            account.Withdraw(10)
        }()
    }

    wg.Wait()

    // Should have $0 left
    if balance := account.Balance(); balance != 0 {
        t.Errorf("Expected balance 0, got %d", balance)
    }
}

Run with: go test -race

The race detector will catch the data race even if the test “passes” due to timing.

Real-World Applications

  1. Financial Systems: Banking, payment processing, cryptocurrency wallets
  2. Inventory Management: E-commerce stock counts, warehouse systems
  3. Gaming: Virtual currency, item trading, player stats
  4. Resource Pools: Database connections, thread pools, memory allocators

Common Pitfalls

  1. Forgetting to lock reads: Even reading shared state requires synchronization
  2. Lock scope too large: Holding locks during I/O or external calls
  3. Lock scope too small: Splitting read-modify-write into separate critical sections
  4. Inconsistent lock ordering: Different functions lock in different orders
  5. Returning pointers to locked data: Caller can modify without lock

Best Practices

  1. Keep critical sections short: Only lock what you must
  2. Consistent lock ordering: Document and enforce lock hierarchy
  3. Avoid calling unknown code with locks held: It might try to acquire the same lock
  4. Consider lock-free alternatives: Atomic operations for simple cases
  5. Use channels when appropriate: Especially for complex coordination
  6. Always use the race detector: go test -race should be in your CI/CD

Advanced: Lock-Free with Atomic Operations

For simple counters, use atomic operations:

import "sync/atomic"

type BankAccount struct {
    balance int64 // Must be 64-bit aligned for atomic ops
}

func (a *BankAccount) Withdraw(amount int64) bool {
    for {
        current := atomic.LoadInt64(&a.balance)
        if current < amount {
            return false
        }

        if atomic.CompareAndSwapInt64(&a.balance, current, current-amount) {
            return true
        }
        // CAS failed, retry
    }
}

This is lock-free but only works for simple operations. Complex operations still need locks or channels.

Conclusion

The bank account drama teaches us that:

  • Simple fixes reveal complex problems: Adding a mutex is easy; handling transfers, deadlocks, and overdrafts is hard
  • Every solution has tradeoffs: Locks are fast but risky; channels are safe but slower
  • Race conditions are subtle: The race detector is your friend
  • Design matters: Sometimes the best fix is redesigning the system

The question isn’t “should I use locks or channels?” but “what properties does my system need?” If you need:

  • Performance: Use mutexes with careful design
  • Safety and simplicity: Use channels
  • Both: Use the right tool for each component

Remember: Correctness first, performance second. A fast system that loses money is worse than a slow system that doesn’t.

Further Reading

← Distributed Tracing | Series Overview | Ticket Seller →