Golang

Go Architecture Patterns Series: ← Previous: CQRS Pattern | Series Overview | Next: Saga Pattern → What is Event Sourcing? Event Sourcing is an architectural pattern where state changes are stored as a sequence of immutable events rather than updating records in place. The current state is derived by replaying all events from the beginning. Key Principles: Event Store: All state changes stored as events Immutable Events: Events are never modified or deleted Event Replay: Current state reconstructed by replaying events Complete Audit Trail: Every change is recorded with full context Temporal Queries: Query state at any point in time Event Versioning: Events can evolve while maintaining history Architecture Overview %%{init: {'theme':'dark', 'themeVariables': {'primaryTextColor':'#e5e7eb','secondaryTextColor':'#e5e7eb','tertiaryTextColor':'#e5e7eb','textColor':'#e5e7eb','nodeTextColor':'#e5e7eb','edgeLabelText':'#e5e7eb','clusterTextColor':'#e5e7eb','actorTextColor':'#e5e7eb'}}}%% graph TB subgraph "Command Side" Command[Command] Aggregate[Aggregate Root] EventStore[(Event Store)] end subgraph "Event Processing" EventBus[Event Bus] Projector[Event Projectors] end subgraph "Query Side" ReadModel1[(Current State View)] ReadModel2[(Analytics View)] ReadModel3[(Audit Log View)] end subgraph "Time Travel" Snapshot[(Snapshots)] Replay[Event Replay] end Command --> Aggregate Aggregate -->|Load Events| EventStore Aggregate -->|Append Events| EventStore EventStore -->|Publish| EventBus EventBus --> Projector Projector --> ReadModel1 Projector --> ReadModel2 Projector --> ReadModel3 EventStore --> Snapshot Snapshot --> Replay Replay --> Aggregate style Aggregate fill:#78350f,color:#fff style EventStore fill:#1e3a5f,color:#fff style Projector fill:#134e4a,color:#fff style ReadModel1 fill:#4c1d95,color:#fff Event Sourcing Flow %%{init: {'theme':'dark', 'themeVariables': {'primaryTextColor':'#e5e7eb','secondaryTextColor':'#e5e7eb','tertiaryTextColor':'#e5e7eb','textColor':'#e5e7eb','nodeTextColor':'#e5e7eb','edgeLabelText':'#e5e7eb','clusterTextColor':'#e5e7eb','actorTextColor':'#e5e7eb'}}}%% sequenceDiagram participant Client participant CommandHandler participant Aggregate participant EventStore participant EventBus participant Projection Client->>CommandHandler: Execute Command CommandHandler->>EventStore: Load Events for Aggregate EventStore-->>CommandHandler: Event Stream CommandHandler->>Aggregate: Replay Events Aggregate->>Aggregate: Rebuild State CommandHandler->>Aggregate: Execute Command Aggregate->>Aggregate: Validate Business Rules Aggregate->>Aggregate: Generate Events CommandHandler->>EventStore: Append New Events EventStore-->>CommandHandler: Events Persisted EventStore->>EventBus: Publish Events EventBus->>Projection: Project Events Projection->>Projection: Update Read Model CommandHandler-->>Client: Command Accepted Real-World Use Cases Banking Systems: Complete transaction history and audit trails E-commerce: Order lifecycle tracking and returns processing Healthcare: Patient medical history and treatment records Trading Systems: Trade execution history and reconciliation Gaming: Player action history and replay functionality Workflow Systems: Process execution history and debugging Event Sourcing Implementation Project Structure event-sourcing-app/ ├── cmd/ │ └── api/ │ └── main.go ├── internal/ │ ├── domain/ │ │ ├── account.go │ │ └── events.go │ ├── eventstore/ │ │ ├── store.go │ │ ├── postgres_store.go │ │ └── snapshot.go │ ├── commands/ │ │ └── handlers.go │ ├── projections/ │ │ └── account_projection.go │ └── queries/ │ └── handlers.go └── go.mod Domain Events // internal/domain/events.go package domain import ( "encoding/json" "time" ) // Event represents a domain event type Event interface { EventType() string AggregateID() string EventVersion() int Timestamp() time.Time } // BaseEvent provides common event fields type BaseEvent struct { Type string `json:"type"` AggrID string `json:"aggregate_id"` Version int `json:"version"` OccurredAt time.Time `json:"occurred_at"` Metadata map[string]string `json:"metadata"` } func (e BaseEvent) EventType() string { return e.Type } func (e BaseEvent) AggregateID() string { return e.AggrID } func (e BaseEvent) EventVersion() int { return e.Version } func (e BaseEvent) Timestamp() time.Time { return e.OccurredAt } // Account Events type AccountCreatedEvent struct { BaseEvent AccountID string `json:"account_id"` Owner string `json:"owner"` Currency string `json:"currency"` } type MoneyDepositedEvent struct { BaseEvent AccountID string `json:"account_id"` Amount float64 `json:"amount"` Balance float64 `json:"balance"` } type MoneyWithdrawnEvent struct { BaseEvent AccountID string `json:"account_id"` Amount float64 `json:"amount"` Balance float64 `json:"balance"` } type AccountClosedEvent struct { BaseEvent AccountID string `json:"account_id"` Reason string `json:"reason"` } // EventEnvelope wraps events for storage type EventEnvelope struct { EventID string `json:"event_id"` EventType string `json:"event_type"` AggregateID string `json:"aggregate_id"` Version int `json:"version"` Data json.RawMessage `json:"data"` Metadata map[string]string `json:"metadata"` CreatedAt time.Time `json:"created_at"` } // Serialize serializes event to envelope func SerializeEvent(event Event) (*EventEnvelope, error) { data, err := json.Marshal(event) if err != nil { return nil, err } return &EventEnvelope{ EventID: generateEventID(), EventType: event.EventType(), AggregateID: event.AggregateID(), Version: event.EventVersion(), Data: data, CreatedAt: event.Timestamp(), }, nil } // DeserializeEvent deserializes envelope to event func DeserializeEvent(envelope *EventEnvelope) (Event, error) { var event Event switch envelope.EventType { case "AccountCreated": event = &AccountCreatedEvent{} case "MoneyDeposited": event = &MoneyDepositedEvent{} case "MoneyWithdrawn": event = &MoneyWithdrawnEvent{} case "AccountClosed": event = &AccountClosedEvent{} default: return nil, fmt.Errorf("unknown event type: %s", envelope.EventType) } if err := json.Unmarshal(envelope.Data, event); err != nil { return nil, err } return event, nil } func generateEventID() string { return fmt.Sprintf("evt_%d", time.Now().UnixNano()) } Aggregate Root // internal/domain/account.go package domain import ( "errors" "fmt" ) var ( ErrInsufficientFunds = errors.New("insufficient funds") ErrAccountClosed = errors.New("account is closed") ErrInvalidAmount = errors.New("invalid amount") ) // Account is an aggregate root that uses event sourcing type Account struct { id string owner string currency string balance float64 isClosed bool version int uncommittedEvents []Event } // NewAccount creates a new account func NewAccount(id, owner, currency string) (*Account, error) { if id == "" || owner == "" || currency == "" { return nil, errors.New("invalid account parameters") } account := &Account{ uncommittedEvents: make([]Event, 0), } // Raise domain event event := AccountCreatedEvent{ BaseEvent: BaseEvent{ Type: "AccountCreated", AggrID: id, Version: 1, OccurredAt: time.Now(), }, AccountID: id, Owner: owner, Currency: currency, } account.raiseEvent(&event) return account, nil } // LoadFromHistory rebuilds account state from events func LoadFromHistory(events []Event) (*Account, error) { if len(events) == 0 { return nil, errors.New("no events to load") } account := &Account{ uncommittedEvents: make([]Event, 0), } for _, event := range events { if err := account.apply(event); err != nil { return nil, err } account.version = event.EventVersion() } return account, nil } // Deposit adds money to account func (a *Account) Deposit(amount float64) error { if a.isClosed { return ErrAccountClosed } if amount <= 0 { return ErrInvalidAmount } newBalance := a.balance + amount event := MoneyDepositedEvent{ BaseEvent: BaseEvent{ Type: "MoneyDeposited", AggrID: a.id, Version: a.version + 1, OccurredAt: time.Now(), }, AccountID: a.id, Amount: amount, Balance: newBalance, } a.raiseEvent(&event) return nil } // Withdraw removes money from account func (a *Account) Withdraw(amount float64) error { if a.isClosed { return ErrAccountClosed } if amount <= 0 { return ErrInvalidAmount } if a.balance < amount { return ErrInsufficientFunds } newBalance := a.balance - amount event := MoneyWithdrawnEvent{ BaseEvent: BaseEvent{ Type: "MoneyWithdrawn", AggrID: a.id, Version: a.version + 1, OccurredAt: time.Now(), }, AccountID: a.id, Amount: amount, Balance: newBalance, } a.raiseEvent(&event) return nil } // Close closes the account func (a *Account) Close(reason string) error { if a.isClosed { return ErrAccountClosed } event := AccountClosedEvent{ BaseEvent: BaseEvent{ Type: "AccountClosed", AggrID: a.id, Version: a.version + 1, OccurredAt: time.Now(), }, AccountID: a.id, Reason: reason, } a.raiseEvent(&event) return nil } // GetUncommittedEvents returns uncommitted events func (a *Account) GetUncommittedEvents() []Event { return a.uncommittedEvents } // MarkEventsAsCommitted clears uncommitted events func (a *Account) MarkEventsAsCommitted() { a.uncommittedEvents = make([]Event, 0) } // raiseEvent adds event to uncommitted and applies it func (a *Account) raiseEvent(event Event) { a.uncommittedEvents = append(a.uncommittedEvents, event) a.apply(event) } // apply applies an event to the aggregate state func (a *Account) apply(event Event) error { switch e := event.(type) { case *AccountCreatedEvent: a.id = e.AccountID a.owner = e.Owner a.currency = e.Currency a.balance = 0 a.isClosed = false case *MoneyDepositedEvent: a.balance = e.Balance case *MoneyWithdrawnEvent: a.balance = e.Balance case *AccountClosedEvent: a.isClosed = true default: return fmt.Errorf("unknown event type: %T", event) } return nil } // Getters func (a *Account) ID() string { return a.id } func (a *Account) Balance() float64 { return a.balance } func (a *Account) Version() int { return a.version } func (a *Account) IsClosed() bool { return a.isClosed } Event Store // internal/eventstore/store.go package eventstore import ( "context" "app/internal/domain" ) // EventStore interface type EventStore interface { SaveEvents(ctx context.Context, aggregateID string, events []domain.Event, expectedVersion int) error GetEvents(ctx context.Context, aggregateID string) ([]domain.Event, error) GetEventsSince(ctx context.Context, aggregateID string, version int) ([]domain.Event, error) GetAllEvents(ctx context.Context) ([]domain.Event, error) } // internal/eventstore/postgres_store.go package eventstore import ( "context" "database/sql" "fmt" "app/internal/domain" ) type PostgresEventStore struct { db *sql.DB } func NewPostgresEventStore(db *sql.DB) *PostgresEventStore { return &PostgresEventStore{db: db} } // SaveEvents appends events to the event store with optimistic concurrency func (s *PostgresEventStore) SaveEvents(ctx context.Context, aggregateID string, events []domain.Event, expectedVersion int) error { tx, err := s.db.BeginTx(ctx, nil) if err != nil { return err } defer tx.Rollback() // Check current version for optimistic locking var currentVersion int err = tx.QueryRowContext(ctx, "SELECT COALESCE(MAX(version), 0) FROM events WHERE aggregate_id = $1", aggregateID, ).Scan(&currentVersion) if err != nil && err != sql.ErrNoRows { return err } if currentVersion != expectedVersion { return fmt.Errorf("concurrency conflict: expected version %d, got %d", expectedVersion, currentVersion) } // Insert events query := ` INSERT INTO events (event_id, event_type, aggregate_id, version, data, metadata, created_at) VALUES ($1, $2, $3, $4, $5, $6, $7) ` for _, event := range events { envelope, err := domain.SerializeEvent(event) if err != nil { return err } _, err = tx.ExecContext(ctx, query, envelope.EventID, envelope.EventType, envelope.AggregateID, envelope.Version, envelope.Data, envelope.Metadata, envelope.CreatedAt, ) if err != nil { return err } } return tx.Commit() } // GetEvents retrieves all events for an aggregate func (s *PostgresEventStore) GetEvents(ctx context.Context, aggregateID string) ([]domain.Event, error) { query := ` SELECT event_id, event_type, aggregate_id, version, data, metadata, created_at FROM events WHERE aggregate_id = $1 ORDER BY version ASC ` rows, err := s.db.QueryContext(ctx, query, aggregateID) if err != nil { return nil, err } defer rows.Close() var events []domain.Event for rows.Next() { envelope := &domain.EventEnvelope{} err := rows.Scan( &envelope.EventID, &envelope.EventType, &envelope.AggregateID, &envelope.Version, &envelope.Data, &envelope.Metadata, &envelope.CreatedAt, ) if err != nil { return nil, err } event, err := domain.DeserializeEvent(envelope) if err != nil { return nil, err } events = append(events, event) } return events, nil } // GetEventsSince retrieves events after a specific version func (s *PostgresEventStore) GetEventsSince(ctx context.Context, aggregateID string, version int) ([]domain.Event, error) { query := ` SELECT event_id, event_type, aggregate_id, version, data, metadata, created_at FROM events WHERE aggregate_id = $1 AND version > $2 ORDER BY version ASC ` rows, err := s.db.QueryContext(ctx, query, aggregateID, version) if err != nil { return nil, err } defer rows.Close() var events []domain.Event for rows.Next() { envelope := &domain.EventEnvelope{} err := rows.Scan( &envelope.EventID, &envelope.EventType, &envelope.AggregateID, &envelope.Version, &envelope.Data, &envelope.Metadata, &envelope.CreatedAt, ) if err != nil { return nil, err } event, err := domain.DeserializeEvent(envelope) if err != nil { return nil, err } events = append(events, event) } return events, nil } // GetAllEvents retrieves all events from the store func (s *PostgresEventStore) GetAllEvents(ctx context.Context) ([]domain.Event, error) { query := ` SELECT event_id, event_type, aggregate_id, version, data, metadata, created_at FROM events ORDER BY created_at ASC ` rows, err := s.db.QueryContext(ctx, query) if err != nil { return nil, err } defer rows.Close() var events []domain.Event for rows.Next() { envelope := &domain.EventEnvelope{} err := rows.Scan( &envelope.EventID, &envelope.EventType, &envelope.AggregateID, &envelope.Version, &envelope.Data, &envelope.Metadata, &envelope.CreatedAt, ) if err != nil { return nil, err } event, err := domain.DeserializeEvent(envelope) if err != nil { return nil, err } events = append(events, event) } return events, nil } Snapshots for Performance // internal/eventstore/snapshot.go package eventstore import ( "context" "database/sql" "encoding/json" "time" ) // Snapshot represents a point-in-time state type Snapshot struct { AggregateID string Version int State json.RawMessage CreatedAt time.Time } type SnapshotStore struct { db *sql.DB } func NewSnapshotStore(db *sql.DB) *SnapshotStore { return &SnapshotStore{db: db} } // SaveSnapshot saves a snapshot func (s *SnapshotStore) SaveSnapshot(ctx context.Context, snapshot *Snapshot) error { query := ` INSERT INTO snapshots (aggregate_id, version, state, created_at) VALUES ($1, $2, $3, $4) ON CONFLICT (aggregate_id) DO UPDATE SET version = $2, state = $3, created_at = $4 ` _, err := s.db.ExecContext(ctx, query, snapshot.AggregateID, snapshot.Version, snapshot.State, snapshot.CreatedAt, ) return err } // GetSnapshot retrieves the latest snapshot func (s *SnapshotStore) GetSnapshot(ctx context.Context, aggregateID string) (*Snapshot, error) { query := ` SELECT aggregate_id, version, state, created_at FROM snapshots WHERE aggregate_id = $1 ` snapshot := &Snapshot{} err := s.db.QueryRowContext(ctx, query, aggregateID).Scan( &snapshot.AggregateID, &snapshot.Version, &snapshot.State, &snapshot.CreatedAt, ) if err == sql.ErrNoRows { return nil, nil } return snapshot, err } Command Handlers // internal/commands/handlers.go package commands import ( "context" "fmt" "app/internal/domain" "app/internal/eventstore" ) // CreateAccountCommand type CreateAccountCommand struct { AccountID string Owner string Currency string } type CreateAccountHandler struct { eventStore eventstore.EventStore } func NewCreateAccountHandler(store eventstore.EventStore) *CreateAccountHandler { return &CreateAccountHandler{eventStore: store} } func (h *CreateAccountHandler) Handle(ctx context.Context, cmd *CreateAccountCommand) error { // Create new aggregate account, err := domain.NewAccount(cmd.AccountID, cmd.Owner, cmd.Currency) if err != nil { return err } // Save events events := account.GetUncommittedEvents() if err := h.eventStore.SaveEvents(ctx, cmd.AccountID, events, 0); err != nil { return fmt.Errorf("failed to save events: %w", err) } account.MarkEventsAsCommitted() return nil } // DepositMoneyCommand type DepositMoneyCommand struct { AccountID string Amount float64 } type DepositMoneyHandler struct { eventStore eventstore.EventStore } func NewDepositMoneyHandler(store eventstore.EventStore) *DepositMoneyHandler { return &DepositMoneyHandler{eventStore: store} } func (h *DepositMoneyHandler) Handle(ctx context.Context, cmd *DepositMoneyCommand) error { // Load aggregate from events events, err := h.eventStore.GetEvents(ctx, cmd.AccountID) if err != nil { return err } account, err := domain.LoadFromHistory(events) if err != nil { return err } // Execute command if err := account.Deposit(cmd.Amount); err != nil { return err } // Save new events newEvents := account.GetUncommittedEvents() if err := h.eventStore.SaveEvents(ctx, cmd.AccountID, newEvents, account.Version()); err != nil { return fmt.Errorf("failed to save events: %w", err) } account.MarkEventsAsCommitted() return nil } // WithdrawMoneyCommand type WithdrawMoneyCommand struct { AccountID string Amount float64 } type WithdrawMoneyHandler struct { eventStore eventstore.EventStore } func NewWithdrawMoneyHandler(store eventstore.EventStore) *WithdrawMoneyHandler { return &WithdrawMoneyHandler{eventStore: store} } func (h *WithdrawMoneyHandler) Handle(ctx context.Context, cmd *WithdrawMoneyCommand) error { events, err := h.eventStore.GetEvents(ctx, cmd.AccountID) if err != nil { return err } account, err := domain.LoadFromHistory(events) if err != nil { return err } if err := account.Withdraw(cmd.Amount); err != nil { return err } newEvents := account.GetUncommittedEvents() if err := h.eventStore.SaveEvents(ctx, cmd.AccountID, newEvents, account.Version()); err != nil { return fmt.Errorf("failed to save events: %w", err) } account.MarkEventsAsCommitted() return nil } Projections // internal/projections/account_projection.go package projections import ( "context" "database/sql" "app/internal/domain" ) // AccountReadModel represents the current state type AccountReadModel struct { ID string Owner string Currency string Balance float64 IsClosed bool TotalDeposits float64 TotalWithdrawals float64 TransactionCount int } type AccountProjection struct { db *sql.DB } func NewAccountProjection(db *sql.DB) *AccountProjection { return &AccountProjection{db: db} } // Project processes events and updates read model func (p *AccountProjection) Project(ctx context.Context, event domain.Event) error { switch e := event.(type) { case *domain.AccountCreatedEvent: return p.handleAccountCreated(ctx, e) case *domain.MoneyDepositedEvent: return p.handleMoneyDeposited(ctx, e) case *domain.MoneyWithdrawnEvent: return p.handleMoneyWithdrawn(ctx, e) case *domain.AccountClosedEvent: return p.handleAccountClosed(ctx, e) } return nil } func (p *AccountProjection) handleAccountCreated(ctx context.Context, event *domain.AccountCreatedEvent) error { query := ` INSERT INTO account_read_model (id, owner, currency, balance, is_closed, total_deposits, total_withdrawals, transaction_count) VALUES ($1, $2, $3, $4, $5, $6, $7, $8) ` _, err := p.db.ExecContext(ctx, query, event.AccountID, event.Owner, event.Currency, 0, false, 0, 0, 0) return err } func (p *AccountProjection) handleMoneyDeposited(ctx context.Context, event *domain.MoneyDepositedEvent) error { query := ` UPDATE account_read_model SET balance = $2, total_deposits = total_deposits + $3, transaction_count = transaction_count + 1 WHERE id = $1 ` _, err := p.db.ExecContext(ctx, query, event.AccountID, event.Balance, event.Amount) return err } func (p *AccountProjection) handleMoneyWithdrawn(ctx context.Context, event *domain.MoneyWithdrawnEvent) error { query := ` UPDATE account_read_model SET balance = $2, total_withdrawals = total_withdrawals + $3, transaction_count = transaction_count + 1 WHERE id = $1 ` _, err := p.db.ExecContext(ctx, query, event.AccountID, event.Balance, event.Amount) return err } func (p *AccountProjection) handleAccountClosed(ctx context.Context, event *domain.AccountClosedEvent) error { query := `UPDATE account_read_model SET is_closed = true WHERE id = $1` _, err := p.db.ExecContext(ctx, query, event.AccountID) return err } // RebuildProjections rebuilds all projections from events func (p *AccountProjection) RebuildProjections(ctx context.Context, eventStore eventstore.EventStore) error { // Clear existing projections if _, err := p.db.ExecContext(ctx, "TRUNCATE account_read_model"); err != nil { return err } // Replay all events events, err := eventStore.GetAllEvents(ctx) if err != nil { return err } for _, event := range events { if err := p.Project(ctx, event); err != nil { return err } } return nil } Main Application // cmd/api/main.go package main import ( "context" "database/sql" "encoding/json" "log" "net/http" "github.com/gorilla/mux" _ "github.com/lib/pq" "app/internal/commands" "app/internal/eventstore" "app/internal/projections" ) func main() { db, err := sql.Open("postgres", "postgres://user:pass@localhost/eventsourcing?sslmode=disable") if err != nil { log.Fatal(err) } defer db.Close() // Initialize stores eventStore := eventstore.NewPostgresEventStore(db) projection := projections.NewAccountProjection(db) // Initialize command handlers createAccountHandler := commands.NewCreateAccountHandler(eventStore) depositHandler := commands.NewDepositMoneyHandler(eventStore) withdrawHandler := commands.NewWithdrawMoneyHandler(eventStore) router := mux.NewRouter() // Command endpoints router.HandleFunc("/accounts", func(w http.ResponseWriter, r *http.Request) { var cmd commands.CreateAccountCommand if err := json.NewDecoder(r.Body).Decode(&cmd); err != nil { http.Error(w, err.Error(), http.StatusBadRequest) return } if err := createAccountHandler.Handle(r.Context(), &cmd); err != nil { http.Error(w, err.Error(), http.StatusBadRequest) return } w.WriteHeader(http.StatusCreated) }).Methods("POST") router.HandleFunc("/accounts/{id}/deposit", func(w http.ResponseWriter, r *http.Request) { vars := mux.Vars(r) var req struct { Amount float64 `json:"amount"` } if err := json.NewDecoder(r.Body).Decode(&req); err != nil { http.Error(w, err.Error(), http.StatusBadRequest) return } cmd := commands.DepositMoneyCommand{ AccountID: vars["id"], Amount: req.Amount, } if err := depositHandler.Handle(r.Context(), &cmd); err != nil { http.Error(w, err.Error(), http.StatusBadRequest) return } w.WriteHeader(http.StatusOK) }).Methods("POST") // Event replay endpoint router.HandleFunc("/admin/rebuild-projections", func(w http.ResponseWriter, r *http.Request) { if err := projection.RebuildProjections(r.Context(), eventStore); err != nil { http.Error(w, err.Error(), http.StatusInternalServerError) return } w.WriteHeader(http.StatusOK) w.Write([]byte("Projections rebuilt successfully")) }).Methods("POST") log.Println("Server starting on :8080") log.Fatal(http.ListenAndServe(":8080", router)) } Best Practices Immutable Events: Never modify or delete events Event Versioning: Version events for schema evolution Snapshots: Use snapshots to optimize replay performance Idempotency: Ensure event handlers are idempotent Event Schema: Keep events small and focused Testing: Test event replay and projections Monitoring: Track event store size and replay performance Documentation: Document all event types and their purpose Common Pitfalls Mutable Events: Modifying or deleting events Missing Events: Not capturing all state changes Large Events: Storing too much data in events No Snapshots: Performance degradation from replaying all events Tight Coupling: Events containing implementation details No Versioning: Breaking changes to event schemas Synchronous Projections: Slowing down command processing When to Use Event Sourcing Use When: ...

The RPC Evolution: From gRPC to ConnectRPC While gRPC has been the go-to choice for high-performance RPC communication, ConnectRPC is emerging as a compelling alternative that addresses many of gRPC’s pain points while maintaining compatibility with existing ecosystems. What is ConnectRPC? ConnectRPC (formerly known as Connect) is a protocol that provides a simpler, more flexible approach to RPC. It’s designed to be: HTTP/1.1 and HTTP/2 compatible - Works with existing infrastructure Browser-friendly - No need for special proxies gRPC compatible - Can interoperate with gRPC services Developer-friendly - Simpler tooling and better debugging Why Teams Choose ConnectRPC Over gRPC 1. Browser Compatibility Without Proxies gRPC requires special proxies (like grpc-web) to work with browsers because it relies on HTTP/2-specific features. ConnectRPC works directly in browsers using standard fetch APIs. ...

Go Architecture Patterns Series: Series Overview | Next: Clean Architecture → What is Layered Architecture? Layered Architecture is one of the most common and fundamental architectural patterns in software development. It organizes code into horizontal layers, where each layer has a specific responsibility and only communicates with adjacent layers. Key Principles: Separation of Concerns: Each layer handles a specific aspect of the application Layer Independence: Layers are loosely coupled and can be changed independently Unidirectional Dependencies: Dependencies flow in one direction (top-down) Clear Boundaries: Well-defined interfaces between layers Testability: Each layer can be tested in isolation Architecture Overview %%{init: {'theme':'dark', 'themeVariables': {'primaryTextColor':'#e5e7eb','secondaryTextColor':'#e5e7eb','tertiaryTextColor':'#e5e7eb','textColor':'#e5e7eb','nodeTextColor':'#e5e7eb','edgeLabelText':'#e5e7eb','clusterTextColor':'#e5e7eb','actorTextColor':'#e5e7eb'}}}%% graph TD A[Presentation Layer] --> B[Business Logic Layer] B --> C[Data Access Layer] C --> D[(Database)] style A fill:#1e3a5f,color:#fff style B fill:#78350f,color:#fff style C fill:#134e4a,color:#fff style D fill:#4c1d95,color:#fff Traditional 3-Tier Layered Architecture %%{init: {'theme':'dark', 'themeVariables': {'primaryTextColor':'#e5e7eb','secondaryTextColor':'#e5e7eb','tertiaryTextColor':'#e5e7eb','textColor':'#e5e7eb','nodeTextColor':'#e5e7eb','edgeLabelText':'#e5e7eb','clusterTextColor':'#e5e7eb','actorTextColor':'#e5e7eb'}}}%% graph LR subgraph "Presentation Layer" A1[HTTP Handlers] A2[Templates/Views] A3[Request/Response DTOs] end subgraph "Business Logic Layer" B1[Services] B2[Business Rules] B3[Domain Models] end subgraph "Data Access Layer" C1[Repositories] C2[Database Access] C3[Data Models] end A1 --> B1 A2 --> B1 B1 --> C1 C1 --> C2 style A1 fill:#1e3a5f,color:#fff style B1 fill:#78350f,color:#fff style C1 fill:#134e4a,color:#fff Real-World Use Cases Web Applications: RESTful APIs and web services Enterprise Applications: Business management systems CRUD Applications: Standard create-read-update-delete operations Monolithic Applications: Traditional single-deployment applications Internal Tools: Admin panels and dashboards Legacy System Modernization: Refactoring existing codebases Basic Layered Architecture Implementation Project Structure ├── cmd/ │ └── api/ │ └── main.go ├── internal/ │ ├── handlers/ # Presentation Layer │ │ ├── user_handler.go │ │ └── product_handler.go │ ├── services/ # Business Logic Layer │ │ ├── user_service.go │ │ └── product_service.go │ ├── repositories/ # Data Access Layer │ │ ├── user_repository.go │ │ └── product_repository.go │ └── models/ # Domain Models │ ├── user.go │ └── product.go └── go.mod Layer 1: Domain Models package models import "time" // User represents a user in the system type User struct { ID int64 `json:"id"` Email string `json:"email"` Name string `json:"name"` CreatedAt time.Time `json:"created_at"` UpdatedAt time.Time `json:"updated_at"` } // Product represents a product in the system type Product struct { ID int64 `json:"id"` Name string `json:"name"` Description string `json:"description"` Price float64 `json:"price"` Stock int `json:"stock"` CreatedAt time.Time `json:"created_at"` UpdatedAt time.Time `json:"updated_at"` } // Order represents an order in the system type Order struct { ID int64 `json:"id"` UserID int64 `json:"user_id"` ProductID int64 `json:"product_id"` Quantity int `json:"quantity"` Total float64 `json:"total"` Status string `json:"status"` CreatedAt time.Time `json:"created_at"` } Layer 2: Data Access Layer (Repository) package repositories import ( "context" "database/sql" "fmt" "time" "myapp/internal/models" ) // UserRepository handles user data access type UserRepository struct { db *sql.DB } // NewUserRepository creates a new user repository func NewUserRepository(db *sql.DB) *UserRepository { return &UserRepository{db: db} } // Create creates a new user func (r *UserRepository) Create(ctx context.Context, user *models.User) error { query := ` INSERT INTO users (email, name, created_at, updated_at) VALUES ($1, $2, $3, $4) RETURNING id ` now := time.Now() user.CreatedAt = now user.UpdatedAt = now err := r.db.QueryRowContext( ctx, query, user.Email, user.Name, user.CreatedAt, user.UpdatedAt, ).Scan(&user.ID) if err != nil { return fmt.Errorf("failed to create user: %w", err) } return nil } // GetByID retrieves a user by ID func (r *UserRepository) GetByID(ctx context.Context, id int64) (*models.User, error) { query := ` SELECT id, email, name, created_at, updated_at FROM users WHERE id = $1 ` user := &models.User{} err := r.db.QueryRowContext(ctx, query, id).Scan( &user.ID, &user.Email, &user.Name, &user.CreatedAt, &user.UpdatedAt, ) if err == sql.ErrNoRows { return nil, fmt.Errorf("user not found") } if err != nil { return nil, fmt.Errorf("failed to get user: %w", err) } return user, nil } // GetByEmail retrieves a user by email func (r *UserRepository) GetByEmail(ctx context.Context, email string) (*models.User, error) { query := ` SELECT id, email, name, created_at, updated_at FROM users WHERE email = $1 ` user := &models.User{} err := r.db.QueryRowContext(ctx, query, email).Scan( &user.ID, &user.Email, &user.Name, &user.CreatedAt, &user.UpdatedAt, ) if err == sql.ErrNoRows { return nil, fmt.Errorf("user not found") } if err != nil { return nil, fmt.Errorf("failed to get user: %w", err) } return user, nil } // Update updates a user func (r *UserRepository) Update(ctx context.Context, user *models.User) error { query := ` UPDATE users SET email = $1, name = $2, updated_at = $3 WHERE id = $4 ` user.UpdatedAt = time.Now() result, err := r.db.ExecContext( ctx, query, user.Email, user.Name, user.UpdatedAt, user.ID, ) if err != nil { return fmt.Errorf("failed to update user: %w", err) } rows, err := result.RowsAffected() if err != nil { return fmt.Errorf("failed to get affected rows: %w", err) } if rows == 0 { return fmt.Errorf("user not found") } return nil } // Delete deletes a user func (r *UserRepository) Delete(ctx context.Context, id int64) error { query := `DELETE FROM users WHERE id = $1` result, err := r.db.ExecContext(ctx, query, id) if err != nil { return fmt.Errorf("failed to delete user: %w", err) } rows, err := result.RowsAffected() if err != nil { return fmt.Errorf("failed to get affected rows: %w", err) } if rows == 0 { return fmt.Errorf("user not found") } return nil } // List retrieves all users func (r *UserRepository) List(ctx context.Context, limit, offset int) ([]*models.User, error) { query := ` SELECT id, email, name, created_at, updated_at FROM users ORDER BY id DESC LIMIT $1 OFFSET $2 ` rows, err := r.db.QueryContext(ctx, query, limit, offset) if err != nil { return nil, fmt.Errorf("failed to list users: %w", err) } defer rows.Close() var users []*models.User for rows.Next() { user := &models.User{} err := rows.Scan( &user.ID, &user.Email, &user.Name, &user.CreatedAt, &user.UpdatedAt, ) if err != nil { return nil, fmt.Errorf("failed to scan user: %w", err) } users = append(users, user) } return users, nil } // ProductRepository handles product data access type ProductRepository struct { db *sql.DB } // NewProductRepository creates a new product repository func NewProductRepository(db *sql.DB) *ProductRepository { return &ProductRepository{db: db} } // UpdateStock updates product stock func (r *ProductRepository) UpdateStock(ctx context.Context, productID int64, quantity int) error { query := ` UPDATE products SET stock = stock + $1, updated_at = $2 WHERE id = $3 ` result, err := r.db.ExecContext(ctx, query, quantity, time.Now(), productID) if err != nil { return fmt.Errorf("failed to update stock: %w", err) } rows, err := result.RowsAffected() if err != nil { return fmt.Errorf("failed to get affected rows: %w", err) } if rows == 0 { return fmt.Errorf("product not found") } return nil } Layer 3: Business Logic Layer (Service) package services import ( "context" "fmt" "regexp" "myapp/internal/models" "myapp/internal/repositories" ) // UserService handles user business logic type UserService struct { userRepo *repositories.UserRepository } // NewUserService creates a new user service func NewUserService(userRepo *repositories.UserRepository) *UserService { return &UserService{ userRepo: userRepo, } } // CreateUser creates a new user with validation func (s *UserService) CreateUser(ctx context.Context, email, name string) (*models.User, error) { // Business rule: Validate email format if !isValidEmail(email) { return nil, fmt.Errorf("invalid email format") } // Business rule: Name must not be empty if name == "" { return nil, fmt.Errorf("name cannot be empty") } // Business rule: Email must be unique existingUser, err := s.userRepo.GetByEmail(ctx, email) if err == nil && existingUser != nil { return nil, fmt.Errorf("email already exists") } user := &models.User{ Email: email, Name: name, } if err := s.userRepo.Create(ctx, user); err != nil { return nil, fmt.Errorf("failed to create user: %w", err) } return user, nil } // GetUser retrieves a user by ID func (s *UserService) GetUser(ctx context.Context, id int64) (*models.User, error) { return s.userRepo.GetByID(ctx, id) } // UpdateUser updates a user with validation func (s *UserService) UpdateUser(ctx context.Context, id int64, email, name string) (*models.User, error) { // Business rule: Validate email format if !isValidEmail(email) { return nil, fmt.Errorf("invalid email format") } // Business rule: Name must not be empty if name == "" { return nil, fmt.Errorf("name cannot be empty") } user, err := s.userRepo.GetByID(ctx, id) if err != nil { return nil, fmt.Errorf("user not found: %w", err) } // Business rule: If email is changing, check uniqueness if user.Email != email { existingUser, err := s.userRepo.GetByEmail(ctx, email) if err == nil && existingUser != nil { return nil, fmt.Errorf("email already exists") } } user.Email = email user.Name = name if err := s.userRepo.Update(ctx, user); err != nil { return nil, fmt.Errorf("failed to update user: %w", err) } return user, nil } // DeleteUser deletes a user func (s *UserService) DeleteUser(ctx context.Context, id int64) error { return s.userRepo.Delete(ctx, id) } // ListUsers retrieves users with pagination func (s *UserService) ListUsers(ctx context.Context, page, pageSize int) ([]*models.User, error) { // Business rule: Validate pagination parameters if page < 1 { page = 1 } if pageSize < 1 || pageSize > 100 { pageSize = 20 } offset := (page - 1) * pageSize return s.userRepo.List(ctx, pageSize, offset) } // isValidEmail validates email format func isValidEmail(email string) bool { emailRegex := regexp.MustCompile(`^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$`) return emailRegex.MatchString(email) } // OrderService handles order business logic type OrderService struct { userRepo *repositories.UserRepository productRepo *repositories.ProductRepository } // NewOrderService creates a new order service func NewOrderService( userRepo *repositories.UserRepository, productRepo *repositories.ProductRepository, ) *OrderService { return &OrderService{ userRepo: userRepo, productRepo: productRepo, } } // PlaceOrder places a new order with business validation func (s *OrderService) PlaceOrder(ctx context.Context, userID, productID int64, quantity int) (*models.Order, error) { // Business rule: Validate user exists user, err := s.userRepo.GetByID(ctx, userID) if err != nil { return nil, fmt.Errorf("user not found: %w", err) } // Business rule: Validate quantity if quantity <= 0 { return nil, fmt.Errorf("quantity must be positive") } // Business rule: Check stock availability // (In a real implementation, this would be in ProductRepository) // Business rule: Calculate total // (In a real implementation, this would fetch product price) order := &models.Order{ UserID: user.ID, ProductID: productID, Quantity: quantity, Status: "pending", } return order, nil } Layer 4: Presentation Layer (HTTP Handlers) package handlers import ( "encoding/json" "net/http" "strconv" "myapp/internal/services" ) // UserHandler handles HTTP requests for users type UserHandler struct { userService *services.UserService } // NewUserHandler creates a new user handler func NewUserHandler(userService *services.UserService) *UserHandler { return &UserHandler{ userService: userService, } } // CreateUserRequest represents the request to create a user type CreateUserRequest struct { Email string `json:"email"` Name string `json:"name"` } // CreateUser handles POST /users func (h *UserHandler) CreateUser(w http.ResponseWriter, r *http.Request) { var req CreateUserRequest if err := json.NewDecoder(r.Body).Decode(&req); err != nil { respondError(w, http.StatusBadRequest, "invalid request body") return } user, err := h.userService.CreateUser(r.Context(), req.Email, req.Name) if err != nil { respondError(w, http.StatusBadRequest, err.Error()) return } respondJSON(w, http.StatusCreated, user) } // GetUser handles GET /users/:id func (h *UserHandler) GetUser(w http.ResponseWriter, r *http.Request) { // Extract ID from URL (implementation depends on router) id, err := extractIDFromPath(r) if err != nil { respondError(w, http.StatusBadRequest, "invalid user ID") return } user, err := h.userService.GetUser(r.Context(), id) if err != nil { respondError(w, http.StatusNotFound, "user not found") return } respondJSON(w, http.StatusOK, user) } // UpdateUser handles PUT /users/:id func (h *UserHandler) UpdateUser(w http.ResponseWriter, r *http.Request) { id, err := extractIDFromPath(r) if err != nil { respondError(w, http.StatusBadRequest, "invalid user ID") return } var req CreateUserRequest if err := json.NewDecoder(r.Body).Decode(&req); err != nil { respondError(w, http.StatusBadRequest, "invalid request body") return } user, err := h.userService.UpdateUser(r.Context(), id, req.Email, req.Name) if err != nil { respondError(w, http.StatusBadRequest, err.Error()) return } respondJSON(w, http.StatusOK, user) } // DeleteUser handles DELETE /users/:id func (h *UserHandler) DeleteUser(w http.ResponseWriter, r *http.Request) { id, err := extractIDFromPath(r) if err != nil { respondError(w, http.StatusBadRequest, "invalid user ID") return } if err := h.userService.DeleteUser(r.Context(), id); err != nil { respondError(w, http.StatusNotFound, err.Error()) return } w.WriteHeader(http.StatusNoContent) } // ListUsers handles GET /users func (h *UserHandler) ListUsers(w http.ResponseWriter, r *http.Request) { page, _ := strconv.Atoi(r.URL.Query().Get("page")) pageSize, _ := strconv.Atoi(r.URL.Query().Get("page_size")) users, err := h.userService.ListUsers(r.Context(), page, pageSize) if err != nil { respondError(w, http.StatusInternalServerError, "failed to list users") return } respondJSON(w, http.StatusOK, users) } // Helper functions func respondJSON(w http.ResponseWriter, status int, data interface{}) { w.Header().Set("Content-Type", "application/json") w.WriteHeader(status) json.NewEncoder(w).Encode(data) } func respondError(w http.ResponseWriter, status int, message string) { respondJSON(w, status, map[string]string{"error": message}) } func extractIDFromPath(r *http.Request) (int64, error) { // This is a simplified version - use your router's method // For example with chi: chi.URLParam(r, "id") return 1, nil } Main Application package main import ( "database/sql" "log" "net/http" _ "github.com/lib/pq" "myapp/internal/handlers" "myapp/internal/repositories" "myapp/internal/services" ) func main() { // Initialize database connection db, err := sql.Open("postgres", "postgres://user:pass@localhost/dbname?sslmode=disable") if err != nil { log.Fatal(err) } defer db.Close() // Initialize layers from bottom to top // Data Access Layer userRepo := repositories.NewUserRepository(db) productRepo := repositories.NewProductRepository(db) // Business Logic Layer userService := services.NewUserService(userRepo) orderService := services.NewOrderService(userRepo, productRepo) // Presentation Layer userHandler := handlers.NewUserHandler(userService) // Setup routes mux := http.NewServeMux() mux.HandleFunc("POST /users", userHandler.CreateUser) mux.HandleFunc("GET /users/{id}", userHandler.GetUser) mux.HandleFunc("PUT /users/{id}", userHandler.UpdateUser) mux.HandleFunc("DELETE /users/{id}", userHandler.DeleteUser) mux.HandleFunc("GET /users", userHandler.ListUsers) // Start server log.Println("Server starting on :8080") if err := http.ListenAndServe(":8080", mux); err != nil { log.Fatal(err) } } Testing Layered Architecture package services_test import ( "context" "testing" "myapp/internal/models" "myapp/internal/services" ) // MockUserRepository for testing type MockUserRepository struct { users map[int64]*models.User nextID int64 } func NewMockUserRepository() *MockUserRepository { return &MockUserRepository{ users: make(map[int64]*models.User), nextID: 1, } } func (m *MockUserRepository) Create(ctx context.Context, user *models.User) error { user.ID = m.nextID m.nextID++ m.users[user.ID] = user return nil } func (m *MockUserRepository) GetByID(ctx context.Context, id int64) (*models.User, error) { user, exists := m.users[id] if !exists { return nil, fmt.Errorf("user not found") } return user, nil } func (m *MockUserRepository) GetByEmail(ctx context.Context, email string) (*models.User, error) { for _, user := range m.users { if user.Email == email { return user, nil } } return nil, fmt.Errorf("user not found") } func TestUserService_CreateUser(t *testing.T) { mockRepo := NewMockUserRepository() service := services.NewUserService(mockRepo) tests := []struct { name string email string userName string wantErr bool }{ { name: "valid user", email: "[email protected]", userName: "Test User", wantErr: false, }, { name: "invalid email", email: "invalid-email", userName: "Test User", wantErr: true, }, { name: "empty name", email: "[email protected]", userName: "", wantErr: true, }, } for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { user, err := service.CreateUser(context.Background(), tt.email, tt.userName) if tt.wantErr { if err == nil { t.Errorf("expected error but got none") } return } if err != nil { t.Errorf("unexpected error: %v", err) return } if user.Email != tt.email { t.Errorf("expected email %s, got %s", tt.email, user.Email) } }) } } Best Practices Clear Layer Boundaries: Define clear interfaces between layers Dependency Direction: Always depend downward, never upward Keep Layers Thin: Avoid bloated service layers with too much logic Use Dependency Injection: Inject dependencies rather than creating them Interface Segregation: Define minimal interfaces for layer communication Error Handling: Handle errors appropriately at each layer Testing: Test each layer independently with mocks Common Pitfalls Layer Violation: Skipping layers (e.g., handler directly accessing repository) Anemic Domain Models: Models with no behavior, only data Fat Service Layer: Putting all logic in the service layer Tight Coupling: Layers knowing too much about each other God Objects: Services or repositories handling too many responsibilities Inconsistent Abstractions: Different patterns in different layers When to Use Layered Architecture Use When: ...

Go Architecture Patterns Series: ← Layered Architecture | Series Overview | Next: Hexagonal Architecture → What is Clean Architecture? Clean Architecture, popularized by Robert C. Martin (Uncle Bob), is an architectural pattern that emphasizes separation of concerns and independence from frameworks, databases, and external agencies. It organizes code in concentric circles with dependencies pointing inward. Key Principles: Independence of Frameworks: Architecture doesn’t depend on frameworks Testability: Business rules can be tested without UI, database, or external elements Independence of UI: UI can change without changing business rules Independence of Database: Business rules don’t know about the database Independence of External Agency: Business rules don’t know about the outside world Dependency Rule: Dependencies only point inward toward higher-level policies Architecture Overview %%{init: {'theme':'dark', 'themeVariables': {'primaryTextColor':'#e5e7eb','secondaryTextColor':'#e5e7eb','tertiaryTextColor':'#e5e7eb','textColor':'#e5e7eb','nodeTextColor':'#e5e7eb','edgeLabelText':'#e5e7eb','clusterTextColor':'#e5e7eb','actorTextColor':'#e5e7eb'}}}%% graph TD subgraph "External Layer (Frameworks & Drivers)" A1[Web Framework] A2[Database] A3[External APIs] end subgraph "Interface Adapters" B1[Controllers] B2[Presenters] B3[Gateways] end subgraph "Use Cases (Application Business Rules)" C1[Interactors] C2[Use Case Logic] end subgraph "Entities (Enterprise Business Rules)" D1[Domain Models] D2[Business Logic] end A1 --> B1 A2 --> B3 B1 --> C1 B3 --> C1 C1 --> D1 style D1 fill:#78350f,color:#fff style C1 fill:#1e3a5f,color:#fff style B1 fill:#134e4a,color:#fff style A1 fill:#4c1d95,color:#fff Clean Architecture Circles %%{init: {'theme':'dark', 'themeVariables': {'primaryTextColor':'#e5e7eb','secondaryTextColor':'#e5e7eb','tertiaryTextColor':'#e5e7eb','textColor':'#e5e7eb','nodeTextColor':'#e5e7eb','edgeLabelText':'#e5e7eb','clusterTextColor':'#e5e7eb','actorTextColor':'#e5e7eb'}}}%% graph LR subgraph "Layer 1: Entities" E[Core BusinessRules & Models] end subgraph "Layer 2: Use Cases" U[ApplicationBusiness Rules] end subgraph "Layer 3: Interface Adapters" I[ControllersPresentersGateways] end subgraph "Layer 4: Frameworks & Drivers" F[WebDBUI] end F -.->|depends on| I I -.->|depends on| U U -.->|depends on| E style E fill:#78350f,color:#fff style U fill:#1e3a5f,color:#fff style I fill:#134e4a,color:#fff style F fill:#4c1d95,color:#fff Dependency Flow %%{init: {'theme':'dark', 'themeVariables': {'primaryTextColor':'#e5e7eb','secondaryTextColor':'#e5e7eb','tertiaryTextColor':'#e5e7eb','textColor':'#e5e7eb','nodeTextColor':'#e5e7eb','edgeLabelText':'#e5e7eb','clusterTextColor':'#e5e7eb','actorTextColor':'#e5e7eb'}}}%% graph TB subgraph "Outer Layers (Low-level Details)" direction LR DB[Database] Web[Web Server] API[External APIs] end subgraph "Interface Adapters" direction LR Repo[RepositoryImplementation] Controller[HTTP Controller] Gateway[API Gateway] end subgraph "Use Cases" direction LR UC1[Create UserUse Case] UC2[Get UserUse Case] end subgraph "Entities (Core)" direction LR Entity[User Entity] Rules[Business Rules] end DB --> Repo Web --> Controller API --> Gateway Repo -.->|implements| UC1 Controller -.->|calls| UC1 Gateway -.->|implements| UC2 UC1 -.->|uses| Entity UC2 -.->|uses| Entity style Entity fill:#78350f,stroke:#fb923c,stroke-width:3px,color:#fff style UC1 fill:#1e3a5f,color:#fff style Repo fill:#134e4a,color:#fff style DB fill:#4c1d95,color:#fff Real-World Use Cases Enterprise Applications: Complex business logic that needs isolation Long-lived Systems: Applications that need to evolve over time Multi-platform Applications: Same core logic, different interfaces Testing-Critical Systems: Financial, healthcare, or mission-critical apps API-first Applications: Where business logic is reused across interfaces Microservices: Each service following clean architecture principles Project Structure ├── cmd/ │ └── api/ │ └── main.go ├── internal/ │ ├── entity/ # Layer 1: Entities │ │ ├── user.go │ │ └── errors.go │ ├── usecase/ # Layer 2: Use Cases │ │ ├── user_usecase.go │ │ ├── interfaces.go # Repository & Presenter interfaces │ │ └── user_interactor.go │ ├── adapter/ # Layer 3: Interface Adapters │ │ ├── repository/ │ │ │ └── user_repository.go │ │ ├── presenter/ │ │ │ └── user_presenter.go │ │ └── controller/ │ │ └── user_controller.go │ └── infrastructure/ # Layer 4: Frameworks & Drivers │ ├── database/ │ │ └── postgres.go │ ├── router/ │ │ └── router.go │ └── config/ │ └── config.go └── go.mod Layer 1: Entities (Core Business Rules) package entity import ( "errors" "regexp" "time" ) // User represents the core user entity with business rules type User struct { ID string Email string Name string Age int Status UserStatus CreatedAt time.Time UpdatedAt time.Time } // UserStatus represents user account status type UserStatus string const ( UserStatusActive UserStatus = "active" UserStatusInactive UserStatus = "inactive" UserStatusSuspended UserStatus = "suspended" ) // Business rule validation errors var ( ErrInvalidEmail = errors.New("invalid email format") ErrInvalidName = errors.New("name must not be empty") ErrInvalidAge = errors.New("age must be between 0 and 150") ErrUserNotFound = errors.New("user not found") ErrUserAlreadyExists = errors.New("user already exists") ErrUnauthorized = errors.New("unauthorized action") ) // NewUser creates a new user with business rule validation func NewUser(email, name string, age int) (*User, error) { user := &User{ Email: email, Name: name, Age: age, Status: UserStatusActive, CreatedAt: time.Now(), UpdatedAt: time.Now(), } if err := user.Validate(); err != nil { return nil, err } return user, nil } // Validate validates the user entity according to business rules func (u *User) Validate() error { // Business Rule: Email must be valid format if !isValidEmail(u.Email) { return ErrInvalidEmail } // Business Rule: Name must not be empty if u.Name == "" { return ErrInvalidName } // Business Rule: Age must be realistic if u.Age < 0 || u.Age > 150 { return ErrInvalidAge } return nil } // UpdateEmail updates user email with validation func (u *User) UpdateEmail(email string) error { if !isValidEmail(email) { return ErrInvalidEmail } u.Email = email u.UpdatedAt = time.Now() return nil } // UpdateName updates user name with validation func (u *User) UpdateName(name string) error { if name == "" { return ErrInvalidName } u.Name = name u.UpdatedAt = time.Now() return nil } // Activate activates the user account func (u *User) Activate() { u.Status = UserStatusActive u.UpdatedAt = time.Now() } // Deactivate deactivates the user account func (u *User) Deactivate() { u.Status = UserStatusInactive u.UpdatedAt = time.Now() } // Suspend suspends the user account func (u *User) Suspend() { u.Status = UserStatusSuspended u.UpdatedAt = time.Now() } // IsActive checks if user is active func (u *User) IsActive() bool { return u.Status == UserStatusActive } // CanPerformAction checks if user can perform actions (business rule) func (u *User) CanPerformAction() bool { return u.Status == UserStatusActive } // isValidEmail validates email format func isValidEmail(email string) bool { emailRegex := regexp.MustCompile(`^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$`) return emailRegex.MatchString(email) } // Product entity with business rules type Product struct { ID string Name string Description string Price Money Stock int CreatedAt time.Time UpdatedAt time.Time } // Money represents monetary value (value object) type Money struct { Amount int64 // in cents Currency string } // NewMoney creates a new money value object func NewMoney(amount int64, currency string) Money { return Money{ Amount: amount, Currency: currency, } } // Add adds two money values func (m Money) Add(other Money) (Money, error) { if m.Currency != other.Currency { return Money{}, errors.New("currency mismatch") } return Money{ Amount: m.Amount + other.Amount, Currency: m.Currency, }, nil } // Multiply multiplies money by a factor func (m Money) Multiply(factor int) Money { return Money{ Amount: m.Amount * int64(factor), Currency: m.Currency, } } Layer 2: Use Cases (Application Business Rules) Use Case Interfaces package usecase import ( "context" "myapp/internal/entity" ) // UserRepository defines the interface for user data access // This interface is defined in the use case layer but implemented in outer layers type UserRepository interface { Create(ctx context.Context, user *entity.User) error GetByID(ctx context.Context, id string) (*entity.User, error) GetByEmail(ctx context.Context, email string) (*entity.User, error) Update(ctx context.Context, user *entity.User) error Delete(ctx context.Context, id string) error List(ctx context.Context, offset, limit int) ([]*entity.User, error) } // UserPresenter defines the interface for presenting user data type UserPresenter interface { PresentUser(user *entity.User) interface{} PresentUsers(users []*entity.User) interface{} PresentError(err error) interface{} } // EmailService defines the interface for email operations type EmailService interface { SendWelcomeEmail(ctx context.Context, user *entity.User) error SendPasswordResetEmail(ctx context.Context, user *entity.User, token string) error } // IDGenerator defines the interface for generating IDs type IDGenerator interface { Generate() string } Use Case Implementation package usecase import ( "context" "fmt" "myapp/internal/entity" ) // CreateUserInput represents input for creating a user type CreateUserInput struct { Email string Name string Age int } // UpdateUserInput represents input for updating a user type UpdateUserInput struct { ID string Email string Name string Age int } // UserInteractor implements user use cases type UserInteractor struct { repo UserRepository emailService EmailService idGen IDGenerator } // NewUserInteractor creates a new user interactor func NewUserInteractor( repo UserRepository, emailService EmailService, idGen IDGenerator, ) *UserInteractor { return &UserInteractor{ repo: repo, emailService: emailService, idGen: idGen, } } // CreateUser creates a new user (use case) func (i *UserInteractor) CreateUser(ctx context.Context, input CreateUserInput) (*entity.User, error) { // Use case logic: Check if user already exists existingUser, err := i.repo.GetByEmail(ctx, input.Email) if err == nil && existingUser != nil { return nil, entity.ErrUserAlreadyExists } // Use case logic: Create new user entity user, err := entity.NewUser(input.Email, input.Name, input.Age) if err != nil { return nil, fmt.Errorf("failed to create user entity: %w", err) } // Use case logic: Generate ID user.ID = i.idGen.Generate() // Use case logic: Save user if err := i.repo.Create(ctx, user); err != nil { return nil, fmt.Errorf("failed to save user: %w", err) } // Use case logic: Send welcome email (async in real system) if err := i.emailService.SendWelcomeEmail(ctx, user); err != nil { // Log error but don't fail the use case fmt.Printf("failed to send welcome email: %v\n", err) } return user, nil } // GetUser retrieves a user by ID (use case) func (i *UserInteractor) GetUser(ctx context.Context, id string) (*entity.User, error) { user, err := i.repo.GetByID(ctx, id) if err != nil { return nil, fmt.Errorf("failed to get user: %w", err) } if user == nil { return nil, entity.ErrUserNotFound } return user, nil } // UpdateUser updates a user (use case) func (i *UserInteractor) UpdateUser(ctx context.Context, input UpdateUserInput) (*entity.User, error) { // Use case logic: Get existing user user, err := i.repo.GetByID(ctx, input.ID) if err != nil { return nil, entity.ErrUserNotFound } // Use case logic: Check if email is changing and if it already exists if user.Email != input.Email { existingUser, err := i.repo.GetByEmail(ctx, input.Email) if err == nil && existingUser != nil { return nil, entity.ErrUserAlreadyExists } } // Use case logic: Update user fields if err := user.UpdateEmail(input.Email); err != nil { return nil, err } if err := user.UpdateName(input.Name); err != nil { return nil, err } user.Age = input.Age if err := user.Validate(); err != nil { return nil, err } // Use case logic: Save updated user if err := i.repo.Update(ctx, user); err != nil { return nil, fmt.Errorf("failed to update user: %w", err) } return user, nil } // DeleteUser deletes a user (use case) func (i *UserInteractor) DeleteUser(ctx context.Context, id string) error { // Use case logic: Verify user exists user, err := i.repo.GetByID(ctx, id) if err != nil { return entity.ErrUserNotFound } // Use case logic: Check if user can be deleted (business rule) if !user.CanPerformAction() { return entity.ErrUnauthorized } // Use case logic: Delete user if err := i.repo.Delete(ctx, id); err != nil { return fmt.Errorf("failed to delete user: %w", err) } return nil } // ListUsers lists users with pagination (use case) func (i *UserInteractor) ListUsers(ctx context.Context, page, pageSize int) ([]*entity.User, error) { // Use case logic: Validate pagination if page < 1 { page = 1 } if pageSize < 1 || pageSize > 100 { pageSize = 20 } offset := (page - 1) * pageSize // Use case logic: Get users users, err := i.repo.List(ctx, offset, pageSize) if err != nil { return nil, fmt.Errorf("failed to list users: %w", err) } return users, nil } // ActivateUser activates a user account (use case) func (i *UserInteractor) ActivateUser(ctx context.Context, id string) error { user, err := i.repo.GetByID(ctx, id) if err != nil { return entity.ErrUserNotFound } user.Activate() if err := i.repo.Update(ctx, user); err != nil { return fmt.Errorf("failed to activate user: %w", err) } return nil } Layer 3: Interface Adapters Repository Implementation package repository import ( "context" "database/sql" "fmt" "time" "myapp/internal/entity" ) // PostgresUserRepository implements UserRepository interface type PostgresUserRepository struct { db *sql.DB } // NewPostgresUserRepository creates a new Postgres user repository func NewPostgresUserRepository(db *sql.DB) *PostgresUserRepository { return &PostgresUserRepository{db: db} } // Create creates a new user in the database func (r *PostgresUserRepository) Create(ctx context.Context, user *entity.User) error { query := ` INSERT INTO users (id, email, name, age, status, created_at, updated_at) VALUES ($1, $2, $3, $4, $5, $6, $7) ` _, err := r.db.ExecContext( ctx, query, user.ID, user.Email, user.Name, user.Age, user.Status, user.CreatedAt, user.UpdatedAt, ) if err != nil { return fmt.Errorf("failed to create user: %w", err) } return nil } // GetByID retrieves a user by ID func (r *PostgresUserRepository) GetByID(ctx context.Context, id string) (*entity.User, error) { query := ` SELECT id, email, name, age, status, created_at, updated_at FROM users WHERE id = $1 ` var user entity.User var status string err := r.db.QueryRowContext(ctx, query, id).Scan( &user.ID, &user.Email, &user.Name, &user.Age, &status, &user.CreatedAt, &user.UpdatedAt, ) if err == sql.ErrNoRows { return nil, entity.ErrUserNotFound } if err != nil { return nil, fmt.Errorf("failed to get user: %w", err) } user.Status = entity.UserStatus(status) return &user, nil } // GetByEmail retrieves a user by email func (r *PostgresUserRepository) GetByEmail(ctx context.Context, email string) (*entity.User, error) { query := ` SELECT id, email, name, age, status, created_at, updated_at FROM users WHERE email = $1 ` var user entity.User var status string err := r.db.QueryRowContext(ctx, query, email).Scan( &user.ID, &user.Email, &user.Name, &user.Age, &status, &user.CreatedAt, &user.UpdatedAt, ) if err == sql.ErrNoRows { return nil, nil } if err != nil { return nil, fmt.Errorf("failed to get user: %w", err) } user.Status = entity.UserStatus(status) return &user, nil } // Update updates a user func (r *PostgresUserRepository) Update(ctx context.Context, user *entity.User) error { query := ` UPDATE users SET email = $1, name = $2, age = $3, status = $4, updated_at = $5 WHERE id = $6 ` result, err := r.db.ExecContext( ctx, query, user.Email, user.Name, user.Age, user.Status, time.Now(), user.ID, ) if err != nil { return fmt.Errorf("failed to update user: %w", err) } rows, err := result.RowsAffected() if err != nil { return fmt.Errorf("failed to get affected rows: %w", err) } if rows == 0 { return entity.ErrUserNotFound } return nil } // Delete deletes a user func (r *PostgresUserRepository) Delete(ctx context.Context, id string) error { query := `DELETE FROM users WHERE id = $1` result, err := r.db.ExecContext(ctx, query, id) if err != nil { return fmt.Errorf("failed to delete user: %w", err) } rows, err := result.RowsAffected() if err != nil { return fmt.Errorf("failed to get affected rows: %w", err) } if rows == 0 { return entity.ErrUserNotFound } return nil } // List retrieves users with pagination func (r *PostgresUserRepository) List(ctx context.Context, offset, limit int) ([]*entity.User, error) { query := ` SELECT id, email, name, age, status, created_at, updated_at FROM users ORDER BY created_at DESC LIMIT $1 OFFSET $2 ` rows, err := r.db.QueryContext(ctx, query, limit, offset) if err != nil { return nil, fmt.Errorf("failed to list users: %w", err) } defer rows.Close() var users []*entity.User for rows.Next() { var user entity.User var status string err := rows.Scan( &user.ID, &user.Email, &user.Name, &user.Age, &status, &user.CreatedAt, &user.UpdatedAt, ) if err != nil { return nil, fmt.Errorf("failed to scan user: %w", err) } user.Status = entity.UserStatus(status) users = append(users, &user) } return users, nil } Controller Implementation package controller import ( "encoding/json" "net/http" "strconv" "myapp/internal/entity" "myapp/internal/usecase" ) // UserController handles HTTP requests for users type UserController struct { interactor *usecase.UserInteractor presenter usecase.UserPresenter } // NewUserController creates a new user controller func NewUserController( interactor *usecase.UserInteractor, presenter usecase.UserPresenter, ) *UserController { return &UserController{ interactor: interactor, presenter: presenter, } } // CreateUserRequest represents the HTTP request for creating a user type CreateUserRequest struct { Email string `json:"email"` Name string `json:"name"` Age int `json:"age"` } // CreateUser handles POST /users func (c *UserController) CreateUser(w http.ResponseWriter, r *http.Request) { var req CreateUserRequest if err := json.NewDecoder(r.Body).Decode(&req); err != nil { c.respondError(w, http.StatusBadRequest, err) return } input := usecase.CreateUserInput{ Email: req.Email, Name: req.Name, Age: req.Age, } user, err := c.interactor.CreateUser(r.Context(), input) if err != nil { switch err { case entity.ErrInvalidEmail, entity.ErrInvalidName, entity.ErrInvalidAge: c.respondError(w, http.StatusBadRequest, err) case entity.ErrUserAlreadyExists: c.respondError(w, http.StatusConflict, err) default: c.respondError(w, http.StatusInternalServerError, err) } return } c.respond(w, http.StatusCreated, c.presenter.PresentUser(user)) } // GetUser handles GET /users/:id func (c *UserController) GetUser(w http.ResponseWriter, r *http.Request) { id := r.PathValue("id") user, err := c.interactor.GetUser(r.Context(), id) if err != nil { if err == entity.ErrUserNotFound { c.respondError(w, http.StatusNotFound, err) } else { c.respondError(w, http.StatusInternalServerError, err) } return } c.respond(w, http.StatusOK, c.presenter.PresentUser(user)) } // UpdateUser handles PUT /users/:id func (c *UserController) UpdateUser(w http.ResponseWriter, r *http.Request) { id := r.PathValue("id") var req CreateUserRequest if err := json.NewDecoder(r.Body).Decode(&req); err != nil { c.respondError(w, http.StatusBadRequest, err) return } input := usecase.UpdateUserInput{ ID: id, Email: req.Email, Name: req.Name, Age: req.Age, } user, err := c.interactor.UpdateUser(r.Context(), input) if err != nil { switch err { case entity.ErrUserNotFound: c.respondError(w, http.StatusNotFound, err) case entity.ErrInvalidEmail, entity.ErrInvalidName, entity.ErrInvalidAge: c.respondError(w, http.StatusBadRequest, err) case entity.ErrUserAlreadyExists: c.respondError(w, http.StatusConflict, err) default: c.respondError(w, http.StatusInternalServerError, err) } return } c.respond(w, http.StatusOK, c.presenter.PresentUser(user)) } // DeleteUser handles DELETE /users/:id func (c *UserController) DeleteUser(w http.ResponseWriter, r *http.Request) { id := r.PathValue("id") if err := c.interactor.DeleteUser(r.Context(), id); err != nil { if err == entity.ErrUserNotFound { c.respondError(w, http.StatusNotFound, err) } else { c.respondError(w, http.StatusInternalServerError, err) } return } w.WriteHeader(http.StatusNoContent) } // ListUsers handles GET /users func (c *UserController) ListUsers(w http.ResponseWriter, r *http.Request) { page, _ := strconv.Atoi(r.URL.Query().Get("page")) pageSize, _ := strconv.Atoi(r.URL.Query().Get("page_size")) users, err := c.interactor.ListUsers(r.Context(), page, pageSize) if err != nil { c.respondError(w, http.StatusInternalServerError, err) return } c.respond(w, http.StatusOK, c.presenter.PresentUsers(users)) } // Helper methods func (c *UserController) respond(w http.ResponseWriter, status int, data interface{}) { w.Header().Set("Content-Type", "application/json") w.WriteHeader(status) json.NewEncoder(w).Encode(data) } func (c *UserController) respondError(w http.ResponseWriter, status int, err error) { w.Header().Set("Content-Type", "application/json") w.WriteHeader(status) json.NewEncoder(w).Encode(c.presenter.PresentError(err)) } Presenter Implementation package presenter import ( "myapp/internal/entity" ) // UserJSONPresenter presents users as JSON type UserJSONPresenter struct{} // NewUserJSONPresenter creates a new user JSON presenter func NewUserJSONPresenter() *UserJSONPresenter { return &UserJSONPresenter{} } // UserResponse represents the JSON response for a user type UserResponse struct { ID string `json:"id"` Email string `json:"email"` Name string `json:"name"` Age int `json:"age"` Status string `json:"status"` CreatedAt string `json:"created_at"` UpdatedAt string `json:"updated_at"` } // PresentUser presents a single user func (p *UserJSONPresenter) PresentUser(user *entity.User) interface{} { return UserResponse{ ID: user.ID, Email: user.Email, Name: user.Name, Age: user.Age, Status: string(user.Status), CreatedAt: user.CreatedAt.Format("2006-01-02T15:04:05Z"), UpdatedAt: user.UpdatedAt.Format("2006-01-02T15:04:05Z"), } } // PresentUsers presents multiple users func (p *UserJSONPresenter) PresentUsers(users []*entity.User) interface{} { responses := make([]UserResponse, len(users)) for i, user := range users { responses[i] = UserResponse{ ID: user.ID, Email: user.Email, Name: user.Name, Age: user.Age, Status: string(user.Status), CreatedAt: user.CreatedAt.Format("2006-01-02T15:04:05Z"), UpdatedAt: user.UpdatedAt.Format("2006-01-02T15:04:05Z"), } } return map[string]interface{}{ "users": responses, "count": len(responses), } } // ErrorResponse represents an error response type ErrorResponse struct { Error string `json:"error"` Code string `json:"code,omitempty"` } // PresentError presents an error func (p *UserJSONPresenter) PresentError(err error) interface{} { return ErrorResponse{ Error: err.Error(), } } Layer 4: Main Application (Dependency Injection) package main import ( "database/sql" "log" "net/http" _ "github.com/lib/pq" "github.com/google/uuid" "myapp/internal/adapter/controller" "myapp/internal/adapter/presenter" "myapp/internal/adapter/repository" "myapp/internal/usecase" ) // UUIDGenerator implements IDGenerator type UUIDGenerator struct{} func (g *UUIDGenerator) Generate() string { return uuid.New().String() } // MockEmailService implements EmailService type MockEmailService struct{} func (s *MockEmailService) SendWelcomeEmail(ctx context.Context, user *entity.User) error { log.Printf("Sending welcome email to %s", user.Email) return nil } func (s *MockEmailService) SendPasswordResetEmail(ctx context.Context, user *entity.User, token string) error { log.Printf("Sending password reset email to %s with token %s", user.Email, token) return nil } func main() { // Initialize database db, err := sql.Open("postgres", "postgres://user:pass@localhost/cleanarch?sslmode=disable") if err != nil { log.Fatal(err) } defer db.Close() // Layer 4: Initialize adapters (implementations) userRepo := repository.NewPostgresUserRepository(db) emailService := &MockEmailService{} idGen := &UUIDGenerator{} userPresenter := presenter.NewUserJSONPresenter() // Layer 2: Initialize use cases (with dependency injection) userInteractor := usecase.NewUserInteractor(userRepo, emailService, idGen) // Layer 3: Initialize controllers userController := controller.NewUserController(userInteractor, userPresenter) // Setup routes mux := http.NewServeMux() mux.HandleFunc("POST /users", userController.CreateUser) mux.HandleFunc("GET /users/{id}", userController.GetUser) mux.HandleFunc("PUT /users/{id}", userController.UpdateUser) mux.HandleFunc("DELETE /users/{id}", userController.DeleteUser) mux.HandleFunc("GET /users", userController.ListUsers) // Start server log.Println("Server starting on :8080") if err := http.ListenAndServe(":8080", mux); err != nil { log.Fatal(err) } } Testing in Clean Architecture package usecase_test import ( "context" "testing" "myapp/internal/entity" "myapp/internal/usecase" ) // Mock implementations type MockUserRepository struct { users map[string]*entity.User } func NewMockUserRepository() *MockUserRepository { return &MockUserRepository{ users: make(map[string]*entity.User), } } func (m *MockUserRepository) Create(ctx context.Context, user *entity.User) error { m.users[user.ID] = user return nil } func (m *MockUserRepository) GetByID(ctx context.Context, id string) (*entity.User, error) { user, exists := m.users[id] if !exists { return nil, entity.ErrUserNotFound } return user, nil } func (m *MockUserRepository) GetByEmail(ctx context.Context, email string) (*entity.User, error) { for _, user := range m.users { if user.Email == email { return user, nil } } return nil, nil } type MockEmailService struct { sentEmails []string } func (m *MockEmailService) SendWelcomeEmail(ctx context.Context, user *entity.User) error { m.sentEmails = append(m.sentEmails, user.Email) return nil } type MockIDGenerator struct { nextID int } func (m *MockIDGenerator) Generate() string { m.nextID++ return fmt.Sprintf("user-%d", m.nextID) } func TestCreateUser(t *testing.T) { // Arrange repo := NewMockUserRepository() emailService := &MockEmailService{} idGen := &MockIDGenerator{} interactor := usecase.NewUserInteractor(repo, emailService, idGen) input := usecase.CreateUserInput{ Email: "[email protected]", Name: "Test User", Age: 25, } // Act user, err := interactor.CreateUser(context.Background(), input) // Assert if err != nil { t.Fatalf("expected no error, got %v", err) } if user.Email != input.Email { t.Errorf("expected email %s, got %s", input.Email, user.Email) } if len(emailService.sentEmails) != 1 { t.Errorf("expected 1 welcome email, got %d", len(emailService.sentEmails)) } } Best Practices Dependency Rule: Always point dependencies inward Interface Segregation: Define minimal interfaces in use case layer Dependency Injection: Inject all dependencies explicitly Entity Purity: Keep entities free from framework dependencies Use Case Focus: Each use case should have a single responsibility Test Independence: Test each layer independently Avoid Anemic Models: Put business logic in entities Common Pitfalls Breaking Dependency Rule: Outer layers should not be imported by inner layers Leaking Infrastructure: Database or framework details leaking into entities Fat Use Cases: Use cases doing too much or too little Ignoring Presenters: Directly returning entities from controllers Over-engineering: Applying Clean Architecture to simple CRUD apps Missing Boundaries: Not clearly defining layer boundaries When to Use Clean Architecture Use When: ...

The Database Layer Dilemma Choosing the right data access layer is critical for Go applications. Get it wrong, and you’ll spend countless hours debugging runtime errors, dealing with SQL injection vulnerabilities, and wrestling with inconsistent data models. Two excellent options have emerged in the Go ecosystem: Ent - Facebook’s entity framework with code generation and type safety Bun - A SQL-first ORM with excellent query building capabilities This guide explores both, showing you when to use each and how to build reliable data layers. ...

Go Architecture Patterns Series: ← Clean Architecture | Series Overview | Next: Domain-Driven Design → What is Hexagonal Architecture? Hexagonal Architecture, also known as Ports and Adapters pattern, was introduced by Alistair Cockburn. It emphasizes separating the core business logic from external concerns by defining clear boundaries through ports (interfaces) and adapters (implementations). Key Principles: Core Domain Isolation: Business logic is completely isolated from external dependencies Ports: Interfaces that define how the application communicates with the outside world Adapters: Concrete implementations of ports for specific technologies Symmetry: No distinction between “front-end” and “back-end” - all external systems are treated equally Testability: Core can be tested in isolation without any external dependencies Pluggability: Adapters can be swapped without changing the core Architecture Overview %%{init: {'theme':'dark', 'themeVariables': {'primaryTextColor':'#e5e7eb','secondaryTextColor':'#e5e7eb','tertiaryTextColor':'#e5e7eb','textColor':'#e5e7eb','nodeTextColor':'#e5e7eb','edgeLabelText':'#e5e7eb','clusterTextColor':'#e5e7eb','actorTextColor':'#e5e7eb'}}}%% graph TB subgraph "Primary Adapters (Driving)" HTTP[HTTP REST API] CLI[CLI Interface] GRPC[gRPC Server] end subgraph "Hexagon (Core Domain)" PORT1[Primary PortsDriving Interfaces] CORE[Business LogicDomain ModelsUse Cases] PORT2[Secondary PortsDriven Interfaces] end subgraph "Secondary Adapters (Driven)" DB[PostgreSQL Adapter] CACHE[Redis Adapter] MSG[Message Queue Adapter] EXT[External API Adapter] end HTTP --> PORT1 CLI --> PORT1 GRPC --> PORT1 PORT1 --> CORE CORE --> PORT2 PORT2 --> DB PORT2 --> CACHE PORT2 --> MSG PORT2 --> EXT style CORE fill:#78350f,color:#fff style PORT1 fill:#1e3a5f,color:#fff style PORT2 fill:#1e3a5f,color:#fff style HTTP fill:#134e4a,color:#fff style DB fill:#4c1d95,color:#fff Ports and Adapters Visualization %%{init: {'theme':'dark', 'themeVariables': {'primaryTextColor':'#e5e7eb','secondaryTextColor':'#e5e7eb','tertiaryTextColor':'#e5e7eb','textColor':'#e5e7eb','nodeTextColor':'#e5e7eb','edgeLabelText':'#e5e7eb','clusterTextColor':'#e5e7eb','actorTextColor':'#e5e7eb'}}}%% graph LR subgraph "Primary Side (Driving)" REST[REST Adapter] GraphQL[GraphQL Adapter] end subgraph "Core Application" IP[Input PortInterface] BL[Business Logic] OP[Output PortInterface] end subgraph "Secondary Side (Driven)" DBAdapter[Database Adapter] EmailAdapter[Email Adapter] end REST -.->|implements| IP GraphQL -.->|implements| IP IP --> BL BL --> OP OP -.->|implemented by| DBAdapter OP -.->|implemented by| EmailAdapter style BL fill:#78350f,color:#fff style IP fill:#1e3a5f,color:#fff style OP fill:#1e3a5f,color:#fff style REST fill:#134e4a,color:#fff style DBAdapter fill:#4c1d95,color:#fff Complete Hexagonal Flow %%{init: {'theme':'dark', 'themeVariables': {'primaryTextColor':'#e5e7eb','secondaryTextColor':'#e5e7eb','tertiaryTextColor':'#e5e7eb','textColor':'#e5e7eb','nodeTextColor':'#e5e7eb','edgeLabelText':'#e5e7eb','clusterTextColor':'#e5e7eb','actorTextColor':'#e5e7eb'}}}%% sequenceDiagram participant Client participant HTTPAdapter participant InputPort participant Core participant OutputPort participant DBAdapter participant Database Client->>HTTPAdapter: HTTP Request HTTPAdapter->>InputPort: Call Method InputPort->>Core: Execute Business Logic Core->>OutputPort: Request Data OutputPort->>DBAdapter: Interface Call DBAdapter->>Database: SQL Query Database-->>DBAdapter: Data DBAdapter-->>OutputPort: Domain Object OutputPort-->>Core: Result Core-->>InputPort: Response InputPort-->>HTTPAdapter: DTO HTTPAdapter-->>Client: HTTP Response Real-World Use Cases API Gateways: Multiple protocols (REST, gRPC, GraphQL) with same core logic Multi-tenant Applications: Different adapters for different tenants Legacy System Integration: Adapter for each legacy system Testing-Critical Systems: Easy mocking of all external dependencies Cloud-Native Applications: Easy switching between cloud providers Evolutionary Architecture: System that needs to adapt over time Project Structure ├── cmd/ │ ├── http/ │ │ └── main.go # HTTP server entry point │ └── cli/ │ └── main.go # CLI entry point ├── internal/ │ ├── core/ │ │ ├── domain/ # Domain entities and value objects │ │ │ ├── user.go │ │ │ └── order.go │ │ ├── port/ # Ports (interfaces) │ │ │ ├── input.go # Primary/Driving ports │ │ │ └── output.go # Secondary/Driven ports │ │ └── service/ # Business logic │ │ └── user_service.go │ └── adapter/ │ ├── input/ # Primary/Driving adapters │ │ ├── http/ │ │ │ └── user_handler.go │ │ └── grpc/ │ │ └── user_server.go │ └── output/ # Secondary/Driven adapters │ ├── persistence/ │ │ └── postgres_user_repository.go │ └── notification/ │ └── email_service.go └── go.mod Core Domain Layer Domain Entities package domain import ( "errors" "time" ) // User represents a user entity in the domain type User struct { ID string Email Email Name string Status UserStatus CreatedAt time.Time UpdatedAt time.Time } // Email is a value object representing an email address type Email struct { value string } // NewEmail creates a new email value object with validation func NewEmail(email string) (Email, error) { if !isValidEmail(email) { return Email{}, errors.New("invalid email format") } return Email{value: email}, nil } // String returns the email string value func (e Email) String() string { return e.value } // UserStatus represents the status of a user type UserStatus string const ( UserStatusActive UserStatus = "active" UserStatusInactive UserStatus = "inactive" UserStatusSuspended UserStatus = "suspended" ) // NewUser creates a new user with business rule validation func NewUser(email, name string) (*User, error) { emailVO, err := NewEmail(email) if err != nil { return nil, err } if name == "" { return nil, errors.New("name cannot be empty") } return &User{ Email: emailVO, Name: name, Status: UserStatusActive, CreatedAt: time.Now(), UpdatedAt: time.Now(), }, nil } // Activate activates the user func (u *User) Activate() { u.Status = UserStatusActive u.UpdatedAt = time.Now() } // Suspend suspends the user func (u *User) Suspend() { u.Status = UserStatusSuspended u.UpdatedAt = time.Now() } // IsActive returns true if user is active func (u *User) IsActive() bool { return u.Status == UserStatusActive } func isValidEmail(email string) bool { // Simplified validation return len(email) > 3 && contains(email, "@") } func contains(s, substr string) bool { for i := 0; i <= len(s)-len(substr); i++ { if s[i:i+len(substr)] == substr { return true } } return false } // Order represents an order in the domain type Order struct { ID string UserID string Items []OrderItem Total Money Status OrderStatus CreatedAt time.Time } // OrderItem represents a line item in an order type OrderItem struct { ProductID string Quantity int Price Money } // OrderStatus represents the status of an order type OrderStatus string const ( OrderStatusPending OrderStatus = "pending" OrderStatusConfirmed OrderStatus = "confirmed" OrderStatusShipped OrderStatus = "shipped" OrderStatusDelivered OrderStatus = "delivered" OrderStatusCancelled OrderStatus = "cancelled" ) // Money represents monetary value type Money struct { Amount int64 Currency string } // NewMoney creates a new money value object func NewMoney(amount int64, currency string) Money { return Money{ Amount: amount, Currency: currency, } } // Add adds two money values func (m Money) Add(other Money) (Money, error) { if m.Currency != other.Currency { return Money{}, errors.New("cannot add different currencies") } return Money{ Amount: m.Amount + other.Amount, Currency: m.Currency, }, nil } Ports (Interfaces) Primary Ports (Driving/Input) package port import ( "context" "myapp/internal/core/domain" ) // UserService defines the input port for user operations // This is what drives the application (called by adapters) type UserService interface { CreateUser(ctx context.Context, email, name string) (*domain.User, error) GetUser(ctx context.Context, id string) (*domain.User, error) UpdateUser(ctx context.Context, id, email, name string) (*domain.User, error) DeleteUser(ctx context.Context, id string) error ListUsers(ctx context.Context, offset, limit int) ([]*domain.User, error) ActivateUser(ctx context.Context, id string) error SuspendUser(ctx context.Context, id string) error } // OrderService defines the input port for order operations type OrderService interface { CreateOrder(ctx context.Context, userID string, items []OrderItemInput) (*domain.Order, error) GetOrder(ctx context.Context, id string) (*domain.Order, error) CancelOrder(ctx context.Context, id string) error GetUserOrders(ctx context.Context, userID string) ([]*domain.Order, error) } // OrderItemInput represents input for creating an order item type OrderItemInput struct { ProductID string Quantity int } Secondary Ports (Driven/Output) package port import ( "context" "myapp/internal/core/domain" ) // UserRepository defines the output port for user persistence // This is driven by the application (implemented by adapters) type UserRepository interface { Save(ctx context.Context, user *domain.User) error FindByID(ctx context.Context, id string) (*domain.User, error) FindByEmail(ctx context.Context, email string) (*domain.User, error) Update(ctx context.Context, user *domain.User) error Delete(ctx context.Context, id string) error FindAll(ctx context.Context, offset, limit int) ([]*domain.User, error) } // OrderRepository defines the output port for order persistence type OrderRepository interface { Save(ctx context.Context, order *domain.Order) error FindByID(ctx context.Context, id string) (*domain.Order, error) FindByUserID(ctx context.Context, userID string) ([]*domain.Order, error) Update(ctx context.Context, order *domain.Order) error } // NotificationService defines the output port for notifications type NotificationService interface { SendWelcomeEmail(ctx context.Context, user *domain.User) error SendOrderConfirmation(ctx context.Context, order *domain.Order) error SendOrderCancellation(ctx context.Context, order *domain.Order) error } // IDGenerator defines the output port for ID generation type IDGenerator interface { GenerateID() string } // EventPublisher defines the output port for publishing domain events type EventPublisher interface { PublishUserCreated(ctx context.Context, user *domain.User) error PublishUserActivated(ctx context.Context, user *domain.User) error PublishOrderPlaced(ctx context.Context, order *domain.Order) error } Core Service (Business Logic) package service import ( "context" "errors" "fmt" "myapp/internal/core/domain" "myapp/internal/core/port" ) // userService implements the UserService port type userService struct { repo port.UserRepository notification port.NotificationService idGen port.IDGenerator eventPub port.EventPublisher } // NewUserService creates a new user service func NewUserService( repo port.UserRepository, notification port.NotificationService, idGen port.IDGenerator, eventPub port.EventPublisher, ) port.UserService { return &userService{ repo: repo, notification: notification, idGen: idGen, eventPub: eventPub, } } // CreateUser creates a new user func (s *userService) CreateUser(ctx context.Context, email, name string) (*domain.User, error) { // Check if user already exists existingUser, err := s.repo.FindByEmail(ctx, email) if err == nil && existingUser != nil { return nil, errors.New("user already exists") } // Create user entity with business rules user, err := domain.NewUser(email, name) if err != nil { return nil, fmt.Errorf("failed to create user: %w", err) } // Generate ID user.ID = s.idGen.GenerateID() // Save user if err := s.repo.Save(ctx, user); err != nil { return nil, fmt.Errorf("failed to save user: %w", err) } // Send notification (best effort, don't fail on error) if err := s.notification.SendWelcomeEmail(ctx, user); err != nil { // Log error but don't fail fmt.Printf("failed to send welcome email: %v\n", err) } // Publish event if err := s.eventPub.PublishUserCreated(ctx, user); err != nil { // Log error but don't fail fmt.Printf("failed to publish user created event: %v\n", err) } return user, nil } // GetUser retrieves a user by ID func (s *userService) GetUser(ctx context.Context, id string) (*domain.User, error) { user, err := s.repo.FindByID(ctx, id) if err != nil { return nil, fmt.Errorf("failed to get user: %w", err) } if user == nil { return nil, errors.New("user not found") } return user, nil } // UpdateUser updates a user func (s *userService) UpdateUser(ctx context.Context, id, email, name string) (*domain.User, error) { user, err := s.repo.FindByID(ctx, id) if err != nil { return nil, fmt.Errorf("user not found: %w", err) } // Update email if changed if email != "" && email != user.Email.String() { newEmail, err := domain.NewEmail(email) if err != nil { return nil, err } // Check if new email is already taken existingUser, _ := s.repo.FindByEmail(ctx, email) if existingUser != nil && existingUser.ID != id { return nil, errors.New("email already exists") } user.Email = newEmail } // Update name if changed if name != "" && name != user.Name { user.Name = name } // Save updated user if err := s.repo.Update(ctx, user); err != nil { return nil, fmt.Errorf("failed to update user: %w", err) } return user, nil } // DeleteUser deletes a user func (s *userService) DeleteUser(ctx context.Context, id string) error { user, err := s.repo.FindByID(ctx, id) if err != nil { return errors.New("user not found") } if !user.IsActive() { return errors.New("cannot delete inactive user") } return s.repo.Delete(ctx, id) } // ListUsers lists users with pagination func (s *userService) ListUsers(ctx context.Context, offset, limit int) ([]*domain.User, error) { if limit <= 0 || limit > 100 { limit = 20 } if offset < 0 { offset = 0 } return s.repo.FindAll(ctx, offset, limit) } // ActivateUser activates a user func (s *userService) ActivateUser(ctx context.Context, id string) error { user, err := s.repo.FindByID(ctx, id) if err != nil { return errors.New("user not found") } user.Activate() if err := s.repo.Update(ctx, user); err != nil { return fmt.Errorf("failed to activate user: %w", err) } // Publish event if err := s.eventPub.PublishUserActivated(ctx, user); err != nil { fmt.Printf("failed to publish user activated event: %v\n", err) } return nil } // SuspendUser suspends a user func (s *userService) SuspendUser(ctx context.Context, id string) error { user, err := s.repo.FindByID(ctx, id) if err != nil { return errors.New("user not found") } user.Suspend() return s.repo.Update(ctx, user) } Primary Adapters (Driving/Input) HTTP Adapter package http import ( "encoding/json" "net/http" "myapp/internal/core/port" ) // UserHandler is the HTTP adapter for user operations type UserHandler struct { service port.UserService } // NewUserHandler creates a new HTTP user handler func NewUserHandler(service port.UserService) *UserHandler { return &UserHandler{service: service} } // CreateUserRequest represents the HTTP request type CreateUserRequest struct { Email string `json:"email"` Name string `json:"name"` } // UserResponse represents the HTTP response type UserResponse struct { ID string `json:"id"` Email string `json:"email"` Name string `json:"name"` Status string `json:"status"` CreatedAt string `json:"created_at"` } // CreateUser handles POST /users func (h *UserHandler) CreateUser(w http.ResponseWriter, r *http.Request) { var req CreateUserRequest if err := json.NewDecoder(r.Body).Decode(&req); err != nil { respondError(w, http.StatusBadRequest, "invalid request") return } user, err := h.service.CreateUser(r.Context(), req.Email, req.Name) if err != nil { respondError(w, http.StatusBadRequest, err.Error()) return } respondJSON(w, http.StatusCreated, toUserResponse(user)) } // GetUser handles GET /users/{id} func (h *UserHandler) GetUser(w http.ResponseWriter, r *http.Request) { id := r.PathValue("id") user, err := h.service.GetUser(r.Context(), id) if err != nil { respondError(w, http.StatusNotFound, "user not found") return } respondJSON(w, http.StatusOK, toUserResponse(user)) } // UpdateUser handles PUT /users/{id} func (h *UserHandler) UpdateUser(w http.ResponseWriter, r *http.Request) { id := r.PathValue("id") var req CreateUserRequest if err := json.NewDecoder(r.Body).Decode(&req); err != nil { respondError(w, http.StatusBadRequest, "invalid request") return } user, err := h.service.UpdateUser(r.Context(), id, req.Email, req.Name) if err != nil { respondError(w, http.StatusBadRequest, err.Error()) return } respondJSON(w, http.StatusOK, toUserResponse(user)) } // DeleteUser handles DELETE /users/{id} func (h *UserHandler) DeleteUser(w http.ResponseWriter, r *http.Request) { id := r.PathValue("id") if err := h.service.DeleteUser(r.Context(), id); err != nil { respondError(w, http.StatusBadRequest, err.Error()) return } w.WriteHeader(http.StatusNoContent) } // Helper functions func toUserResponse(user *domain.User) UserResponse { return UserResponse{ ID: user.ID, Email: user.Email.String(), Name: user.Name, Status: string(user.Status), CreatedAt: user.CreatedAt.Format("2006-01-02T15:04:05Z"), } } func respondJSON(w http.ResponseWriter, status int, data interface{}) { w.Header().Set("Content-Type", "application/json") w.WriteHeader(status) json.NewEncoder(w).Encode(data) } func respondError(w http.ResponseWriter, status int, message string) { respondJSON(w, status, map[string]string{"error": message}) } Secondary Adapters (Driven/Output) Database Adapter package persistence import ( "context" "database/sql" "errors" "time" "myapp/internal/core/domain" "myapp/internal/core/port" ) // postgresUserRepository implements the UserRepository port type postgresUserRepository struct { db *sql.DB } // NewPostgresUserRepository creates a new Postgres user repository func NewPostgresUserRepository(db *sql.DB) port.UserRepository { return &postgresUserRepository{db: db} } // Save saves a user func (r *postgresUserRepository) Save(ctx context.Context, user *domain.User) error { query := ` INSERT INTO users (id, email, name, status, created_at, updated_at) VALUES ($1, $2, $3, $4, $5, $6) ` _, err := r.db.ExecContext( ctx, query, user.ID, user.Email.String(), user.Name, user.Status, user.CreatedAt, user.UpdatedAt, ) return err } // FindByID finds a user by ID func (r *postgresUserRepository) FindByID(ctx context.Context, id string) (*domain.User, error) { query := ` SELECT id, email, name, status, created_at, updated_at FROM users WHERE id = $1 ` var ( userID string email string name string status string createdAt time.Time updatedAt time.Time ) err := r.db.QueryRowContext(ctx, query, id).Scan( &userID, &email, &name, &status, &createdAt, &updatedAt, ) if err == sql.ErrNoRows { return nil, nil } if err != nil { return nil, err } emailVO, _ := domain.NewEmail(email) return &domain.User{ ID: userID, Email: emailVO, Name: name, Status: domain.UserStatus(status), CreatedAt: createdAt, UpdatedAt: updatedAt, }, nil } // FindByEmail finds a user by email func (r *postgresUserRepository) FindByEmail(ctx context.Context, email string) (*domain.User, error) { query := ` SELECT id, email, name, status, created_at, updated_at FROM users WHERE email = $1 ` var ( userID string emailStr string name string status string createdAt time.Time updatedAt time.Time ) err := r.db.QueryRowContext(ctx, query, email).Scan( &userID, &emailStr, &name, &status, &createdAt, &updatedAt, ) if err == sql.ErrNoRows { return nil, nil } if err != nil { return nil, err } emailVO, _ := domain.NewEmail(emailStr) return &domain.User{ ID: userID, Email: emailVO, Name: name, Status: domain.UserStatus(status), CreatedAt: createdAt, UpdatedAt: updatedAt, }, nil } // Update updates a user func (r *postgresUserRepository) Update(ctx context.Context, user *domain.User) error { query := ` UPDATE users SET email = $1, name = $2, status = $3, updated_at = $4 WHERE id = $5 ` result, err := r.db.ExecContext( ctx, query, user.Email.String(), user.Name, user.Status, time.Now(), user.ID, ) if err != nil { return err } rows, _ := result.RowsAffected() if rows == 0 { return errors.New("user not found") } return nil } // Delete deletes a user func (r *postgresUserRepository) Delete(ctx context.Context, id string) error { query := `DELETE FROM users WHERE id = $1` result, err := r.db.ExecContext(ctx, query, id) if err != nil { return err } rows, _ := result.RowsAffected() if rows == 0 { return errors.New("user not found") } return nil } // FindAll finds all users with pagination func (r *postgresUserRepository) FindAll(ctx context.Context, offset, limit int) ([]*domain.User, error) { query := ` SELECT id, email, name, status, created_at, updated_at FROM users ORDER BY created_at DESC LIMIT $1 OFFSET $2 ` rows, err := r.db.QueryContext(ctx, query, limit, offset) if err != nil { return nil, err } defer rows.Close() var users []*domain.User for rows.Next() { var ( userID string email string name string status string createdAt time.Time updatedAt time.Time ) if err := rows.Scan(&userID, &email, &name, &status, &createdAt, &updatedAt); err != nil { return nil, err } emailVO, _ := domain.NewEmail(email) users = append(users, &domain.User{ ID: userID, Email: emailVO, Name: name, Status: domain.UserStatus(status), CreatedAt: createdAt, UpdatedAt: updatedAt, }) } return users, nil } Email Notification Adapter package notification import ( "context" "fmt" "log" "myapp/internal/core/domain" "myapp/internal/core/port" ) // emailService implements the NotificationService port type emailService struct { smtpHost string smtpPort int fromAddr string } // NewEmailService creates a new email notification service func NewEmailService(smtpHost string, smtpPort int, fromAddr string) port.NotificationService { return &emailService{ smtpHost: smtpHost, smtpPort: smtpPort, fromAddr: fromAddr, } } // SendWelcomeEmail sends a welcome email to a new user func (s *emailService) SendWelcomeEmail(ctx context.Context, user *domain.User) error { // In production, use actual email service log.Printf("Sending welcome email to %s", user.Email.String()) return s.sendEmail(ctx, user.Email.String(), "Welcome!", "Welcome to our service!") } // SendOrderConfirmation sends an order confirmation email func (s *emailService) SendOrderConfirmation(ctx context.Context, order *domain.Order) error { log.Printf("Sending order confirmation for order %s", order.ID) return nil } // SendOrderCancellation sends an order cancellation email func (s *emailService) SendOrderCancellation(ctx context.Context, order *domain.Order) error { log.Printf("Sending order cancellation for order %s", order.ID) return nil } func (s *emailService) sendEmail(ctx context.Context, to, subject, body string) error { // Simulate email sending fmt.Printf("Email sent to %s: %s\n", to, subject) return nil } Main Application with Dependency Injection package main import ( "database/sql" "log" "net/http" _ "github.com/lib/pq" "github.com/google/uuid" "myapp/internal/adapter/input/http" "myapp/internal/adapter/output/notification" "myapp/internal/adapter/output/persistence" "myapp/internal/core/port" "myapp/internal/core/service" ) // UUIDGenerator implements IDGenerator port type UUIDGenerator struct{} func (g *UUIDGenerator) GenerateID() string { return uuid.New().String() } // MockEventPublisher implements EventPublisher port type MockEventPublisher struct{} func (p *MockEventPublisher) PublishUserCreated(ctx context.Context, user *domain.User) error { log.Printf("Event: User created - %s", user.ID) return nil } func (p *MockEventPublisher) PublishUserActivated(ctx context.Context, user *domain.User) error { log.Printf("Event: User activated - %s", user.ID) return nil } func (p *MockEventPublisher) PublishOrderPlaced(ctx context.Context, order *domain.Order) error { log.Printf("Event: Order placed - %s", order.ID) return nil } func main() { // Initialize database db, err := sql.Open("postgres", "postgres://user:pass@localhost/hexarch?sslmode=disable") if err != nil { log.Fatal(err) } defer db.Close() // Initialize secondary adapters (driven/output) userRepo := persistence.NewPostgresUserRepository(db) emailService := notification.NewEmailService("smtp.example.com", 587, "[email protected]") idGen := &UUIDGenerator{} eventPub := &MockEventPublisher{} // Initialize core service userService := service.NewUserService(userRepo, emailService, idGen, eventPub) // Initialize primary adapters (driving/input) httpHandler := httpAdapter.NewUserHandler(userService) // Setup routes mux := http.NewServeMux() mux.HandleFunc("POST /users", httpHandler.CreateUser) mux.HandleFunc("GET /users/{id}", httpHandler.GetUser) mux.HandleFunc("PUT /users/{id}", httpHandler.UpdateUser) mux.HandleFunc("DELETE /users/{id}", httpHandler.DeleteUser) // Start server log.Println("Server starting on :8080") if err := http.ListenAndServe(":8080", mux); err != nil { log.Fatal(err) } } Testing with Mock Adapters package service_test import ( "context" "testing" "myapp/internal/core/domain" "myapp/internal/core/service" ) // Mock repository type mockUserRepository struct { users map[string]*domain.User } func newMockUserRepository() *mockUserRepository { return &mockUserRepository{ users: make(map[string]*domain.User), } } func (m *mockUserRepository) Save(ctx context.Context, user *domain.User) error { m.users[user.ID] = user return nil } func (m *mockUserRepository) FindByID(ctx context.Context, id string) (*domain.User, error) { return m.users[id], nil } func (m *mockUserRepository) FindByEmail(ctx context.Context, email string) (*domain.User, error) { for _, user := range m.users { if user.Email.String() == email { return user, nil } } return nil, nil } // Mock notification service type mockNotificationService struct { sentEmails []string } func (m *mockNotificationService) SendWelcomeEmail(ctx context.Context, user *domain.User) error { m.sentEmails = append(m.sentEmails, user.Email.String()) return nil } // Mock ID generator type mockIDGenerator struct { id int } func (m *mockIDGenerator) GenerateID() string { m.id++ return fmt.Sprintf("user-%d", m.id) } // Mock event publisher type mockEventPublisher struct{} func (m *mockEventPublisher) PublishUserCreated(ctx context.Context, user *domain.User) error { return nil } func TestCreateUser(t *testing.T) { // Arrange repo := newMockUserRepository() notif := &mockNotificationService{} idGen := &mockIDGenerator{} eventPub := &mockEventPublisher{} service := service.NewUserService(repo, notif, idGen, eventPub) // Act user, err := service.CreateUser(context.Background(), "[email protected]", "Test User") // Assert if err != nil { t.Fatalf("expected no error, got %v", err) } if user.Email.String() != "[email protected]" { t.Errorf("expected email [email protected], got %s", user.Email.String()) } if len(notif.sentEmails) != 1 { t.Errorf("expected 1 welcome email, got %d", len(notif.sentEmails)) } } Best Practices Port Definition: Define ports (interfaces) in the core domain Adapter Independence: Adapters should not know about each other Domain First: Design domain model before thinking about adapters Single Responsibility: Each adapter handles one external concern Configuration Injection: Inject configuration into adapters, not core Error Handling: Let domain define error types Testing: Use mock adapters for testing core logic Common Pitfalls Adapter Coupling: Adapters depending on each other directly Leaky Abstractions: Infrastructure details leaking into core Anemic Ports: Ports that are too thin or just data transfer Adapter in Core: Importing adapter packages in core Forgetting Symmetry: Treating primary and secondary adapters differently Over-abstraction: Creating too many small ports When to Use Hexagonal Architecture Use When: ...

The API Dilemma: REST vs RPC? For years, teams have debated REST vs RPC as if they were mutually exclusive choices. The truth? You need both. Modern applications benefit from a two-tier API strategy that leverages REST for external clients and RPC for internal services. This isn’t about choosing sides-it’s about using the right tool for each job. Understanding the Two Tiers Tier 1: REST for External APIs (The Public Face) Use REST when: ...

The Aha Moment: Every Cell Is a Node If you’ve ever felt stuck staring at a grid problem, unsure where to start, you’re not alone. The breakthrough comes when you realize: a matrix is just a graph in disguise. Each cell in a 2D array is a node. The connections between adjacent cells (up, down, left, right, and sometimes diagonals) are the edges. Once this clicks, an entire category of problems becomes approachable using familiar graph algorithms. ...

Go Concurrency Patterns Series: ← Request/Response | Series Overview | Mutex Patterns → What is the Worker Pool Pattern? The Worker Pool pattern manages a fixed number of worker goroutines that process jobs from a shared queue. This pattern is essential for controlling resource usage, preventing system overload, and ensuring predictable performance under varying loads. Key Components: Job Queue: Channel containing work to be processed Worker Pool: Fixed number of worker goroutines Result Channel: Optional channel for collecting results Dispatcher: Coordinates job distribution to workers Real-World Use Cases Image Processing: Resize/compress images with limited CPU cores Database Operations: Limit concurrent database connections API Rate Limiting: Control outbound API call rates File Processing: Process files with bounded I/O operations Web Scraping: Limit concurrent HTTP requests Background Jobs: Process queued tasks with resource limits Basic Worker Pool Implementation package main import ( "fmt" "math/rand" "sync" "time" ) // Job represents work to be processed type Job struct { ID int Data interface{} } // Result represents the outcome of processing a job type Result struct { JobID int Output interface{} Error error } // WorkerPool manages a pool of workers type WorkerPool struct { workerCount int jobQueue chan Job resultQueue chan Result quit chan bool wg sync.WaitGroup } // NewWorkerPool creates a new worker pool func NewWorkerPool(workerCount, jobQueueSize int) *WorkerPool { return &WorkerPool{ workerCount: workerCount, jobQueue: make(chan Job, jobQueueSize), resultQueue: make(chan Result, jobQueueSize), quit: make(chan bool), } } // Start initializes and starts all workers func (wp *WorkerPool) Start() { for i := 0; i < wp.workerCount; i++ { wp.wg.Add(1) go wp.worker(i) } } // worker processes jobs from the job queue func (wp *WorkerPool) worker(id int) { defer wp.wg.Done() for { select { case job := <-wp.jobQueue: fmt.Printf("Worker %d processing job %d\n", id, job.ID) result := wp.processJob(job) wp.resultQueue <- result case <-wp.quit: fmt.Printf("Worker %d stopping\n", id) return } } } // processJob simulates job processing func (wp *WorkerPool) processJob(job Job) Result { // Simulate work time.Sleep(time.Duration(rand.Intn(100)) * time.Millisecond) // Process the job (example: square the number) if num, ok := job.Data.(int); ok { return Result{ JobID: job.ID, Output: num * num, } } return Result{ JobID: job.ID, Error: fmt.Errorf("invalid job data"), } } // Submit adds a job to the queue func (wp *WorkerPool) Submit(job Job) { wp.jobQueue <- job } // Results returns the result channel func (wp *WorkerPool) Results() <-chan Result { return wp.resultQueue } // Stop gracefully shuts down the worker pool func (wp *WorkerPool) Stop() { close(wp.quit) wp.wg.Wait() close(wp.jobQueue) close(wp.resultQueue) } func main() { // Create worker pool with 3 workers pool := NewWorkerPool(3, 10) pool.Start() defer pool.Stop() // Submit jobs go func() { for i := 1; i <= 10; i++ { job := Job{ ID: i, Data: i * 10, } pool.Submit(job) } }() // Collect results for i := 0; i < 10; i++ { result := <-pool.Results() if result.Error != nil { fmt.Printf("Job %d failed: %v\n", result.JobID, result.Error) } else { fmt.Printf("Job %d result: %v\n", result.JobID, result.Output) } } } Advanced Worker Pool with Context package main import ( "context" "fmt" "sync" "time" ) // ContextJob includes context for cancellation type ContextJob struct { ID string Data interface{} Context context.Context } // ContextResult includes timing and context information type ContextResult struct { JobID string Output interface{} Error error Duration time.Duration WorkerID int } // AdvancedWorkerPool supports context cancellation and monitoring type AdvancedWorkerPool struct { workerCount int jobQueue chan ContextJob resultQueue chan ContextResult ctx context.Context cancel context.CancelFunc wg sync.WaitGroup metrics *PoolMetrics } // PoolMetrics tracks worker pool performance type PoolMetrics struct { mu sync.RWMutex jobsProcessed int64 jobsFailed int64 totalDuration time.Duration activeWorkers int } func (pm *PoolMetrics) RecordJob(duration time.Duration, success bool) { pm.mu.Lock() defer pm.mu.Unlock() if success { pm.jobsProcessed++ } else { pm.jobsFailed++ } pm.totalDuration += duration } func (pm *PoolMetrics) SetActiveWorkers(count int) { pm.mu.Lock() defer pm.mu.Unlock() pm.activeWorkers = count } func (pm *PoolMetrics) GetStats() (processed, failed int64, avgDuration time.Duration, active int) { pm.mu.RLock() defer pm.mu.RUnlock() processed = pm.jobsProcessed failed = pm.jobsFailed active = pm.activeWorkers if pm.jobsProcessed > 0 { avgDuration = pm.totalDuration / time.Duration(pm.jobsProcessed) } return } // NewAdvancedWorkerPool creates a new advanced worker pool func NewAdvancedWorkerPool(ctx context.Context, workerCount, queueSize int) *AdvancedWorkerPool { poolCtx, cancel := context.WithCancel(ctx) return &AdvancedWorkerPool{ workerCount: workerCount, jobQueue: make(chan ContextJob, queueSize), resultQueue: make(chan ContextResult, queueSize), ctx: poolCtx, cancel: cancel, metrics: &PoolMetrics{}, } } // Start begins processing with all workers func (awp *AdvancedWorkerPool) Start() { awp.metrics.SetActiveWorkers(awp.workerCount) for i := 0; i < awp.workerCount; i++ { awp.wg.Add(1) go awp.worker(i) } // Start metrics reporter go awp.reportMetrics() } // worker processes jobs with context support func (awp *AdvancedWorkerPool) worker(id int) { defer awp.wg.Done() for { select { case job := <-awp.jobQueue: start := time.Now() result := awp.processContextJob(job, id) duration := time.Since(start) awp.metrics.RecordJob(duration, result.Error == nil) select { case awp.resultQueue <- result: case <-awp.ctx.Done(): return } case <-awp.ctx.Done(): fmt.Printf("Worker %d shutting down\n", id) return } } } // processContextJob handles job processing with context func (awp *AdvancedWorkerPool) processContextJob(job ContextJob, workerID int) ContextResult { start := time.Now() // Check if job context is already cancelled select { case <-job.Context.Done(): return ContextResult{ JobID: job.ID, Error: job.Context.Err(), Duration: time.Since(start), WorkerID: workerID, } default: } // Simulate work that respects context cancellation workDone := make(chan interface{}, 1) workErr := make(chan error, 1) go func() { // Simulate processing time time.Sleep(time.Duration(50+rand.Intn(100)) * time.Millisecond) if num, ok := job.Data.(int); ok { workDone <- num * num } else { workErr <- fmt.Errorf("invalid data type") } }() select { case result := <-workDone: return ContextResult{ JobID: job.ID, Output: result, Duration: time.Since(start), WorkerID: workerID, } case err := <-workErr: return ContextResult{ JobID: job.ID, Error: err, Duration: time.Since(start), WorkerID: workerID, } case <-job.Context.Done(): return ContextResult{ JobID: job.ID, Error: job.Context.Err(), Duration: time.Since(start), WorkerID: workerID, } case <-awp.ctx.Done(): return ContextResult{ JobID: job.ID, Error: awp.ctx.Err(), Duration: time.Since(start), WorkerID: workerID, } } } // Submit adds a job to the queue func (awp *AdvancedWorkerPool) Submit(job ContextJob) error { select { case awp.jobQueue <- job: return nil case <-awp.ctx.Done(): return awp.ctx.Err() } } // Results returns the result channel func (awp *AdvancedWorkerPool) Results() <-chan ContextResult { return awp.resultQueue } // reportMetrics periodically reports pool statistics func (awp *AdvancedWorkerPool) reportMetrics() { ticker := time.NewTicker(2 * time.Second) defer ticker.Stop() for { select { case <-ticker.C: processed, failed, avgDuration, active := awp.metrics.GetStats() fmt.Printf("Pool Stats - Processed: %d, Failed: %d, Avg Duration: %v, Active Workers: %d\n", processed, failed, avgDuration, active) case <-awp.ctx.Done(): return } } } // Stop gracefully shuts down the worker pool func (awp *AdvancedWorkerPool) Stop() { awp.cancel() awp.wg.Wait() close(awp.jobQueue) close(awp.resultQueue) } func main() { ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) defer cancel() pool := NewAdvancedWorkerPool(ctx, 4, 20) pool.Start() defer pool.Stop() // Submit jobs with individual timeouts go func() { for i := 1; i <= 15; i++ { jobCtx, jobCancel := context.WithTimeout(ctx, 200*time.Millisecond) job := ContextJob{ ID: fmt.Sprintf("job-%d", i), Data: i * 5, Context: jobCtx, } if err := pool.Submit(job); err != nil { fmt.Printf("Failed to submit job %d: %v\n", i, err) jobCancel() break } // Cancel some jobs early to demonstrate cancellation if i%5 == 0 { go func() { time.Sleep(50 * time.Millisecond) jobCancel() }() } else { defer jobCancel() } } }() // Collect results resultCount := 0 for result := range pool.Results() { resultCount++ if result.Error != nil { fmt.Printf("Job %s failed (worker %d): %v (took %v)\n", result.JobID, result.WorkerID, result.Error, result.Duration) } else { fmt.Printf("Job %s completed (worker %d): %v (took %v)\n", result.JobID, result.WorkerID, result.Output, result.Duration) } if resultCount >= 15 { break } } } Dynamic Worker Pool package main import ( "context" "fmt" "sync" "sync/atomic" "time" ) // DynamicWorkerPool can scale workers up and down based on load type DynamicWorkerPool struct { minWorkers int maxWorkers int currentWorkers int64 jobQueue chan Job resultQueue chan Result ctx context.Context cancel context.CancelFunc wg sync.WaitGroup workerControl chan int // +1 to add worker, -1 to remove worker metrics *DynamicMetrics } // DynamicMetrics tracks load and performance for scaling decisions type DynamicMetrics struct { mu sync.RWMutex queueLength int64 avgProcessingTime time.Duration lastScaleTime time.Time scaleUpThreshold int scaleDownThreshold int } func (dm *DynamicMetrics) UpdateQueueLength(length int) { atomic.StoreInt64(&dm.queueLength, int64(length)) } func (dm *DynamicMetrics) GetQueueLength() int { return int(atomic.LoadInt64(&dm.queueLength)) } func (dm *DynamicMetrics) ShouldScaleUp(currentWorkers int, maxWorkers int) bool { dm.mu.RLock() defer dm.mu.RUnlock() return currentWorkers < maxWorkers && dm.GetQueueLength() > dm.scaleUpThreshold && time.Since(dm.lastScaleTime) > 5*time.Second } func (dm *DynamicMetrics) ShouldScaleDown(currentWorkers int, minWorkers int) bool { dm.mu.RLock() defer dm.mu.RUnlock() return currentWorkers > minWorkers && dm.GetQueueLength() < dm.scaleDownThreshold && time.Since(dm.lastScaleTime) > 10*time.Second } func (dm *DynamicMetrics) RecordScale() { dm.mu.Lock() defer dm.mu.Unlock() dm.lastScaleTime = time.Now() } // NewDynamicWorkerPool creates a new dynamic worker pool func NewDynamicWorkerPool(ctx context.Context, minWorkers, maxWorkers, queueSize int) *DynamicWorkerPool { poolCtx, cancel := context.WithCancel(ctx) return &DynamicWorkerPool{ minWorkers: minWorkers, maxWorkers: maxWorkers, currentWorkers: 0, jobQueue: make(chan Job, queueSize), resultQueue: make(chan Result, queueSize), ctx: poolCtx, cancel: cancel, workerControl: make(chan int, maxWorkers), metrics: &DynamicMetrics{ scaleUpThreshold: queueSize / 2, scaleDownThreshold: queueSize / 4, }, } } // Start initializes the pool with minimum workers func (dwp *DynamicWorkerPool) Start() { // Start with minimum workers for i := 0; i < dwp.minWorkers; i++ { dwp.addWorker() } // Start the scaler go dwp.scaler() // Start queue monitor go dwp.queueMonitor() } // addWorker creates and starts a new worker func (dwp *DynamicWorkerPool) addWorker() { workerID := atomic.AddInt64(&dwp.currentWorkers, 1) dwp.wg.Add(1) go func(id int64) { defer dwp.wg.Done() defer atomic.AddInt64(&dwp.currentWorkers, -1) fmt.Printf("Worker %d started\n", id) for { select { case job := <-dwp.jobQueue: start := time.Now() result := dwp.processJob(job) duration := time.Since(start) fmt.Printf("Worker %d processed job %d in %v\n", id, job.ID, duration) select { case dwp.resultQueue <- result: case <-dwp.ctx.Done(): return } case <-dwp.ctx.Done(): fmt.Printf("Worker %d stopping\n", id) return } } }(workerID) } // processJob simulates job processing func (dwp *DynamicWorkerPool) processJob(job Job) Result { // Simulate variable processing time time.Sleep(time.Duration(50+rand.Intn(200)) * time.Millisecond) if num, ok := job.Data.(int); ok { return Result{ JobID: job.ID, Output: num * 2, } } return Result{ JobID: job.ID, Error: fmt.Errorf("invalid job data"), } } // scaler monitors load and adjusts worker count func (dwp *DynamicWorkerPool) scaler() { ticker := time.NewTicker(3 * time.Second) defer ticker.Stop() for { select { case <-ticker.C: currentWorkers := int(atomic.LoadInt64(&dwp.currentWorkers)) queueLength := dwp.metrics.GetQueueLength() fmt.Printf("Scaler check - Workers: %d, Queue: %d\n", currentWorkers, queueLength) if dwp.metrics.ShouldScaleUp(currentWorkers, dwp.maxWorkers) { fmt.Printf("Scaling up: adding worker (current: %d)\n", currentWorkers) dwp.addWorker() dwp.metrics.RecordScale() } else if dwp.metrics.ShouldScaleDown(currentWorkers, dwp.minWorkers) { fmt.Printf("Scaling down: removing worker (current: %d)\n", currentWorkers) // Signal one worker to stop by closing context // In a real implementation, you might use a more sophisticated approach dwp.metrics.RecordScale() } case <-dwp.ctx.Done(): return } } } // queueMonitor tracks queue length for scaling decisions func (dwp *DynamicWorkerPool) queueMonitor() { ticker := time.NewTicker(1 * time.Second) defer ticker.Stop() for { select { case <-ticker.C: queueLength := len(dwp.jobQueue) dwp.metrics.UpdateQueueLength(queueLength) case <-dwp.ctx.Done(): return } } } // Submit adds a job to the queue func (dwp *DynamicWorkerPool) Submit(job Job) error { select { case dwp.jobQueue <- job: return nil case <-dwp.ctx.Done(): return dwp.ctx.Err() } } // Results returns the result channel func (dwp *DynamicWorkerPool) Results() <-chan Result { return dwp.resultQueue } // Stop gracefully shuts down the pool func (dwp *DynamicWorkerPool) Stop() { dwp.cancel() dwp.wg.Wait() close(dwp.jobQueue) close(dwp.resultQueue) } func main() { ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second) defer cancel() pool := NewDynamicWorkerPool(ctx, 2, 6, 20) pool.Start() defer pool.Stop() // Submit jobs in bursts to trigger scaling go func() { // Initial burst for i := 1; i <= 10; i++ { job := Job{ID: i, Data: i * 10} if err := pool.Submit(job); err != nil { fmt.Printf("Failed to submit job %d: %v\n", i, err) break } } time.Sleep(8 * time.Second) // Second burst for i := 11; i <= 25; i++ { job := Job{ID: i, Data: i * 10} if err := pool.Submit(job); err != nil { fmt.Printf("Failed to submit job %d: %v\n", i, err) break } } time.Sleep(5 * time.Second) // Final smaller batch for i := 26; i <= 30; i++ { job := Job{ID: i, Data: i * 10} if err := pool.Submit(job); err != nil { fmt.Printf("Failed to submit job %d: %v\n", i, err) break } } }() // Collect results resultCount := 0 for result := range pool.Results() { resultCount++ if result.Error != nil { fmt.Printf("Job %d failed: %v\n", result.JobID, result.Error) } else { fmt.Printf("Job %d completed: %v\n", result.JobID, result.Output) } if resultCount >= 30 { break } } } Best Practices Right-Size the Pool: Match worker count to available resources Monitor Performance: Track queue length, processing times, and throughput Handle Backpressure: Implement proper queue management Graceful Shutdown: Ensure all workers complete current jobs Error Handling: Isolate worker failures from the pool Resource Cleanup: Properly close channels and cancel contexts Load Balancing: Distribute work evenly across workers Common Pitfalls Too Many Workers: Creating more workers than CPU cores for CPU-bound tasks Unbounded Queues: Memory issues with unlimited job queues Worker Leaks: Not properly shutting down workers Blocking Operations: Long-running jobs blocking other work No Backpressure: Not handling queue overflow situations Testing Worker Pools package main import ( "context" "testing" "time" ) func TestWorkerPool(t *testing.T) { ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) defer cancel() pool := NewAdvancedWorkerPool(ctx, 2, 5) pool.Start() defer pool.Stop() // Submit test jobs jobCount := 5 for i := 1; i <= jobCount; i++ { job := ContextJob{ ID: fmt.Sprintf("test-%d", i), Data: i, Context: ctx, } if err := pool.Submit(job); err != nil { t.Fatalf("Failed to submit job: %v", err) } } // Collect results results := make(map[string]ContextResult) for i := 0; i < jobCount; i++ { select { case result := <-pool.Results(): results[result.JobID] = result case <-time.After(2 * time.Second): t.Fatal("Timeout waiting for results") } } // Verify all jobs completed if len(results) != jobCount { t.Errorf("Expected %d results, got %d", jobCount, len(results)) } // Verify results are correct for i := 1; i <= jobCount; i++ { jobID := fmt.Sprintf("test-%d", i) result, exists := results[jobID] if !exists { t.Errorf("Missing result for job %s", jobID) continue } if result.Error != nil { t.Errorf("Job %s failed: %v", jobID, result.Error) continue } expected := i * i if result.Output != expected { t.Errorf("Job %s: expected %d, got %v", jobID, expected, result.Output) } } } The Worker Pool pattern is essential for building scalable, resource-efficient concurrent applications in Go. It provides controlled concurrency, predictable resource usage, and excellent performance characteristics for both CPU-bound and I/O-bound workloads. ...

Go Concurrency Patterns Series: ← Mutex Patterns | Series Overview | Once Pattern → What is the WaitGroup Pattern? The WaitGroup pattern uses sync.WaitGroup to coordinate the completion of multiple goroutines. It acts as a counter that blocks until all registered goroutines have finished executing, making it perfect for implementing barriers and waiting for parallel tasks to complete. Key Operations: Add(n): Increment the counter by n Done(): Decrement the counter by 1 (usually called with defer) Wait(): Block until counter reaches zero Real-World Use Cases Parallel Processing: Wait for all workers to complete Batch Operations: Process multiple items concurrently Service Initialization: Wait for all services to start Data Collection: Gather results from multiple sources Cleanup Operations: Ensure all cleanup tasks finish Testing: Coordinate test goroutines Basic WaitGroup Usage package main import ( "fmt" "math/rand" "sync" "time" ) // Task represents work to be done type Task struct { ID int Name string } // processTask simulates processing a task func processTask(task Task, wg *sync.WaitGroup) { defer wg.Done() // Always call Done when goroutine finishes fmt.Printf("Starting task %d: %s\n", task.ID, task.Name) // Simulate work duration := time.Duration(rand.Intn(1000)) * time.Millisecond time.Sleep(duration) fmt.Printf("Completed task %d: %s (took %v)\n", task.ID, task.Name, duration) } func main() { tasks := []Task{ {1, "Process images"}, {2, "Send emails"}, {3, "Update database"}, {4, "Generate reports"}, {5, "Backup files"}, } var wg sync.WaitGroup fmt.Println("Starting parallel task processing...") // Start all tasks for _, task := range tasks { wg.Add(1) // Increment counter for each goroutine go processTask(task, &wg) } // Wait for all tasks to complete wg.Wait() fmt.Println("All tasks completed!") } WaitGroup with Error Handling package main import ( "fmt" "math/rand" "sync" "time" ) // Result represents the outcome of a task type Result struct { TaskID int Data interface{} Error error } // TaskProcessor handles tasks with error collection type TaskProcessor struct { wg sync.WaitGroup results chan Result errors []error mu sync.Mutex } // NewTaskProcessor creates a new task processor func NewTaskProcessor(bufferSize int) *TaskProcessor { return &TaskProcessor{ results: make(chan Result, bufferSize), } } // processTaskWithError simulates task processing that might fail func (tp *TaskProcessor) processTaskWithError(taskID int, data interface{}) { defer tp.wg.Done() fmt.Printf("Processing task %d\n", taskID) // Simulate work time.Sleep(time.Duration(rand.Intn(500)) * time.Millisecond) // Simulate random failures if rand.Float32() < 0.3 { err := fmt.Errorf("task %d failed", taskID) tp.results <- Result{TaskID: taskID, Error: err} // Collect error tp.mu.Lock() tp.errors = append(tp.errors, err) tp.mu.Unlock() fmt.Printf("Task %d failed\n", taskID) return } // Success result := fmt.Sprintf("Result from task %d", taskID) tp.results <- Result{TaskID: taskID, Data: result} fmt.Printf("Task %d completed successfully\n", taskID) } // ProcessTasks processes multiple tasks and collects results func (tp *TaskProcessor) ProcessTasks(taskCount int) ([]Result, []error) { // Start all tasks for i := 1; i <= taskCount; i++ { tp.wg.Add(1) go tp.processTaskWithError(i, fmt.Sprintf("data-%d", i)) } // Close results channel when all tasks complete go func() { tp.wg.Wait() close(tp.results) }() // Collect results var results []Result for result := range tp.results { results = append(results, result) } tp.mu.Lock() errors := make([]error, len(tp.errors)) copy(errors, tp.errors) tp.mu.Unlock() return results, errors } func main() { processor := NewTaskProcessor(10) fmt.Println("Starting task processing with error handling...") results, errors := processor.ProcessTasks(8) fmt.Printf("\nProcessing complete!\n") fmt.Printf("Successful tasks: %d\n", len(results)-len(errors)) fmt.Printf("Failed tasks: %d\n", len(errors)) if len(errors) > 0 { fmt.Println("\nErrors:") for _, err := range errors { fmt.Printf(" - %v\n", err) } } fmt.Println("\nResults:") for _, result := range results { if result.Error == nil { fmt.Printf(" Task %d: %v\n", result.TaskID, result.Data) } } } Nested WaitGroups for Hierarchical Tasks package main import ( "fmt" "sync" "time" ) // Department represents a department with multiple teams type Department struct { Name string Teams []Team } // Team represents a team with multiple workers type Team struct { Name string Workers []string } // processDepartment processes all teams in a department func processDepartment(dept Department, wg *sync.WaitGroup) { defer wg.Done() fmt.Printf("Department %s starting work\n", dept.Name) var teamWG sync.WaitGroup // Process all teams in parallel for _, team := range dept.Teams { teamWG.Add(1) go processTeam(team, &teamWG) } // Wait for all teams to complete teamWG.Wait() fmt.Printf("Department %s completed all work\n", dept.Name) } // processTeam processes all workers in a team func processTeam(team Team, wg *sync.WaitGroup) { defer wg.Done() fmt.Printf(" Team %s starting work\n", team.Name) var workerWG sync.WaitGroup // Process all workers in parallel for _, worker := range team.Workers { workerWG.Add(1) go processWorker(worker, &workerWG) } // Wait for all workers to complete workerWG.Wait() fmt.Printf(" Team %s completed all work\n", team.Name) } // processWorker simulates worker processing func processWorker(worker string, wg *sync.WaitGroup) { defer wg.Done() fmt.Printf(" Worker %s working...\n", worker) time.Sleep(time.Duration(100+rand.Intn(200)) * time.Millisecond) fmt.Printf(" Worker %s finished\n", worker) } func main() { departments := []Department{ { Name: "Engineering", Teams: []Team{ { Name: "Backend", Workers: []string{"Alice", "Bob", "Charlie"}, }, { Name: "Frontend", Workers: []string{"Diana", "Eve"}, }, }, }, { Name: "Marketing", Teams: []Team{ { Name: "Digital", Workers: []string{"Frank", "Grace"}, }, { Name: "Content", Workers: []string{"Henry", "Ivy", "Jack"}, }, }, }, } var deptWG sync.WaitGroup fmt.Println("Starting company-wide project...") // Process all departments in parallel for _, dept := range departments { deptWG.Add(1) go processDepartment(dept, &deptWG) } // Wait for all departments to complete deptWG.Wait() fmt.Println("Company-wide project completed!") } WaitGroup with Timeout package main import ( "context" "fmt" "sync" "time" ) // TimedTaskRunner runs tasks with timeout support type TimedTaskRunner struct { timeout time.Duration } // NewTimedTaskRunner creates a new timed task runner func NewTimedTaskRunner(timeout time.Duration) *TimedTaskRunner { return &TimedTaskRunner{timeout: timeout} } // RunWithTimeout runs tasks with a timeout func (ttr *TimedTaskRunner) RunWithTimeout(tasks []func()) error { ctx, cancel := context.WithTimeout(context.Background(), ttr.timeout) defer cancel() var wg sync.WaitGroup done := make(chan struct{}) // Start all tasks for i, task := range tasks { wg.Add(1) go func(taskID int, taskFunc func()) { defer wg.Done() fmt.Printf("Starting task %d\n", taskID) taskFunc() fmt.Printf("Completed task %d\n", taskID) }(i+1, task) } // Wait for completion in separate goroutine go func() { wg.Wait() close(done) }() // Wait for either completion or timeout select { case <-done: fmt.Println("All tasks completed successfully") return nil case <-ctx.Done(): fmt.Println("Tasks timed out") return ctx.Err() } } // simulateTask creates a task that takes a specific duration func simulateTask(duration time.Duration, name string) func() { return func() { fmt.Printf(" %s working for %v\n", name, duration) time.Sleep(duration) fmt.Printf(" %s finished\n", name) } } func main() { runner := NewTimedTaskRunner(2 * time.Second) // Test with tasks that complete within timeout fmt.Println("Test 1: Tasks that complete within timeout") tasks1 := []func(){ simulateTask(300*time.Millisecond, "Quick task 1"), simulateTask(500*time.Millisecond, "Quick task 2"), simulateTask(400*time.Millisecond, "Quick task 3"), } if err := runner.RunWithTimeout(tasks1); err != nil { fmt.Printf("Error: %v\n", err) } fmt.Println("\nTest 2: Tasks that exceed timeout") tasks2 := []func(){ simulateTask(800*time.Millisecond, "Slow task 1"), simulateTask(1500*time.Millisecond, "Slow task 2"), simulateTask(2000*time.Millisecond, "Very slow task"), } if err := runner.RunWithTimeout(tasks2); err != nil { fmt.Printf("Error: %v\n", err) } } Dynamic WaitGroup Management package main import ( "fmt" "sync" "time" ) // DynamicTaskManager manages tasks that can spawn other tasks type DynamicTaskManager struct { wg sync.WaitGroup taskChan chan func() quit chan struct{} active sync.WaitGroup } // NewDynamicTaskManager creates a new dynamic task manager func NewDynamicTaskManager() *DynamicTaskManager { return &DynamicTaskManager{ taskChan: make(chan func(), 100), quit: make(chan struct{}), } } // Start begins processing tasks func (dtm *DynamicTaskManager) Start() { go dtm.taskProcessor() } // taskProcessor processes tasks from the channel func (dtm *DynamicTaskManager) taskProcessor() { for { select { case task := <-dtm.taskChan: dtm.active.Add(1) go func() { defer dtm.active.Done() task() }() case <-dtm.quit: return } } } // AddTask adds a task to be processed func (dtm *DynamicTaskManager) AddTask(task func()) { select { case dtm.taskChan <- task: case <-dtm.quit: } } // Wait waits for all active tasks to complete func (dtm *DynamicTaskManager) Wait() { dtm.active.Wait() } // Stop stops the task manager func (dtm *DynamicTaskManager) Stop() { close(dtm.quit) dtm.Wait() } // recursiveTask demonstrates a task that spawns other tasks func recursiveTask(manager *DynamicTaskManager, depth int, maxDepth int, id string) func() { return func() { fmt.Printf("Task %s (depth %d) starting\n", id, depth) time.Sleep(100 * time.Millisecond) if depth < maxDepth { // Spawn child tasks for i := 0; i < 2; i++ { childID := fmt.Sprintf("%s.%d", id, i+1) manager.AddTask(recursiveTask(manager, depth+1, maxDepth, childID)) } } fmt.Printf("Task %s (depth %d) completed\n", id, depth) } } func main() { manager := NewDynamicTaskManager() manager.Start() defer manager.Stop() fmt.Println("Starting dynamic task processing...") // Add initial tasks that will spawn more tasks for i := 0; i < 3; i++ { taskID := fmt.Sprintf("root-%d", i+1) manager.AddTask(recursiveTask(manager, 0, 2, taskID)) } // Wait for all tasks (including dynamically created ones) to complete manager.Wait() fmt.Println("All tasks completed!") } Best Practices Always Use defer: Call Done() with defer to ensure it’s called even if panic occurs Add Before Starting: Call Add() before starting goroutines to avoid race conditions Don’t Reuse WaitGroups: Create new WaitGroup for each batch of operations Handle Panics: Use recover in goroutines to prevent panic from affecting WaitGroup Avoid Negative Counters: Don’t call Done() more times than Add() Use Timeouts: Combine with context for timeout handling Consider Alternatives: Use channels for complex coordination scenarios Common Pitfalls 1. Race Condition with Add/Done // Bad: Race condition func badExample() { var wg sync.WaitGroup for i := 0; i < 5; i++ { go func() { wg.Add(1) // Race: might be called after Wait() defer wg.Done() // do work }() } wg.Wait() // Might not wait for all goroutines } // Good: Add before starting goroutines func goodExample() { var wg sync.WaitGroup for i := 0; i < 5; i++ { wg.Add(1) // Add before starting goroutine go func() { defer wg.Done() // do work }() } wg.Wait() } 2. Forgetting to Call Done // Bad: Missing Done() call func badTask(wg *sync.WaitGroup) { // do work if someCondition { return // Forgot to call Done()! } wg.Done() } // Good: Always use defer func goodTask(wg *sync.WaitGroup) { defer wg.Done() // Always called // do work if someCondition { return // Done() still called } } Testing WaitGroup Patterns package main import ( "sync" "testing" "time" ) func TestWaitGroupCompletion(t *testing.T) { var wg sync.WaitGroup completed := make([]bool, 5) for i := 0; i < 5; i++ { wg.Add(1) go func(index int) { defer wg.Done() time.Sleep(10 * time.Millisecond) completed[index] = true }(i) } wg.Wait() // Verify all tasks completed for i, done := range completed { if !done { t.Errorf("Task %d did not complete", i) } } } func TestWaitGroupWithTimeout(t *testing.T) { var wg sync.WaitGroup done := make(chan struct{}) wg.Add(1) go func() { defer wg.Done() time.Sleep(50 * time.Millisecond) }() go func() { wg.Wait() close(done) }() select { case <-done: // Success case <-time.After(100 * time.Millisecond): t.Error("WaitGroup did not complete within timeout") } } The WaitGroup pattern is essential for coordinating goroutines in Go. It provides a simple yet powerful way to wait for multiple concurrent operations to complete, making it perfect for parallel processing, batch operations, and synchronization barriers. ...