Event-Driven Note Sharing: Building Real-Time Microservices with NATS, Go, and Vue.js

    Building a real-time note-sharing application is a perfect use case for exploring event-driven architecture. In this comprehensive guide, we’ll build a production-ready system with two microservices (User and Note services) that communicate through NATS, delivering instant updates to a Vue.js frontend. Why Event-Driven Architecture? Traditional request-response patterns create tight coupling between services. When your Note service needs to notify users about changes, you don’t want to make synchronous HTTP calls to every service that cares about notes. Event-driven architecture solves this with loose coupling: ...

    January 27, 2025 · 22 min · Rafiul Alam

    Microservices Architecture in Go: Building Distributed Systems

    Go Architecture Patterns Series: ← Previous: Modular Monolith | Series Overview | Next: Event-Driven Architecture → What is Microservices Architecture? Microservices Architecture is an approach where an application is composed of small, independent services that communicate over a network. Each service is self-contained, owns its data, and can be deployed, scaled, and updated independently. Key Principles: Service Independence: Each service is deployed and scaled independently Single Responsibility: Each service handles a specific business capability Decentralized Data: Each service owns its database API-First Design: Services communicate through well-defined APIs Resilience: Services handle failures gracefully Technology Diversity: Services can use different technologies Architecture Overview %%{init: {'theme':'dark', 'themeVariables': {'primaryTextColor':'#e5e7eb','secondaryTextColor':'#e5e7eb','tertiaryTextColor':'#e5e7eb','textColor':'#e5e7eb','nodeTextColor':'#e5e7eb','edgeLabelText':'#e5e7eb','clusterTextColor':'#e5e7eb','actorTextColor':'#e5e7eb'}}}%% graph TB subgraph "Client Layer" Client[Web/Mobile Client] end subgraph "API Gateway" Gateway[API Gateway / Load Balancer] end subgraph "Service Mesh" UserService[User Service] ProductService[Product Service] OrderService[Order Service] PaymentService[Payment Service] NotificationService[Notification Service] end subgraph "Data Layer" UserDB[(User DB)] ProductDB[(Product DB)] OrderDB[(Order DB)] PaymentDB[(Payment DB)] end subgraph "Infrastructure" MessageBroker[Message Broker] ServiceRegistry[Service Discovery] ConfigServer[Config Server] end Client --> Gateway Gateway --> UserService Gateway --> ProductService Gateway --> OrderService Gateway --> PaymentService UserService --> UserDB ProductService --> ProductDB OrderService --> OrderDB PaymentService --> PaymentDB OrderService -.->|HTTP/gRPC| UserService OrderService -.->|HTTP/gRPC| ProductService OrderService -.->|HTTP/gRPC| PaymentService OrderService -.->|Async| MessageBroker PaymentService -.->|Async| MessageBroker NotificationService -.->|Subscribe| MessageBroker UserService -.-> ServiceRegistry ProductService -.-> ServiceRegistry OrderService -.-> ServiceRegistry style UserService fill:#1e3a5f,color:#fff style ProductService fill:#78350f,color:#fff style OrderService fill:#134e4a,color:#fff style PaymentService fill:#4c1d95,color:#fff style NotificationService fill:#4a1e3a,color:#fff Service Communication Patterns %%{init: {'theme':'dark', 'themeVariables': {'primaryTextColor':'#e5e7eb','secondaryTextColor':'#e5e7eb','tertiaryTextColor':'#e5e7eb','textColor':'#e5e7eb','nodeTextColor':'#e5e7eb','edgeLabelText':'#e5e7eb','clusterTextColor':'#e5e7eb','actorTextColor':'#e5e7eb'}}}%% sequenceDiagram participant Client participant Gateway participant OrderSvc as Order Service participant UserSvc as User Service participant ProductSvc as Product Service participant PaymentSvc as Payment Service participant Queue as Message Queue participant NotifySvc as Notification Service Client->>Gateway: Create Order Request Gateway->>OrderSvc: POST /orders OrderSvc->>UserSvc: GET /users/{id} UserSvc-->>OrderSvc: User Data OrderSvc->>ProductSvc: GET /products/{id} ProductSvc-->>OrderSvc: Product Data OrderSvc->>ProductSvc: POST /products/reserve ProductSvc-->>OrderSvc: Stock Reserved OrderSvc->>PaymentSvc: POST /payments PaymentSvc-->>OrderSvc: Payment Success OrderSvc->>Queue: Publish OrderCreated Event Queue->>NotifySvc: OrderCreated Event NotifySvc->>NotifySvc: Send Email/SMS OrderSvc-->>Gateway: Order Created Gateway-->>Client: Response Real-World Use Cases E-commerce Platforms: Amazon, eBay with separate services for products, orders, payments Streaming Services: Netflix with services for recommendations, playback, billing Ride-Sharing Apps: Uber with services for riders, drivers, payments, routing Financial Systems: Banking apps with separate services for accounts, transactions, loans Social Media: Twitter with services for posts, timelines, notifications, messages Cloud Platforms: AWS-like platforms with independent service offerings Microservices Implementation Project Structure (Multi-Repository) microservices/ ├── user-service/ │ ├── cmd/ │ │ └── server/ │ │ └── main.go │ ├── internal/ │ │ ├── domain/ │ │ ├── handlers/ │ │ ├── repository/ │ │ └── service/ │ ├── proto/ │ │ └── user.proto │ └── go.mod ├── product-service/ │ ├── cmd/ │ │ └── server/ │ │ └── main.go │ ├── internal/ │ │ ├── domain/ │ │ ├── handlers/ │ │ ├── repository/ │ │ └── service/ │ └── go.mod ├── order-service/ │ ├── cmd/ │ │ └── server/ │ │ └── main.go │ ├── internal/ │ │ ├── domain/ │ │ ├── handlers/ │ │ ├── repository/ │ │ ├── service/ │ │ └── clients/ │ └── go.mod └── api-gateway/ ├── cmd/ │ └── server/ │ └── main.go └── go.mod Service 1: User Service // user-service/internal/domain/user.go package domain import ( "context" "errors" "time" ) type User struct { ID string `json:"id"` Email string `json:"email"` Name string `json:"name"` Active bool `json:"active"` CreatedAt time.Time `json:"created_at"` } var ( ErrUserNotFound = errors.New("user not found") ErrUserExists = errors.New("user already exists") ) type Repository interface { Create(ctx context.Context, user *User) error GetByID(ctx context.Context, id string) (*User, error) GetByEmail(ctx context.Context, email string) (*User, error) Update(ctx context.Context, user *User) error } // user-service/internal/service/user_service.go package service import ( "context" "fmt" "time" "user-service/internal/domain" ) type UserService struct { repo domain.Repository } func NewUserService(repo domain.Repository) *UserService { return &UserService{repo: repo} } func (s *UserService) CreateUser(ctx context.Context, email, name string) (*domain.User, error) { existing, _ := s.repo.GetByEmail(ctx, email) if existing != nil { return nil, domain.ErrUserExists } user := &domain.User{ ID: generateID(), Email: email, Name: name, Active: true, CreatedAt: time.Now(), } if err := s.repo.Create(ctx, user); err != nil { return nil, fmt.Errorf("failed to create user: %w", err) } return user, nil } func (s *UserService) GetUser(ctx context.Context, id string) (*domain.User, error) { return s.repo.GetByID(ctx, id) } func (s *UserService) ValidateUser(ctx context.Context, id string) (bool, error) { user, err := s.repo.GetByID(ctx, id) if err != nil { return false, err } return user.Active, nil } func generateID() string { return fmt.Sprintf("user_%d", time.Now().UnixNano()) } // user-service/internal/handlers/http_handler.go package handlers import ( "encoding/json" "net/http" "github.com/gorilla/mux" "user-service/internal/service" ) type HTTPHandler struct { service *service.UserService } func NewHTTPHandler(service *service.UserService) *HTTPHandler { return &HTTPHandler{service: service} } type CreateUserRequest struct { Email string `json:"email"` Name string `json:"name"` } func (h *HTTPHandler) CreateUser(w http.ResponseWriter, r *http.Request) { var req CreateUserRequest if err := json.NewDecoder(r.Body).Decode(&req); err != nil { respondError(w, http.StatusBadRequest, "invalid request") return } user, err := h.service.CreateUser(r.Context(), req.Email, req.Name) if err != nil { respondError(w, http.StatusBadRequest, err.Error()) return } respondJSON(w, http.StatusCreated, user) } func (h *HTTPHandler) GetUser(w http.ResponseWriter, r *http.Request) { vars := mux.Vars(r) id := vars["id"] user, err := h.service.GetUser(r.Context(), id) if err != nil { respondError(w, http.StatusNotFound, "user not found") return } respondJSON(w, http.StatusOK, user) } func (h *HTTPHandler) ValidateUser(w http.ResponseWriter, r *http.Request) { vars := mux.Vars(r) id := vars["id"] valid, err := h.service.ValidateUser(r.Context(), id) if err != nil { respondError(w, http.StatusNotFound, "user not found") return } respondJSON(w, http.StatusOK, map[string]bool{"valid": valid}) } func respondJSON(w http.ResponseWriter, status int, data interface{}) { w.Header().Set("Content-Type", "application/json") w.WriteHeader(status) json.NewEncoder(w).Encode(data) } func respondError(w http.ResponseWriter, status int, message string) { respondJSON(w, status, map[string]string{"error": message}) } // user-service/cmd/server/main.go package main import ( "database/sql" "log" "net/http" "os" "github.com/gorilla/mux" _ "github.com/lib/pq" "user-service/internal/handlers" "user-service/internal/repository" "user-service/internal/service" ) func main() { dbURL := os.Getenv("DATABASE_URL") if dbURL == "" { dbURL = "postgres://user:pass@localhost/users?sslmode=disable" } db, err := sql.Open("postgres", dbURL) if err != nil { log.Fatal(err) } defer db.Close() repo := repository.NewPostgresRepository(db) svc := service.NewUserService(repo) handler := handlers.NewHTTPHandler(svc) router := mux.NewRouter() router.HandleFunc("/users", handler.CreateUser).Methods("POST") router.HandleFunc("/users/{id}", handler.GetUser).Methods("GET") router.HandleFunc("/users/{id}/validate", handler.ValidateUser).Methods("GET") router.HandleFunc("/health", func(w http.ResponseWriter, r *http.Request) { w.WriteHeader(http.StatusOK) w.Write([]byte("OK")) }) port := os.Getenv("PORT") if port == "" { port = "8081" } log.Printf("User service starting on port %s", port) if err := http.ListenAndServe(":"+port, router); err != nil { log.Fatal(err) } } Service 2: Product Service // product-service/internal/domain/product.go package domain import ( "context" "errors" "time" ) type Product struct { ID string `json:"id"` Name string `json:"name"` Description string `json:"description"` Price float64 `json:"price"` Stock int `json:"stock"` CreatedAt time.Time `json:"created_at"` } var ( ErrProductNotFound = errors.New("product not found") ErrInsufficientStock = errors.New("insufficient stock") ) type Repository interface { Create(ctx context.Context, product *Product) error GetByID(ctx context.Context, id string) (*Product, error) Update(ctx context.Context, product *Product) error ReserveStock(ctx context.Context, id string, quantity int) error } // product-service/internal/service/product_service.go package service import ( "context" "fmt" "time" "product-service/internal/domain" ) type ProductService struct { repo domain.Repository } func NewProductService(repo domain.Repository) *ProductService { return &ProductService{repo: repo} } func (s *ProductService) CreateProduct(ctx context.Context, name, desc string, price float64, stock int) (*domain.Product, error) { product := &domain.Product{ ID: generateID(), Name: name, Description: desc, Price: price, Stock: stock, CreatedAt: time.Now(), } if err := s.repo.Create(ctx, product); err != nil { return nil, fmt.Errorf("failed to create product: %w", err) } return product, nil } func (s *ProductService) GetProduct(ctx context.Context, id string) (*domain.Product, error) { return s.repo.GetByID(ctx, id) } func (s *ProductService) ReserveStock(ctx context.Context, id string, quantity int) error { product, err := s.repo.GetByID(ctx, id) if err != nil { return err } if product.Stock < quantity { return domain.ErrInsufficientStock } return s.repo.ReserveStock(ctx, id, quantity) } func generateID() string { return fmt.Sprintf("product_%d", time.Now().UnixNano()) } // product-service/cmd/server/main.go package main import ( "database/sql" "encoding/json" "log" "net/http" "os" "github.com/gorilla/mux" _ "github.com/lib/pq" "product-service/internal/repository" "product-service/internal/service" ) func main() { dbURL := os.Getenv("DATABASE_URL") if dbURL == "" { dbURL = "postgres://user:pass@localhost/products?sslmode=disable" } db, err := sql.Open("postgres", dbURL) if err != nil { log.Fatal(err) } defer db.Close() repo := repository.NewPostgresRepository(db) svc := service.NewProductService(repo) router := mux.NewRouter() router.HandleFunc("/products/{id}", func(w http.ResponseWriter, r *http.Request) { vars := mux.Vars(r) product, err := svc.GetProduct(r.Context(), vars["id"]) if err != nil { http.Error(w, err.Error(), http.StatusNotFound) return } json.NewEncoder(w).Encode(product) }).Methods("GET") router.HandleFunc("/products/reserve", func(w http.ResponseWriter, r *http.Request) { var req struct { ProductID string `json:"product_id"` Quantity int `json:"quantity"` } if err := json.NewDecoder(r.Body).Decode(&req); err != nil { http.Error(w, err.Error(), http.StatusBadRequest) return } if err := svc.ReserveStock(r.Context(), req.ProductID, req.Quantity); err != nil { http.Error(w, err.Error(), http.StatusBadRequest) return } w.WriteHeader(http.StatusOK) json.NewEncoder(w).Encode(map[string]string{"status": "reserved"}) }).Methods("POST") router.HandleFunc("/health", func(w http.ResponseWriter, r *http.Request) { w.WriteHeader(http.StatusOK) }) port := os.Getenv("PORT") if port == "" { port = "8082" } log.Printf("Product service starting on port %s", port) if err := http.ListenAndServe(":"+port, router); err != nil { log.Fatal(err) } } Service 3: Order Service (Orchestrator) // order-service/internal/clients/user_client.go package clients import ( "context" "encoding/json" "fmt" "net/http" "time" ) type UserClient struct { baseURL string httpClient *http.Client } func NewUserClient(baseURL string) *UserClient { return &UserClient{ baseURL: baseURL, httpClient: &http.Client{ Timeout: 5 * time.Second, }, } } func (c *UserClient) ValidateUser(ctx context.Context, userID string) (bool, error) { url := fmt.Sprintf("%s/users/%s/validate", c.baseURL, userID) req, err := http.NewRequestWithContext(ctx, "GET", url, nil) if err != nil { return false, err } resp, err := c.httpClient.Do(req) if err != nil { return false, fmt.Errorf("failed to call user service: %w", err) } defer resp.Body.Close() if resp.StatusCode != http.StatusOK { return false, fmt.Errorf("user service returned status %d", resp.StatusCode) } var result struct { Valid bool `json:"valid"` } if err := json.NewDecoder(resp.Body).Decode(&result); err != nil { return false, err } return result.Valid, nil } // order-service/internal/clients/product_client.go package clients import ( "bytes" "context" "encoding/json" "fmt" "net/http" "time" ) type Product struct { ID string `json:"id"` Name string `json:"name"` Price float64 `json:"price"` Stock int `json:"stock"` } type ProductClient struct { baseURL string httpClient *http.Client } func NewProductClient(baseURL string) *ProductClient { return &ProductClient{ baseURL: baseURL, httpClient: &http.Client{ Timeout: 5 * time.Second, }, } } func (c *ProductClient) GetProduct(ctx context.Context, productID string) (*Product, error) { url := fmt.Sprintf("%s/products/%s", c.baseURL, productID) req, err := http.NewRequestWithContext(ctx, "GET", url, nil) if err != nil { return nil, err } resp, err := c.httpClient.Do(req) if err != nil { return nil, fmt.Errorf("failed to call product service: %w", err) } defer resp.Body.Close() if resp.StatusCode != http.StatusOK { return nil, fmt.Errorf("product not found") } var product Product if err := json.NewDecoder(resp.Body).Decode(&product); err != nil { return nil, err } return &product, nil } func (c *ProductClient) ReserveStock(ctx context.Context, productID string, quantity int) error { url := fmt.Sprintf("%s/products/reserve", c.baseURL) reqBody := map[string]interface{}{ "product_id": productID, "quantity": quantity, } body, err := json.Marshal(reqBody) if err != nil { return err } req, err := http.NewRequestWithContext(ctx, "POST", url, bytes.NewBuffer(body)) if err != nil { return err } req.Header.Set("Content-Type", "application/json") resp, err := c.httpClient.Do(req) if err != nil { return fmt.Errorf("failed to reserve stock: %w", err) } defer resp.Body.Close() if resp.StatusCode != http.StatusOK { return fmt.Errorf("failed to reserve stock: status %d", resp.StatusCode) } return nil } // order-service/internal/service/order_service.go package service import ( "context" "fmt" "time" "order-service/internal/clients" "order-service/internal/domain" ) type OrderService struct { repo domain.Repository userClient *clients.UserClient productClient *clients.ProductClient } func NewOrderService( repo domain.Repository, userClient *clients.UserClient, productClient *clients.ProductClient, ) *OrderService { return &OrderService{ repo: repo, userClient: userClient, productClient: productClient, } } func (s *OrderService) CreateOrder(ctx context.Context, userID string, items []domain.OrderItem) (*domain.Order, error) { // Validate user via User Service valid, err := s.userClient.ValidateUser(ctx, userID) if err != nil { return nil, fmt.Errorf("failed to validate user: %w", err) } if !valid { return nil, fmt.Errorf("user is not valid") } // Validate products and calculate total var total float64 for i, item := range items { product, err := s.productClient.GetProduct(ctx, item.ProductID) if err != nil { return nil, fmt.Errorf("failed to get product: %w", err) } items[i].Price = product.Price total += product.Price * float64(item.Quantity) } // Reserve stock via Product Service for _, item := range items { if err := s.productClient.ReserveStock(ctx, item.ProductID, item.Quantity); err != nil { return nil, fmt.Errorf("failed to reserve stock: %w", err) } } order := &domain.Order{ ID: generateID(), UserID: userID, Items: items, Total: total, Status: "pending", CreatedAt: time.Now(), } if err := s.repo.Create(ctx, order); err != nil { return nil, fmt.Errorf("failed to create order: %w", err) } return order, nil } func generateID() string { return fmt.Sprintf("order_%d", time.Now().UnixNano()) } // order-service/cmd/server/main.go package main import ( "database/sql" "encoding/json" "log" "net/http" "os" "github.com/gorilla/mux" _ "github.com/lib/pq" "order-service/internal/clients" "order-service/internal/domain" "order-service/internal/repository" "order-service/internal/service" ) func main() { dbURL := os.Getenv("DATABASE_URL") if dbURL == "" { dbURL = "postgres://user:pass@localhost/orders?sslmode=disable" } userServiceURL := os.Getenv("USER_SERVICE_URL") if userServiceURL == "" { userServiceURL = "http://localhost:8081" } productServiceURL := os.Getenv("PRODUCT_SERVICE_URL") if productServiceURL == "" { productServiceURL = "http://localhost:8082" } db, err := sql.Open("postgres", dbURL) if err != nil { log.Fatal(err) } defer db.Close() repo := repository.NewPostgresRepository(db) userClient := clients.NewUserClient(userServiceURL) productClient := clients.NewProductClient(productServiceURL) svc := service.NewOrderService(repo, userClient, productClient) router := mux.NewRouter() router.HandleFunc("/orders", func(w http.ResponseWriter, r *http.Request) { var req struct { UserID string `json:"user_id"` Items []domain.OrderItem `json:"items"` } if err := json.NewDecoder(r.Body).Decode(&req); err != nil { http.Error(w, err.Error(), http.StatusBadRequest) return } order, err := svc.CreateOrder(r.Context(), req.UserID, req.Items) if err != nil { http.Error(w, err.Error(), http.StatusBadRequest) return } w.Header().Set("Content-Type", "application/json") w.WriteHeader(http.StatusCreated) json.NewEncoder(w).Encode(order) }).Methods("POST") router.HandleFunc("/health", func(w http.ResponseWriter, r *http.Request) { w.WriteHeader(http.StatusOK) }) port := os.Getenv("PORT") if port == "" { port = "8083" } log.Printf("Order service starting on port %s", port) if err := http.ListenAndServe(":"+port, router); err != nil { log.Fatal(err) } } Docker Compose Setup version: '3.8' services: user-service: build: ./user-service ports: - "8081:8081" environment: - DATABASE_URL=postgres://user:pass@user-db:5432/users?sslmode=disable - PORT=8081 depends_on: - user-db product-service: build: ./product-service ports: - "8082:8082" environment: - DATABASE_URL=postgres://user:pass@product-db:5432/products?sslmode=disable - PORT=8082 depends_on: - product-db order-service: build: ./order-service ports: - "8083:8083" environment: - DATABASE_URL=postgres://user:pass@order-db:5432/orders?sslmode=disable - USER_SERVICE_URL=http://user-service:8081 - PRODUCT_SERVICE_URL=http://product-service:8082 - PORT=8083 depends_on: - order-db - user-service - product-service user-db: image: postgres:15 environment: - POSTGRES_USER=user - POSTGRES_PASSWORD=pass - POSTGRES_DB=users product-db: image: postgres:15 environment: - POSTGRES_USER=user - POSTGRES_PASSWORD=pass - POSTGRES_DB=products order-db: image: postgres:15 environment: - POSTGRES_USER=user - POSTGRES_PASSWORD=pass - POSTGRES_DB=orders Best Practices Service Boundaries: Define clear service boundaries based on business capabilities API Contracts: Use API versioning and maintain backward compatibility Service Discovery: Implement service registry for dynamic service location Circuit Breakers: Prevent cascading failures with circuit breaker pattern Distributed Tracing: Implement tracing to debug cross-service calls Health Checks: Provide health endpoints for monitoring Configuration Management: Externalize configuration Security: Implement service-to-service authentication Common Pitfalls Distributed Monolith: Services too tightly coupled, defeating the purpose Chatty Services: Too many synchronous calls between services Shared Database: Multiple services accessing the same database Ignoring Network Failures: Not handling network errors gracefully No Service Versioning: Breaking changes without versioning Data Consistency Issues: Not handling eventual consistency Over-Engineering: Creating too many small services When to Use Microservices Architecture Use When: ...

    January 25, 2025 · 13 min · Rafiul Alam

    Visual Guide to Distributed Systems Patterns

    Introduction Building robust distributed systems requires understanding fundamental patterns that solve common challenges like consensus, fault tolerance, request distribution, and asynchronous communication. This comprehensive guide uses visual diagrams to illustrate how these patterns work, making complex distributed systems concepts easier to understand and implement. We’ll explore: Raft Consensus Algorithm: How distributed systems agree on shared state Circuit Breaker Pattern: Preventing cascading failures in microservices Load Balancing Algorithms: Distributing traffic efficiently across servers Message Queue Patterns: Asynchronous communication strategies Part 1: Raft Consensus Algorithm The Raft consensus algorithm ensures that a cluster of servers maintains a consistent, replicated log even in the face of failures. It’s designed to be more understandable than Paxos while providing the same guarantees. ...

    January 17, 2025 · 24 min · Rafiul Alam

    ConnectRPC: Why Smart Teams Are Betting on ConnectRPC Over Standard gRPC

    The RPC Evolution: From gRPC to ConnectRPC While gRPC has been the go-to choice for high-performance RPC communication, ConnectRPC is emerging as a compelling alternative that addresses many of gRPC’s pain points while maintaining compatibility with existing ecosystems. What is ConnectRPC? ConnectRPC (formerly known as Connect) is a protocol that provides a simpler, more flexible approach to RPC. It’s designed to be: HTTP/1.1 and HTTP/2 compatible - Works with existing infrastructure Browser-friendly - No need for special proxies gRPC compatible - Can interoperate with gRPC services Developer-friendly - Simpler tooling and better debugging Why Teams Choose ConnectRPC Over gRPC 1. Browser Compatibility Without Proxies gRPC requires special proxies (like grpc-web) to work with browsers because it relies on HTTP/2-specific features. ConnectRPC works directly in browsers using standard fetch APIs. ...

    January 15, 2025 · 8 min · Rafiul Alam

    The Two-Tier API Strategy: Why You Need Both REST and RPC (and How to Manage It)

    The API Dilemma: REST vs RPC? For years, teams have debated REST vs RPC as if they were mutually exclusive choices. The truth? You need both. Modern applications benefit from a two-tier API strategy that leverages REST for external clients and RPC for internal services. This isn’t about choosing sides-it’s about using the right tool for each job. Understanding the Two Tiers Tier 1: REST for External APIs (The Public Face) Use REST when: ...

    January 10, 2025 · 12 min · Rafiul Alam

    Distributed Tracing in Go

    Go Concurrency Patterns Series: ← Go Generics Patterns | Series Overview What is Distributed Tracing? Distributed tracing tracks requests as they flow through microservices, providing visibility into performance bottlenecks, service dependencies, and error propagation in distributed systems. Key Concepts: Trace: End-to-end journey of a request across services Span: Single unit of work within a trace Context Propagation: Carrying trace information across boundaries Sampling: Controlling which traces to collect Why OpenTelemetry? OpenTelemetry (OTel) is the industry standard for observability: ...

    June 29, 2024 · 10 min · Rafiul Alam

    Context Propagation Patterns in Go

    Go Concurrency Patterns Series: ← Circuit Breaker | Series Overview | Memory Model → What is Context Propagation? Context propagation is the practice of threading context.Context through your application to carry cancellation signals, deadlines, and request-scoped values across API boundaries, goroutines, and service boundaries. This is critical for building observable, responsive distributed systems. Key Capabilities: Distributed Tracing: Propagate trace IDs across services Cancellation Cascades: Cancel entire request trees Deadline Enforcement: Ensure requests complete within time budgets Request-Scoped Values: Carry metadata without polluting function signatures Real-World Use Cases Microservices: Trace requests across multiple services API Gateways: Propagate timeouts and user context Database Layers: Cancel queries when requests are abandoned Message Queues: Propagate processing deadlines HTTP Middleware: Extract and inject trace headers gRPC Services: Automatic context propagation Basic Context Propagation Propagating Through Function Calls package main import ( "context" "fmt" "time" ) // ServiceA calls ServiceB which calls ServiceC // Context propagates through all layers func ServiceA(ctx context.Context, userID string) error { // Add request-scoped value ctx = context.WithValue(ctx, "user_id", userID) ctx = context.WithValue(ctx, "request_id", generateRequestID()) fmt.Printf("[ServiceA] Processing request for user: %s\n", userID) // Propagate context to next service return ServiceB(ctx) } func ServiceB(ctx context.Context) error { // Retrieve values from context userID := ctx.Value("user_id").(string) requestID := ctx.Value("request_id").(string) fmt.Printf("[ServiceB] User: %s, Request: %s\n", userID, requestID) // Add timeout for downstream call ctx, cancel := context.WithTimeout(ctx, 2*time.Second) defer cancel() return ServiceC(ctx) } func ServiceC(ctx context.Context) error { userID := ctx.Value("user_id").(string) requestID := ctx.Value("request_id").(string) fmt.Printf("[ServiceC] Processing for User: %s, Request: %s\n", userID, requestID) // Simulate work select { case <-time.After(1 * time.Second): fmt.Println("[ServiceC] Work completed") return nil case <-ctx.Done(): fmt.Printf("[ServiceC] Cancelled: %v\n", ctx.Err()) return ctx.Err() } } func generateRequestID() string { return fmt.Sprintf("req-%d", time.Now().UnixNano()) } func main() { ctx := context.Background() err := ServiceA(ctx, "user-123") if err != nil { fmt.Printf("Error: %v\n", err) } } Output: ...

    June 24, 2024 · 11 min · Rafiul Alam