Key Concepts in Distributed Computing Architectures

Why This Matters

Understanding distributed computing architectures is about recognizing why different systems are designed the way they are and when to apply each model. You'll be tested on your ability to analyze trade-offs between centralization vs. decentralization, latency vs. throughput, and scalability vs. complexity. These architectures form the backbone of everything from web applications to scientific simulations, and exam questions frequently ask you to compare approaches or recommend solutions for specific scenarios.

Each architecture solves a particular set of problems while introducing its own limitations. Whether you're dealing with fault tolerance, resource sharing, real-time processing, or service modularity, the architecture you choose determines your system's behavior under load, failure, and growth. Don't just memorize what each architecture does. Know what problem it solves and what trade-offs it accepts.

---

Centralized vs. Decentralized Control Models

The fundamental architectural decision in distributed systems is where control and coordination live. Centralized models simplify management but create bottlenecks and single points of failure; decentralized models distribute risk but increase coordination complexity.

Client-Server Architecture

• Centralized request-response model — clients initiate requests, dedicated servers process and respond, creating a clear separation of concerns. Think of a web browser (client) talking to a web server: the server holds the data and logic, the client just renders results.
• Scalability through server replication — horizontal scaling adds servers behind load balancers to handle increased demand. The clients don't need to know how many servers exist behind the scenes.
• Simplified security and data management — centralized storage enables consistent access control, backup policies, and data integrity checks. Having one authoritative data source avoids the synchronization headaches of distributed data.

Peer-to-Peer (P2P) Architecture

• Fully decentralized model — every node acts as both client and server, eliminating hierarchical dependencies. There's no "master" node that the system depends on.
• No single point of failure — system resilience increases as peers join; the network survives individual node failures because no single node is essential.
• Self-scaling resource pool — each new peer contributes storage, bandwidth, or compute capacity, so the system grows stronger with adoption. BitTorrent file sharing and blockchain networks are classic examples.

The trade-off: P2P systems are harder to secure and govern precisely because there's no central authority. Malicious nodes can join freely, and enforcing consistent policies across all peers is difficult.

Cluster Computing

• Tightly coupled nodes functioning as a unified system — multiple machines (usually in the same data center or rack) share workloads through coordinated scheduling and load balancing.
• High availability through redundancy — if one node fails, others absorb its tasks without service interruption. Failover is fast because nodes are closely connected.
• Optimized for HPC workloads — scientific simulations, rendering, and large-scale data processing benefit from parallel execution across nodes with high-speed interconnects (like InfiniBand) between them.

---

Service-Based Decomposition Patterns

Modern distributed systems often break applications into discrete services that communicate over networks. The granularity and coupling of these services determine flexibility, deployment speed, and operational complexity.

Service-Oriented Architecture (SOA)

• Coarse-grained services communicating via standardized protocols — SOAP, REST, or message queues enable interoperability across platforms. "Coarse-grained" means each service handles a broad chunk of functionality (e.g., an entire "Order Management" service).
• Promotes reusability across applications — services are designed as shared components that multiple systems can invoke. An authentication service, for instance, might serve five different internal applications.
• Legacy system integration — SOA's protocol standardization makes it ideal for connecting older systems with modern applications, which is why it's common in large enterprises with decades of existing software.

Microservices Architecture

• Fine-grained, independently deployable services — each microservice owns a specific business function and its own data store. For example, a "Shopping Cart" microservice has its own database, separate from the "User Profile" microservice.
• Continuous deployment enabled — teams can update individual services without redeploying the entire application. This is a huge advantage for organizations shipping features frequently.
• Technology diversity permitted — different services can use different languages, frameworks, and databases based on their specific needs. Your recommendation engine could run Python with a graph database while your payment service runs Java with a relational database.

The cost: microservices introduce significant operational complexity. You now need service discovery, distributed tracing, inter-service communication handling, and strategies for dealing with partial failures across dozens or hundreds of services.

Distributed Object Architecture

• Object-oriented principles applied to networked components — encapsulation, inheritance, and polymorphism work across machine boundaries. The system treats remote components as objects with defined interfaces.
• Remote method invocation (RMI) — objects call methods on remote objects as if they were local, abstracting away network communication. Java RMI and CORBA are well-known implementations.
• Enhanced modularity — distributed objects can be developed, tested, and reused independently across systems. However, this architecture has largely been superseded by SOA and microservices for new projects.

---

Location-Aware Processing Architectures

Where computation happens matters enormously for latency, bandwidth, and real-time responsiveness. These architectures represent a spectrum from fully centralized (cloud) to fully distributed (edge) processing.

Cloud Computing

• On-demand resources delivered over the internet — the pay-as-you-go model eliminates upfront infrastructure investment. You rent compute, storage, and networking from providers like AWS, Azure, or GCP.
• Elastic scalability — resources scale up or down automatically based on demand, optimizing cost and performance. During a traffic spike, new virtual machines spin up; when traffic drops, they shut down.
• Multiple service models:
  • IaaS (Infrastructure as a Service) provides raw virtual machines, storage, and networking
  • PaaS (Platform as a Service) offers managed development platforms (databases, app hosting)
  • SaaS (Software as a Service) delivers complete applications (Gmail, Slack)

Fog Computing

• Intermediate layer between cloud and edge — computation occurs at network aggregation points (like local servers or smart routers) closer to data sources than the cloud but not directly on the devices themselves.
• Reduces cloud-bound traffic — local preprocessing filters and aggregates data before transmission, lowering bandwidth costs. Instead of sending millions of raw sensor readings to the cloud, a fog node might send hourly summaries.
• Supports IoT at scale — handles the coordination layer for thousands of edge devices while maintaining cloud connectivity for centralized analytics and storage.

Edge Computing

• Processing at or near data generation sources — computation happens on devices, gateways, or local servers rather than remote data centers.
• Minimizes latency for real-time applications — autonomous vehicles, industrial automation, and AR/VR require sub-millisecond response times. Sending data to a cloud server hundreds of miles away and waiting for a response simply isn't fast enough.
• Enhanced data privacy — sensitive information stays local, reducing exposure during transmission and storage. A security camera with on-device face detection never needs to send video to the cloud.

---

Resource Pooling and Shared Infrastructure

Some architectures focus primarily on aggregating distributed resources into unified pools for large-scale computation. The key mechanism is abstracting heterogeneous resources into a coherent, schedulable whole.

Grid Computing

• Geographically distributed resource sharing — connects heterogeneous systems across institutions into a virtual supercomputer. A university in Tokyo and a lab in Geneva can contribute idle machines to the same computational job.
• Designed for compute-intensive batch workloads — scientific simulations, genomics analysis, and physics modeling benefit from massive parallelism. Jobs are typically submitted to a queue and don't require real-time responses.
• Middleware-coordinated scheduling — job schedulers (like HTCondor or Apache YARN) distribute tasks across available nodes based on resource requirements and availability.

---

Quick Reference Table

---

Self-Check Questions

1. Which two architectures both eliminate single points of failure but differ in how tightly coupled their nodes are? Explain the trade-off.

2. A hospital needs to process patient monitoring data in real-time while maintaining long-term records in a central system. Which combination of architectures would you recommend, and why?

3. Compare and contrast SOA and Microservices: What problem does each solve best, and when would you choose one over the other?

4. An FRQ describes a system that must scale automatically with demand, minimize infrastructure costs, and support rapid feature deployment. Which architectures address each requirement?

5. Why might an autonomous vehicle system use edge computing rather than cloud computing for obstacle detection, even though cloud computing offers more computational power?