BSc CSIT (TU) Science Cloud Computing (BSc CSIT, CSC465) Question Paper 2078 Nepal

MapReduce Programming Model

MapReduce is a programming model (introduced by Google) for processing very large data sets in parallel across a distributed cluster of commodity machines. The programmer expresses computation as two functions:

• Map(k1, v1) -> list(k2, v2): processes an input split and emits intermediate key/value pairs.
• Reduce(k2, list(v2)) -> list(v3): aggregates all values that share the same intermediate key.

Between the two phases the framework automatically performs shuffle and sort, grouping all intermediate values by key and sending each group to a reducer. The framework also handles partitioning, scheduling, parallelism, and fault tolerance, so the developer only writes the two functions.

Example: Word Count

# Map: emit (word, 1) for each word in a line
def map(key, line):
    for word in line.split():
        emit(word, 1)

# Reduce: sum the counts for each word
def reduce(word, counts):
    emit(word, sum(counts))

Input line "the cat the dog" -> Map emits (the,1),(cat,1),(the,1),(dog,1) -> after shuffle the->[1,1], cat->[1], dog->[1] -> Reduce outputs the=2, cat=1, dog=1.

Hadoop Architecture

Apache Hadoop is the open-source implementation of MapReduce + a distributed file system. Its core has two layers:

1. HDFS (storage layer) – stores data reliably across the cluster.
2. YARN / MapReduce (processing layer) – schedules and runs jobs.

Daemons (with YARN):

• NameNode – master that holds the filesystem namespace and metadata (which blocks belong to which file, and where each block replica is stored).
• DataNodes – slaves that actually store data blocks and serve read/write requests.
• ResourceManager – cluster-wide scheduler that allocates resources (containers).
• NodeManager – per-node agent that launches and monitors containers.
• ApplicationMaster – per-job manager that negotiates resources and coordinates map/reduce tasks.

HDFS (Hadoop Distributed File System)

HDFS is designed for write-once, read-many access to huge files on commodity hardware:

• Files are split into large blocks (default 128 MB).
• Each block is replicated (default factor 3) on different DataNodes for fault tolerance.
• The NameNode stores metadata in memory; DataNodes store actual blocks and send periodic heartbeats and block reports.
• If a DataNode fails, the NameNode re-replicates its blocks elsewhere to maintain the replication factor.
• It follows data locality – computation is moved to the node holding the data, reducing network traffic.

This combination lets Hadoop process petabyte-scale big data economically in the cloud, with built-in scalability and fault tolerance.

Security Issues and Challenges in Cloud Computing

Because data and applications are hosted on infrastructure owned by a third party and shared by many tenants, cloud computing introduces several security concerns:

• Loss of control / data location: users do not know exactly where data is physically stored, raising jurisdiction and compliance issues.
• Multi-tenancy & isolation failure: other tenants share the same physical resources; weak isolation can lead to data leakage or side-channel attacks.
• Data breaches and data loss: misconfiguration or attacks can expose or destroy data.
• Insecure interfaces/APIs exposed to the Internet.
• Account or service hijacking (stolen credentials, phishing).
• Insider threats from provider staff.
• Compliance and legal challenges (GDPR, audit, vendor lock-in).
• Denial-of-Service (DoS) attacks against shared infrastructure.

Data Security

Protecting data in three states:

• Data at rest: encrypt stored data (e.g., AES-256), use encrypted volumes and key management.
• Data in transit: use TLS/SSL, VPN, IPSec to protect data moving over the network.
• Data in use: access controls, tokenization, and emerging techniques like homomorphic/confidential computing.

Other measures include data classification, backups, secure deletion, and data masking.

Identity and Access Management (IAM)

IAM ensures the right entities get the right access to the right resources:

• Authentication – verifying identity (passwords, Multi-Factor Authentication, single sign-on, federated identity such as SAML/OAuth/OpenID).
• Authorization – granting permissions via RBAC (Role-Based Access Control) or ABAC, applying the principle of least privilege.
• Accounting/Auditing – logging and monitoring who accessed what and when.

Ensuring Confidentiality and Integrity

• Confidentiality: symmetric/asymmetric encryption, key management services (KMS), access control, tokenization.
• Integrity: hash functions (SHA-256), Message Authentication Codes (MAC/HMAC), and digital signatures to detect tampering; checksums and versioning for stored objects.
• Availability is preserved through redundancy and DoS protection (completing the CIA triad).

Together these controls give defense-in-depth that protects cloud data against unauthorized access and modification.

Service Level Agreement (SLA)

A Service Level Agreement (SLA) is a formal, contractual document between a cloud service provider and a customer that defines the level of service to be delivered, the metrics by which it is measured, and the remedies or penalties if the agreed levels are not met. It sets clear, measurable expectations and is the basis for trust between both parties.

Components of an SLA

1. Service description – the services covered and their scope.
2. Service-level objectives (SLOs) – measurable targets such as availability/uptime (e.g., 99.9%), response time, throughput, and latency.
3. Performance metrics & measurement method – how each metric is measured and reported.
4. Roles and responsibilities of provider and customer.
5. Penalties / service credits – compensation (often as credits) when SLOs are violated.
6. Exclusions – conditions not covered (e.g., scheduled maintenance, force majeure).
7. Security, compliance and data-handling terms.
8. Reporting, review and termination clauses.

Managing and Monitoring QoS

Quality of Service (QoS) is the measured ability of the cloud to meet the agreed SLOs (availability, performance, reliability). It is managed and monitored as follows:

• Monitoring & metering: continuous collection of metrics (CPU, memory, latency, response time, uptime) using monitoring tools/agents and dashboards (e.g., CloudWatch-style services).
• SLA monitoring: comparing live metrics against SLO thresholds; alerts are raised on violations.
• Auto-scaling & load balancing: dynamically add/remove resources to keep performance within targets under varying load.
• Resource provisioning & admission control: allocate resources to honour QoS guarantees.
• Redundancy & failover: maintain availability targets.
• Reporting & penalties: periodic QoS reports; service credits applied when SLAs are breached.

Effective QoS management therefore combines real-time monitoring, automated elasticity, and contractual enforcement to ensure the promised SLA is consistently delivered.

Elasticity is the ability of a cloud system to automatically and dynamically provision and de-provision resources (CPU, memory, instances) in real time to match the current workload, so that capacity closely follows demand — scaling out when load rises and scaling back in when load falls. This avoids both over-provisioning and under-provisioning, and underpins the pay-as-you-go model.

Difference from scalability:

In short, scalability is the capability of a system to grow to handle increased load, while elasticity is the automatic, on-demand matching of resources to load at any moment.

Challenges in Migrating an Application to the Cloud

• Application compatibility / re-architecting: legacy or monolithic apps may need refactoring (the "rehost vs re-platform vs re-architect" choice) to run well in the cloud.
• Data migration: moving large volumes of data is slow and risky; requires bandwidth planning, downtime minimization, and integrity validation.
• Security and compliance: ensuring data protection, encryption, and meeting regulatory/data-residency requirements in the new environment.
• Vendor lock-in: dependence on provider-specific services and APIs makes future portability difficult.
• Downtime and business continuity: minimizing service interruption during cutover.
• Performance and latency: network latency between users, services, and data can degrade performance.
• Cost estimation & management: unexpected costs from data transfer, storage, and running instances.
• Skill gap and management complexity: staff may lack cloud expertise; need new monitoring and operations practices.
• Integration and dependencies: existing on-premise integrations must keep working during/after migration.

A phased strategy (assessment, pilot, migrate, optimize) with testing and rollback plans helps overcome these challenges.

Cloud Storage

Cloud storage is a model in which data is stored on remote servers maintained by a cloud provider and accessed over the network/Internet, offered on demand with scalability, durability (via replication), and pay-as-you-go pricing, with management handled by the provider.

Types of Cloud Storage

1. Block Storage

• Data is stored in fixed-size blocks, each with an address but no metadata; the OS treats it like a raw disk volume.
• Offers high performance and low latency; good for databases, transactional apps, and boot/OS volumes.
• Example: Amazon EBS. Attached to a single VM at a time.

2. File Storage

• Data is stored as files in a hierarchy of folders/directories and accessed via file protocols (NFS, SMB/CIFS).
• Easy to share across multiple clients; good for shared file systems and content repositories.
• Example: Amazon EFS, Azure Files.

3. Object Storage

• Data is stored as objects in a flat namespace; each object bundles the data + rich metadata + a unique ID, accessed via HTTP/REST APIs.
• Highly scalable and durable, ideal for unstructured data (images, videos, backups, big data). Not suited for frequent in-place edits.
• Example: Amazon S3, Azure Blob Storage.

Summary: block = raw high-performance volumes, file = shared hierarchical file systems, object = massively scalable metadata-rich storage for unstructured data.

Google App Engine (PaaS)

Google App Engine (GAE) is a Platform-as-a-Service (PaaS) offering from Google Cloud that lets developers build and deploy web applications and backends without managing the underlying servers, OS, or infrastructure — Google handles provisioning, patching, and scaling.

Key features:

• Fully managed serverless platform: developers upload code; Google runs and scales it.
• Automatic scaling: scales instances up or down automatically based on traffic (down to zero with no load).
• Multi-language support: Python, Java, Go, Node.js, PHP, Ruby, etc. (Standard and Flexible environments).
• Built-in services: integrates with Datastore/Firestore, Cloud SQL, Memcache, Task Queues, Cloud Storage, and user authentication.
• Pay-as-you-go: billed for resources actually consumed (instance hours, requests).
• Versioning & traffic splitting: deploy multiple versions and route traffic for testing/rollback.

Advantages: fast development, no server management, automatic load balancing and scaling, high availability.

Limitations: some vendor lock-in to Google APIs, restrictions in the Standard environment (sandbox), and less low-level control than IaaS.

Thus GAE typifies PaaS by providing a complete application platform so developers focus only on code.

Architecture of a Cloud Data Center

A cloud data center is a large facility of networked computing, storage, and network resources that delivers cloud services. Its architecture is organized in layers:

1. Physical / Facility layer

• Building with power supply (UPS, generators), cooling (HVAC), fire suppression, and physical security.
• Redundant power and cooling for high availability.

2. Compute layer

• Racks of servers grouped into clusters/pods. Each server runs a hypervisor that hosts many virtual machines / containers (virtualization enables multi-tenancy and resource pooling).

3. Storage layer

• Storage Area Networks (SAN), NAS, and distributed/object storage providing block, file, and object storage with replication for durability.

4. Network layer

• A typically hierarchical / leaf-spine topology: servers connect to Top-of-Rack (ToR) switches, which connect to aggregation/spine switches, then to core routers and the Internet.
• Includes load balancers, firewalls, and SDN for flexible, high-bandwidth, low-latency connectivity.

5. Virtualization & management layer

• Virtualization software, orchestration, and a management/control plane that handle provisioning, scheduling, monitoring, billing, auto-scaling, and fault management.

Design goals: scalability, high availability/redundancy, fault tolerance, energy efficiency (PUE), and security. Modern designs use modular/pod-based layouts so capacity can grow incrementally.

Multi-Tenancy

Multi-tenancy is an architecture in which a single instance of an application and its supporting infrastructure serves multiple customers (tenants) simultaneously, while keeping each tenant's data and configuration logically isolated from the others. It is a defining characteristic of the SaaS model and enables efficient resource sharing and lower cost per tenant.

Implementation Approaches in SaaS

Data isolation is commonly achieved at one of three levels:

1. Separate databases – each tenant has its own database (strong isolation, higher cost).
2. Shared database, separate schemas – one database, a schema per tenant.
3. Shared database, shared schema – one set of tables with a tenant_id column distinguishing rows (most efficient, hardest to isolate).

Implementation Issues / Challenges

• Data isolation and security: preventing one tenant from accessing another's data; a single bug can leak data across tenants.
• Performance & noisy neighbour: one heavy tenant can degrade performance for others; needs throttling/quotas.
• Customization: tenants want different configurations/branding on a shared codebase.
• Scalability & resource allocation: fairly distributing shared resources as tenants grow.
• Maintenance & upgrades: a single update affects all tenants at once; must avoid breaking any tenant.
• Backup, recovery and SLA per tenant: per-tenant compliance and data-residency requirements.

Balancing efficiency (sharing) against isolation/security is the central challenge of multi-tenant SaaS design.

Pay-As-You-Go Billing and Metering

Pay-as-you-go (PAYG), also called utility/consumption-based pricing, is a cloud billing model in which customers pay only for the resources they actually consume, with no large upfront cost or long-term commitment — similar to paying for electricity or water.

Metering

Metering is the process of continuously measuring and recording resource usage per customer. Typical metered units include:

• Compute: instance/CPU hours (or per-second), vCPU-hours.
• Storage: GB-months stored.
• Network: GB of data transfer (especially egress).
• Requests/operations: number of API calls, function invocations.
• Other services: database transactions, IP addresses, etc.

The metering subsystem collects these usage records (often via monitoring agents) reliably and accurately.

Billing

The billing system applies the provider's rate/price per unit to the metered usage over the billing period and generates an invoice:

Benefits

• No upfront capital expense (CapEx -> OpEx).
• Cost efficiency: pay only for what is used; idle resources cost nothing.
• Elasticity-aligned: scales cost with demand.
• Transparency: detailed usage/cost reports.

Variations include reserved/spot pricing and free tiers, but the core idea is consumption-based, metered charging.

Fault Tolerance and High Availability in the Cloud

Fault tolerance is the ability of a system to keep operating correctly even when some components fail, while high availability (HA) means the service remains accessible for a very high percentage of time (e.g., 99.99% uptime). Clouds achieve these through:

• Redundancy: duplicate servers, storage, network paths and power so there is no single point of failure (N+1 / 2N).
• Replication: data is replicated across multiple nodes and Availability Zones / regions so a copy survives a failure.
• Multiple Availability Zones & geographic distribution: deploying across independent zones protects against datacenter-level outages.
• Load balancing: distributes traffic across healthy instances and routes around failed ones.
• Automatic failover: standby resources take over when the primary fails, often transparently.
• Health monitoring & auto-healing: continuous health checks detect failures and automatically restart/replace unhealthy instances (auto-scaling groups).
• Backups, snapshots and checkpointing: enable recovery of data/state.
• Stateless / distributed design: stateless services and clustering let any node serve any request.
• Disaster recovery (DR) plans with defined RTO/RPO.

Together, redundancy + replication + automatic failover + monitoring let the cloud mask hardware/software failures and deliver continuous, highly available service.

Containerization

Containerization is a lightweight OS-level virtualization technique that packages an application together with its dependencies, libraries, and configuration into a single portable unit called a container. Containers share the host operating system kernel but run in isolated user spaces, so they are fast, portable, and consistent across environments ("build once, run anywhere"). Docker is the common runtime and Kubernetes is the popular orchestrator.

Containers vs Virtual Machines

Summary: A VM virtualizes the hardware and runs a complete guest OS via a hypervisor, giving strong isolation but high overhead. A container virtualizes the OS, sharing the host kernel, giving much lower overhead, faster startup, and greater density — at the cost of slightly weaker isolation. Containers and VMs are often used together (containers running inside VMs in the cloud).