Designing Idempotent REST APIs in Java & Spring Boot

 

“In distributed systems, things will fail. Idempotency is how you survive that failure gracefully.”

---

1. What Is Idempotency?

In mathematics, an operation is idempotent if applying it multiple times produces the same result as applying it once. The formal definition:

f(f(x)) = f(x)

In REST API design, idempotency means:

Calling the same API endpoint with the same request payload multiple times should produce the same server-side state and (logically) the same response, no matter how many times it’s executed.

This is not just theory — it’s baked into the HTTP specification (RFC 9110). Here’s a breakdown of HTTP method idempotency:

HTTP Method	Idempotent	Safe	Description
GET	✅ Yes	✅ Yes	Read-only, no side effects
HEAD	✅ Yes	✅ Yes	Like GET, no body
OPTIONS	✅ Yes	✅ Yes	Metadata retrieval
PUT	✅ Yes	❌ No	Full resource replace
DELETE	✅ Yes	❌ No	Remove resource
POST	❌ No	❌ No	Creates new resource (by default)
PATCH	❌ No*	❌ No	Partial update (context-dependent)

• PATCH can be made idempotent by design (e.g., SET quantity=5 is idempotent; INCREMENT quantity by 1 is not).

Safe vs. Idempotent — Don’t Confuse Them

• Safe = No server-side state mutation (read-only)
• Idempotent = Can be retried safely; same net effect on state

DELETE /orders/123 is idempotent (first call deletes; subsequent calls still result in “order 123 does not exist”) but NOT safe (it mutates state on first call).

---

2. The Problem This Solves

In modern microservices architectures, network calls fail. A lot. Consider this scenario:

Client ──→ API Gateway ──→ Order Service ──→ Payment Service
                                                    ↓
                                              (Network timeout!)
                                              Did the payment go through?

The client receives a timeout error. It doesn’t know if:

• The request never reached the Payment Service
• The Payment Service processed it but the response was lost
• The request is still being processed

Without idempotency: Retrying creates duplicate payments. Customer is charged twice.

With idempotency: Retrying is safe. The server recognizes “I’ve seen this exact request before” and returns the original result without re-processing.

Real-World Consequences of Non-Idempotent APIs

• Double charges in payment systems (Stripe, PayPal lost millions before enforcing this)
• Duplicate orders in e-commerce platforms
• Ghost records in inventory systems during retry storms
• Data inconsistency across microservices during partial failures
• Cascading failures when upstream services retry without idempotency guards

Industry Standards That Enforce This

• Stripe API uses Idempotency-Key headers for all mutation endpoints
• PayPal REST API uses PayPal-Request-Id
• AWS S3 PUT operations are inherently idempotent
• Kubernetes uses resource version tokens for optimistic concurrency

---

3. Implementation Guide

Project Setup — Spring Boot Microservice

<!-- pom.xml -->
<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-data-jpa</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-data-redis</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-validation</artifactId>
    </dependency>
    <dependency>
        <groupId>org.projectlombok</groupId>
        <artifactId>lombok</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-test</artifactId>
        <scope>test</scope>
    </dependency>
</dependencies>

---

Step 1 — The Idempotency Key Model

// IdempotencyRecord.java
package com.example.idempotency.model;

import jakarta.persistence.*;
import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
import lombok.NoArgsConstructor;

import java.time.Instant;

/**
 * Persists idempotency keys and their associated responses.
 * This acts as a "request journal" — if we've seen this key before,
 * we return the cached result without re-processing.
 */
@Entity
@Table(name = "idempotency_records",
       indexes = @Index(name = "idx_idempotency_key", columnList = "idempotencyKey", unique = true))
@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
public class IdempotencyRecord {

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    @Column(nullable = false, unique = true, length = 128)
    private String idempotencyKey;

    @Column(nullable = false)
    private String requestHash;      // SHA-256 of request body to detect payload mismatch

    @Column(nullable = false, columnDefinition = "TEXT")
    private String responseBody;     // Cached serialized response

    @Column(nullable = false)
    private Integer httpStatus;      // Cached HTTP status code

    @Column(nullable = false)
    private Instant createdAt;

    @Column(nullable = false)
    private Instant expiresAt;       // TTL — clean up old records

    @Enumerated(EnumType.STRING)
    @Column(nullable = false)
    private ProcessingStatus status; // PROCESSING | COMPLETED | FAILED

    public enum ProcessingStatus {
        PROCESSING,  // Request is in-flight (guards against concurrent duplicates)
        COMPLETED,   // Successfully processed and response cached
        FAILED       // Processing failed — allow retry
    }
}

---

Step 2 — Repository Layer

// IdempotencyRecordRepository.java
package com.example.idempotency.repository;

import com.example.idempotency.model.IdempotencyRecord;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.data.jpa.repository.Modifying;
import org.springframework.data.jpa.repository.Query;
import org.springframework.stereotype.Repository;

import java.time.Instant;
import java.util.Optional;

@Repository
public interface IdempotencyRecordRepository extends JpaRepository<IdempotencyRecord, Long> {

    Optional<IdempotencyRecord> findByIdempotencyKey(String idempotencyKey);

    /**
     * Clean up expired records — run via a scheduled job.
     * Prevents the idempotency table from growing unbounded.
     */
    @Modifying
    @Query("DELETE FROM IdempotencyRecord r WHERE r.expiresAt < :now")
    void deleteExpiredRecords(Instant now);
}

---

Step 3 — The Core Idempotency Service

// IdempotencyService.java
package com.example.idempotency.service;

import com.example.idempotency.exception.IdempotencyConflictException;
import com.example.idempotency.exception.IdempotencyKeyMismatchException;
import com.example.idempotency.model.IdempotencyRecord;
import com.example.idempotency.repository.IdempotencyRecordRepository;
import com.fasterxml.jackson.databind.ObjectMapper;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.http.ResponseEntity;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.time.Instant;
import java.time.temporal.ChronoUnit;
import java.util.Base64;
import java.util.Optional;
import java.util.function.Supplier;

/**
 * Central idempotency service.
 *
 * Strategy:
 * 1. Check if idempotency key exists
 * 2. If PROCESSING  → throw 409 Conflict (concurrent duplicate)
 * 3. If COMPLETED   → return cached response
 * 4. If not found   → mark as PROCESSING, execute, cache result
 * 5. If FAILED      → allow retry (treat as new request)
 */
@Service
@RequiredArgsConstructor
@Slf4j
public class IdempotencyService {

    private final IdempotencyRecordRepository repository;
    private final ObjectMapper objectMapper;

    private static final long TTL_HOURS = 24; // Keys expire after 24 hours

    /**
     * Execute an operation idempotently.
     *
     * @param idempotencyKey  Client-provided unique key (UUID recommended)
     * @param requestBody     Raw request payload (for hash validation)
     * @param operation       The actual business logic to execute
     * @return ResponseEntity (either cached or freshly computed)
     */
    @Transactional
    public ResponseEntity<Object> executeIdempotently(
            String idempotencyKey,
            Object requestBody,
            Supplier<ResponseEntity<Object>> operation) {

        String requestHash = computeHash(requestBody);

        Optional<IdempotencyRecord> existing = repository.findByIdempotencyKey(idempotencyKey);

        if (existing.isPresent()) {
            IdempotencyRecord record = existing.get();

            // Guard: same key but DIFFERENT payload is a client error
            if (!record.getRequestHash().equals(requestHash)) {
                log.warn("Idempotency key reuse with different payload. Key: {}", idempotencyKey);
                throw new IdempotencyKeyMismatchException(
                    "Idempotency key '" + idempotencyKey + "' was used with a different request payload."
                );
            }

            switch (record.getStatus()) {
                case PROCESSING -> {
                    // Another thread/pod is processing the same request
                    log.warn("Concurrent request detected for key: {}", idempotencyKey);
                    throw new IdempotencyConflictException(
                        "Request with key '" + idempotencyKey + "' is currently being processed."
                    );
                }
                case COMPLETED -> {
                    // Cache hit — return stored response
                    log.info("Cache hit for idempotency key: {}", idempotencyKey);
                    return deserializeResponse(record);
                }
                case FAILED -> {
                    // Previous attempt failed — allow retry, update record
                    log.info("Retrying previously failed request. Key: {}", idempotencyKey);
                    record.setStatus(IdempotencyRecord.ProcessingStatus.PROCESSING);
                    repository.save(record);
                }
            }
        } else {
            // First time seeing this key — lock it
            IdempotencyRecord newRecord = IdempotencyRecord.builder()
                    .idempotencyKey(idempotencyKey)
                    .requestHash(requestHash)
                    .responseBody("")
                    .httpStatus(0)
                    .status(IdempotencyRecord.ProcessingStatus.PROCESSING)
                    .createdAt(Instant.now())
                    .expiresAt(Instant.now().plus(TTL_HOURS, ChronoUnit.HOURS))
                    .build();
            repository.save(newRecord);
        }

        // Execute the actual business logic
        try {
            ResponseEntity<Object> response = operation.get();
            cacheResponse(idempotencyKey, requestHash, response);
            return response;
        } catch (Exception e) {
            // Mark as FAILED so the client can safely retry
            repository.findByIdempotencyKey(idempotencyKey).ifPresent(r -> {
                r.setStatus(IdempotencyRecord.ProcessingStatus.FAILED);
                repository.save(r);
            });
            throw e;
        }
    }

    private void cacheResponse(String key, String hash, ResponseEntity<Object> response) {
        try {
            String serialized = objectMapper.writeValueAsString(response.getBody());
            repository.findByIdempotencyKey(key).ifPresent(record -> {
                record.setResponseBody(serialized);
                record.setHttpStatus(response.getStatusCode().value());
                record.setStatus(IdempotencyRecord.ProcessingStatus.COMPLETED);
                repository.save(record);
            });
        } catch (Exception e) {
            log.error("Failed to cache idempotency response for key: {}", key, e);
        }
    }

    private ResponseEntity<Object> deserializeResponse(IdempotencyRecord record) {
        try {
            Object body = objectMapper.readValue(record.getResponseBody(), Object.class);
            return ResponseEntity.status(record.getHttpStatus()).body(body);
        } catch (Exception e) {
            log.error("Failed to deserialize cached response for key: {}", record.getIdempotencyKey(), e);
            throw new RuntimeException("Failed to reconstruct cached response", e);
        }
    }

    private String computeHash(Object payload) {
        try {
            String json = objectMapper.writeValueAsString(payload);
            MessageDigest digest = MessageDigest.getInstance("SHA-256");
            byte[] hashBytes = digest.digest(json.getBytes(StandardCharsets.UTF_8));
            return Base64.getEncoder().encodeToString(hashBytes);
        } catch (NoSuchAlgorithmException e) {
            throw new RuntimeException("SHA-256 not available", e);
        } catch (Exception e) {
            throw new RuntimeException("Failed to compute request hash", e);
        }
    }
}

---

Step 4 — The REST Controller (Order Service Example)

// OrderController.java
package com.example.idempotency.controller;

import com.example.idempotency.dto.CreateOrderRequest;
import com.example.idempotency.dto.OrderResponse;
import com.example.idempotency.service.IdempotencyService;
import com.example.idempotency.service.OrderService;
import jakarta.validation.Valid;
import jakarta.validation.constraints.NotBlank;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.validation.annotation.Validated;
import org.springframework.web.bind.annotation.*;

/**
 * Order REST Controller demonstrating idempotent POST endpoint.
 *
 * The client MUST provide an `Idempotency-Key` header (UUID v4 recommended).
 * This ensures safe retries in the face of network failures.
 */
@RestController
@RequestMapping("/api/v1/orders")
@RequiredArgsConstructor
@Validated
@Slf4j
public class OrderController {

    private final OrderService orderService;
    private final IdempotencyService idempotencyService;

    /**
     * POST /api/v1/orders
     *
     * Creates a new order. This endpoint is made idempotent via the
     * Idempotency-Key header. Duplicate requests with the same key
     * return the original response without creating another order.
     *
     * @param idempotencyKey  Required: UUID from client (e.g., UUID.randomUUID())
     * @param request         Order creation payload
     */
    @PostMapping
    public ResponseEntity<Object> createOrder(
            @RequestHeader("Idempotency-Key")
            @NotBlank(message = "Idempotency-Key header is required")
            String idempotencyKey,

            @Valid @RequestBody CreateOrderRequest request) {

        log.info("Received order request. Idempotency-Key: {}", idempotencyKey);

        return idempotencyService.executeIdempotently(
                idempotencyKey,
                request,
                () -> {
                    // This block only runs if the key is new or a retry of a failed request
                    OrderResponse order = orderService.createOrder(request);
                    return ResponseEntity.status(HttpStatus.CREATED).body(order);
                }
        );
    }

    /**
     * GET /api/v1/orders/{orderId}
     * GET is inherently idempotent — no special handling needed.
     */
    @GetMapping("/{orderId}")
    public ResponseEntity<OrderResponse> getOrder(@PathVariable Long orderId) {
        return ResponseEntity.ok(orderService.getOrder(orderId));
    }

    /**
     * PUT /api/v1/orders/{orderId}
     * PUT is idempotent by HTTP spec — full replacement is naturally safe to retry.
     */
    @PutMapping("/{orderId}")
    public ResponseEntity<OrderResponse> updateOrder(
            @PathVariable Long orderId,
            @Valid @RequestBody CreateOrderRequest request) {
        return ResponseEntity.ok(orderService.updateOrder(orderId, request));
    }
}

---

Step 5 — DTOs

// CreateOrderRequest.java
package com.example.idempotency.dto;

import jakarta.validation.constraints.*;
import lombok.Data;

import java.math.BigDecimal;

@Data
public class CreateOrderRequest {

    @NotBlank(message = "Customer ID is required")
    private String customerId;

    @NotBlank(message = "Product ID is required")
    private String productId;

    @Min(value = 1, message = "Quantity must be at least 1")
    private int quantity;

    @NotNull(message = "Amount is required")
    @DecimalMin(value = "0.01", message = "Amount must be positive")
    private BigDecimal amount;

    @NotBlank(message = "Currency is required")
    @Size(min = 3, max = 3, message = "Currency must be a 3-letter ISO code")
    private String currency;
}

// OrderResponse.java
package com.example.idempotency.dto;

import lombok.Builder;
import lombok.Data;

import java.math.BigDecimal;
import java.time.Instant;

@Data
@Builder
public class OrderResponse {
    private Long orderId;
    private String customerId;
    private String productId;
    private int quantity;
    private BigDecimal amount;
    private String currency;
    private String status;
    private Instant createdAt;
}

---

Step 6 — Global Exception Handler

// GlobalExceptionHandler.java
package com.example.idempotency.exception;

import lombok.extern.slf4j.Slf4j;
import org.springframework.http.HttpStatus;
import org.springframework.http.ProblemDetail;
import org.springframework.web.bind.annotation.ExceptionHandler;
import org.springframework.web.bind.annotation.RestControllerAdvice;

import java.net.URI;
import java.time.Instant;

@RestControllerAdvice
@Slf4j
public class GlobalExceptionHandler {

    /**
     * 422 Unprocessable Entity — Same key, different payload.
     * The client is misusing the idempotency key contract.
     */
    @ExceptionHandler(IdempotencyKeyMismatchException.class)
    public ProblemDetail handleKeyMismatch(IdempotencyKeyMismatchException ex) {
        ProblemDetail problem = ProblemDetail.forStatusAndDetail(
                HttpStatus.UNPROCESSABLE_ENTITY, ex.getMessage());
        problem.setType(URI.create("https://api.example.com/errors/idempotency-key-mismatch"));
        problem.setTitle("Idempotency Key Mismatch");
        problem.setProperty("timestamp", Instant.now());
        return problem;
    }

    /**
     * 409 Conflict — Concurrent request with the same key in-flight.
     * Client should wait and retry after a short delay.
     */
    @ExceptionHandler(IdempotencyConflictException.class)
    public ProblemDetail handleConflict(IdempotencyConflictException ex) {
        ProblemDetail problem = ProblemDetail.forStatusAndDetail(
                HttpStatus.CONFLICT, ex.getMessage());
        problem.setType(URI.create("https://api.example.com/errors/concurrent-request"));
        problem.setTitle("Concurrent Request Conflict");
        problem.setProperty("timestamp", Instant.now());
        problem.setProperty("retryAfterMs", 500);
        return problem;
    }
}

// Custom exceptions
public class IdempotencyKeyMismatchException extends RuntimeException {
    public IdempotencyKeyMismatchException(String message) { super(message); }
}

public class IdempotencyConflictException extends RuntimeException {
    public IdempotencyConflictException(String message) { super(message); }
}

---

Step 7 — Scheduled Cleanup Job

// IdempotencyCleanupJob.java
package com.example.idempotency.scheduler;

import com.example.idempotency.repository.IdempotencyRecordRepository;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;
import org.springframework.transaction.annotation.Transactional;

import java.time.Instant;

/**
 * Periodically purges expired idempotency records to prevent unbounded table growth.
 * Runs every hour. In high-volume systems, consider batch deletes with pagination.
 */
@Component
@RequiredArgsConstructor
@Slf4j
public class IdempotencyCleanupJob {

    private final IdempotencyRecordRepository repository;

    @Scheduled(fixedRateString = "${idempotency.cleanup.interval-ms:3600000}")
    @Transactional
    public void purgeExpiredRecords() {
        log.info("Running idempotency record cleanup at {}", Instant.now());
        repository.deleteExpiredRecords(Instant.now());
        log.info("Idempotency cleanup complete");
    }
}

---

Step 8 — Test Cases

// OrderControllerIdempotencyTest.java
package com.example.idempotency.controller;

import com.example.idempotency.dto.CreateOrderRequest;
import com.example.idempotency.dto.OrderResponse;
import com.example.idempotency.service.IdempotencyService;
import com.example.idempotency.service.OrderService;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTest;
import org.springframework.boot.test.mock.mockito.MockBean;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.test.web.servlet.MockMvc;

import java.math.BigDecimal;
import java.time.Instant;
import java.util.UUID;
import java.util.function.Supplier;

import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.eq;
import static org.mockito.Mockito.*;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.post;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.*;

@WebMvcTest(OrderController.class)
class OrderControllerIdempotencyTest {

    @Autowired
    private MockMvc mockMvc;

    @Autowired
    private ObjectMapper objectMapper;

    @MockBean
    private OrderService orderService;

    @MockBean
    private IdempotencyService idempotencyService;

    private CreateOrderRequest buildRequest() {
        CreateOrderRequest req = new CreateOrderRequest();
        req.setCustomerId("CUST-001");
        req.setProductId("PROD-XYZ");
        req.setQuantity(2);
        req.setAmount(new BigDecimal("99.99"));
        req.setCurrency("USD");
        return req;
    }

    private OrderResponse buildResponse() {
        return OrderResponse.builder()
                .orderId(101L)
                .customerId("CUST-001")
                .productId("PROD-XYZ")
                .quantity(2)
                .amount(new BigDecimal("99.99"))
                .currency("USD")
                .status("CREATED")
                .createdAt(Instant.now())
                .build();
    }

    @Test
    @DisplayName("First request — creates order and returns 201")
    void testFirstRequest_ReturnsCreated() throws Exception {
        String idempotencyKey = UUID.randomUUID().toString();
        OrderResponse expectedResponse = buildResponse();

        when(idempotencyService.executeIdempotently(
                eq(idempotencyKey), any(), any(Supplier.class)
        )).thenReturn(ResponseEntity.status(201).body(expectedResponse));

        mockMvc.perform(post("/api/v1/orders")
                .header("Idempotency-Key", idempotencyKey)
                .contentType(MediaType.APPLICATION_JSON)
                .content(objectMapper.writeValueAsString(buildRequest())))
                .andExpect(status().isCreated())
                .andExpect(jsonPath("$.orderId").value(101))
                .andExpect(jsonPath("$.status").value("CREATED"));

        verify(idempotencyService, times(1))
                .executeIdempotently(eq(idempotencyKey), any(), any(Supplier.class));
    }

    @Test
    @DisplayName("Duplicate request with same key — returns cached 201 without re-processing")
    void testDuplicateRequest_ReturnsCachedResponse() throws Exception {
        String idempotencyKey = UUID.randomUUID().toString();
        OrderResponse cachedResponse = buildResponse();

        // Both calls return same cached response (idempotency service handles dedup)
        when(idempotencyService.executeIdempotently(
                eq(idempotencyKey), any(), any(Supplier.class)
        )).thenReturn(ResponseEntity.status(201).body(cachedResponse));

        // First call
        mockMvc.perform(post("/api/v1/orders")
                .header("Idempotency-Key", idempotencyKey)
                .contentType(MediaType.APPLICATION_JSON)
                .content(objectMapper.writeValueAsString(buildRequest())))
                .andExpect(status().isCreated())
                .andExpect(jsonPath("$.orderId").value(101));

        // Second call — same key, same payload
        mockMvc.perform(post("/api/v1/orders")
                .header("Idempotency-Key", idempotencyKey)
                .contentType(MediaType.APPLICATION_JSON)
                .content(objectMapper.writeValueAsString(buildRequest())))
                .andExpect(status().isCreated())
                .andExpect(jsonPath("$.orderId").value(101)); // Same order ID!

        // OrderService should only be involved once (via idempotency service)
        verify(idempotencyService, times(2))
                .executeIdempotently(eq(idempotencyKey), any(), any(Supplier.class));
    }

    @Test
    @DisplayName("Missing Idempotency-Key header — returns 400")
    void testMissingHeader_ReturnsBadRequest() throws Exception {
        mockMvc.perform(post("/api/v1/orders")
                .contentType(MediaType.APPLICATION_JSON)
                .content(objectMapper.writeValueAsString(buildRequest())))
                .andExpect(status().isBadRequest());
    }

    @Test
    @DisplayName("Invalid request body — returns 400 with validation errors")
    void testInvalidBody_ReturnsBadRequest() throws Exception {
        CreateOrderRequest invalidRequest = new CreateOrderRequest();
        // Missing all required fields

        mockMvc.perform(post("/api/v1/orders")
                .header("Idempotency-Key", UUID.randomUUID().toString())
                .contentType(MediaType.APPLICATION_JSON)
                .content(objectMapper.writeValueAsString(invalidRequest)))
                .andExpect(status().isBadRequest());
    }
}

---

Step 9 — application.yml Configuration

# application.yml
spring:
  datasource:
    url: jdbc:postgresql://localhost:5432/orderdb
    username: ${DB_USERNAME}
    password: ${DB_PASSWORD}
  jpa:
    hibernate:
      ddl-auto: validate
    show-sql: false
  data:
    redis:
      host: localhost
      port: 6379
      timeout: 2000ms

# Idempotency-specific config
idempotency:
  cleanup:
    interval-ms: 3600000  # 1 hour
  ttl-hours: 24           # Keys valid for 24 hours

logging:
  level:
    com.example.idempotency: INFO

---

Architecture Diagram: Idempotency Flow

                    ┌─────────────────────────────────────────────────────────┐
                    │                  API Gateway / Load Balancer             │
                    └──────────────────────────┬──────────────────────────────┘
                                               │  POST /api/v1/orders
                                               │  Idempotency-Key: abc-123
                                               ▼
                    ┌─────────────────────────────────────────────────────────┐
                    │                   OrderController                        │
                    │         Validates header + request body                  │
                    └──────────────────────────┬──────────────────────────────┘
                                               │
                                               ▼
                    ┌─────────────────────────────────────────────────────────┐
                    │                  IdempotencyService                      │
                    │                                                           │
                    │  ┌──────────────────────────────────────────────────┐   │
                    │  │  Check DB: Does key "abc-123" exist?              │   │
                    │  └────────────────────┬─────────────────────────────┘   │
                    │           ┌───────────┼────────────────┐                 │
                    │        YES│           │NO               │PROCESSING       │
                    │           ▼           ▼                 ▼                 │
                    │    ┌──────────┐ ┌─────────────┐ ┌──────────────┐        │
                    │    │ Check    │ │ Insert key  │ │ Return 409   │        │
                    │    │ Status   │ │ PROCESSING  │ │  Conflict    │        │
                    │    └──────────┘ └──────┬──────┘ └──────────────┘        │
                    │    COMPLETED│          │                                  │
                    │           ▼           ▼                                  │
                    │    ┌──────────┐ ┌─────────────────────────┐             │
                    │    │ Return   │ │ Execute Business Logic  │             │
                    │    │ Cached   │ │ (OrderService.create)   │             │
                    │    │ Response │ └──────────┬──────────────┘             │
                    │    └──────────┘            │                              │
                    │                            ▼                              │
                    │                  ┌──────────────────┐                    │
                    │                  │ Cache response   │                    │
                    │                  │ Mark COMPLETED   │                    │
                    │                  └──────────────────┘                    │
                    └─────────────────────────────────────────────────────────┘

---

4. Pros & Cons

Advantages

1. Fault Tolerance and Retry Safety Distributed systems fail. Idempotent APIs let clients retry with confidence. Network timeouts, pod restarts, and transient errors no longer cause data corruption. This is foundational to building resilient microservices.

2. Simplifies Client Logic Without idempotency guarantees, clients must implement complex state tracking: “Did my request succeed? Should I retry? Will retrying cause duplicates?” With idempotency keys, the answer is always: “Just retry with the same key.”

3. Prevents Costly Business Logic Duplication In payment, order, and inventory systems, double-processing is a business disaster. Idempotency provides a technical guarantee that matches the business requirement: exactly-once execution semantics.

4. Aligns with HTTP Specification Properly implementing idempotency makes your API compliant with RFC 9110. This improves interoperability with HTTP clients, proxies, CDNs, and API gateways that may automatically retry safe requests.

---

Disadvantages

1. Storage Overhead and Cleanup Complexity Every unique request stores a record. High-volume APIs generate millions of idempotency records. You need TTL policies, scheduled cleanup jobs, and an indexed database table — adding operational complexity.

2. Not a Silver Bullet for Distributed Transactions Idempotency handles retry safety at the API layer. It does NOT solve distributed consistency across multiple services. If an order service is idempotent but the downstream inventory service is not, you still have problems. Each hop needs its own idempotency strategy.

3. Payload Mismatch Ambiguity What should happen when the same key arrives with a different payload? Is it a client bug or a legitimate re-use? You must define a clear contract (this implementation returns 422), but this edge case adds complexity to client implementations and error handling.

4. Concurrency Window and Race Conditions Between the “check if key exists” and “insert key” operations, concurrent requests can slip through without strict database-level uniqueness constraints. Relying purely on application-level checks is insufficient — you must enforce uniqueness at the database constraint level and handle DataIntegrityViolationException gracefully.

---

5. Top 5 Interview Questions & Answers

---

Q1. What is idempotency in REST APIs, and why does it matter in microservices?

Answer:

Idempotency means that performing the same operation multiple times produces the same result as performing it once, with no additional side effects. In REST, methods like GET, PUT, and DELETE are idempotent by specification. POST is not idempotent by default.

In microservices, this matters because distributed networks are inherently unreliable. When a client sends a request and receives a timeout, it cannot know whether the server processed the request. Without idempotency, retrying can lead to duplicate orders, double charges, or corrupted inventory counts. With idempotency keys, a client can safely retry any number of times, knowing the server will only execute the business logic once.

The implementation typically involves:

• A client-generated UUID sent as a header (Idempotency-Key)
• Server-side persistence of the key and its response
• A lookup on every incoming request before execution

---

Q2. How would you implement idempotency for a POST endpoint that creates payments? What happens if two identical requests arrive simultaneously?

Answer (Scenario-Based):

For a payment endpoint, I’d implement the following strategy:

1. Require an Idempotency-Key header (UUID) from the client on every POST /payments call.
2. Before processing, query an idempotency_records table using the key.
3. If key exists and COMPLETED, return the cached response — payment already processed.
4. If key exists and PROCESSING, return 409 Conflict — concurrent duplicate detected.
5. If key is new, insert a record with PROCESSING status (using a unique DB constraint to handle race conditions atomically), process the payment, and update status to COMPLETED with the serialized response.

For the simultaneous request scenario, the database unique constraint on idempotency_key column ensures only one insertion succeeds. The second concurrent thread gets a DataIntegrityViolationException, which I catch and translate into a 409 Conflict response, telling the client to retry after a brief delay.

This is exactly how Stripe handles their payment API — their documentation explicitly states that idempotency keys should be unique per logical operation and that concurrent requests with the same key receive a 429 or 409 response.

---

Q3. Is PUT inherently idempotent? What about PATCH? Give examples.

Answer:

PUT is idempotent by HTTP specification. It performs a full resource replacement. For example:

PUT /users/42
{ "name": "Alice", "email": "alice@example.com" }

Calling this 10 times results in exactly the same state — user 42 always has name “Alice” and email “alice@example.com”. The first call creates/replaces the resource; subsequent calls replace it with the same data.

PATCH is NOT inherently idempotent — it depends on the operation semantics:

• PATCH /accounts/1 { "balance": 500 } (SET operation) → Idempotent — always sets balance to 500
• PATCH /accounts/1 { "balanceDelta": +100 } (INCREMENT operation) → NOT idempotent — balance grows with each call

For production APIs, I always design PATCH operations to be idempotent by using absolute values (SET semantics) rather than relative changes (INCREMENT semantics) for critical fields. If relative changes are required, I enforce idempotency keys on those endpoints.

---

Q4. How do you handle idempotency key expiration? What are the trade-offs?

Answer:

Idempotency keys should have a TTL — typically 24 to 48 hours. Here’s the reasoning and trade-offs:

Why expire keys?

• Prevents unbounded growth of the idempotency_records table
• Old keys are no longer needed once the client has received a response and confirmed success

Implementation:

• Store an expires_at timestamp on each record
• Run a scheduled cleanup job (e.g., Spring @Scheduled) to delete expired records
• For high-scale systems, use Redis with native TTL support as a cache layer, backed by a database for durability

Trade-offs:

TTL Duration	Pros	Cons
Very short (1 hour)	Smaller storage footprint	Client retries might fail if too slow
Medium (24 hours)	Balanced, industry standard (Stripe uses 24h)	Moderate storage growth
Long (7 days)	Very safe for slow clients	Significant storage cost at scale

For payment systems, I align the TTL with the maximum expected retry window — usually 24 hours. For high-frequency, low-criticality APIs, shorter TTLs with Redis as the backing store are more cost-effective.

---

Q5. What’s the difference between idempotency and exactly-once delivery? Can you guarantee exactly-once execution in a distributed system?

Answer:

This is a subtle but critical distinction:

Idempotency is a property of an operation: calling it multiple times is equivalent to calling it once. It’s a guarantee about correctness when retried.

Exactly-once delivery is a messaging guarantee: a message is delivered to the consumer precisely once, no more, no fewer. It’s a guarantee about how many times a message is dispatched.

The key insight: you cannot guarantee exactly-once delivery in a distributed system (this is a well-established impossibility result under network partitions — see the CAP theorem and Two Generals Problem). Networks can duplicate messages; systems can crash and recover.

However, you can guarantee exactly-once execution by combining:

1. At-least-once delivery (accept that retries will happen)
2. Idempotent consumers (design your processing logic to be safe on duplicates)

This is the standard pattern in Apache Kafka (idempotent producers + transactional consumers), AWS SQS (at-least-once + idempotent handlers), and all production microservices at scale.

So in an interview context: the correct answer is “exactly-once delivery is a myth in distributed systems; we achieve exactly-once semantics through idempotent processing combined with at-least-once delivery.”

---

6. Conclusion

Idempotency is not a feature — it is a fundamental design contract that every production REST API must honor, especially in microservices architectures where partial failures are the norm, not the exception.

The implementation we walked through establishes a clean separation of concerns:

• The controller enforces the header contract at the HTTP boundary
• The idempotency service owns the lookup, locking, and caching logic
• The database constraint provides the atomic safety net for concurrent requests
• The cleanup scheduler manages operational hygiene

Beyond the code, the most important shift is conceptual: stop designing APIs that assume requests always succeed on the first attempt. Start designing APIs that assume requests will fail, be duplicated, and be retried — and make sure your system handles all of those scenarios correctly.

In the words of the distributed systems community: “Design for failure. Idempotency is how you make failure invisible to your users.”

The patterns in this guide align with how industry leaders like Stripe, Twilio, and AWS design their APIs. When you internalize idempotency as a default — not an afterthought — you will build systems that are resilient, trustworthy, and production-grade from day one.

---

Built with Java 21, Spring Boot 3.x, Spring Data JPA, PostgreSQL, and Spring Scheduler. All code follows industry best practices and is production-ready with appropriate modifications for your environment.