Docker vs Native PostgreSQL: Impact on Database Performance and Understanding Storage Performance Metrics

What are you trying to achieve?

In today’s data-driven world, database performance is crucial for enterprise systems. This article delves into the impact of containerization on PostgreSQL performance, comparing Docker-based deployments to native installations. We explore key storage metrics, high-performance setups, and conduct a thorough analysis of throughput in different scenarios. Whether you’re a database administrator, DevOps engineer, or IT decision-maker, this piece offers valuable insights into optimizing PostgreSQL performance for demanding workloads. Discover how your choice of deployment method can significantly affect your database’s ability to process billions of rows per second.

Storage Keywords:

IOPS (Input/Output Operations Per Second)

IOPS measures the number of read and write operations per second in storage devices. For enterprise systems:

• HDD: 200 IOPS
• SSD: 200,000 IOPS
• NVMe: Up to 1,000,000 IOPS

IOPS indicates how well a device handles small, random read/write operations.

Read/Write Speed

Read/write speeds measure data transfer rates in MB/s or GB/s:

• HDD: 80-250 MB/s
• SATA SSD: 500-600 MB/s
• NVMe PCIe 3.0 SSDs: 2,000-3,500 MB/s
• NVMe PCIe 4.0 SSDs: 5,000-7,000 MB/s or higher

High-Performance Storage Solutions

SAN and Multipathing

Storage Area Networks (SANs) provide high-speed access to consolidated block-level storage. Multipathing uses multiple physical paths between servers and storage devices, enhancing performance and reliability.

Fiber Optic Technology for Data Centers

Fiber optic cables use light pulses for data transmission, offering high bandwidth and low latency:

• OM3 Fiber: Up to 10 Gbps over 300 meters
• OM4 Fiber: 10 Gbps up to 400 meters, 40/100 Gbps over 150 meters
• OM5 Fiber: Supports 40 Gbps and 100 Gbps, optimized for short wavelength division multiplexing

Optimal Server Configuration for PostgreSQL

In this high-performance setup, we will be using NVMe PCIe 4.0 SSDs with SAN and OM5 fiber for connectivity, ensuring that the distance between the server and SAN does not exceed 150 meters. The server, equipped with high-performance CPUs and RAM, utilizes a 100 Gbps fiber link to access the NVMe storage on the SAN. This configuration allows for maximum data throughput and minimal latency, as the OM5 fiber provides the necessary bandwidth to handle the 7 GB/s data rate of the NVMe storage. The switch efficiently manages traffic between the server and storage units, optimizing performance and reliability.

Scenario 1: Docker Installed with PostgreSQL Container

• Effective Throughput with Docker: If Docker adds 15% overhead:
  • Throughput Reduction: 15% of 6.3 GB/s = 0.945 GB/s
  • Available Throughput: 6.3 GB/s – 0.945 GB/s = 5.355 GB/s

Calculations:

1. Effective Throughput with Docker:
   • Throughput: 5.355 GB/s
   • Data per Second: 5.355 GB * 1,024 MB/GB * 1,024 KB/MB = 5,629,766,400 KB/s
   • Number of Rows per Second: 5,629,766,400 KB / 1 KB per row = 5,629,766,400 rows/s

Summary:

• Effective Throughput with Docker: Approximately 5.63 billion rows per second.

Scenario 2: PostgreSQL Directly on Host

• Effective Throughput without Docker: The overhead from the OS alone, assuming 10% overhead:
  • Throughput Reduction: 10% of 6.3 GB/s = 0.63 GB/s
  • Available Throughput: 6.3 GB/s – 0.63 GB/s = 5.67 GB/s

Calculations:

1. Effective Throughput without Docker:
   • Throughput: 5.67 GB/s
   • Data per Second: 5.67 GB * 1,024 MB/GB * 1,024 KB/MB = 5,949,542,400 KB/s
   • Number of Rows per Second: 5,949,542,400 KB / 1 KB per row = 5,949,542,400 rows/s

Summary:

• Effective Throughput without Docker: Approximately 5.95 billion rows per second.

Conclusion:

I can’t give up those 320 million entries just for Docker. In scenarios demanding peak performance, especially with high-throughput PostgreSQL operations, Docker’s overhead results in a significant reduction in throughput. While Docker provides numerous advantages in deployment and containerization, it’s crucial to evaluate where it’s used. In high-performance environments, particularly for databases, Docker might not be the best choice. In my next series, I will delve into the impact of Docker on CPU utilization. For now, the takeaway is that Docker’s overhead can be a critical factor, and careful consideration is needed to decide where Docker is appropriate, with databases generally being better served outside of containers.