Deploying AWS RDS Instances with Terraform: MySQL and PostgreSQL - Related to terraform:, mysql, aws, modern, use

Deploying AWS RDS Instances with Terraform: MySQL and PostgreSQL

In this article, we'll walk through how to provision Amazon Web Services (AWS) Relational Database Service (RDS) instances for both MySQL and PostgreSQL using Terraform. By using Infrastructure as Code (IaC), we can automate the process of creating these databases, which will allow for consistent and reproducible infrastructure deployments.

Before we dive into the Terraform configuration, make sure you have the following prerequisites:

Terraform: Install Terraform on your machine. You can follow the official installation guide here. AWS Account: Ensure you have an active AWS account. AWS CLI: Set up AWS CLI and configure your credentials. You can configure it using aws configure . IAM Permissions: Ensure your AWS credentials have sufficient permissions to manage RDS instances.

In this tutorial, we’ll define two AWS RDS instances:

MySQL : A basic MySQL instance using the [website] engine.

: A basic MySQL instance using the engine. PostgreSQL: A PostgreSQL instance using the latest postgres16 engine.

We’ll configure both instances with 20 GB of storage, a [website] instance type, and encryption enabled for data at rest. The instances will also be set to not be publicly accessible and will have a backup retention period of 7 days.

Create a file named [website] . This is where we’ll define our Terraform resources for provisioning the RDS instances.

provider "aws" { region = "us-east-1" # Change to your preferred region } # MySQL RDS Instance resource "aws_db_instance" "mysql" { identifier = "mysql-db-instance" engine = "mysql" instance_class = "[website]" allocated_storage = 20 db_name = "mydb_mysql" username = "my_admin" password = "mysecretpassword" parameter_group_name = "[website]" multi_az = false publicly_accessible = false backup_retention_period = 7 storage_type = "gp2" storage_encrypted = true } # PostgreSQL RDS Instance resource "aws_db_instance" "postgres" { identifier = "postgres-db-instance" engine = "postgres" instance_class = "[website]" allocated_storage = 20 engine_version = "16" db_name = "mydb_postgres" username = "my_admin" password = "mysecretpassword" multi_az = false publicly_accessible = false parameter_group_name = "default.postgres16" backup_retention_period = 7 storage_type = "gp3" storage_encrypted = true } # Outputs: RDS Endpoints output "mysql_endpoint" { value = aws_db_instance . mysql . endpoint } output "postgres_endpoint" { value = aws_db_instance . postgres . endpoint } Enter fullscreen mode Exit fullscreen mode.

provider "aws" { region = "us-east-1" # Change to your preferred region } Enter fullscreen mode Exit fullscreen mode.

This block configures the AWS provider for Terraform, specifying the region in which your resources will be created. You can change the region as per your requirements.

resource "aws_db_instance" "mysql" { identifier = "mysql-db-instance" engine = "mysql" instance_class = "[website]" allocated_storage = 20 db_name = "mydb_mysql" username = "my_admin" password = "mysecretpassword" parameter_group_name = "[website]" multi_az = false publicly_accessible = false backup_retention_period = 7 storage_type = "gp2" storage_encrypted = true } Enter fullscreen mode Exit fullscreen mode.

This resource block creates an RDS instance for MySQL. Here’s a breakdown of some key properties:

engine : Specifies the database engine (MySQL).

: Specifies the database engine (MySQL). allocated_storage : Sets the storage size in GB (20 GB here).

: Sets the storage size in GB (20 GB here). instance_class : Defines the type of instance ( [website] in this case, which is a low-cost instance).

: Defines the type of instance ( in this case, which is a low-cost instance). storage_encrypted : Ensures data is encrypted at rest.

: Ensures data is encrypted at rest. backup_retention_period : Retains backups for 7 days.

resource "aws_db_instance" "postgres" { identifier = "postgres-db-instance" engine = "postgres" instance_class = "[website]" allocated_storage = 20 engine_version = "16" db_name = "mydb_postgres" username = "my_admin" password = "mysecretpassword" multi_az = false publicly_accessible = false parameter_group_name = "default.postgres16" backup_retention_period = 7 storage_type = "gp3" storage_encrypted = true } Enter fullscreen mode Exit fullscreen mode.

This block is similar to the MySQL one but for PostgreSQL. The engine_version property is set to 16, indicating the latest version of PostgreSQL.

output "mysql_endpoint" { value = aws_db_instance . mysql . endpoint } output "postgres_endpoint" { value = aws_db_instance . postgres . endpoint } Enter fullscreen mode Exit fullscreen mode.

These output blocks display the endpoint URLs for the MySQL and PostgreSQL RDS instances once they are created. You can use these endpoints to connect to your databases.

Initialize Terraform: To begin, initialize your Terraform working directory with the following command:

terraform init Enter fullscreen mode Exit fullscreen mode.

Plan the Deployment: Run the following command to preview the resources Terraform will create:

terraform plan Enter fullscreen mode Exit fullscreen mode.

Apply the Configuration: Apply the configuration to create the RDS instances by running:

terraform apply Enter fullscreen mode Exit fullscreen mode.

Terraform will prompt for confirmation before proceeding. Type yes to create the resources.

Once the deployment is complete, Terraform will output the endpoints for the MySQL and PostgreSQL RDS instances. You can connect to these instances using the endpoint URLs along with the username and password specified in the configuration.

In this tutorial, we’ve successfully automated the deployment of MySQL and PostgreSQL RDS instances in AWS using Terraform. By leveraging Terraform's Infrastructure as Code capabilities, we can quickly and efficiently create reproducible environments for our databases.

Feel free to expand upon this configuration by adding more customization options like VPC, security groups, and IAM roles to further secure and optimize your RDS instances.

Ryan is joined by Fynn Glover (CEO) and Ben Papillon (CTO), cofounders of Schematic, for a conv......

We're a place where coders share, stay up-to-date and grow their careers....

System Analysis and Design has to do with studying and examining the existing system in order to understand the problems ......

Docker Bake: A Modern Approach to Container Building

The traditional way of building Docker images using the docker build command is simple and straightforward, but when working with complex applications consisting of multiple components, this process can become tedious and error-prone. This is where Docker Bake comes in — a powerful and flexible tool for organizing multi-stage and parallel image building.

In this article, we'll look at the capabilities of Docker Bake, its advantages over the standard approach, and practical examples of its use for various development scenarios.

Docker Bake is a BuildKit feature that allows you to organize and automate the Docker image-building process using configuration files.

Declarative syntax . Instead of multiple commands in scripts, you describe the desired result in HCL (HashiCorp Configuration Language), JSON, or YAML (Docker Compose files).

. Instead of multiple commands in scripts, you describe the desired result in HCL (HashiCorp Configuration Language), JSON, or YAML (Docker Compose files). Parallel building . BuildKit automatically performs image building in parallel where possible.

. BuildKit automatically performs image building in parallel where possible. Cache reuse . Efficient use of cache between different builds.

. Efficient use of cache between different builds. Grouping and targeted builds . Ability to define groups of images and build only the targets needed at the moment.

. Ability to define groups of images and build only the targets needed at the moment. Variables and inheritance . A powerful system of variables and property inheritance between build targets.

. A powerful system of variables and property inheritance between build targets. CI/CD integration. Easily integrates into continuous integration and delivery pipelines.

Let's look at the main components of a bake file:

Variables allow you to define values that can be used in different parts of the configuration and easily redefined at runtime:

Shell variable "TAG" { default = "latest" } variable "DEBUG" { default = "false" }.

Variables can be used in other parts of the configuration through string interpolation: ${TAG} .

Groups allow you to combine multiple targets for simultaneous building:

Shell group "default" { targets = ["app", "api"] } group "backend" { targets = ["api", "database"] }.

Targets are the main units of building, each defining one Docker image:

Shell target "app" { dockerfile = "[website]" context = "./app" tags = ["myorg/app:${TAG}"] args = { DEBUG = "${DEBUG}" } platforms = ["linux/amd64", "linux/arm64"] }.

– path to the Dockerfile context – build context.

– build context tags – tags for the image.

– tags for the image args – arguments to pass to the Dockerfile.

– arguments to pass to the Dockerfile platforms – platforms for multi-platform building.

– platforms for multi-platform building target – target for multi-stage building in Dockerfile.

– target for multi-stage building in Dockerfile output – where to output the build result.

– where to output the build result cache-from and cache-to – cache settings.

One of the most powerful elements of Bake is the ability to inherit parameters:

Shell target "base" { context = "." args = { BASE_IMAGE = "node:16-alpine" } } target "app" { inherits = ["base"] dockerfile = "app/Dockerfile" tags = ["myapp/app:latest"] }.

The app target will inherit all parameters from the base and overwrite or supplement them with its own.

In HCL, you can define functions for more flexible configuration:

Shell function "tag" { params = [name, version] result = ["${name}:${version}"] } target "app" { tags = tag("myapp/app", "[website]") }.

Docker Bake is part of BuildKit, a modern engine for building Docker images. Starting with Docker [website], BuildKit is enabled by default, so most individuals don't need additional configuration. However, if you're using an older version of Docker or want to make sure BuildKit is activated, follow the instructions below.

Make sure you have an up-to-date version of Docker ([website] or higher). You can check the version with the command:

If your Docker version is outdated, revision it following the official documentation.

Activating BuildKit (for old Docker versions).

For Docker versions below [website], BuildKit needs to be activated manually. This can be done in one of the following ways:

Via environment variable : Shell export DOCKER_BUILDKIT=1 Plain Text In the Docker configuration file: Edit the ~/.docker/[website] file and add the following parameters:

JSON { "capabilities": { "buildkit": true } }.

Via command line: When using the docker build or docker buildx bake command, you can explicitly specify the use of BuildKit: Shell DOCKER_BUILDKIT=1 docker buildx bake.

Docker Buildx is an extension of the Docker CLI that provides additional capabilities for building images, including support for multi-platform building. Starting with Docker [website], Buildx is included with Docker, but for full functionality, it's recommended to ensure it's installed and activated.

If Buildx is not installed, follow the instructions below.

For Linux: Shell mkdir -p ~/.docker/cli-plugins curl -sSL [website] -o ~/.docker/cli-plugins/docker-buildx chmod +x ~/.docker/cli-plugins/docker-buildx.

For macOS (using Homebrew): Shell brew install docker-buildx.

By default, Docker uses the built-in builder, but for full functionality, it's recommended to create a new builder:

Shell docker buildx create --use --name my-builder.

Docker Bake uses configuration files that can be written in HCL (default), JSON, or YAML formats. Standard names for these files:

You can also use [website] with some extensions.

A typical Docker Bake configuration file has the following structure:

Shell // Defining variables variable "TAG" { default = "latest" } // Defining groups group "default" { targets = ["app", "api"] } // Defining common settings target "docker-metadata-action" { tags = ["user/app:${TAG}"] } // Defining build targets target "app" { inherits = ["docker-metadata-action"] dockerfile = "[website]" context = "./app" } target "api" { inherits = ["docker-metadata-action"] dockerfile = "[website]" context = "./api" }.

Build all targets from the default group:

Example 1: Simple Multi-Component Application.

Suppose we have an application consisting of a web frontend, API, and database service. Here's what a [website] file might look like:

Shell variable "TAG" { default = "latest" } group "default" { targets = ["frontend", "api", "db"] } group "services" { targets = ["api", "db"] } target "base" { context = "." args = { BASE_IMAGE = "node:16-alpine" } } target "frontend" { inherits = ["base"] dockerfile = "frontend/Dockerfile" tags = ["myapp/frontend:${TAG}"] args = { API_URL = "[website]:3000" } } target "api" { inherits = ["base"] dockerfile = "api/Dockerfile" tags = ["myapp/api:${TAG}"] args = { DB_HOST = "db" DB_PORT = "5432" } } target "db" { context = "./db" dockerfile = "Dockerfile" tags = ["myapp/db:${TAG}"] }.

One of the powerful aspects of Docker Bake is the ease of setting up multi-platform building:

Shell variable "TAG" { default = "latest" } group "default" { targets = ["app-all"] } target "app" { dockerfile = "Dockerfile" tags = ["myapp/app:${TAG}"] } target "app-linux-amd64" { inherits = ["app"] platforms = ["linux/amd64"] } target "app-linux-arm64" { inherits = ["app"] platforms = ["linux/arm64"] } target "app-all" { inherits = ["app"] platforms = ["linux/amd64", "linux/arm64"] }.

Example 3: Different Development Environments.

Docker Bake makes it easy to manage builds for different environments ([website], development, testing, and production). For this, you can use variables that are overridden via the command line:

Shell variable "ENV" { default = "dev" } group "default" { targets = ["app-${ENV}"] } target "app-base" { dockerfile = "Dockerfile" args = { BASE_IMAGE = "node:16-alpine" } } target "app-dev" { inherits = ["app-base"] tags = ["myapp/app:dev"] args = { NODE_ENV = "development" DEBUG = "true" } } target "app-stage" { inherits = ["app-base"] tags = ["myapp/app:stage"] args = { NODE_ENV = "production" API_URL = "[website]" } } target "app-prod" { inherits = ["app-base"] tags = ["myapp/app:prod", "myapp/app:latest"] args = { NODE_ENV = "production" API_URL = "[website]" } }.

To build an image for a specific environment, use the command:

Docker Bake allows you to define matrices for creating multiple build variants based on parameter combinations:

Shell variable "REGISTRY" { default = "[website]" } target "matrix" { name = "app-${platform}-${version}" matrix = { platform = ["linux/amd64", "linux/arm64"] version = ["[website]", "[website]"] } dockerfile = "Dockerfile" tags = ["${REGISTRY}/app:${version}-${platform}"] platforms = ["${platform}"] args = { VERSION = "${version}" } }.

This code will create four image variants for each combination of platform and version. You can build them all with a single command.

Docker Bake allows you to use external files and functions for more flexible configuration:

Shell // Import variables from a JSON file variable "settings" { default = {} } function "tag" { params = [name, tag] result = ["${name}:${tag}"] } target "app" { dockerfile = "Dockerfile" tags = tag("myapp/app", "[website]") args = { CONFIG = "${settings.app_config}" } }.

Plain Text docker buildx bake --file [website].

Docker Bake can be integrated with Docker Compose, which is especially convenient for existing projects:

YAML # [website] services: app: build: context: ./app dockerfile: Dockerfile args: VERSION: "[website]" image: myapp/app:latest api: build: context: ./api dockerfile: Dockerfile image: myapp/api:latest.

Use case for self-healing tests with a local LLM

In this article, I will discuss the practical application of large language models (LLMs) in combination with traditional automation tools like Python/Selenium to improve test reliability.

The article consists of the following sections:

What are self-healing tests? Hardware configuration Software configuration Verifying the local LLM API Integrating with tests Limitations Future prospects.

Automated tests must be reliable to prevent false positives. Such tests increase trust in their results and enable deeper integration of automated testing into processes. Test reliability can be improved by addressing the key issues that arise during their execution. All these issues lead to false positives:

Changing properties of elements in the tested application. Unstable infrastructure. Excessive speed of the automation tool.

Based on these common issues, we define self-healing tests as those that can automatically adapt their behavior when a problem occurs. The focus of the current implementation is on solving issue #1.

The table below provides measurements of two model performance parameters on our configurations, along with a description of the configurations themselves.

Due to limited hardware availability and experimental usage, we use only one server in the first configuration.

The model lmstudio-community/[website] from HuggingFace (convenient to download via LMStudio - see item 3 below). The OpenAI pip package. LMStudio – an "IDE" that is convenient for prompt debugging, model parameter configuration, offline execution, and setting up a "model server."

(venv) user@MacBook-Pro-Admin-2 web2 % cat tttt from openai import OpenAI LLM_URL = 'http://:1234/v1' LLM_MODEL = '[website]' def call_llm(request): client = OpenAI(base_url=LLM_URL, api_key="lm-studio") completion = [website] model=LLM_MODEL, messages=[ { "role": "user", "content": [ { "type": "text", "text": request, }, ], }, ], ) return completion.choices[0].message.content print(call_llm('Yes or no?')) (venv) user@MacBook-Pro-Admin-2 web2 % (venv) user@MacBook-Pro-Admin-2 web2 % python tttt I'm sorry, but your question is not clear. Could you please provide more details or context? Enter fullscreen mode Exit fullscreen mode.

All the logic begins with the get_object method, which is accessible to the final PageObjects through the base PageObject.

Thanks to RLS (Run-time Locators Storage), all subsequent tests within the test run will not fail on the problematic element. Another key aspect is the organization of the design-time locator storage. These are separate classes that are connected to functional PageObjects. They look something like this:

class LLogin: @staticmethod def L_I18N_TEXTFIELD_LOGIN(lang=''): """login input field""" return ('xpath', f'//*[@e2e-id="I intentionally broke this locator"]') Enter fullscreen mode Exit fullscreen mode.

Thanks to the naming convention for locators (the name starts with the prefix L_) and the convention for working with page objects (access to them is only done through Page.get_object(L_I18N_TEXTFIELD_LOGIN)), we can extract the locator name from the stack trace and save it in the Run-time Locators Storage.

Thanks to the docstring of the locator method ("""login input field"""), we have an elegant solution for where to store the human-readable description of the locator, which is then used for LLM inference.

Below is a raw log demonstrating self-healing in action. See WARNING level steps.

2025-03-02 23:28:57 STEP WEB Client: Pick language 2025-03-02 23:28:57 STEP WEB Client: Authorize 2025-03-02 23:29:03 WARNING Web element with locator "('xpath', '//*[@e2e-id="I intentionally broke it"]')" not found within timeout, trying AI locator 2025-03-02 23:29:03 WARNING Problematic locator is L_I18N_TEXTFIELD_LOGIN for [website] 2025-03-02 23:29:03 WARNING AI will try to find element locator using description: "login input field" 2025-03-02 23:29:36 WARNING Store AI-locator in cache for: "[website]" for subsequent tests 2025-03-02 23:29:36 WARNING Using AI-calculated locator "('xpath', '//input[@e2e-id="[website]"]')" 2025-03-02 23:29:36 STEP WEB Client: Enter code 2025-03-02 23:29:36 STEP Mailbox: Get confirmation code PASSED [ 33%] -------------------------------------------------------------------------------------------- live log teardown -------------------------------------------------------------------------------------------- 2025-03-02 23:29:40 STEP ~~~~~END test_login_2fa~~~~~ tests/[website] --------------------------------------------------------------------------------------------- live log setup ---------------------------------------------------------------------------------------------- 2025-03-02 23:29:40 STEP ~~~~~START test_logout~~~~~ ---------------------------------------------------------------------------------------------- live log call ---------------------------------------------------------------------------------------------- 2025-03-02 23:29:50 STEP WEB Client: Pick language 2025-03-02 23:29:51 STEP WEB Client: Authorize 2025-03-02 23:29:51 WARNING Using AI-calculated locator from cache: "('xpath', '//input[@e2e-id="[website]"]')" 2025-03-02 23:29:55 STEP WEB Client: Open profile 2025-03-02 23:29:55 STEP WEB Client: Logout 2025-03-02 23:29:55 STEP ASSERT: User sees auth page Enter fullscreen mode Exit fullscreen mode.

For now, we are not making automatic commits to replace old locators with new ones generated by the LLM. Instead, we display them in the Allure investigation:

Simultaneous requests to the LLM for inference increase the TTFT (Time To First Token) for subsequent requests. For example, if you make three requests at once on our initial configuration (TTFT=35), the response for the first request will arrive in 35 seconds (OK), for the second in ~70 seconds, and for the third in ~105 seconds. This resembles a queue.

If you decide to use this approach with a configuration that results in a long TTFT, especially when many tests are running or when almost all locators fail, everything will queue up, and your Gitlab-like system will terminate the test pipeline due to a timeout.

Under limited hardware resources, you cannot simply connect all locators to the LLM — doing so could potentially create a queue due to multi-threaded test execution, even if your product is stable and developers allocate special properties for automated tests (like e2e-id in our case). To minimize this issue, a simple strategy can be employed: automatically count how many times each locator is used in a test run and connect only the most frequently used ones. For instance, when almost all your tests start from the login page, you will get the highest usage count for locators related to the login input field, password input field, and the "Login" button. By connecting these, you can prevent the failure of all tests at the very beginning.

Context size refers to the number of tokens in the prompt. The larger the context size, the higher the memory consumption. The smaller the context size, the more you will need to optimize to provide the LLM with a suitable portion of the HTML for inference. In our case, we preprocess the data—we remove unnecessary parts from the entire page_source provided by Selenium, such as style and script tags along with their content. We will likely further reduce the size for pages with a large number of elements. Our context size is 10,000 tokens. 's documentation, 1 token is approximately 3-4 characters of English text.

2024 was an exciting year at Stack Overflow. From the launch of new products and functions that came directly from integrations with global partnership......

Tomasz Jakut reflects on the evolution of web design, recalling the days when tabl......

Look Closer, Inspiration Lies Everywhere (February 2025 Wallpapers Edition).