[DEALS] Microsoft Project 2021 Professional: Lifetime License for Windows (92% off) & Other Deals Up To 98% Off – Offers End Soon! - Related to i, optimization, db, (92%, off)

[DEALS] Microsoft Project 2021 Professional: Lifetime License for Windows (92% off) & Other Deals Up To 98% Off – Offers End Soon!

Withmonthly unique visitors and overauthors we are placed among the top Java related sites around. Constantly being on the lookout for partners; we encourage you to join us. So If you have a blog with unique and interesting content then you should check out ourpartners program. You can also be afor Java Code Geeks and hone your writing skills!

I often wonder what it’s like working for the Chrome team. You must get issued some sort of government-level security clearance for the latest browser......

System Analysis and Design has to do with studying and examining the existing system in order to understand the problems ......

Writing Logs to Files in [website] with LogLayer.

Writing logs to files is a crucial aspect of application monitoring and debugging in production enviro......

Scaling & DB Optimization

Vertical Scaling means increasing the size of your machine so that it can take more load essentially more number of req/sec.

Vertical Scaling in Single Threaded language*.

In languages like NodeJS which are single threaded,.

Is there any benefit of doing Vertical Scaling?

As you can see the program is only running on single core, it's not utilizing other cores.

So it is not advised to vertically scale your NodeJS application more. However some vertical scaling is good as it will increase the memory and storage of our machine.

NodeJS is for IO intensive tasks, like Reading and writing to the DB, Reading & writing to files, making network requests.

If you want to do computational heavy work then you can use something called cluster modules & worker threads . This way your Node JS application can spawn multiple threads.

Vertical Scaling in Multi-Threaded language*.

In languages like Rust/Go/Java we can spawn multiple threads and there will be one main program that handles all that.

How can you support a certain SLA given some traffic?

We would use Auto Scaling Groups for scaling our servers.

If our avg. CPU usage is 50% then we have scale up, if it is less then we need to scale down.

Here is how, we handle the spikes, we have machines all across the regions and maybe different cloud providers also. Every 5 Minutes the server sends the no of requests it is receiving to a aggregator service which then make decision does we have scale it or not.

IN Case of persistant Connections, Like in Chess app.

Here the no of connections a server can keep is very less as they are ws connections. So in case of Scaling up the server the process will be same.

But in case of scaling down, we need to reconnect the individuals as they got disconnected for sometime.

In Streaming Platform we use HLS Protocol [HTTP Live Streaming] and data is received in chunks.

In Case of Apps like Replit & Video Transcoding App.

In case of this we can't use a queue based approach as we need some compute instantly.

So we keep a warm pool of computers ready as they get occupied we increase our servers. YOUTUBE.

The best way is to use a queue but in this case as our queue gets very big we scale our workers/servers.

It is a contract between a service provider and a customer that defined the expected performance and availability of the service.

Deals between Jio Hotstar and AWS, AWS promise [website] uptime.

So there can only be [website] downtime.

Now Let's say IPL Happens 2 time in a year.

So we have approx 25-30 minutes to up system if they crash.

we have to use some monitoring service/ aggregator service that sees and autoscale.

Increasing the numer of instances based on some metric so that our application can support more load.

AWS has concept of Auto Scaling Groups that autoscale the instances based on some metrics.

Another approach would be autoscaling via containers as they spinoff very fast.

Load Balancer : It's the entry point for the user's request after that it forward that to one of many machines [ Target Groups ], It is fully managed by the AWS so we don't need worry about scaling this. It's highly available.

: It's the entry point for the user's request after that it forward that to one of many machines [ ], It is fully managed by the AWS so we don't need worry about scaling this. It's highly available. Images [AMI - Amazon Machine Image] : Snapshot of a machine from which we can create more machine.

: Snapshot of a machine from which we can create more machine. Target Group : A Group of EC2 Machine that a Load Balancer can send request to, when ASG increases the machines it assign that instance to a particular Target Group.

: A Group of EC2 Machine that a Load Balancer can send request to, when ASG increases the machines it assign that instance to a particular Target Group. Launch Template : A Template that can be used to start new machine. [ Same as Docker Compose file ].

: A Template that can be used to start new machine. [ Same as Docker Compose file ]. SSH : Its an network protocol used for securely accessing and managing remote computers.

: Its an network protocol used for securely accessing and managing remote computers. Inbound Rule : They are firewall or security Group rules that control the incoming network traffic to a server.

This of it like Index Page of a Book, with that we can easily access content of the book. Our Reads gets very Fast.

When we create an Index, a new Data Structure is Created usually B-Tree that stores mapping from indexed_column to its location. Search on Index is `Log(n)`, cuz the Indexed_column is sorted, and we just need to do a binary search. But after doing this optimization, our writes will take much time as we have to add that to our Index also. Enter fullscreen mode Exit fullscreen mode.

It is a process of removing redundant data from DB. Redundant Data means data is present more than 1 place.

Normalizing Data : Decomposing table to eliminate data redundancy and data integrity.

1NF 2NF 3NF BCNF (Boyce-Codd Normal Form [advanced form of 3NF]) 4NF 5NF.

The more we move down ,our table get more normalized. But over normalization is also not recommended cuz we have to do more joins.

A single cell must not hold more than one value (atomicity) ### 2NF.

Has 0 partial dependency. [if table has composite primary key and some column is dependent is only one PK then it need to be resolved.].

The world of web development has seen massive changes over the past decade. Traditionally, .NET developers have used the [website] MVC (Model-View-Contr......

The Digital Playbook: A Crucial Counterpart To Your Design System.

Design systems play a crucial role in today......

When we introduced GitHub Copilot back in 2021, we had a clear goal: to make developers’ lives easier with an AI pair programmer that helps them write......

How I Made My Liberty Microservices Load-Resilient

It started with an all-too-familiar problem: As the traffic spiked, our microservices started slowing down, some even crashing under the pressure. consumers experienced frustrating delays and outright bulk API request failures, and reliability took a hit.

IBM Liberty gave us a solid, lightweight foundation, but we needed to do more. To keep things running smoothly, we had to fine-tune our architecture and make our services truly resilient to heavy loads.

This blog is a deep dive into how we optimized various layers of the architecture in a short span of time. We cover various strategies that helped prevent crashes and keep things running smoothly. By the end, you’ll see how we transformed our fragile microservices into a rock-solid, self-healing system that can take on anything we throw at it.

We started with two goals for ourselves:

Increase the throughput to an acceptable level At peak load, the performance needs to gracefully degrade.

Here is a simplified version of the application architecture, highlighting the key components involved in the analysis.

The following are the notable architectural elements and their implementation details:

A service mesh. This is a dedicated software layer that facilitates service-to-service communication using proxies.

Security encrypts service-to-service communication using mTLS (Mutual TLS).

Traffic management uses a virtual service for intelligent routing. Sidecar Proxy (Envoy) controls how traffic flows between services.

controls how traffic flows between services. Public Ingress Gateway exposes services inside the mesh to Cloudflare (a cloud internet service).

Private Ingress Gateway controls access to services within a private network.

Private Egress Gateway manages outgoing traffic from the mesh to other cloud services.

A set of services that help in addressing issues in exposing a microservice to the Internet.

DNS resolved is used for resolving domain names.

CDN (Content Delivery Network) caches website content globally to reduce latency.

WAF (Web Application Firewall) protects against OWASP Top 10 vulnerabilities.

DDoS mitigation prevents large-scale attacks from overwhelming services.

A load balancer distributes traffic across multiple origins.

A set of nodes that help in the orchestration of containerized applications using Kubernetes.

A cluster is used to run all the microservices. The gateway nginx microservice is implemented using the Go language. App server microservice is implemented using Java. This runs on a Liberty Server. To connect to a database, this microservice uses OpenJPA implementation.

The cluster is distributed across three zones to achieve high availability.

The layer in the architecture where all the persistent and ephemeral cache data resides.

PostgreSQL is used as the service storage to store data.

Redis cache is used as an in-memory data store.

The interface layer that customers interact using APIs.

API calls can be made by applications integrating the SDK provided by the service.

API calls can be made by automation scripts.

API calls can be made from browsers or API testing tools like Postman.

A network segment with direct "private" connectivity to the application without going to the Internet.

API calls can be made by automation scripts via private networks.

API calls can be made by applications integrating the SDK provided by the service via a private network.

Following are the technical details of the incident:

As the traffic spiked , the CPU and memory of our Java microservices spiked.

, the CPU and memory of our Java microservices spiked. Our Java microservices threads hung .

. Though we had rate limiting configured in CIS, we noticed the requests landing in our microservice more than what was configured.

configured in CIS, we noticed the requests landing in our microservice more than what was configured. Though the number of requests grew, the number of connections to the database did not grow as expected.

Requests timed out in JMeter when a load was initiated. (Load tested in our staging environment with the same configuration to reproduce the problem.).

in JMeter when a load was initiated. (Load tested in our staging environment with the same configuration to reproduce the problem.) A particular tenant’s load grew exponentially as they were testing their application. And this tenant was connecting using a private endpoint.

As the number of requests increased, the Go-based gateway nginx microservice was stable, but the Liberty-based Java app server hung.

Here are the pre-existing resilience measures that were already implemented before the incident:

Public traffic rate limiting . Public traffic enters from CIS. CIS rate-limiting configurations are enabled to manage the inflow of public endpoint requests.

. Public traffic enters from CIS. CIS rate-limiting configurations are enabled to manage the inflow of public endpoint requests. Microservices were running with a larger number of pods so that the high availability was built in. Gateway microservices were configured to run with three instances (pods). The app server was configured to run with three instances (pods). pgBouncer was configured to run with three instances (pods).

was built in. PgBouncer is a lightweight connection pooler for PostgreSQL that improves database performance, scalability, and resilience. Connection pooling is configured to limit the number of active database connections to prevent PostgreSQL from getting overwhelmed. Transaction and statement-level pooling. Configured with session pooling, where each client gets a dedicated database connection (default mode).

is a lightweight connection pooler for PostgreSQL that improves database performance, scalability, and resilience. Gateway nginx timeout . The timeout configured for proxy_read_timeout was 30 seconds so that if the app server microservice does not return the response on time, the request gets a timeout in 30 seconds.

. The timeout configured for was 30 seconds so that if the app server microservice does not return the response on time, the request gets a timeout in 30 seconds. Istio ingress gateway timeout . The timeout configured for Istio was 60 seconds. If the gateway nginx does not return the response to Istio on time, the request gets timed out in 30 seconds. Timeouts were not in sync, and we noticed many HTTP-504 error codes as responses.

. The configured for Istio was 60 seconds. If the gateway nginx does not return the response to Istio on time, the request gets timed out in 30 seconds. Timeouts were not in sync, and we noticed many error codes as responses. Resource requests and limits were configured for all our microservices correctly.

were configured for all our microservices correctly. Maximum number of allowed connections on PostgreSQL . For enhanced management of connections and enhanced resilience, the total number of connections allowed was set to 400 in PostgreSQL.

. For more effective management of connections and more effective resilience, the total number of connections allowed was set to 400 in PostgreSQL. Data that’s frequently retrieved is also stored in memory cache (Redis) to avoid latency to connect to PostgreSQL for data retrieval.

to avoid latency to connect to PostgreSQL for data retrieval. SDK implements web sockets to get data on demand. SDK also caches the data from the server in the runtime memory of the customer application. SDK makes calls to the server only on any updates that are being made to the application data. On any data modification, the server initiates a WebSocket event for the connected clients. These clients then make a GET call to the server to get up-to-date information.

in the runtime memory of the customer application.

Strategies Introduced to Improve Load Resilience.

Here are the strategies we introduced to improve resilience during the incident:

Liberty thread pool management is enabled with maxTotal (the maximum number of threads) set to control thread spawning. Without this limit, Liberty may continuously create new threads as demand rises, leading to high resource (CPU and memory) consumption. Excessive thread creation increases context-switching overhead, slowing down request processing and increasing the latency. If left uncontrolled, it could also exhaust JVM resources, potentially causing system instability. Along with maxTotal other parameters related to InitialSize , MinIdle , MaxIdle , and MaxWaitMillis were also set.

(the maximum number of threads) set to control thread spawning. Without this limit, Liberty may continuously create new threads as demand rises, leading to high resource (CPU and memory) consumption. Excessive thread creation increases context-switching overhead, slowing down request processing and increasing the latency. If left uncontrolled, it could also exhaust JVM resources, potentially causing system instability. Along with other parameters related to , , , and were also set. The maximum number of HTTP request threads available in Liberty is 200. So, maxTotal is set to 200 to control the number of threads spawned by the server at runtime.

is set to 200 to control the number of threads spawned by the server at runtime. Setting the above configuration helped control the threads spawned, preventing thread hangs.

Pgbouncer thread pool management was configured with pool_mode set to session and max_client_conn set to 200 . However, session mode did not perform as expected for our application, requiring an improvement to transaction mode, which is also the recommended configuration.

set to and set to . However, session mode did not perform as expected for our application, requiring an enhancement to mode, which is also the recommended configuration. With three instances of pgBouncer and max_client_conn set to 200 , up to 600 connections could be established with the PostgreSQL database. However, since PostgreSQL is configured with a maximum of 400 connections for optimal performance, we needed to adjust the max_client_conn to 100 per instance.

set to , up to 600 connections could be established with the PostgreSQL database. However, since PostgreSQL is configured with a maximum of 400 connections for optimal performance, we needed to adjust the to 100 per instance. With the pgBouncer connection pool modification, the connections established to the database were controlled with 300+ connections. Also, with the Liberty thread pool updates, the number of requests handled successfully, without thread hung, and without much latency increased.

Nginx proxy_read_timeout was initially set to 30 seconds, matching the Istio timeout of 30 seconds. To maintain consistency, we increased both timeouts to 60 seconds. As a result, requests now time out at 60 seconds if no response is received from the upstream. This adjustment helped reduce 504 errors from Istio and allowed the server to handle more requests efficiently.

Cloudflare rate limiting was in place for requests coming from the public end point to the service. However, rate limiting was missing on the Istio ingress private gateway. As traffic on the private end point increased, we immediately implemented rate limiting on the Istio private gateway, along with a Retry-After header. This effectively controlled the number of requests reaching the Nginx gateway microservice, ensuring more effective load management.

pgBouncer was running on an older version, so we upgraded to the latest version. While this did not have a direct impact on resilience, we used the opportunity to benefit from the new version.

Overall improvements achieved during the incident with the above configuration updates are:

The latency of a GET request retrieving 122KB of data, which involves approximately 7-9 database calls, was improved from 9 seconds to 2 seconds even under a load of 400 concurrent requests to the API.

even under a load of to the API. The number of requests handled concurrently improved by 5x times .

. Errors reduced drastically. The customer started noticing only 429 (too many requests) if too many requests were hit within a specific period.

Teamwork drives fast recovery . The technical team collaborated effectively, analyzing each layer independently and ensuring a quick resolution to the incident. Cross-functional efforts played a crucial role in restoring stability.

. The technical team collaborated effectively, analyzing each layer independently and ensuring a quick resolution to the incident. Cross-functional efforts played a crucial role in restoring stability. Logging is key . Comprehensive logging across various layers provided critical insights, allowing us to track: The total number of requests initiated The total number of failing requests Thread hangs and failures The number of requests hitting private endpoints.

. Comprehensive logging across various layers provided critical insights, allowing us to track:

These logs helped us pinpoint the root cause swiftly.

Monitoring enables real-time insights . With active monitoring, we could track live database connections, which helped us fine-tune the connection pool configurations accurately, preventing resource exhaustion.

. With active monitoring, we could track live database connections, which helped us fine-tune the connection pool configurations accurately, preventing resource exhaustion. Master what you implement.

Kubernetes expertise allowed us to access pods, tweak thread pools , and observe real-time behavior before rolling out a permanent fix.

allowed us to access pods, tweak thread pools and observe real-time behavior before rolling out a permanent fix.

Istio rate limiting was applied immediately, helping balance the load effectively and preventing service degradation .

was applied immediately, helping balance the load effectively and preventing service degradation Fail gracefully . Returning HTTP 504 Gateway Timeout left the API clients with not much option but to declare failure. Instead, we returned an HTTP 429 Too Many Requests . This gave a more accurate picture, and the API clients can try after some time.

. Returning HTTP left the API clients with not much option but to declare failure. Instead, we returned an HTTP . This gave a more accurate picture, and the API clients can try after some time. Feature flags for dynamic debugging. Running microservices behind feature flags enabled on-demand debugging without requiring server restarts. This played a vital role in identifying bottlenecks, particularly those caused by database connection limits, which in turn reduced MTTR.

Building resilient microservices is not just about choosing the right tools — it's about understanding and fine-tuning every layer of the system to handle unpredictable traffic spikes efficiently. Through this incident, we reinforced the importance of proactive monitoring, optimized configurations, and rapid collaboration in achieving high availability and performance.

By implementing rate limiting, thread pool management, optimized database connections, appropriate failure codes, and real-time observability, we transformed our fragile system into a self-healing, scalable, and fault-tolerant architecture. The key takeaway? Master what you implement — whether it's Kubernetes, Istio, or database tuning — deep expertise helps teams respond quickly and make the right decisions under pressure.

Resilience isn’t a one-time fix — it’s a mindset. Keeping a system healthy means constantly monitoring, learning from failures, and improving configurations. No more crashes — just a system that grows and adapts effortlessly, and in an unlikely situation, degrades gracefully!

Amazon ne veut pas que les candidat(e)s aux offres d'emploi puissent utiliser l'IA / GenIA pour rédiger un CV, une lettre de motivation ou lire des ré......

Meta has in recent times introduced data logs as part of their Download Your Information (DYI) tool, enabling individuals to access additional data about their pro......

Microservice architecture has become the standard for modern IT projects, enabling the creation of autonomous services with independent lifecycles. In......