How RadixTrie helped a client reduce a 30TB Postgres database to 13TB

RadixTrie was asked to help a client improve the performance, size and cost of a large Postgres database used in a critical financial crime prevention environment. The database was 30TB in size and running in a high cost AWS RDS instance, with a replica and three non production databases. Over the first year of RadixTrie’s involvement, the overall database size was reduced from 30TB to 13TB.

Large databases do not only cost money in production. They affect replicas. They affect non production databases. They affect backups. They affect performance. They affect how much work the team has to do to keep the environment stable. In this case, every saving in the production database also carried through to the replica and the three non production databases. That made the impact much bigger than a single database clean up.

The client had a Postgres database used to support a financial crime prevention application. The database had grown to 30TB and was running on AWS RDS. The environment also included one replica and three non production databases, which meant that unnecessary data in production was multiplied across the broader estate. RadixTrie was asked to help reduce the database size, improve performance and reduce cost.

The database had two major areas that needed attention. The first was an 11TB logging table. This was identified as a good target for space reduction, but the data could not simply be deleted without review. RadixTrie first had to work through the correct process with the application vendor and get agreement on what could safely be removed. The second was a 3.8TB table that contained redundant data. Removing this data was more complex. It was not a simple once off delete. The process to remove the data and prevent the same redundant data from coming back was cumbersome and had to be managed carefully over time.

RadixTrie started by identifying where the biggest space saving opportunities were inside the database. The 11TB logging table was reviewed with the application vendor. After deliberation, RadixTrie received permission to purge data older than 12 months. This was done with minimal downtime and reduced the database size by 10TB. The team then focused on the 3.8TB table with redundant data. Because the process was more complex, the work was completed over four weekend sessions across a 12 month period. Through that process, the table was reduced from 3.8TB to 316GB. RadixTrie also helped reduce the future growth rate of the table by 75 per cent. That was important because the goal was not only to clean up old data, but also to slow the rate at which the same issue returned.

The overall database size reduced from 30TB to 13TB over the first year of RadixTrie’s involvement. The saving carried through to the replica and the three non production databases, which created substantial AWS cost savings and performance improvements. The key outcomes were: • Database size reduced from 30TB to 13TB • 10TB removed from the logging table • Redundant data table reduced from 3.8TB to 316GB • Table growth rate reduced by 75 per cent • Lower storage requirement across production, replica and non production databases • Improved performance • Better control over future database growth

Before the work, the database was carrying a large amount of data that no longer needed to sit in the active environment. After the work, the estate was smaller, cleaner and more efficient. The client did not only get a smaller production database. They also benefited from reduced size across the surrounding environments, including the replica and non production databases. This made the platform easier to run, easier to maintain and less expensive to keep in AWS.

Database growth can become expensive quietly. A table grows. Logs build up. Redundant data stays in place. Copies multiply across environments. By the time the cost becomes obvious, the database estate is already carrying years of unnecessary weight. There are a few useful lessons from this project: • Old logging data should be reviewed regularly • Redundant data needs both clean up and prevention • Vendor alignment matters when the application is sensitive or specialised • Weekend work can reduce risk for larger purge activities • Reducing production size also reduces the footprint of replicas and non production databases • Good database optimisation is not only about performance, it is also about cost control

Cost reduction does not always require a new platform or a major migration. Sometimes the biggest win is already inside the database. The work is to understand what is there, what still matters and what can safely be removed. For this client, RadixTrie helped reduce a 30TB Postgres database to 13TB, while improving performance and reducing AWS cost across the wider environment. That is practical database optimisation. Less waste. Better performance. More control.

Related reading: PostgreSQL Support in South Africa: Running Postgres in Production Without the Guesswork · The Database Health Check: What a Good One Covers and What It Finds · Managed DBA Services Explained: What You Hand Over, What You Keep