Our guest bloggers are from Heimdall Data, an AWS partner that offers “DBA assistant” tools (you can find them in the AWS Marketplace). These tools include automated caching that combines Amazon ElastiCache for Redis with relational databases, including MySQL, MariaDB, PostgreSQL, Oracle, and Microsoft SQL Server. Let them step you through how to use their tools to automate the creation and management of a Redis query cache using

Slow-performing applications reduce user engagement and customer satisfaction, and eventually lower revenue. Application-database inefficiency (for example, network latency or slow queries) is a primary cause of performance bottlenecks. This post shows how Heimdall Data’s auto-caching solution with ElastiCache improves performance with no code changes, while preventing stale cache data.

Background
Various database systems provide different features and performance. Systems such as Amazon Aurora provide dynamic scaling and performance. Another AWS Database Blog post covers horizontal (more servers) and vertical (larger servers) scaling and the trade-offs. Database scaling can be costly, especially for commercially licensed databases.

Others have used Amazon ElastiCache to improve responsiveness. Despite the simplicity of ElastiCache, developers are still challenged to know what to cache and what to invalidate, and to ensure that all data is up to date. This requires manual application code changes and database modifications.

We can now automate caching and invalidation in ElastiCache with the Heimdall Data SQL caching solution.

Architecture
The Heimdall Data Access Platform (HDAP) has two components: the Heimdall Central Manager (HCM), and the Heimdall Data Access Layer (HDAL). The Heimdall system is an intelligent query routing and caching data access layer that is installed in a distributed way on each Amazon EC2 instance.

Heimdall Data software packaging includes the following:

• Database Proxy for Amazon RDS: Aurora, MySQL, SQL Server 2008+, PostgreSQL
• JDBC Driver: Gives access to Oracle and any other JDBC-compliant database

For deployment, the only application-level change is to modify the host and port or JDBC URL to route through the Heimdall proxy/driver. The following is a sample architecture diagram for the MySQL proxy configuration:

In proxy mode (the focus of this article), there are two modes of deployment:

• Distributed mode: A proxy resides on each Amazon EC2 application instance, for optimal performance (shown in the preceding diagram).
• Centralized mode: One Amazon EC2 instance proxy services many application servers.

The proxy provides two levels of caching: 1) locally on the application server, and 2) on ElastiCache (akin to an L1/L2 cache). As SQL is sent from the application to the database, the proxy responds from the cache and routes requests to different servers (for use in load-balancing and read/write split). All this functionality is provided by Heimdall Data, requiring zero code changes.

Prerequisites
This post covers how the Heimdall system with Amazon ElastiCache and Amazon RDS can be deployed with an existing application. You can find a demo instance on the Heimdall Data website.

If you’re testing with your own infrastructure, the following is required:

• A Heimdall install instance. Directions on the various installation methods are available on the Heimdall Data site, including via the AWS Marketplace.

Script installation of the Heimdall system
If you’re installing Heimdall via the one-line installation process, it downloads and installs the newest Heimdall Central Manager and Proxy. The default user ID is admin. If an Amazon instance is successfully detected, the password is the instance ID. Otherwise, the default password is heimdall.

Example overview
The example uses an Aurora database that supports a WordPress application. To explore a live site without setting up your own infrastructure, see the Heimdall Live Demo page. Role-based security prevents modifications to the site, but you can explore all features that are provided with the sample application setup. Testing in your own environment requires configuring an instance as detailed earlier.

Getting started
Once the Heimdall Central Manager (HCM) is running on an instance, the web-based configuration interface is accessible with the server URL and port 8087 (see the live demo). For an already configured server, the default tab is the Status tab, which displays current server and system status. For a new installation, go to the Wizard tab.

Using the wizard to configure data sources
Heimdall has a configuration wizard designed for Amazon Web Services. On the Wizard tab, choose AWS Detect. Otherwise, choose Manual Configuration. The goal is to configure information on the database system and caching infrastructure.

If you see a screen requesting AWS IAM credentials, you can add these credentials in this window or add the credentials to the instance as an IAM instance role through the AWS interface. Then, choose AWS Detect again.

Step 1. The Amazon RDS cluster and ElastiCache for Redis are automatically populated once they are detected. In the drop-down list, choose the appropriate Amazon RDS cluster and ElastiCache cluster. If you don’t have an ElastiCache for Redis instance available, you can leave this blank, and later, choose the local cache option. After completing these steps, choose Next.

Step 2. Specify the database server and connection type. This includes the host name, driver, user name, password, and port:

Step 3. Provide the cache configuration. Amazon ElastiCache for Redis is automatically detected in this example.

Step 4. The next screen provides settings on logging and the use of a proxy and advanced settings. If the database is to be used as a proxy (e.g., MySQL, PostgreSQL), then select Enable Proxy and choose a proxy port.

Step 5. The summary confirmation screen allows you to change any of these settings before creating the configuration.

Choose Next, and the system provides a summary of important items to verify to ensure that the application can use the configuration.

After you choose Submit on this final page (which is customized for your setup), the configuration is updated. On the demo system, guest users receive a warning indicating that they can’t update the server configuration. Browsing the configuration tabs provides information about how the system works.

Step 6. The Virtual Databases tab provides connection information for the application. In this case, the application accesses via MySQL proxy on localhost. If you’re using your own instance, make any changes to this information, and then choose Commit to finalize the configuration.

You can also change cache settings on the Virtual Databases tab:

Step 7. The Data Sources tab provides the database connection information and settings, such as connection pooling, load balancing, high availability, and query routing (read/write split). Make sure that this tab has the information for the Amazon RDS or Aurora instance that you want to cache, and then choose Commit to finalize the database configuration.

Step 8. The Rules tab controls how queries are cached, routed, and transformed when they are received from the application. The default rules configuration is to cache all traffic that is not in transactions, forward selected traffic to a read-only source, and log query traffic. You can change rules dynamically at any time without restarting the application or database. For information about how the rules are configured, choose Help. If you’re using your own instance, make any changes to this information, and then choose Commit to finalize the rule configuration.

Step 9. When you’re configuring the application to use the Heimdall proxy, the only required change to an existing application is to change the database configuration that it uses to access the database to match the Heimdall proxy. In the MySQL demo example, the existing MySQL configuration in WordPress was changed to 127.0.0.1:3306. This change is usually straightforward, but it is specific to your application installation. For details on the URL to use for the Heimdall JDBC Driver, see the JDBC section of the Virtual Database details, or for the proxy, in the Proxy Configuration section.

Step 10. The dashboard provides information about query traffic and server performance. For the sample instance, the WordPress site is being accessed with a generated traffic load. Notice in the performance statistics how the average query time for traffic coming from the cache is about 50 microseconds compared to 1000 microseconds from the database. By deploying caching in this application, there is a data layer performance boost of over 20 times!

Other statistics collected include information about the cache hit rate and the query traffic for each server. With an over 90 percent cache hit rate, the load on the database is reduced significantly, allowing for more users to be supported on the same database infrastructure. There were no changes to the application besides the database URL/host + port change—so no coding or database system changes were required.

Conclusion
Heimdall Data automates caching for Amazon ElastiCache. Configuration is simple with zero disruption to the application or the Amazon RDS or Aurora database. Users experience up to 5x improvement in performance and scale.

Heimdall is available as a free trial on the AWS Marketplace. It can also be downloaded at the Heimdall Data website.

Resources and links

About the Author

Darin Briskman (@briskmad) is a technical evangelist at Amazon Web Services. He works with our customers to provide guidance and technical assistance on database projects, helping them improving the value of their solutions when using AWS.