Working with DynamoDB in Laravel

When building your Laravel applications, you may sometimes need to use a NoSQL database to store and retrieve data. One popular choice is Amazon DynamoDB, a fully managed, serverless, and highly scalable NoSQL database service provided by Amazon Web Services (AWS).

In this article, we'll take a brief look at DynamoDB. We'll then delve into how to use DynamoDB as a cache store in Laravel, and how to store Laravel models in DynamoDB using the baopham/laravel-dynamodb package.

By the end of this article, you should feel confident using DynamoDB within your Laravel applications.

What is DynamoDB?

DynamoDB is a NoSQL database service provided by Amazon Web Services (AWS). It's powerful and flexible due to its fully managed, serverless, and highly scalable design.

Because DynamoDB is fully managed, you don't need to worry about maintaining the underlying infrastructure in the same way you might need to with something like a self-hosted database. Instead, you can focus on building your application rather than managing the database.

With its serverless design, you can scale DynamoDB to meet your application's demands. A correctly configured DynamoDB table can handle large amounts of requests as your application grows.

To learn more about DynamoDB, you may want to check out the official AWS DynamoDB documentation.

Alternatively, there's a great video series on YouTube that breaks down the concepts and theories used in DynamoDB: AWS DynamoDB Guides - Everything you need to know about DynamoDB.

Using DynamoDB for caching in Laravel

Now that we have a brief understanding of DynamoDB let's examine how to use it to cache data in Laravel.

To start, you'll need to create access keys in the AWS dashboard so that Laravel can access DynamoDB. If you aren't sure how to do this, you may want to refer to the official "Identity and Access Management for Amazon DynamoDB" documentation. You'll need to keep these keys secure, as they provide API access to your AWS account, and we'll store them in our Laravel application's .env file.

You'll also want to create a new DynamoDB table called "cache" with a primary string key called "key". You may want to do this manually via the AWS dashboard or programmatically using the AWS CLI or AWS SDK.

After creating your access keys, you'll need to add them to your Laravel application's .env file:

CACHE_STORE=dynamodb

AWS_ACCESS_KEY_ID=KEY-GOES-HERE
AWS_SECRET_ACCESS_KEY=ACCESS-KEY-GOES-HERE
AWS_DEFAULT_REGION=us-east-1

In the example, we're also setting the CACHE_STORE environment variable to dynamodb to tell Laravel to use DynamoDB as the cache store.

It's important to remember that you must also set the AWS_DEFAULT_REGION environment variable to the same region as your DynamoDB table. In this particular instance, we're using the us-east-1 region.

Next, for our Laravel application to communicate with DynamoDB, we must install the aws/aws-sdk-php package. You can do this via Composer by running the following command:

composer require aws/aws-sdk-php

Your Laravel application should now be configured and ready to use DynamoDB as a cache store.

Similar to how we've discussed caching in Laravel in previous articles, you can now use the Cache facade to store and retrieve items from the cache.

For example, you can read an item from DynamoDB like so:

$value = Cache::get('key');

And you can store an item in DynamoDB like so:

Cache::put('key', 'value', $seconds);

You can also use the remember method to retrieve an item from the cache or store it if it doesn't exist:

$value = Cache::remember('key', $seconds, function () {
    return 'the-value-to-be-returned';
});

In the example above, we'll first attempt to find an item in DynamoDB with a key of key. If the item exists, we'll return the cached item. Otherwise, we'll execute the closure and store the returned value in DynamoDB with a key of key.

You can also delete items from the cache using the forget method:

Cache::forget('key');

However, it's important to remember that you can't flush an entire table in DynamoDB. This means that code such as Cache::flush() and the command php artisan cache:clear won't work as expected. Attempting to run either of these will result in a RuntimeException being thrown with the error message:

DynamoDb does not support flushing an entire table. Please create a new table.

Storing Laravel models in DynamoDB

There may be times when you want to use DynamoDB to store your Laravel models. Unfortunately, Laravel doesn't support this out of the box like it does with caching. However, we can use the popular baopham/laravel-dynamodb package to achieve this.

At the time of writing, this package has over 3.25 million downloads, 470+ stars on GitHub, and 120 forks. So it's safe to assume that it is well-maintained and popular.

For the rest of this article, we'll cover the package's features that you're most likely to use in your applications. To see all the features the package provides, you may want to check out the documentation on GitHub: https://github.com/baopham/laravel-dynamodb.

Installing the package

To get started, we'll first need to install the package via Composer by running the following command:

composer require baopham/dynamodb

You'll then need to publish the package's configuration file by running the following command:

php artisan vendor:publish --provider 'BaoPham\DynamoDb\DynamoDbServiceProvider'

After this, you'll need to add your DynamoDB access keys to your Laravel application's .env file:

DYNAMODB_KEY=DYNAMO-DB-KEY-HERE
DYNAMODB_SECRET=DYNAMO-DB-SECRET-HERE
DYNAMODB_REGION=us-east-1

It's worth noting that the package doesn't automatically use the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables that we set up earlier. Instead, it uses its own environment variables. But if you'd like to, you can update your published config/dynamodb.php file to use the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables instead.

The package should now be set up and ready to use.

Storing models in DynamoDB

To store a model in DynamoDB, you'll first need to create a table in DynamoDB to store them. You can do this manually via the AWS dashboard or programmatically with the AWS CLI or AWS SDK. For example, if you want to store a Post model in DynamoDB, you'll need to create a table called posts.

Because DynamoDB is designed to be highly scalable and performant, it doesn't support auto-incrementing primary keys like traditional databases. For this reason, you can't rely on an id column being automatically incremented like you would if using a database such as MySQL. For this reason, you may opt to use a UUID as the primary key for your models.

Let's look at a basic example of how we might want to store a Post model in DynamoDB containing information about a blog post. We'll imagine the model has the following fields:

uuid - The unique identifier for the blog post. This will be the "partition key" of the posts table in DynamoDB.
title - The title of the blog post.
slug - The slug of the blog post.
content - The content of the blog post.
created_at - The date and time the blog post was created.
updated_at - The date and time the blog post was last updated.

We'll create our model using the following command:

php artisan make:model Post

This will create a model that looks like so:

namespace App\Models;

use Illuminate\Database\Eloquent\Factories\HasFactory;
use Illuminate\Database\Eloquent\Model;

class Post extends Model
{
    use HasFactory;
}

Since we're going to be using UUIDs as the primary key for our models, we'll update the model to use Laravel's Illuminate\Database\Eloquent\Concerns\HasUuids trait and define the primary key field as uuid:

namespace App\Models;

use Illuminate\Database\Eloquent\Concerns\HasUuids;
use Illuminate\Database\Eloquent\Factories\HasFactory;
use Illuminate\Database\Eloquent\Model;

class Post extends Model
{
    use HasFactory;
    use HasUuids;

    protected $primaryKey = 'uuid';
}

By defining the primary key as uuid, the package will automatically detect that this is our partition key. When we make a query to find a post by its UUID, a "query" operation will be performed. If we don't define the primary key, a "scan" operation will be performed instead, which is less efficient and can be expensive depending on the billing model you've chosen in AWS.

We'll then need to update the Post model to extend the BaoPham\DynamoDb\DynamoDbModel class instead of Laravel's Illuminate\Database\Eloquent\Model class:

namespace App\Models;

use BaoPham\DynamoDb\DynamoDbModel;
use Illuminate\Database\Eloquent\Concerns\HasUuids;
use Illuminate\Database\Eloquent\Factories\HasFactory;

class Post extends DynamoDbModel
{
    use HasFactory;
    use HasUuids;

    protected $primaryKey = 'uuid';
}

Assuming you already have a DynamoDB table called posts set up, you can now read and write Post models to and from DynamoDB using Eloquent-like syntax.

For example, to create a new Post model, you can do the following:

$post = new Post();

$post->uuid = Str::uuid()->toString();
$post->title = 'Hello World';
$post->slug = Str::slug($post->title);
$post->content = 'This is a test post';

$post->save();

Calling $post->save() this will create a new item in the posts table in DynamoDB with the attributes we've set.

You can also query for a Post model by its UUID like so:

$post = Post::find('the-uuid-of-the-post');

This will return the Post model with the UUID of the-uuid-of-the-post.

Using indexes to query models

There will likely be times when you want to query your models based on fields other than the primary/partition key. For example, you may want to find a Post model by its slug field:

$post = Post::query()
    ->where('slug', 'the-slug-goes-here')
    ->first();

If you were to run this, DynamoDB would perform a "scan" operation with a "FilterExpression" to find the item you're looking for. This essentially involves DynamoDB reading every row in the table and finding the rows with a slug attribute equal to the-slug-goes-here. As you might imagine, this can be inefficient, especially if the table contains many rows. Depending on your billing model, this might also be more expensive as you're reading more data than you need to.

To make this more efficient, we can use a Global Secondary Index (GSI) on the slug attribute. This will allow us to perform a "query" operation instead of a "scan" operation, which is much more efficient.

Before we can use the GSI, you'll need to create it in DynamoDB on the posts table. You can do this manually via the AWS dashboard or programmatically using the AWS CLI or AWS SDK.

We can then update our Post model to state that we have an index on the slug attribute using a dynamoDbIndexKeys property:

namespace App\Models;

use BaoPham\DynamoDb\DynamoDbModel;
use Illuminate\Database\Eloquent\Concerns\HasUuids;

class Post extends DynamoDbModel
{
    use HasUuids;

    protected $primaryKey = 'uuid';

    protected $dynamoDbIndexKeys = [
        'slug_index' => [
            'hash' => 'slug'
        ],
    ];
}

In this example, we've specified that we have an index on the posts table called slug_index with a hash key of slug. We can now search for items in the posts table based on the slug attribute using the GSI.

To check that the GSI is being used, the bao-pham/laravel-dynamodb package provides a toDynamoDbQuery method that you can use to dump the query to DynamoDB without sending it:

$query = Post::query()
    ->where('slug', 'hello-world')
    ->toDynamoDbQuery();

The toDynamoDbQuery method returns an instance of BaoPham\DynamoDb\RawDynamoDbQuery, containing the operation and query sent to DynamoDB. If we were to run dd($query) after the above code, we'd see the raw contents of the returned object:

BaoPham\DynamoDb\RawDynamoDbQuery {
  +op: "Scan"
  +query: array:4 [
    "TableName" => "posts"
    "FilterExpression" => "#slug = :a1"
    "ExpressionAttributeNames" => array:1 [
      "#slug" => "slug"
    ]
    "ExpressionAttributeValues" => array:1 [
      ":a1" => array:1 [
        "S" => "hello-world"
      ]
    ]
  ]
}

In the example above, we can see from the op property that DynamoDB will perform a "Scan" operation. We can also see in the query property that we'll search the posts table for items where the slug attribute equals hello-world.

If we were to update the Post model to include the GSI, we'd see something like the following:

BaoPham\DynamoDb\RawDynamoDbQuery {
  +op: "Query"
  +query: array:5 [
    "TableName" => "posts"
    "KeyConditionExpression" => "#slug = :a1"
    "IndexName" => "slug_index"
    "ExpressionAttributeNames" => array:1 [
      "#slug" => "slug"
    ]
    "ExpressionAttributeValues" => array:1 [
      ":a1" => array:1 [
        "S" => "hello-world"
      ]
    ]
  ]
}

This example shows that a "Query" operation would be performed instead of a "Scan" operation.

As you can imagine, the toDynamoDbQuery method is a valuable tool for debugging and can help ensure that you use the GSI correctly when writing your queries.

Syncing models in DynamoDB and a traditional database

So far, we've examined how to use DynamoDB to replace a traditional database. However, there may be times when you want to use both a traditional database and DynamoDB to store a particular model's data.

For example, imagine you're building an application that provides a real-time analytics dashboard for admins. You may want to store the data in a MySQL database so that the main part of your application can read and write from it. You might then store a duplicate of the data in DynamoDB so that the analytics dashboard can read from it quickly and efficiently. This hybrid approach allows you to keep the main part of your application performant while providing a fast and efficient way to read the data for the analytics dashboard.

Let's take a look at an example of how we might convert our Post model to use both a traditional database and DynamoDB. We'll first need to switch the model back to using Laravel's Illuminate\Database\Eloquent\Model class:

namespace App\Models;

use Illuminate\Database\Eloquent\Concerns\HasUuids;
use Illuminate\Database\Eloquent\Model;

class Post extends Model
{
    use HasUuids;

    protected $primaryKey = 'uuid';
}

You may have noticed that we've also removed the $dynamoDbIndexKeys property from the model. This is because we're no longer using DynamoDB as the primary storage for the model, so it's unnecessary.

We'll then need to update the model to use the package's BaoPham\DynamoDb\ModelTrait trait like so:

namespace App\Models;

use BaoPham\DynamoDb\ModelTrait;
use Illuminate\Database\Eloquent\Concerns\HasUuids;
use Illuminate\Database\Eloquent\Model;

class Post extends Model
{
    use HasUuids;
    use ModelTrait;

    protected $primaryKey = 'uuid';
}

By adding the trait to the model, whenever a model is created, updated, or deleted in the traditional database, the same operation will be performed in DynamoDB. This means that the model data in both databases should always be in sync.

Conclusion

In this article, we briefly examined what DynamoDB is. We then discussed how to cache your Laravel application's data using DynamoDB and used the baopham/laravel-dynamodb package to store Laravel models in DynamoDB.

I hope you'll feel confident leveraging DynamoDB in your Laravel applications the next time you need a performant and highly scalable database!