As promised in the last installment of this series, this post will focus upon providing some of the C# implementation details for the index/data table interface we defined for DynamoDB. This post will only cover through defining the abstract class definition for our index/data table model.
For following along with the actual files, you can view the code at https://github.com/SeanCaruthers/DynamoDB-Service-Provider
The main files to references for this post are
https://github.com/SeanCaruthers/DynamoDB-Service-Provider/blob/main/DynamoDBInterfaces.cs
and
https://github.com/SeanCaruthers/DynamoDB-Service-Provider/blob/main/DynamoDBIndexedTableService.cs
Here are the interfaces defined in the last post:
// DynamoDBInterfaces.cs
public interface IAmDataTable{ }
public interface IAmIndexTable{ }
public interface IConvertsToIndex<IndexTableEntry>
{
public IndexTableEntry ToIndex();
}
public interface IHasIndexedDynamoCRUD<IndexTableEntry, DataTableEntry, Partition, Key>
{
public Task<Key> Create(DataTableEntry instance);
public Task<DataTableEntry> Read(Partition partition, Key key);
public Task<List<IndexTableEntry>> Read(Partition partition);
public Task Update(DataTableEntry updatedInstance);
public Task Delete(Partition partition, Key key);
}
public interface IHasDynamoKeys<PartitionKey, RangeKey>{
public PartitionKey GetPartitionKey();
public RangeKey GetRangeKey();
}The basic idea is that we need to map any read all functions to an index table to prevent querying unnecessary data and running into dynamoDB’s 1MB query limit.
This is helpful if we are storing files that are around 100-300KB, as any read all request would pretty quickly reach this limit. Instead we can enforce read all request to returns only the filenames and allow the client to query the actual file data only when it is needed.
One might think that this problem would be solved merely by providing a projection or filter, but the 1MB limit is applied prior to any filter expressions.
So moving on, how do we get started with implementing something like this? First we should define an abstract class that provides default CRUD implementations through the sdk client without specifying any real data types and still enforcing the chosen constraints.
First we will start with the class definition, which has a few parts. This may make it look complicated at first, but breaking it up into pieces should provide some clarity
// DynamoDBIndexedTableService.cs
public abstract class DynamoDBIndexedTableService<
IndexTableEntry, DataTableEntry,
PartitionKey, RangeKey
> : ...
Ok, so we have a basic class definition. Our indexed table service is and abstract class and it takes in a few type arguments. These type arguments are the Index/Data table classes that we want to use as well as their shared Partition and Range keys types. Keeping these things generic will allow us to use this class with any objects that correspond to table entries
Next we add our IHasIndexedDynamoCrud interface to the class definition, passing in the generic types. This forces the compiler to show an error as long as we have not implemented the necessary generic CRUD functions in our interface, marking our progress towards completion of the abstract class.
...
IHasIndexedDynamoCRUD<
IndexTableEntry, DataTableEntry,
PartitionKey, RangeKey
>
...Next we move towards defining constraints for the generic types that we are using. Both the IndexTableEntry and DataTableEntry types should have a partition and range key, while the data table should be convertable to an index entry. This constraint ensures that create calls can create both an index and data entry without direct access to an index instance.
...
where IndexTableEntry :
IHasDynamoKeys<PartitionKey, RangeKey>,
IAmIndexTable
where DataTableEntry :
IHasDynamoKeys<PartitionKey, RangeKey>,
IAmDataTable,
IConvertsToIndex<
IndexTableEntry, DataTableEntry
>
With the class definition and interface constraints out of the way, we move on to the constructor. Note that while the class is abstract, we do define a constructor and data member that instantiate and hold a reference to the DynamoDB client. All further code will be pseudocode in order to avoid getting bogged down in details.
public DynamoDBIndexedTableService(region)
assign sdk client in region to data memberHolding the db context within the base class allows us to define default implementations of the CRUD methods with the correct context, ensuring that we don’t need to touch anything DB related other than passing along the region upon instantiation.
Next we can move to define the generic CRUD methods. I won’t go into all of the details here, but let’s take a look at a few examples.
Prior to that, it is important to understand that the DynamoDB object model for .NET allows us to map our data transfer objects and their attributes directly to DB tables using C# attributes. A basic example of an object stored in an example table might look something like the following.
[DynamoDBTable("Example")]
public class Example
{
[DynamoDBHashKey("UserID");
public string UserID { get; set; }
[DynamoDBRangeKey("filename")]
public string Filename { get; set; }
}By defining the class with the attributes, we can call db methods directly with no further context necessary other than the object’s themselves.
var entry = new Example(user, filename);
var db = // a reference to a dynamoDBContext object
await db.SaveAsync(entry);So as long as we have a valid reference to the db context object, we can save to any table that is defined for an object with the db attributes. As a basis for our crud functions, we can use this with our generic types defined for the abstract class. Here are is create, read one and read all. Note that while most of the transactional/error handling code is stripped out, we can begin to see where the various interfaces come into play.
// create an index and data entry
public async Task<RangeKey Create(DataTableEntry entry)
{
// recall the IConvertsToIndex interface
await _db.SaveAsync(entry.ToIndex());
await _db.SaveAsync(entry)
// recall the has IHasDynamoKeys interface
return entry.GetRangeKey();
}Read all takes in a partition key and returns the index entries that relate to the partition.
public async Task<List<IndexTableEntry>> Read(PartitionKey key)
{
var query = db.QueryAsync<IndexTableEntry>(key);
return await query.GetRemainingAsync();
}Read One gets an actual entry from the data table
public async Task<List<DataTableEntry>> Read(PartitionKey key, RangeKey kye)
{
return await db.LoadAsync<DataTableEntry>(partition, key);
}At this point, the majority of the abstract work is out of the way. If you’ve gotten this far, congratulations, hopefully I didn’t bore you to death. Tune in index week as we begin to define concrete implementations that make use of our generic indexed table service!
