Data Access Layer | DAL / DAO | Why is it needed | How to structure it

The following content is my take on this concept based on my experience.

This article also talks about Data Access Layer assuming MVC design is followed.

Every application that persists data, needs to store the data some place and retrieve it back. Usually, it is into some database through various CRUD operations.

Often, there are complex operations performed on the data before sending it in a response. These operations include merging data from different sources, filtering, validating, etc.

A DAO layer is supposed to do:

1. Simple CRUD operations on the database.
2. Perform more complex data manipulations from multiple sources and return them to the controllers.
3. Perform few data validations.

Basically, a dao layer bridges the gap between Controllers and Database (data persistence).

To perform various CRUD operations, there should be different data structures which the DAO uses to communicate with various tables/collections in the Database. The Controller also should use the same data structures to update data and send it to the DAO layer for CRUD operations.

These data structures are generally stored in Model files and are used by both DAO and Controller.

Typically, the Controllers use a model, update it and send it to DAO layer for various CRUD operations.

These CRUD operations can get more complex, where, for some controller end points, you might need data that is taken from multiple sources, merged, joined, filtered, validated, processed, etc. and then returned. If this complex set of operations are handled in the controller, then, the data access part gets tightly coupled with the Controller (Business Logic) and changing the database will become a nightmare.

Also, the controller can do simple queries from multiple sources and then do the merge, join, filter, process, etc. within the application code itself. But, this is a crappy idea because then:

1. You are bringing huge data in memory and processing it. Which is slow and might not be possible for few cases.
2. You are not using database querying capabilities which make things extremely fast and simple.

You can also use the database querying language within the Controller itself and execute your joins, etc without bringing data in memory. But this tightly couples the database access code with controller code and makes it hard to migrate from one database to another. Also, makes the controller code big and hard to understand.

So, few reasons to use a DAO layer are:

1. Segregating the data access code to allow better maintainability and easy migration of database if needed.
2. Better modularity and easy understanding.

What about data models? How should data models be structured?

I feel, for each domain module (or group of routes example: /auth/<signup, login,etc.>), there should be a set of data models.

In the case of accessing data, these data models are used by the DAO to get data and pass it on to the Controller.

In the case of updates, the controller borrows these data models, updates them and passes them on to the DAO for CRUD operations.

So, if data models change, then:

1. Do we need the Controller to change the way it it updates the data into the model before sending to the DAO?
2. Do we need the DAO to change the way it updates the model before it sends it to the database / controller?

Few cases which can occur here:

1. The structure of the data (schema of a table or json keys pattern in a collection) is changed but the fields are same. Example: Some fields of a json are now re-structured to be in another parent key. In these cases, having setter / getter methods will help. The controller / DAO code need not change. Only the setter getter methods should be updated and the rest of the code will fall into place. Getter setter methods also help in making necessary validations to the data before retrieving or saving them.
2. The structure of the data (schema of a table or json keys pattern in a collection) is changed, new fields are added and old fields are deleted. This means there is some change in business logic (additional validations, filters, etc.), hence, there will be updates needed in Controller, DAO and model.

My Conclusion:

1. Every module (or group of routes example: /auth/<signup, login,etc.>) should have its own DAO layer.
2. Every module (or group of routes example: /auth/<signup, login,etc.>) should have its own set of Models.
3. DAO layer should use database queries as much as possible for various data manipulations.
4. Models should have their own getters and setters functions which should be used by the Controller / DAO for data updation and retrieval.

Thank you!