Mongo Data Repositories

Copies of this entity may be made for Your own use and for distribution to others, provided that You do not charge any fee for such copies and further provided that each copy contains this Copyright Notice, whether distributed in print or electronically.

If You find errors or omissions in this entity, please don’t hesitate to submit an issue or open a pull request with a fix.

The RxMicro framework supports creation of dynamic repositories for interaction with databases.

To interact with Mongo DB, the RxMicro framework provides the rxmicro.data.mongo module.

1. Basic Usage

To use the rxmicro.data.mongo module in the project, the following two steps must be taken:

Inject the rxmicro.data.mongo dependency to the pom.xml file:

<dependency>
    <groupId>io.rxmicro</groupId>
    <artifactId>rxmicro-data-mongo</artifactId>
    <version>${rxmicro.version}</version>
</dependency>
<dependency>
    <groupId>io.projectreactor</groupId>
    <artifactId>reactor-core</artifactId> (1)
    <version>${projectreactor.version}</version>
</dependency>

1	Besides the `rxmicro.data.mongo` dependency, it is also recommended to add the reactive programming library.

Instead of adding a third-party reactive programming library, You can also use the java.util.concurrent library built into the JDK, but often in practice the java.util.concurrent library’s features are not enough.

Therefore, it is recommended to use the Project Reactor or the RxJava library!

Add the rxmicro.data.mongo module to the module-info.java descriptor:

module examples.data.mongo.basic {
    requires rxmicro.data.mongo;
    requires reactor.core;       (1)
}

1	When using a third-party reactive programming library, do not forget to add the corresponding module.

Once the rxmicro.data.mongo module is added, You can create a data model class and a dynamic repository:

public final class Account {

    String firstName;

    String lastName;

    public String getFirstName() {
        return firstName;
    }

    public String getLastName() {
        return lastName;
    }
}

(1)
@MongoRepository(collection = "account")
public interface DataRepository {

    (2)
    @Find(query = "{email: ?}")
    Mono<Account> findByEmail(String email);
}

1 In order for a standard interface to be recognized by the RxMicro framework as a dynamic repository for interaction with Mongo DB, this interface should be annotated with the @MongoRepository annotation.

2 The dynamic repository may contain methods that form a query to the Mongo DB. (The query that used for a request for data uses the JSON format (Specialized request format for Mongo DB) and is specified in the annotation parameters. For each operation supported by Mongo DB, the RxMicro framework defines a separate annotation: The @Find annotation describes the db.collection.find()operation.)

Since the dynamic repository is a RxMicro component, for its testing You need to use the microservice component testing approach:

The common approach recommended for testing dynamic repositories, that interact with Mongo DB, is described in the Section 3.2, “Test Templates”.

@Testcontainers
@RxMicroComponentTest(DataRepository.class)
final class DataRepositoryTest {

    @Container
    private final GenericContainer<?> mongoTestDb =
            new GenericContainer<>("rxmicro/mongo-test-db")
                    .withExposedPorts(27017);

    @WithConfig
    private final MongoConfig mongoConfig = new MongoConfig()
            .setDatabase("rxmicro");

    private DataRepository dataRepository;

    @BeforeEach
    void beforeEach() {
        mongoConfig
                .setHost(mongoTestDb.getHost())
                .setPort(mongoTestDb.getFirstMappedPort());
    }

    @Test
    void Should_find_account() {
        final Account account =
                dataRepository.findByEmail("richard.hendricks@piedpiper.com")
                        .blockOptional()
                        .orElseThrow();

        assertEquals("Richard", account.getFirstName());
        assertEquals("Hendricks", account.getLastName());
    }
}

The project source code used in the current subsection is available at the following link:

https://github.com/rxmicro/rxmicro-usage/tree/master/examples/group-data-mongo/data-mongo-basic

When compiling, the RxMicro framework searches for RxMicro Annotations in the source code and generates additional classes necessary for the integral work of the microservice.

When changing the RxMicro Annotations in the source code, DON’T FORGET to recompile the ALL source code, not just the changed file, for the changes to take effect: mvn clean compile.

2. `RxMicro Annotations`

The RxMicro framework supports the following RxMicro Annotations:

Table 1. Supported `RxMicro Annotations`.
Annotation	Description
`@Column`	Sets mapping between the field name in the `Mongo DB` document and the Java model class field name. (By default, the RxMicro framework uses the Java model class field name as the field name in the `Mongo DB` document. If the name should differ for some reason, (for example, as a field name in the `Mongo DB` document the keyword Java is used), it should be specified using this annotation!)
`@ColumnMappingStrategy`	Sets the strategy of field name formation in the `Mongo DB` document, based on the analysis of the Java model class field names. (If this annotation annotates the Java model class, then the set strategy will be used for all fields in this class. For example, if You set the default `LOWERCASE_WITH_UNDERSCORED` strategy, then the `parentId` field in the Java class will correspond to the `parent_id` field in the `Mongo DB` document.)
`@DataRepositoryGeneratorConfig`	Allows You to configure the repository generation process.
`@RepeatParameter`	Allows setting mapping between one method parameter marked with this annotation and several universal placeholders that are used in the request to `Mongo DB`.
`@Find`	Denotes a repository method that must execute a `db.collection.find()` operation.
`@Aggregate`	Denotes a repository method that must execute a `db.collection.aggregate()` operation.
`@Distinct`	Denotes a repository method that must execute a `db.collection.distinct()` operation.
`@CountDocuments`	Denotes a repository method that must execute a `db.collection.countDocuments()` operation.
`@EstimatedDocumentCount`	Denotes a repository method that must execute a `db.collection.estimatedDocumentCount()` operation.
`@Insert`	Denotes a repository method that must execute a `db.collection.insertOne()` operation.
`@Update`	Denotes a repository method that must execute a `db.collection.updateOne()` operation.
`@Delete`	Denotes a repository method that must execute a `db.collection.deleteOne()` operation.
`@DocumentId`	Denotes a model field that must be used as document unique identifier.
`@MongoRepository`	Denotes that an interface is a dynamic generated Mongo data repository.
`@PartialImplementation`	Denotes an abstract class that contains a partial implementation of the annotated by this annotation a Mongo Data Repository interface.

3. Repositories Testing

For successful functional testing of dynamic repositories, that interact with Mongo DB, it is required:

Presence of a script that creates a test database.
Mechanism for preparing a database for testing: creating a database before starting the test and deleting a database after completing the test.

3.1. Test Database

A test database was created for testing the rxmicro.data.mongo, module features, which are described in this section.

The test database consists of one account collection, which contains 6 documents describing the accounts of the test users:

Figure 1. Mongo Test Database

Scripts for creating a test database are available at the following link:

https://github.com/rxmicro/rxmicro-usage/blob/master/examples/docker-image-mongo-test-db/src/main/js/

The following classes of Java models correspond to the documents created in the test database:

public class Account {
    @DocumentId
    Long id;

    String email;

    String firstName;

    String lastName;

    BigDecimal balance;

    Role role;
}

public enum Role {

    CEO,

    Systems_Architect,

    Lead_Engineer,

    Engineer
}

For ease of studying the rxmicro.data.mongo module, You can use the ready-made Mongo DB image with the rxmicro/mongo-test-db test database.

The source code of the project used as a base for building this docker image, is available at the following link:

https://github.com/rxmicro/rxmicro-usage/tree/master/examples/docker-image-mongo-test-db

3.2. Test Templates

As a mechanism for preparing a database for testing (creating a database before starting the test and deleting a database after completing the test), it is most convenient to use docker.

To start docker containers in the functional test it is convenient to use the Testcontainers Java library:

(1)
@Testcontainers
(2)
@RxMicroComponentTest(DataRepository.class)
final class DataRepositoryTestTemplate1 {

    (3)
    @Container
    private static final GenericContainer<?> MONGO_TEST_DB =
            new GenericContainer<>("rxmicro/mongo-test-db")
                    .withExposedPorts(27017); (4)

    (5)
    @WithConfig
    private static final MongoConfig MONGO_CONFIG = new MongoConfig()
            .setDatabase("rxmicro"); (6)

    @BeforeAll
    static void beforeAll() {
        MONGO_TEST_DB.start(); (7)
        MONGO_CONFIG
                .setHost(MONGO_TEST_DB.getHost()) (8)
                .setPort(MONGO_TEST_DB.getFirstMappedPort());
    }

    private DataRepository dataRepository; (9)

    // ... test methods must be here

    @AfterAll
    static void afterAll() {
        MONGO_TEST_DB.stop(); (10)
    }
}

1	The `@Testcontainers` annotation activates the start and stop of the `docker` containers to be used in this test.
2	Since the dynamic repository is a RxMicro component, for its testing You need to use the microservice component testing approach.
3	The `@Container` annotation indicates the `docker` container that will be used in this test. As an image on the basis of which it is necessary to create the `docker` container, the `Mongo DB` ready-made image with the `rxmicro/mongo-test-db` test database is used.
4	When starting the `docker` container, You need to open the standard port for `Mongo DB`.
5	Using the `@WithConfig` annotation, the configuration available only during the test is declared.
6	Setting up the configuration to interact with the test database.
7	Before running all tests, You must start the `docker` container.
8	After starting the `docker` container, You need to read the random IP address and port that will be used when connecting to the running `docker` container.
9	When testing microservice components, it is necessary to specify a reference to the component in which the RxMicro framework will inject the tested component.
10	After completing all the tests, You must stop the `docker` container.

The main advantage of this template is the speed of testing. Since the docker container is created once before starting all test methods, the total runtime of all test methods is reduced. The main disadvantage of this template is that if any test method changes the Mongo DB state, the following test method may end with an error.

Therefore, this functional test template should be used for queries to Mongo DB that do not change the database state!

If You need to test methods that change the Mongo DB state, You should use another template:

(1)
@Testcontainers
(2)
@RxMicroComponentTest(DataRepository.class)
final class DataRepositoryTestTemplate2 {

    (3)
    @Container
    private final GenericContainer<?> mongoTestDb =
            new GenericContainer<>("rxmicro/mongo-test-db")
                    .withExposedPorts(27017); (4)

    (5)
    @WithConfig
    private final MongoConfig mongoConfig = new MongoConfig()
            .setDatabase("rxmicro"); (6)

    private DataRepository dataRepository; (7)

    @BeforeEach
    void beforeEach() {
        mongoConfig
                .setHost(mongoTestDb.getHost()) (8)
                .setPort(mongoTestDb.getFirstMappedPort());
    }

    // ... test methods must be here
}

1	The `@Testcontainers` annotation activates the start and stop of the `docker` containers to be used in this test.
2	Since the dynamic repository is a RxMicro component, for its testing You need to use the microservice component testing approach
3	The `@Container` annotation indicates the `docker` container that will be used in this test. As an image on the basis of which it is necessary to create the `docker` container, the `Mongo DB` ready-made image with the `rxmicro/mongo-test-db` test database is used.
4	When starting the `docker` container, You need to open the standard port for `Mongo DB`.
5	Using the `@WithConfig` annotation, the configuration available only during the test is declared.
6	Setting up the configuration to interact with the test database.
7	When testing microservice components, it is necessary to specify a reference to the component in which the RxMicro framework will inject the tested component.
8	After starting the `docker` container, You need to read the random IP address and port that will be used when connecting to the running `docker` container.

This template for each test method will create and drop the docker container, which may increase the total runtime of all test methods.

Therefore, select the most appropriate functional test template based on the requirements of the tested functionality!

The Testcontainers library starts the docker container before running the test and stops the docker container automatically after completing the test!

So You should start and stop the docker container manually only if You want to use one docker container for all test methods!

4. Universal Placeholder

The RxMicro framework recommends using the universal placeholder (?) as parameter value placeholder in the Mongo DB queries:

@Find(query = "{email: ?}")
Mono<Account> findByEmail(String email);

If this method invoked with the following parameter:
repository.findByEmail("welcome@rxmicro.io");,

the RxMicro framework will generate the following Mongo DB query:
{email: "welcome@rxmicro.io"}.

5. `@RepeatParameter` Annotation

The universal placeholder (?) is the simplest type of placeholders.

But unfortunately, it has one disadvantage: if a query parameter must be repeated, a developer must define a copy of this parameter:

@Find(query = "{$or: [{firstName: ?}, {lastName: ?}]}")
Mono<Account> findByFirstOrLastNames(String name1, String name2);

The @RepeatParameter annotation fixes this disadvantage.

The following code is an equivalent to the code with a copy of the name parameter:

@Find(query = "{$or: [{firstName: ?}, {lastName: ?}]}")
Mono<Account> findByFirstOrLastNames(@RepeatParameter(2) String name);

6. Mongo Operations

6.1. `@Find`

The rxmicro.data.mongo module supports the db.collection.find() operation:

@Find(query = "{_id: ?}")
Mono<Account> findById(long id);

@Find(query = "{_id: ?}", projection = "{firstName: 1, lastName: 1}")
Mono<Account> findWithProjectionById(long id);

@Find(query = "{role: ?}", sort = "{role: 1, balance: 1}")
Flux<Account> findByRole(Role role, Pageable pageable);

@Find(query = "{role: ?}", sort = "{role: ?, balance: ?}")
Flux<Account> findByRole(Role role, @RepeatParameter(2) SortOrder sortOrder, Pageable pageable);

For more information, we recommend that You familiarize yourself with the following examples:

The project source code used in the current subsection is available at the following link:

https://github.com/rxmicro/rxmicro-usage/tree/master/examples/group-data-mongo/data-mongo-find

When compiling, the RxMicro framework searches for RxMicro Annotations in the source code and generates additional classes necessary for the integral work of the microservice.

When changing the RxMicro Annotations in the source code, DON’T FORGET to recompile the ALL source code, not just the changed file, for the changes to take effect: mvn clean compile.

6.2. `@Aggregate`

The rxmicro.data.mongo module supports the db.collection.aggregate() operation:

public final class Report {

    @DocumentId
    Role id;

    BigDecimal total;
}

@Aggregate(pipeline = {
        "{ $group : { _id: '$role', total : { $sum: '$balance'}} }",
        "{ $sort: {total: -1, _id: -1} }"
})
Flux<Report> aggregate();

For more information, we recommend that You familiarize yourself with the following examples:

The project source code used in the current subsection is available at the following link:

https://github.com/rxmicro/rxmicro-usage/tree/master/examples/group-data-mongo/data-mongo-aggregate

When compiling, the RxMicro framework searches for RxMicro Annotations in the source code and generates additional classes necessary for the integral work of the microservice.

When changing the RxMicro Annotations in the source code, DON’T FORGET to recompile the ALL source code, not just the changed file, for the changes to take effect: mvn clean compile.

6.3. `@Distinct`

The rxmicro.data.mongo module supports the db.collection.distinct() operation:

@Distinct(field = "email", query = "{_id: ?}")
Mono<String> getEmailById(long id);

@Distinct(field = "role")
Flux<Role> getAllUsedRoles();

For more information, we recommend that You familiarize yourself with the following examples:

The project source code used in the current subsection is available at the following link:

https://github.com/rxmicro/rxmicro-usage/tree/master/examples/group-data-mongo/data-mongo-distinct

When compiling, the RxMicro framework searches for RxMicro Annotations in the source code and generates additional classes necessary for the integral work of the microservice.

When changing the RxMicro Annotations in the source code, DON’T FORGET to recompile the ALL source code, not just the changed file, for the changes to take effect: mvn clean compile.

6.4. `@CountDocuments`

The rxmicro.data.mongo module supports the db.collection.countDocuments() operation:

@CountDocuments
Mono<Long> countDocuments();

@CountDocuments(query = "{role:?}", skip = 0, limit = 100)
Mono<Long> countDocuments(Role role);

For more information, we recommend that You familiarize yourself with the following examples:

The project source code used in the current subsection is available at the following link:

https://github.com/rxmicro/rxmicro-usage/tree/master/examples/group-data-mongo/data-mongo-count-documents

When compiling, the RxMicro framework searches for RxMicro Annotations in the source code and generates additional classes necessary for the integral work of the microservice.

When changing the RxMicro Annotations in the source code, DON’T FORGET to recompile the ALL source code, not just the changed file, for the changes to take effect: mvn clean compile.

6.5. `@EstimatedDocumentCount`

The rxmicro.data.mongo module supports the db.collection.estimatedDocumentCount() operation:

@EstimatedDocumentCount
Mono<Long> estimatedDocumentCount();

For more information, we recommend that You familiarize yourself with the following examples:

All Supported Return Types.

The project source code used in the current subsection is available at the following link:

https://github.com/rxmicro/rxmicro-usage/tree/master/examples/group-data-mongo/data-mongo-estimated-document-count

When compiling, the RxMicro framework searches for RxMicro Annotations in the source code and generates additional classes necessary for the integral work of the microservice.

When changing the RxMicro Annotations in the source code, DON’T FORGET to recompile the ALL source code, not just the changed file, for the changes to take effect: mvn clean compile.

6.6. `@Insert`

The rxmicro.data.mongo module supports the db.collection.insertOne() operation:

@Insert
Mono<Void> insert(Account account);

For more information, we recommend that You familiarize yourself with the following examples:

All Supported Return Types.

The project source code used in the current subsection is available at the following link:

https://github.com/rxmicro/rxmicro-usage/tree/master/examples/group-data-mongo/data-mongo-insert

When compiling, the RxMicro framework searches for RxMicro Annotations in the source code and generates additional classes necessary for the integral work of the microservice.

When changing the RxMicro Annotations in the source code, DON’T FORGET to recompile the ALL source code, not just the changed file, for the changes to take effect: mvn clean compile.

6.7. `@Update`

The rxmicro.data.mongo module supports the db.collection.updateOne() operation:

@Update
Mono<Boolean> updateEntity(AccountEntity accountEntity);

@Update(filter = "{_id: ?}")
Mono<Void> updateDocument(AccountDocument accountDocument, long id);

@Update(update = "{$set: {balance: ?}}", filter = "{_id: ?}")
Mono<Long> updateById(BigDecimal balance, long id);

@Update(update = "{$set: {balance: ?}}", filter = "{role: ?}")
Mono<UpdateResult> updateByRole(BigDecimal balance, Role role);

@Update(update = "{$set: {balance: ?}}")
Mono<UpdateResult> updateAll(BigDecimal balance);

For more information, we recommend that You familiarize yourself with the following examples:

The project source code used in the current subsection is available at the following link:

https://github.com/rxmicro/rxmicro-usage/tree/master/examples/group-data-mongo/data-mongo-update

When compiling, the RxMicro framework searches for RxMicro Annotations in the source code and generates additional classes necessary for the integral work of the microservice.

When changing the RxMicro Annotations in the source code, DON’T FORGET to recompile the ALL source code, not just the changed file, for the changes to take effect: mvn clean compile.

6.8. `@Delete`

The rxmicro.data.mongo module supports the db.collection.deleteOne() and db.collection.deleteMany() operations:

@Delete
Mono<Boolean> deleteById(Long id);

@Delete(filter = "{role: ?}")
Mono<Long> deleteByRole(Role role);

For more information, we recommend that You familiarize yourself with the following examples:

The project source code used in the current subsection is available at the following link:

https://github.com/rxmicro/rxmicro-usage/tree/master/examples/group-data-mongo/data-mongo-delete

When compiling, the RxMicro framework searches for RxMicro Annotations in the source code and generates additional classes necessary for the integral work of the microservice.

When changing the RxMicro Annotations in the source code, DON’T FORGET to recompile the ALL source code, not just the changed file, for the changes to take effect: mvn clean compile.

7. Partial Implementation

If the Mongo data repository generated by the RxMicro Annotation Processor contains errors, incorrect or non-optimized logic, the developer can use the Partial Implementation feature.

This feature allows You to implement methods for the Mongo data repository on Your own, instead of generating them by the RxMicro framework.

To activate this feature, You need to use the @PartialImplementation annotation, and specify an abstract class that contains a partial implementation of method(s) for Mongo data repository:

@MongoRepository(collection = DataRepository.COLLECTION_NAME)
(1)
@PartialImplementation(AbstractDataRepository.class)
public interface DataRepository {

    String COLLECTION_NAME = "account";

    @CountDocuments
    CompletableFuture<Long> generatedMethod();

    CompletableFuture<Long> userDefinedMethod();
}

1	Using the `@PartialImplementation` annotation, the `AbstractDataRepository` class is specified.

An AbstractDataRepository contains the following content:

public abstract class AbstractDataRepository extends AbstractMongoRepository
        implements DataRepository {

    protected AbstractDataRepository(final Class<?> repositoryClass,
                                     final MongoCollection<Document> collection) {
        super(repositoryClass, collection);
    }

    @Override
    public CompletableFuture<Long> userDefinedMethod() {
        return CompletableFuture.completedFuture(100L);
    }
}

An abstract class that contains a partial implementation must meet the following requirements:

The class must be an abstract one.
The class must extend the AbstractMongoRepository one.
The class must implement the Mongo data repository interface.
The class must contain an implementation of all methods that are not generated automatically.

In terms of infrastructure, the repository methods generated and defined by the developer for Mongo data repository do not differ:

@Test
void generatedMethod() {
    assertEquals(6L, dataRepository.generatedMethod().join());
}

@Test
void userDefinedMethod() {
    assertEquals(100L, dataRepository.userDefinedMethod().join());
}

The project source code used in the current subsection is available at the following link:

https://github.com/rxmicro/rxmicro-usage/tree/master/examples/group-data-mongo/data-mongo-partial-implementation

When compiling, the RxMicro framework searches for RxMicro Annotations in the source code and generates additional classes necessary for the integral work of the microservice.

When changing the RxMicro Annotations in the source code, DON’T FORGET to recompile the ALL source code, not just the changed file, for the changes to take effect: mvn clean compile.

8. Logging

Mongo Data Repositories use the MongoDB Async Driver, so in order to activate database request logging to the Mongo DB, You must configure the MongoDB Async Driver Logger:

http://mongodb.github.io/mongo-java-driver/4.0/driver/reference/logging/.

For example, if to the classpath of the current project add the jul.properties resource:

org.mongodb.driver.protocol.level=TRACE

,then Mongo Data Repositories will generate request logs to the Mongo DB while working:

2020-03-08 13:15:03.912 [DEBUG] org.mongodb.driver.protocol.command : Sending command '{"find": "account", "filter": {"_id": 1}, "batchSize": 2147483647, "$db": "rxmicro"}' with request id 6 to database rxmicro on connection [connectionId{localValue:2, serverValue:4}] to server localhost:27017

2020-03-08 13:15:03.914 [DEBUG] org.mongodb.driver.protocol.command : Execution of command with request id 6 completed successfully in 3.11 ms on connection [connectionId{localValue:2, serverValue:4}] to server localhost:27017

Postgre SQL Data Repositories

Home

Contexts and Dependency Injection