Refactoring Friday BOT with NestJS, Domain-driven Design (DDD), and CQRS

Friday BOT has been refactored to using NestJS framework, by following the Domain-driven Design (DDD) and the Command Query Responsibility Segregation (CQRS) this week, by merging this HUGE(307 files changed) Pull Request:

Background

For a long time, I’m planning to build Wechaty bot with an Event-Driven Architecture (EDA). The first EDA cloud service for Wechaty I have designed is the Wechaty token discovery service 5 years ago, it helps Wechaty instances to discover each other by their token, and make remote calls to each other by JSON-RPC payload, via a center WebSocket server provided by the community.

The most beautiful part of the EDA is that it can decouple the sub-systems, and make the system more flexible and scalable. In the past 5 years, I’ve built a lot of EDA services, like @chatie/io server and wechaty/io-client, I’ve learned a lot of things from them.

Besides the EDA, we are doing our best to make the Wechaty API to be concise and easy to be understand and use, by using the most intuitive names for all the APIs. We built a plugin layer based on the basic APIs, and the plugin layer successfully helps the Wechaty to be more flexible and scalable.

The Friday BOT

Friday BOT is a Wechaty bot, which is in charge of help us to manage the community. It has lots of features, for example, the most basic features are automatically accept new friend requests, sending them a welcome message, and sending them a invitation to join the community chat room.

Some of more advanced features are:

Forward messages across the 20+ different chat rooms on multiple Instant Messaging (IM) platforms, current includes 10+ WeChat groups, one Gitter channel, one QQ group, one WhatsApp group, with 3600+ developers.
Recognize the message with different languages like English and Chinese, and forward the message to the right chat room.
Help the BOT5 Club to organize the seminar and meetup, in the WeChat room, with the power of the bot5-assistant plugin.
Answer the FAQ in the chat room, with the power of the Azure Congnitive Service QnAMaker plugin.
Update the chatie statuspage to record the the community status.
Provide a Math Master Game powered by the Wechaty Vorpal pugin
Connect the Freshdesk & Intercom tickets to the Wechaty, with the power of the Freshdesk plugin and Intercom plugin.
Provide a Voting to Kickout plugin to kickout the bad guys in the community, by counting the thumb-down votes from the community.
and more …

With Friday BOT get more and more features, the code base is growing, and the community is growing.

Domain-driven Design (DDD)

Domain-driven Design easily explained

Image source: DOMAIN-DRIVEN DESIGN: STRUCTURED MODELING OF COMPLEX SOFTWARE SYSTEMS IN BANKING

Domain Driven Design (DDD) is about mapping business domain concepts into software artifacts.

— InfoQ

In my understanding, the soul of DDD is a bridge to connect the business people with the software engineers, and make sure they are in the same page and talking about the same things, efficiently.

Some useful resources:

Event Storming

Event-storming

Source: EventStorming Glossary & Cheat sheet

Event Storming is a sprint workshop for a group of both business and engineer people who, together, to analyze the business domain by drawning events flows.

Command Query Responsibility Segregation (CQRS)

CQRS

Source: Introduction to CQRS and Event Sourcing

Command Query Responsibility Segregation (CQRS) is a software architecture pattern that separates the command(write) and query(read) layers.

NestJS

Nest (NestJS) is a framework for building efficient, scalable Node.js server-side applications. It uses progressive TypeScript and combines elements of OOP (Object Oriented Programming), FP (Functional Programming), and FRP (Functional Reactive Programming).

Nest provides an out-of-the-box application architecture which allows developers and teams to create highly testable, scalable, loosely coupled, and easily maintainable applications. The architecture is heavily inspired by Angular.

It has Controlers to handle the HTTP requests, Providers to be injected as dependency (Inversify of Control, IOC), and CQRS which works out-of-the-box.

The New Architecture of Friday BOT

With Friday BOT version 1.13+ and the new architecture, we have modulized the system into a few modules, and we have a lot of features to support the new architecture.

DDD Perspective

Repository: provide the WeChat, WhatsApp, QQ, Gitter, OfficialAccount, and WXWork Wechaty instances.
Model/AggregateRoot: each Wechaty instance is treated as a model/aggregate root.
Commands & Queries & Events: they are defined to support the business logic, and they are the core of the system.

Domain Saga: Event to Command

graph LR
  classDef event fill:DarkGoldenRod
  classDef command fill:blue
  classDef query fill:green

  ME(PuppetMessageReceivedEvent):::event
  GTE[fab:fa-gitter GitterCommunityMessageReceivedEvent]:::event
  WCE[fab:fa-weixin WeChatCommunityMessageReceivedEvent]:::event
  WAE[fab:fa-whatsapp WhatsAppCommunityMessageReceivedEvent]:::event
  QQE[fab:fa-qq QQCommunityMessageReceivedEvent]:::event

  WAC[fab:fa-whatsapp WhatsAppCommunitySendMessageCommand]:::command
  QQC[fab:fa-qq QQCommunitySendMessageCommand]:::command
  WCC[fab:fa-weixin WeChatCommunitySendMessageCommand]:::command
  GTC[fab:fa-gitter GitterCommunitySendMessageCommand]:::command

  S{Saga}
  GTS{fab:fa-gitter}
  WCS{fab:fa-weixin}
  WAS{fab:fa-whatsapp}
  QQS{fab:fa-qq}

  ME==>S

  subgraph Events Saga
    S-->GTE
    S-->WCE
    S-->QQE
    S-->WAE
  end

  subgraph Commands Saga
    GTE-->GTS
    GTS-->WAC
    GTS-->QQC
    GTS-->WCC
    
    WCE-->WCS
    WCS-->WAC
    WCS-->QQC
    WCS-->GTC
    
    WAE-->WAS
    WAS-->GTC
    WAS-->QQC
    WAS-->WCC

    QQE-->QQS
    QQS-->WAC
    QQS-->GTC
    QQS-->WCC
  end

Text Message

graph LR
  subgraph Community Send Message Command Saga
    classDef event fill:DarkGoldenRod
    classDef command fill:blue
    classDef query fill:green

    CSC[CommunitySendTextMessageCommand]:::command
    SMC_S[fa:fa-signature SendMessageCommand Signature + Text]:::command
    
    TNQ[GetTalkerNameQuery]:::query
    CNQ[GetChannelNameQuery]:::query

    CSC-->CNQ & TNQ
    CNQ & TNQ --> SMC_S
  end

Non-text Message

graph LR
  subgraph Community Send Message Command Saga
    classDef event fill:DarkGoldenRod
    classDef command fill:blue
    classDef query fill:green

    CSC[CommunitySendMessageCommand]:::command
    SMC_S[fa:fa-signature SendMessageCommand Signature]:::command
    SMC_N[fa:fa-photo-film SendMessageCommand Non-text]:::command
    
    TNQ[GetTalkerNameQuery]:::query
    CNQ[GetChannelNameQuery]:::query

    CSC-->CNQ & TNQ
    CNQ & TNQ --> SMC_S
    CSC-->SMC_N
  end

Application & Domain

sequenceDiagram
    participant User
    participant Application
    participant Domain
    User->>Application: original message
    Application->>Domain: PuppetMessageReceivedEvent
    Domain->>Domain: XXXCommunityMessageReceivedEvent
    Domain->>Domain: XXXCommunitySendMessageCommand
    Domain->>Application: XXXQuery
    Application->>Domain: 
    Domain->>Application: PuppetSendMessageCommand
    Application->>User: forwarded message

Lays & Dependencies in DDD

Most enterprise applications with significant business and technical complexity are defined by multiple layers. The layers are a logical artifact, and are not related to the deployment of the service. They exist to help developers manage the complexity in the code. Different layers (like the domain model layer versus the presentation layer, etc.) might have different types, which mandate translations between those types.

DDD Layers

Domain Layer: Responsible for representing concepts of the business, information about the business situation, and business rules. S
Application Layer: Defines the jobs the software is supposed to do and directs the expressive domain objects to work out problems.
Infrastructure Layer: How the data that is initially held in domain entities (in memory) is persisted in databases or another persistent store.

Dependencies:

Dependencies between Layers in DDD

Source: Design a DDD-oriented microservice, Microsoft Docs

NestJS Perspective

Dependency Injection

Devopedia. 2022. “Dependency Injection.” Version 5, February 15. Accessed 2022-02-15.

Controller: The controller is the entry point of the HTTP sub-system.
Application: 3rd party APIs.
Core: business logics, such as statuspage and sync-community-rooms
Infrastructure: low-level supporting services, such as EnvVar, finis, etc.
WechatyEvents: the Event-driven architecture of Wechaty, such as PuppetMessageReceivedEvent, PuppetMessageSentEvent, etc.
WechatyRepository: provide the Wechaty instances.
WechatySettings: provide the settings from pre-defined properties and environment variables.
FridayModule: main module of Friday BOT, where the whole system is assembled.
Sagas: using RxJS to convert Event to Commands
Unit test: using marble to test the event streaming system

All the above modules are in the src/ folder, they are working under the Inverse of Control (IOC) pattern powered by NestJS.

To-do List

Design the CQRS system for Wechaty, and publish it as wechaty-cqrs NPM module.
Finish the Event-Driven Architecture (EDA) module for Wechaty, and publish it as wechaty-actor NPM module.
Publish a NestJS module wechaty-nest for easily integrating the wechaty-actor NPM module to NestJS

Source Code

The source code of Friday BOT is available on GitHub

You can join our Gitter network if you aren’t already a member.