Bluebird Transport Management System - Backend Part

Backend

Tech Stack Golang, Fiber, GRPC, Google Pub/Sub

Type Project

Client Bluebird

Categories Backend

Project Brief

The Bluebird Transport Management System backend is a complex golang microservice structured architecture interfaced with fiber for HTTP REST API and GRPC for inter services communication. This project delivered to one of the biggest public taxi transportation vendor in Indonesia that is going to expand their business domain into package shipment services. This backend part is architected with microservice architecture with up to 9 backend services, this backend is designed to be performant, scalable, and maintainable.

Key Features

Dashboard

Allows user to monitor the performance of transactional data, the charts shown are using recharts library.

Multi Application

One frontend project to host 2 mode application that serve different menus based on selected logged in application mode. The modes available is Headquarter mode and Branch. The headquarter focus on managing masterdata including master branche, and monitoring transactional data. The branch focus on managing order transaction and managing package transition between first mile, middle mile, and last mile.

Order Transaction and Transition (First Mile, Middle Mile, and Last Mile)

The form which the transaction came from, branch users will fill the form with the package information and the package will be processed later in different menus in branch mode application. The delivery process includes a first mile, an optional middle mile, and a last mile.

Airwaybill Information Printing

The airwaybill information printing feature allows users to print the airwaybill information for a specific package. The airwaybill information includes the package details, such as the package airwaybill number, the package sender and receiver, and the package weight. This feature utilize react-to-print package to convert rendered react dom to be inside an iframe and then print that iframe.

My Involvement

Technical Contributions

- Architecting the project by using clean architecture directory structure below:


.
├── .vscode/                 # VSCode configuration
├── assets/                  # Static assets and configurations
├── cmd/                     # Application entry points
│   ├── cron/                # Cron job commands
│   ├── database/            # Database related commands
│   ├── event/               # Event handling commands
│   ├── grpc/                # gRPC service commands
│   ├── http/                # HTTP service commands
│   └── start/               # Application startup commands
├── config/                  # Configuration management
│   ├── file/
│   │   ├── config.local.yaml
│   │   └── config.local.yaml.example
│   └── config.go
├── docs/                    # Generated Swagger documentation from Swaggo
│   ├── openapi.yaml         # OpenAPI/Swagger documentation
│   └── swagger.json         # Generated Swagger documentation
├── handler/                 # Request handlers
│   ├── cron/                # Cron job handlers
│   ├── event/               # Event handlers
│   ├── grpc/                # gRPC service handlers
│   └── rest/                # REST API handlers
├── init/                    # Initialization code
│   ├── assembler/           # Component assembly
│   ├── initiator/           # Service initialization
│   └── service/             # Definition of handler, repo, usecase, infrastructure, etc
├── internal/                # Internal packages
│   ├── app/
│   │   ├── integration/     # Public api to be consumed by another application
│   │   └── main/v1/         # Handler for REST and GRPC endpoints
│   ├── config/              # Internal configuration
│   ├── db/                  # Database
│   │   ├── entity/          # Database entities
│   │   └── migration/       # Database migration sql files
│   └── domain/              # Domain logic includes dto, repository and usecase
├── resource/                # Additional resources, e.g. translation files
├── Dockerfile               # Docker configuration
├── go.mod                   # Go module definition
├── go.sum                   # Go module checksums
├── main.go                  # Main application entry point
└── README.md                # Project readme

- Directing the database structure and flow of specific cases like integration to Keycloak, Bluebird Notification Service, and client's partner KGX Shipment API.
- Documenting flow of the entire application and integration between each service by using C4 Component Diagram.
- Create an optimal size and cacheable Dockerfile for the deployment.
- Architecting standard REST API response structure for all of the REST API endpoints.
- Develop the SAP Integration Journaling integration.
- Do bug fixing.

Non Technical Contributions

- Onboard the development team (engineer and QA) to the project.
- Led daily meetings to align team with the project timeline.
- Attend client's steering committee meetings to show the latest progress of the project and get feedback.
- Bridge client's technical requirements with my development team.
- Document the high level project overview by using C4 Diagram to let my development team understand the bigger picture of the project.
- Meet with client's technical team to coordinate for integration between client's system and my development team's system (Keycloak SSO, SAP Accounting System, Email Notification Service, and API Gateway).
- Coordinate with client's partner to integrate my development team's system to the partner system (KGX API Shipment Integration).

Project Notes

What I Learned

- The developers was not fully handling GRPC errors, this lead to an incosistent data between services. For a better error handling, the GRPC call can be called inside db transaction to roll back the transaction if an error occurs. But if the grpc call success but transaction failed, the related data in another service must be deleted. Otherwise for safer option use event sourcing with outbox pattern.
- Event sourcing made easy by using Golang Watermill package, this package helps a lot to define middleware when processing messages also changing supported pub/sub engine (Kafka / Google Pub/Sub / RabbitMQ, etc.) is quite easy and fast.
- Database is better defined without foreign key, so if a service is later refactored and some of the domain of its service moved to a new service, the migration of the data will be easy because we will ignoring the foreign key. And also eliminiating foreign key means we eliminate data integrity check when inserting, updating, or deleting the data.
- Microservice is complex because it has inter services communications, so the contract between service must be clear and the error messages must be clear to inform from which service and which process the error came from.