Shipdeo App - Backend Part
Backend
Tech Stack
Node.JS, NestJS, Nx Monorepo, Kafka, Google Pub/Sub
Type
Product
Categories
Backend
Shipdeo is a comprehensive 3PL (Third-Party Logistics) aggregator platform that empowers Indonesian online businesses to efficiently manage their shipping operations. The platform aggregates multiple courier services, provides discounts starting at 20%, minimizes return rates, offers transparent COD handling with a fixed 3% fee, and seamlessly integrates with various e-commerce platforms including Shopify and WooCommerce.
Key Features
Dashboard
Allows user to monitor the performance of their shipment data, this charts showing shipment performance based on different metrics. The metrics are mainly compares progress of packages between their status differences based on date range.
Order Package Management
Shipdeo has its own portal application to let users manage their own order information, and later after submitted and pickup requested to the logistic partner, the courier will pickup the package to the user's pickup request address.
Shipment Pricing Check & Comparison
Shipdeo aggregates shipping rates from all supported couriers and compares them to provide the best shipping option for customers.
Airwaybill Tracking
Airwaybill tracking feature allows users to track the status of their packages in real-time. This feature provides users with the ability to monitor the progress of their packages and receive updates on their delivery status.
Shopify Integration
Shopify integration allows users to manage their orders and packages from within Shopify Admin. This plugin also enable Shopify carrier calculated shipping strategy which showing users available courier calculated shippping price on checkout page.
Airwaybill Printing
Airwaybill printing feature allows users to print the airwaybill information for a specific package. The airwaybill information includes the package details, such as the package airwaybill number, the package sender and receiver, and the package weight.
My Involvement
Technical Contributions
- Developed the base project by using architecture directory structure below:
- Setup database connectivity and configuration by using Typegoose (a MongoDB ODM).
- Developed the authentication mechanism by using oauth2-server.
- Developed library for handling Kafka messages and Google Cloud Pub/Sub messages, because at that time i didn't find a good library for this such Watermill.io for Golang.
- Developed library for sending trace transaction to Jaeger by using Opentelemetry.
.
├── Dockerfile # Dockerfile
├── README.md # README.md
├── apps
│ ├── auth-api # Auth API defined in a NestJS dedicated project inside this monorepo
│ ├── event-partner # Event Partner defined in a NestJS dedicated project inside this monorepo
│ ├── kafka-processor # Kafka Processor defined in a NestJS dedicated project inside this monorepo
│ ├── main-api # Main API defined in a NestJS dedicated project inside this monorepo
│ ├── partner-api # Partner API defined in a NestJS dedicated project inside this monorepo
│ └── portal-api # Portal API defined in a NestJS dedicated project inside this monorepo
├── jest.config.js # Jest configuration file
├── libs
│ ├── clodeo-library-js # Code libraries for javascript only helpers/services
│ ├── clodeo-library-microservice # Code libraries for microservice helpers/services
│ ├── clodeo-library-nestjs # Code libraries for NestJS helpers/services
│ ├── clodeo-library-node # Code libraries for Node.js helpers/services
│ ├── clodeo-mongo-logging-db # Code libraries for MongoDB logging database
│ ├── shipdeo-db # Shipdeo database defined in a library so apps can access this database including its entities
│ └── shipdeo-shared # Shipdeo shared services that related to business logic
├── nx.json # Nx configuration file
├── package-lock.json # Package lock file
├── package.json # Package configuration file
├── tsconfig.json # TypeScript configuration file
├── tslint.json # TypeScript linting configuration file
└── workspace.json # Nx workspace configuration file
- Developed libraries including javascript only helpers/services, microservice helpers/services, NestJS helpers/services, Node.js helpers/services.- Setup database connectivity and configuration by using Typegoose (a MongoDB ODM).
- Developed the authentication mechanism by using oauth2-server.
- Developed library for handling Kafka messages and Google Cloud Pub/Sub messages, because at that time i didn't find a good library for this such Watermill.io for Golang.
- Developed library for sending trace transaction to Jaeger by using Opentelemetry.
Non Technical Contributions
- Onboard the development team (engineer and QA) to the project.
- Assigned Engineering Manager to the team.
- Direct the technical direction of the project such as using Levenshtein Distance Algorithm for address matching, optimize MongoDB queries, using Google Cloud Pub/Sub for heavy operations based.
- Attend weekly meeting to watch and give feedback about the product weekly progress.
- Do one on one meeting with the team member to check the condition either performance or working atmosphere of the tribe's team member.
- Assigned Engineering Manager to the team.
- Direct the technical direction of the project such as using Levenshtein Distance Algorithm for address matching, optimize MongoDB queries, using Google Cloud Pub/Sub for heavy operations based.
- Attend weekly meeting to watch and give feedback about the product weekly progress.
- Do one on one meeting with the team member to check the condition either performance or working atmosphere of the tribe's team member.
Project Notes
What I Learned
- Biggest failure in this project is that i implemented this project by using monorepo. Because this project is a large scaled project that shouldn't use monorepo. If it is just a simple to medium scaled project, monorepo is a good choice. But for a large scaled project, monorepo is not a good choice. And refactoring moving this project to non monorepo is a big task.
- Another thing that i considered as a failure is that this project doesn't utilize NestJS Dependency Injection feature, this backend uses a lot singleton for faster development speed, but this is not a good practice. The downside is that classes are hard to be mocked for unit testing, so the testing are only run for integration test method only.
- This proejct had used Lerna with a Hoisted Dependency feature, but it is not a good choice for a large scaled project. Because hosting dependencies will make package versions inconsistent if one or more package has same package name but with different version, also hoisting is taking time. So then i merged all package.json files in sub packages into one package.json file and removed Lerna, the result is a more controllable package versioning inside a single package.json and faster dependency installation.
- MongoDB is pretty good for this project because this project not so much relying on database transactions, the only transaction need is document locking and MongoDB 4 at that time supports this already.
- Another thing that i considered as a failure is that this project doesn't utilize NestJS Dependency Injection feature, this backend uses a lot singleton for faster development speed, but this is not a good practice. The downside is that classes are hard to be mocked for unit testing, so the testing are only run for integration test method only.
- This proejct had used Lerna with a Hoisted Dependency feature, but it is not a good choice for a large scaled project. Because hosting dependencies will make package versions inconsistent if one or more package has same package name but with different version, also hoisting is taking time. So then i merged all package.json files in sub packages into one package.json file and removed Lerna, the result is a more controllable package versioning inside a single package.json and faster dependency installation.
- MongoDB is pretty good for this project because this project not so much relying on database transactions, the only transaction need is document locking and MongoDB 4 at that time supports this already.