Course Overview
This hackathon teaches learners how to build and deploy a microservice application that can handle complex delivery scenarios using Azure Event Hubs or HDInsight Kafka and modern DevOps tools. The challenges focus on implementing asynchronous communication between microservices with the power of message brokers, containerizing all microservices and their deployment to a Kubernetes cluster, automating the CI/CD process, and connecting the legacy system with the modern one through message communication.
Course Content
Microservice Application
The main resource with which the team will start their work is a microservice application that has already been built for them. The application consists of a frontend and a certain number of decoupled services that communicate with each other in various ways supported in the microservice architecture, one of which must be completely asynchronous communication using the chosen message broker.
The “New World Distribution” Application represents a new digital system of a large global consumer goods distributor. By using this system, the company plans to conquer the world and accelerate the delivery process of its goods. At the same time, the old system, an application that runs at multiple locations, must continue to work uninterrupted, but the delivery of all orders should work through the new central system.
Online Shop
Online Shop is a web frontend service developed in Angular. The online shop needs to be able to communicate with the Product Catalog and Shopping Cart APIs.
Product Catalog
The Product Catalog service is responsible for information about products and their prices, categories, manufacturers, etc. It enables standard CRUD operations, including searching for products and pagination when retrieving larger amounts of data. The service stores data in its own database. Changing basic product information (such as prices) needs to be communicated asynchronously to the Shopping Cart service, which needs to update all active items in the cart.
Shopping Cart
The Shopping Cart service takes care of all currently active carts. It enables standard CRUD operations. The service stores data in its own database and must be able to receive a message about any changes to basic product information within the Product Catalog service and update the information in active carts accordingly.
Shipping Service
The Shipping service represents the central place for processing orders that can come from the new system or legacy applications. The service processes data that comes to it via message broker and stores processed orders in its own database. When a larger amount of data arrives, the service must be able to scale.
Legacy System
The legacy system allows ordering from various locations. Each system implementation at a particular location uses a local database (SQL Server). The system does not have APIs for communication and represents a black box, which means that it cannot be changed. However, It is possible to access the database and implement a system to monitor changes in certain tables and send them as a stream to message broker (Change Data Capture – CDC pattern).
Course Objectives
Learning Outcomes
- Understand and implement messaging services (Azure EventHub, HDInsight Kafka) for microservices.
- Create and manage topics; establish communication strategies between APIs.
- Develop skills in writing Docker files, testing containerized applications, and using Container Registries.
- Deploy applications using Kubernetes, manage clusters, and test deployments.
- Implement CI/CD pipelines with GitHub Actions or Azure DevOps, ensuring code validation and automated deployment.
- Set up monitoring tools (Prometheus/Grafana or cloud-native systems), create dashboards, and set alert policies.
- Understand legacy applications, implement Change Data Capture tools, and integrate with Kafka for real-time data processing.
Outline: Microservices in Azure Hackathon (MCSAZ-HACK)
Challenges
Challenge 1: Microservices and Asynchronous communication.
Choose between Azure EventHub and HDInsight Kafka. Create service/cluster. Create a client machine and test the connection to the cluster. Create a topic and test it by producing and consuming data. Explore the application, design a communication strategy, and create the topic(s) for production. Establishing communication between the Shopping Cart API, EventHubs/HD Insight Kafka (Queue), and Shipping API. Development of the Shipping API. Connecting the Shopping Cart and Shipping API with EventHubs/HD Insight Kafka.
Challenge 2: Containerization.
Writing Docker files for all microservice components. Local testing of the containerized application. Pushing container images to the team’s Container Registry.
Challenge 3: Kubernetes.
Creating a team Kubernetes cluster on the selected cloud, deploying the complete application on the cluster, and testing the deployment.
Challenge 4: CI/CD.
Implementation of the continuous delivery process using GitHub Actions or Azure DevOps tools. Creation of a build pipeline as a combination of code validation (executing unit tests) and creating and pushing new container images to the Container Registry. Creation of a deployment pipeline for deploying new application versions into production.
Challenge 5: Monitoring.
Establishing tools for monitoring the implemented system (Prometheus/Grafana or the native monitoring system of the cloud vendor). Creating dashboards and alert policies.
Challenge 6: CDC Pattern
Researching the Legacy system's database. Implementing Change Data Capture tools. Connecting CDC tools with Kafka. System testing.