Let’s come up with a technical task and design a system on paper. First of all, we will write down the functionality so that it is clear what will turn out in the end.
Key features:
registration and authorization in the system,
creating / changing / deleting notes,
file attachment,
categorization and tagging of notes,
full-text search.We will make a version for the browser, applications for Android and iOS may appear in the future.
Main UI pages:
authorization,
check in,
list of notes,
note card.Notes must support Markdown format and consist of a title and a body. In the TK there are intentionally no non-functional requirements, such as performance, reliability, disaster recovery, and so on, so as not to complicate the system.
System design
So, we know the functionality, let’s design. Let’s start by choosing the infrastructure.
CI / CD
We will build the system and deploy the code using CircleCI or GitHub Actions. All services will be packaged in Docker containers and run using Docker-compose.
Version control system
There is already a repository on GitHub. It will be used as a mono-repository. This will speed up development and reduce the amount of extra work with the git. All code will be freely available under the GPLv3 license.
Zipkin
We need to debug the networking between services, so we use the Opentracing standard and take Zipkin as our implementation. He has many ready-made libraries for both Python and Golang.
Search and filter
For full-text search and filtering, we’ll take the ELK stack, namely Elasticsearch and Logstash. At this stage, it’s like shooting a sparrow with a cannon, it was quite possible to organize this functionality on the storage, but:
as a training and demonstration of the capabilities of microservices, this would be a great example;
if people start using the service, then we have already thought about scaling and performance.To fill Elasticsearch with data, we will configure ETL processes and use Apache Airflow.
Logs
Since we already have Elasticsearch, we will upload the logs of all services there. To track the intersection of business process logs for different services, we will add TraceId to the log, which will remain from Zipkin. The logs will be collected by the Telegraf or FileBeat agent.
Service Discovering and Configuration
For Service Discovering, we use Consul and its DNS server. The logic is as follows: the service refers to another service by its hostname, and Consul will resolve the hostname.
