Apache Kafka για Έμπειρους Μηχανικούς*

This training provides the needed information in order to use Apache Kafka in production and build robust and performant applications on top of it. We highlight the popular use cases: message bus for event-driven microservices, stream-processing applications, and large-scale data pipelines. We also focus on making the training general and comprehensive enough so it will be useful to anyone using Kafka, no matter the use case or architecture. Topics such as how to install and configure Kafka and how to use the Kafka APIs are covered. Finally, Kafka’s design principles are presented and discussed.

The main audience is software engineers who develop applications that use Kafka’s APIs and production engineers (also called SREs, devops, or sysadmins) who install, configure, tune, and monitor Kafka in production. Furthermore this training can be useful for data architects and data engineers in mind, those responsible for designing and building an organization’s entire data infrastructure. 

1. Χρησιμοποιεί Apache Kafka στην παραγωγή και στην υποστήριξη αποδοτικών εφαρμογών.
2. Εγκαθιστά και παραμετροποιεί Apache Kafka.
3. Αναπτύσσει επαγγελματικό λογισμικό.
4. Σχεδιάζει και υλοποιεί ανεξάρτητα της περίπτωσης ή της αρχιτεκτονικής.

Διαδικασία Υλοποίησης της Επιμόρφωσης
Μεικτή Μάθηση.

Μέθοδοι Υλοποίησης της Επιμόρφωσης
Το εκπαιδευτικό υλικό θα αποτελείται από βιντεοσκοπημένες ενότητες και παραδείγματα προγραμματισμού. Θα χρησιμοποιηθεί η μέθοδος της μεικτής τηλεκπαίδευσης. Στην διάρκεια των δύο εβδομάδων του εντατικού προγράμματος θα υπάρξουν έξι συνεδριάσεις σύγχρονης εκπαίδευσης (3 ανά εβδομάδα) με διάρκεια από 90 έως 120 λεπτά. Το υλικό θα διατίθεται σταδιακά.

Μέθοδοι Αξιολόγησης & Κριτήρια Απόδοσης Πιστοποιητικού Επιμόρφωσης
Η μέθοδος αξιολόγησης θα αποτελείται από εργασίες με την μορφή προγραμματισμού σε συχνή βάση.

Από το αποτέλεσμα των εργασιών θα προκύψει και ο τελικός βαθμός των εκπαιδευομένων.

Το πρόγραμμα δεν θα τηρεί απουσιολόγιο.

Έναρξη – Λήξη: 01/11/2021 – 02/11/2021
Σύνολο φόρτου εργασίας: 3  ώρες
Ακαδημαϊκά Υπεύθυνος: Χαράλαμπος Σκιάνης

In this session, we provide the important information for trainees to understand the concept of publish/subscribe messaging and why it is important. Publish/subscribe messaging is a pattern that is characterized by the sender (publisher) of a piece of data (message) not specifically directing it to a receiver. Instead, the sender (publisher) classifies the message somehow, and the receiver (subscriber) subscribes to receive certain classes of messages. Pub/sub systems often have a broker, a central point where messages are published, to facilitate this. Furthermore, we will describe how to get started with the Apache Kafka broker, including how to set up Apache Zookeeper, which is used by Kafka for storing metadata for the brokers. It will also cover the basic configuration options for a Kafka deployment, as well as criteria for selecting the correct hardware to run the brokers on. Finally, it will cover how to install multiple Kafka brokers as part of a single cluster and some specific concerns when using Kafka in a production environment.

• Εβδομαδιαία συνεδρία: Introduction to Kafka & Kafka Installation
  Διδάσκων: Θωμάς Πλιάκας

Έναρξη – Λήξη: 03/11/2021 – 04/11/2021
Σύνολο φόρτου εργασίας: 3 ώρες
Ακαδημαϊκά Υπεύθυνος: Χαράλαμπος Σκιάνης

This session provides information on how to use the Kafka producer, starting with an overview of its design and components. It will show how to create KafkaProducer and ProducerRecord objects, how to send records to Kafka, and how to handle the errors that Kafka may return. We’ll then review the most important configuration options used to control the producer behavior. Finally, a deeper look at how to use different partitioning methods and serializers, and how to write your own serializers and partitioners. At the consumer level we will present how applications that need to read data from Kafka use a KafkaConsumer to subscribe to Kafka topics and receive messages from these topics. Since, reading data from Kafka is a bit different than reading data from other messaging systems, and there are few unique concepts and ideas involved. We’ll start by explaining some of the important concepts, and then we’ll go through some examples that show the different ways consumer APIs can be used to implement applications with varying requirements.

• Εβδομαδιαία συνεδρία: Kafka Producers & Consumers
  Διδάσκων: Θωμάς Πλιάκας

Έναρξη – Λήξη: 05/11/2021 – 07/11/2021
Σύνολο φόρτου εργασίας: 4 ώρες
Ακαδημαϊκά Υπεύθυνος: Χαράλαμπος Σκιάνης

In this session, we focus on three topics that are especially relevant to Kafka practitioners: (1) How Kafka replication works; (2) How Kafka handles requests from producers and consumers; (3) How Kafka handles storage such as file format and indexes. By understanding these topics in-depth will be especially useful when tuning Kafka and understanding the mechanisms that the tuning knobs control goes a long way toward using them with precise intent rather than fiddling with them randomly.

• Εβδομαδιαία συνεδρία: Kafka Internals
  Διδάσκων: Θωμάς Πλιάκας

Έναρξη – Λήξη: 08/11/2021 – 09/11/2021
Σύνολο φόρτου εργασίας: 3 ώρες
Ακαδημαϊκά Υπεύθυνος: Χαράλαμπος Σκιάνης

In this session, we will start by talking about different kinds of reliability and what they mean in the context of Apache Kafka. Then we will talk about Kafka’s replication mechanism and how it contributes to the reliability of the system. Furthermore, we will discuss Kafka’s brokers and topics and how they should be configured for different use cases. Then we will discuss the clients, producer, and consumer, and how they should be used in different reliability scenarios. Last, we will discuss the topic of validating the system reliability, because it is not enough to believe a system is reliable.

• Εβδομαδιαία συνεδρία: Reliable Data Delivery
  Διδάσκων: Θωμάς Πλιάκας

Έναρξη – Λήξη: 10/11/2021 – 11/11/2021
Σύνολο φόρτου εργασίας: 3 ώρες
Ακαδημαϊκά Υπεύθυνος: Χαράλαμπος Σκιάνης

In this session,  we will present cross-cluster mirroring concepts of all or part of the data, including some use cases and architectures that are used to implement these use cases and provide the pros and cons of each architecture pattern. Furthermore, we will  discuss MirrorMaker, and how to use it, including operational tips, including deployment and performance tuning. Finally, we will also present some other alternatives

• Εβδομαδιαία συνεδρία: Cross Clustering Mirroring
  Διδάσκων: Θωμάς Πλιάκας

Έναρξη – Λήξη: 12/11/2021 – 15/11/2021
Σύνολο φόρτου εργασίας: 4 ώρες
Ακαδημαϊκά Υπεύθυνος: Χαράλαμπος Σκιάνης

In this session, we will present how to use several command-line interface (CLI) utilities that are useful for making administrative changes to the Kafka clusters. The tools are implemented in Java classes, and a set of scripts are provided to call those classes properly. These tools provide basic functions, but you may find they are lacking for more complex operations. Furthermore, we will present the most critical metrics to monitor Apache Kafka all the time, and how to respond to them. We’ll also describe some of the more important metrics to have on hand when debugging problems. This is not an exhaustive list of the metrics that are available, however, because the list changes frequently, and many will only be informative to a hardcode Kafka developer.

• Εβδομαδιαία συνεδρία: Administering & Monitoring Kafka
  Διδάσκων: Θωμάς Πλιάκας