使用kafka提高休息时间的质量

Who is the best? To answer this question once and for all we created a tool with which we can track scores, rankings, and statistics. Why you might ask, it is simple, playing kicker, foosball or töggele is in our DNA. So is creating awesome tools with awesome components.

谁是最好的？ 为了一劳永逸地回答这个问题，我们创建了一个工具，可以用来跟踪得分，排名和统计数据。 为什么您可能会问，这很简单，我们的基因里包含踢脚，桌上足球或töggele。 用很棒的组件创建很棒的工具也是如此。

by Julian Stampfli

朱利安·斯坦普利(Julian Stampfli)

Who does not like breaks? I do like them. One could say that I love them. There does not one day go by in our office when we don’t go to the töggelikasten to play a game. Even more so, I like them so much that I convinced the guys at Confluent that it makes perfect sense to talk about breaks together with Kafka at the KafkaSummit.

谁不喜欢休息？ 我喜欢他们 可以说我爱他们。 没有一天，我们不去töggelikasten玩游戏。 更是如此，我非常喜欢他们，以至于我说服了Confluent的家伙， 在KafkaSummit与Kafka一起讨论休息是很有意义的。

What is that game, you ask? It is famously played all around Europe and also (maybe even especially) in Switzerland. In essence, it is a fast-paced, semi-competitive, 2v2 game that crudely imitates football (ehrm, soccer). It is aptly named foosball or table soccer, depending on who you ask, and in swiss german, it is called töggele (from the noun töggel, which means figurine).

你问那是什么游戏？ 它在欧洲乃至整个瑞士都享有盛名。 从本质上讲，它是一种快节奏，半竞争性的2v2游戏，可以粗略地模仿足球(ehrm，足球)。 根据您要问的人，它被恰当地命名为桌上足球或桌上足球，在瑞士德语中，它被称为töggele(源自名词töggel，意为小雕像)。

However, playing such a game very often leads to some inconveniences.

但是，玩这样的游戏经常会带来一些不便。

Firstly, people tend to have a favorite position, either defense or offense. The issue starts when more than two players have a strong preference for one spot because they need to agree who plays in what place. The second issue is when the skill level is vastly different. Thus fair teambuilding is hard for people. It turns out this is a child’s play for a computer. Simply randomizing the teams leads to good results and no discussions apart from the occasional glare from people who were assigned the same role repeatedly.

首先，人们倾向于拥有最喜欢的位置，无论是防守还是进攻。 当两个以上的玩家强烈希望获得一个位置时，便开始出现问题，因为他们需要同意谁在哪个位置打球。 第二个问题是技能水平差异很大时。 因此，公平的团队建设对人们来说很难。 事实证明，这是孩子玩电脑的游戏。 只是随机分配团队会产生良好的结果，除了偶尔被反复分配相同角色的人眩光之外，没有讨论。

Secondly, I like to know how good I am. And for this, I want statistics, rankings, or generally, data. Thus creating a ranking seemed obvious. Additionally, to a simple ranking, I think it is vital that people who are ranked higher should receive fewer points than people who are ranked lower. Such that the spread does not get too big. Implementing some diminishing return functionality seemed crucial.

其次，我想知道自己的能力。 为此，我需要统计数据，排名或一般数据。 因此，创建排名似乎很明显。 此外，对于简单的排名，我认为至关重要的是，排名较高的人应该获得比排名较低的人更少的积分。 这样价差不会太大。 实现一些递减的退货功能似乎至关重要。

To alleviate us from those pains, we decided to band together and create an application called the Töggelomat during a hackathon.

为了减轻我们的痛苦，我们决定在黑客马拉松期间联合起来创建一个名为Töggelomat的应用程序。

So what the hell does this game have to do with Kafka? As you probably already know, Kafka is a streaming platform for event sourcing and processing, among other use-cases. Generally, most people would not consider using Kafka for such an application and would instead go for a relational database. We are not most people. For us, it makes perfect sense to think of matches as events and process them accordingly. Also, there were no time or budget restrictions because we did it in our spare time and were free to use the technology we love and learn more about it at the same time. : heart eyes:

那么这个游戏和卡夫卡到底有什么关系呢？ 您可能已经知道，Kafka是一个用于事件源和处理以及其他用例的流媒体平台。 通常，大多数人不会考虑将Kafka用于此类应用程序，而会选择关系数据库。 我们不是大多数人。 对于我们来说，将匹配视为事件并进行相应处理是非常有意义的。 另外，没有时间或预算限制，因为我们在业余时间做到了这一点，并且可以自由使用我们喜欢的技术并同时了解更多信息。 心眼：

There is one minor downside to using Kafka Streams for such a small tool. Operating a Kafka cluster is not trivial, and it makes no sense to do this overhead for a small application like this. There are two choices available to fix this. Either use an already existing cluster and write the data there or go for Confluent Cloud, which has very decent pricing for small use cases. For me, it makes perfect sense to use Confluent Cloud for that because it feels natural.

使用Kafka Streams这么小的工具有一个小的缺点。 操作Kafka集群并非易事，对于像这样的小型应用程序，这样做的开销是没有意义的。 有两种选择可解决此问题。 要么使用一个已经存在的集群并在其中写入数据，要么选择Confluent Cloud，对于小型用例来说，价格非常合理。 对我而言，为此使用Confluent Cloud非常合情合理，因为它感觉很自然。

Other things that we wanted to try out were Quarkus and ksqlDB. Quarkus is an alternative to spring boot. It has been made to build applications for the cloud and includes several libraries that work well together. ksqlDB seems like an obvious choice when you want to experiment with Kafka and use the newest and fanciest tools. I had no previous experience with it and was eager to try it out.

我们想要尝试的其他东西是Quarkus和ksqlDB。 Quarkus是Spring Boot的替代方案。 它是为云计算构建的应用程序，它包括几个可以很好地协同工作的库。 当您想尝试使用Kafka并使用最新和最先进的工具时，ksqlDB似乎是一个显而易见的选择。 我以前没有使用过它的经验，渴望尝试一下。

Given those conditions, let me tell you the story of our data. It is a story of little Emily. She applied to SPOUD and got the job. She was delighted. When she then came to the office, everyone welcomed her, and she was happy. Her start at SPOUD caused an event to be triggered, which writes into the spoud employee topic. This topic has many uses, but for this, it is used to create the players. With ksqlDB, we listen to the employee stream and devise players from them.

在这些条件下，让我告诉您我们数据的故事。 这是小艾米丽的故事。 她申请了 SPOUD 并获得了工作。 她很高兴。 当她到办公室时，每个人都欢迎她，她很高兴。 她从SPOUD开始时触发了一个事件，该事件写入了spoud员工主题。 本主题有许多用途，但为此，它用于创建播放器。 使用ksqlDB，我们可以聆听员工流并从中设计出球员。

Emily had a fun first day, with coffee, water, and a lot of snacks. At about nine, the team invited her to play a game of kicker. She was enthusiastic. She is not so good, but she likes to play regardless. The team filled out the application, and she was assigned a position. She tried her best and scored some points.

艾米丽(Emily)玩得很开心，喝咖啡，水和很多小吃。 大约九点钟，球队邀请她踢足球。 她很热情。 她不是很好，但是无论如何她都喜欢玩。 团队填写了申请表，并为她分配了一个职位。 她尽了最大的努力，并获得了一些分数。

Nevertheless, she lost the game. The team entered the score and went back to their places. The result of the match is another event. It gets published to match the results topic with the current state of the players.

然而，她输了比赛。 球队进入比分并回到他们的位置。 比赛的结果是另一个事件。 它被发布以使结果主题与玩家的当前状态相匹配。

Emily feels a bit sad because she lost and wants to check how many points she lost. In the records, she sees the entry of her last match with the calculated points. She lost 20 points. For the evaluation of the scores, winners, and losers, the service listens to the match result topic’s events. After the calculation, it publishes these specifications into another topic called enriched results.

艾米莉(Emily)感到有点难过，因为她输了，想检查一下她输了多少分。 在记录中，她会看到她的最后一场比赛与计算出的得分的分录。 她输了20分。 为了评估得分，获胜者和失败者，服务会监听比赛结果主题的事件。 计算之后，它将这些规范发布到另一个称为“充实结果”的主题中。

Emily wanted to check where she is on the ranking and noticed that she only lost 19 points. After exploring, she finds that her points changed differently than what she first expected because each player is always rewarded one point for playing, she was delighted. After publishing the enriched results, a ksqlDB definition calculates the score changes for each player. These are published in a different topic called point change. This topic is needed to enable multiple sources of points like betting or achievements. Those changes are then applied to the players, and the player stream gets updated with the new state.

艾米莉(Emily)想检查自己在排名中的位置，并注意到她只输了19分。 在探索之后，她发现自己的积分变化与她最初的预期有所不同，因为每位玩家在比赛中总是获得一个积分，她很高兴。 发布丰富的结果后，ksqlDB定义将计算每个玩家的得分变化。 这些内容发布在另一个称为“点更改”的主题中。 需要这个主题才能启用多种来源的积分，例如投注或成就。 然后将这些更改应用于播放器，并用新状态更新播放器流。

After a while, Emily wants to regain her points. She plays again with the chaps. To everyone’s surprise, Emily wins and is very happy. After checking the history, she notices that she gained more points than she lost before. The current state of the player influences the scores that are given or taken. Meaning if two lower-ranked players play against two higher-ranked ones, the latter can only win a small number of points while the former can earn a lot. This is what happened when Emily played again because she is ranked lower; she could win more points.

不久后，艾米丽(Emily)想重新获得积分。 她再次扮演小伙子。 令所有人惊讶的是，艾米丽(Emily)获胜，并感到非常高兴。 在检查历史记录后，她发现自己获得的积分比以前减少的积分还要多。 玩家的当前状态会影响给出或取得的分数。 这意味着，如果两个排名较低的玩家与两个排名较高的玩家对战，则后者只能赢得少量积分，而前者则可以赚取很多积分。 这是艾米丽再次出演的原因，因为她的排名较低； 她可以赢得更多积分。

The story of Emily ends here. And it shows very well how the data flows through the application. I used her as an example of how we used Apache Kafka to navigate through the töggelomat. This use case is rather simple and easily understandable, but the same techniques can be applied to more complex workflows. The producer of data (in our case, the entry of match results) should not have to care about what is done with it afterward. The whole process does not need to be coupled. With this architecture and data flow, we can easily extend it to other implementations. Some guys want to add a SlackBot for writing the match results; I want to add achievements and betting, some other guy probably wants to add statistics to know which side is generally better. All of that can easily be integrated by consuming and producing events to the correct topics. Finding those topics can be hard at times, but even for that, there are solutions.

艾米丽的故事到此结束。 它很好地显示了数据如何流经应用程序。 我以她为例，说明了如何使用Apache Kafka导航töggelomat。 该用例非常简单，易于理解，但是相同的技术也可以应用于更复杂的工作流程。 数据的产生者(在我们的例子中，是匹配结果的输入)应该不必担心以后要做什么。 整个过程不需要耦合。 有了这种架构和数据流，我们可以轻松地将其扩展到其他实现。 有些人想添加一个SlackBot来编写比赛结果。 我想添加成绩和投注，其他一些人可能想添加统计信息以了解哪一方通常更好。 所有这些都可以通过消耗和产生正确主题的事件来轻松集成。 有时很难找到这些主题，但是即使如此，也有解决方案。

There are some pitfalls and learnings that I think are important:

我认为有一些陷阱和学习很重要：

1. writing complex objects using ksqlDB is challenging. I would personally advise you, for now, to use different sources if you need to write nested objects.

1.使用ksqlDB编写复杂的对象具有挑战性。 我现在个人建议您，如果需要编写嵌套对象，请使用其他资源。

2. it is crucial to keep in mind that event sourcing is asynchronous. Data is not immediately available to the front end after sending it back

2.牢记事件源是异步的，这一点至关重要。 将数据发送回后，前端无法立即使用

3. cyclic dependencies of data can bite you. In our use case, the current state of the player influences the points this player can gain, which then, in turn, influences the scores the player wins. To avoid wrong reprocessing, we added the current state of the players directly within the match event. This solution works perfectly fine for us because realistically, the frequency of match events will never exceed one per five minutes. Your use case might need a different answer.

3.数据的循环依赖性可能会叮咬您。 在我们的用例中，玩家的当前状态影响该玩家可以获取的积分，进而影响该玩家赢得的分数。 为了避免错误的重新处理，我们在比赛事件中直接添加了球员的当前状态。 该解决方案对我们来说非常完美，因为实际上，比赛事件的发生频率永远不会超过每五分钟一分钟。 您的用例可能需要一个不同的答案。

With these things in mind, I think everyone can benefit from writing a simple use case with Kafka first. You will gain some experience and know-how to solve business problems fast, without sacrificing extendability. Kafka is a tool that is well suited for a lot of situations, not only for ones that need an incredible amount of load. Writing distributed systems with event sourcing leads to a decoupling of use cases. This is one of the major benefits of using Apache Kafka to improve your data culture.

考虑到这些事情，我认为每个人都可以从首先使用Kafka编写一个简单的用例中受益。 您将获得一些经验和专门知识，可以在不牺牲可扩展性的情况下快速解决业务问题。 Kafka是一种非常适合许多情况的工具，不仅适用于需要大量负载的情况。 使用事件源编写分布式系统会导致用例解耦。 这是使用Apache Kafka改善数据文化的主要好处之一。

To learn more about the Töggelomat check out my talk on the KafkaSummit or contact us on www.agoora. We can also talk with you about other Apache Kafka related solutions.

要了解更多关于Töggelomat 看看我讲的KafkaSummit或联系我们www.agoora 。 我们还可以与您讨论其他与Apache Kafka相关的解决方案。