apache风暴单元测试活动拓扑中的螺栓(apache storm unit testing a bolt in a live topology)

apache风暴单元测试活动拓扑中的螺栓(apache storm unit testing a bolt in a live topology)

我有一个风暴拓扑结构，它有一个连接到kafka队列的喷口，并将元组转发给我的螺栓进行处理。 我只想在螺栓上进行单元测试，而不是从kafka -> spout -> bolt的整个部分。 但是，我也想测试风暴拓扑实例中的螺栓，而不仅仅是它的纯功能。 原因是螺栓实际上将处理后的数据发送到cassandra数据库。

因此，实现这一目标的一种方法是制作一个测试喷口，将其连接到螺栓，并将测试元组通过测试喷口发送到螺栓。 但是，这似乎可能太多的工作，为测试。 有一个更好的方法吗？ 像劫持测试中的原始喷口发送一些测试元组？

I have a storm topology that have a spout that connects to a kafka queue and the forwards the tuple to my bolt for processing. I want to do unit testing on the bolt only, not the whole segment from kafka -> spout -> bolt. However, I also want to test the bolt in a storm topology instance, not just the pure functionality of it. The reason is that the bolt actually sends the processed data to a cassandra database.

So one way for me to achieve this is to make a test spout, connect it to the bolt, and send test tuples through the test spout to the bolt. However, that seems like maybe too much work for a test. Is there a better way to do this? Like hijacking the original spout in test to send some test tuples?

最满意答案

您可能会考虑用https://storm.apache.org/releases/0.9.6/javadocs/backtype/storm/testing/FixedTupleSpout.html或https://storm.apache.org/releases/替换您的Kafka spout 0.9.6 / javadocs / backtype / storm / testing / FeederSpout.html进行测试。

使用FixedTupleSpout进行测试的一个好处是它实现了https://github.com/apache/storm/blob/a4afacd9617d620f50cf026fc599821f7ac25c79/storm-client/src/jvm/org/apache/storm/testing/CompletableSpout.java ，所以可以与completeTopology一起使用https://github.com/apache/storm/blob/64e29f365c9b5d3e15b33f33ab64e200345333e4/storm-server/src/main/java/org/apache/storm/Testing.java#L405 。 这可以让你编写最初设置一些元组的测试，运行拓扑，直到所有的元组都出现/失败，然后让你声明例如数据写入Cassandra并且所有的元组都被占用。

有一个使用completeTopology的例子https://github.com/xumingming/storm-lib/blob/master/src/jvm/storm/TestingApiDemo.java#L83 。

You might look at replacing your Kafka spout with either https://storm.apache.org/releases/0.9.6/javadocs/backtype/storm/testing/FixedTupleSpout.html or https://storm.apache.org/releases/0.9.6/javadocs/backtype/storm/testing/FeederSpout.html for tests.

A benefit of using FixedTupleSpout for testing is that it implements https://github.com/apache/storm/blob/a4afacd9617d620f50cf026fc599821f7ac25c79/storm-client/src/jvm/org/apache/storm/testing/CompletableSpout.java, so can be used with completeTopology https://github.com/apache/storm/blob/64e29f365c9b5d3e15b33f33ab64e200345333e4/storm-server/src/main/java/org/apache/storm/Testing.java#L405. This can let you write tests that set up some tuples initially, runs the topology until all tuples are acked/failed, and then lets you assert e.g. that data was written to Cassandra and all tuples were acked.

There's an example that uses completeTopology here https://github.com/xumingming/storm-lib/blob/master/src/jvm/storm/TestingApiDemo.java#L83.