Cassandra In

Cassandra In-Memory选项(Cassandra In-Memory option)

DataStax Enterprise 4.0在Cassandra中引入了In-Memory选项： http ：//www.datastax.com/documentation/datastax_enterprise/4.0/datastax_enterprise/inMemory.html但是内存表的大小限制为1GB。

有人知道考虑为什么限制它为1GB？ 并且可能扩展到大容量的内存表，例如64GB？

There is an In-Memory option introduced in the Cassandra by DataStax Enterprise 4.0: http://www.datastax.com/documentation/datastax_enterprise/4.0/datastax_enterprise/inMemory.html But with 1GB size limited for an in-memory table.

Anyone know the consideration why limited it as 1GB? And possible extend to a large size of in-memory table, such as 64GB?

最满意答案

回答你的问题：今天不可能绕过这个限制。 内存表存储在JVM堆中，无论单个节点上可用的内存量是多少都不建议分配给JVM堆。 这种限制的主要原因是Java垃圾收集器在处理大量内存时会减慢速度。

但是，如果你认为Cassandra是一个分布式系统，1GB并不是真正的限制。

（节点* allocated_memory）/ ReplicationFactor

allocated_memory最大为1GB - 因此您的表可能包含在不同节点中分配的内存中的许多GB。

我认为将来某些东西会有所改善但是在内存中处理64GB时，当你需要刷新磁盘上的数据时，这可能是一个真正的问题。 另一个创造限制的考虑因素：在使用内存表时避免使用TTL。 TTL创建墓碑，在GCGraceSeconds周期过去之前不会释放墓碑 - 因此，考虑到默认值10天，每个墓碑将使内存部分忙碌且不可用，可能持续很长时间。

HTH，卡罗

To answer your question: today it's not possible to bypass this limitation. In-Memory tables are stored within the JVM Heap, regardless the amount of memory available on single node allocating more than 8GB to JVM Heap is not recommended. The main reason of this limitation is that Java Garbage Collector slow down when dealing with huge memory amount.

However if you consider Cassandra as a distributed system 1GB is not the real limitation.

(nodes*allocated_memory)/ReplicationFactor

allocated_memory is max 1GB -- So your table may contains many GB in memory allocated in different nodes.

I think that in future something will improve but dealing with 64GB in memory it could be a real problem when you need to flush data on disk. One more consideration that creates limitation: avoid TTL when working with In-Memory tables. TTL creates tombstones, a tombstone is not deallocated until the GCGraceSeconds period passes -- so considering a default value of 10 days each tombstone will keep the portion of memory busy and unavailable, possibly for long time.

HTH, Carlo