Big Redis设置(Big Redis setup)

Big Redis设置(Big Redis setup)

我需要在Redis 3.0.1中存储32M记录，每个记录需要大约422KB。 总共制作大约13GB的信息。

这些信息存储在压缩哈希列表中的光盘中，并在微笑杰克逊中序列化。 我正在使用Java 6，Jedis和AIX。

我有几个问题：

这是否意味着Redis进程需要13GB或RAM？ 这对于单个实例来说是一个可管理的大小，还是您要进行群集设置？ 我认为我们最多可以有4台服务器。 这意味着重新审视整个项目和日期，所以请考虑其他管理对此问题的影响。 有更好的方法来存储这些数据量吗？

谢谢

卡洛斯

I need to store 32M records in Redis 3.0.1, each record needs around 422KB. Making a total of around 13GB of information.

The information is stored in disc in a zipped hashlist, and serialized in smile jackson. I'm using Java 6, Jedis and AIX.

I have a few questions:

Does that mean that the Redis process needs 13GB or RAM? Is this a manageable size for a single instance or would you go for a cluster setup? I think we can have up to 4 servers. This would mean revisit the whole project and dates, so please consider other management impacts on this question. Is there a better way of storing this amount of data?

Thanks

Carlos

最满意答案

即使您使用Redis群集，您的所有数据都应该放在内存中。 有了13TB的数据，正如Alex所指出的那样，并且如你所说的那样仅限于4台服务器，这意味着每台服务器应该拥有超过4TB的RAM ......

此外，Redis以一种为速度优化的格式将数据存储在内存中，因此不会尽力减少它们的大小。 所以在实践中可能需要超过13TB。 这就是为什么我不会在这种情况下推荐Redis，或者至少不是Redis。 也许您应该考虑另一种NoSQL数据库，它提供快速响应时间，尽管它将数据存储在磁盘上，例如Couchbase（它使用Redis internaly作为缓存）。

或者，如果您的用例允许，最简单的解决方案是将Redis缓存添加到当前体系结构，而无需更改您使用的当前数据库。 它将显着提高缓存中数据的访问速度（但不会减少第一次访问）。 这取决于数据是否可能在短时间内被请求多次。

Even if you use a Redis Cluster, all of your data should fit in memory. With 13TB of data, as pointed by Alex, and limited to 4 servers as you said, it means each server should have more than 4TB of RAM...

Moreover Redis stores the data in memory in a format that is optimised for speed, and so does not try very hard to reduce their size. So it may take more than 13TB in practice. That's why I would not recommend Redis in this case, or at least not Redis only. Maybe you should consider an alternative NoSQL Database, that offer fast response time although it stores data on disk, like Couchbase (it uses Redis internaly as a cache).

Or, if your use case allows it, an easiest solution would be to add a Redis cache to your current architecture, without changing the current database you use. It will dramaticaly improve the access speed to data in cache (but wont reduce the first access). It depends if the data are likely to be requested more than once in a short period of time, or not.