Running Multiple Replicas of Backend

Hello,

We are POC’ing Cube.js as a backend caching/pre-aggregations layer and for managing data schemas. I just created a Helm chart (I couldn’t find one online) and started deploying to Kubernetes. I was wondering under what circumstances it is safe to run multiple Cube.js pods in a deployment. This is mostly for high availability and horizontal scaling (perf). It seems like caching should be fine, given the use of Redis, but I am worried that weird things could happen w.r.t. pre-aggregations when running many pods. Specifically, I think the pre-agg refreshers could step on each others toes.

Is it safe to run multiple instances in a single deployment? If not, what is needed to do so?

Thanks!
-kevin

1 Like

Hello @kmgreen,

Cube.js uses Redis not only as a cache layer. It uses Redis as query queue and cache. Any query executes in the queue, which is inside Redis, and it’s available for all instances of Cube.js. In this fact, one Cube.js instance can receive a request, and another instance will execute it.

Related to Scheduled Refresh, you should disable it by default for routers (all instances that accept traffic) and introduce only one instance that will process Scheduled Refresh.

Router instances:

  • CUBEJS_SCHEDULED_REFRESH_TIMER=false

Scheduler instance:

  • CUBEJS_SCHEDULED_REFRESH_TIMER=30

Thanks

2 Likes

Thanks for the quick response! Very helpful.

2 Likes