Create a RemoteInfinispanClusterManager Option for Vert.x Clustering

[murphye]

Describe the feature

InfinispanClusterManager enforces the use of Embedded Infinispan cluster (i.e. EmbeddedMultimapCacheManager).

By offering a RemoteInfinispanClusterManager that would give some flexibility for how Infinispan is deployed. This version would use different APIs (i.e. RemoteMultimapCacheManager)

Use cases

• Deploying Infinispan in a sidecar container along with the Clustered Vert.x application.
• Deploying Infinispan in a pod with the same node affinity as the Clustered Vert.x application.
• Sharing a Infinispan instance with multiple Clustered Vert.x application instances (using different Vert.x cache configs)

Looking to experiment with this for Serverless applications utilizing Vert.x Clustering.

Contribution

Would be a change to go along with the upgrade to Infinispan 10.

[murphye]

Just found this on the Vert.x Roadmap:

Infinispan cluster manager improvements (4)
Provide an alternative cluster manager implementation as a client of a separate data grid:

JGroups for discovery/membership

Infinispan client for data storage

This should improve startup time (no rebalancing) and memory footprint (no data stored on the node) by a great margin.

Sounds very similar to my request here. Or at least serves the same use case. Interested to hear details.

This is a blocker for me for a PoC I'd like to do for using Vert.x Clustering in a serverless context.

[tsegismont]

@murphye thanks for sharing your use cases.

Currently Infinispan is used for discovery/membership management so it must run in embedded mode, otherwise we can't have access to the underlying JGroups transport.

In Vert.x 4.0, our priority is to adhere to the new clustering SPI (see #73) with the existing (embedded) implementation.

After 4.0 is released, we want to look into an alternative cluster manager implementations for K8S. In addition to the memory footprint and startup time improvements, we're looking for flexibility because the current solution requires a lot of care from users scaling up/down pods or performing rolling upgrades.

I haven't investigated much yet, but I had in mind that a user would:

• spin-up an Infinispan cluster with the Infinispan Operator
• put the cluster connection details in Vert.x options

Before that is possible, I need to talk with @karesti and the rest of the ISPN team to check if multimaps, counters and locks are all available remotely in ISPN 10.

About using the same ISPN cluster for multiple Vert.x clusters, that sounds like a very good idea to me (again, it's less problems for the users). We need to take security into account though (a Vert.x node from app A must not be able to see async map data created by nodes in app B).

[murphye]

Great info. I would be interested in having a conversation on this topic in the near future.