Single Site/Multisite Clusters for Splunk Disaster Recovery

Recently, I had an inquiry come in from a fellow consultant about setting up a disaster recovery site for a customer. The customer was curious about how long it would take to replicate all of their data from their current index cluster to their new DR site and how much bandwidth it was going to take.

I didn’t have to ask how much data it was, the answer is zero minutes, and zero MB/s.

Many people don’t realize that when configuring their first index cluster, single index cluster buckets are not replicable to a second site when you later try to convert to a multisite cluster. Without getting too far into the technical specifics of why that is, I’m going to give you a way around this little problem. The solution is surprisingly simple.

The Single-site Multisite Cluster

When you set up your initial cluster, set up a multisite cluster. And then only build one site. You already have an index cluster? Convert it to a multi-site cluster, one that just happens to only have one site.

How, you ask? Luckily for you, the Queen of the index cluster is here to help you out. It’s all about those sweet, sweet configuration files.

In the server.conf of your cluster master, you work to configure a multisite cluster in accordance with Splunk documentation; except that in this case, you will only have one site defined.

Here is the example of the server.conf for your Cluster Master:

[clustering]
multisite = true
site = site1
available_sites = site1
site_replication_factor = origin:2, total:2
site_search_factor = origin:1, total:1

Be sure that each node in the cluster is assigned to site1.

Boom. You are now the proud owner of a multisite cluster. One that just happens to only (currently) have one site. Not sure if you’re EVER going to need another site? Consider this keeping your options open. Trust me, going thru this extra little step is far less painful than manually migrating all your buckets to a Continuity of Operations (COOP) site manually three years from now when you have 300TB of data indexed!

Perhaps you’re in the early stages of planning your COOP site. You haven’t purchase servers, let alone racked them yet. Go ahead and use this method to convert your cluster to a multisite now. When you are finally ready to start bringing that second site up, you’ll have some of your data already ready to go.

Calling in the Professionals

Of course, you may not feel comfortable digging around in those configuration files yourself; and we totally respect that. It’s why we’re here. August Schell Splunk engineers are experts at performance tuning installations of all sizes.

Whether you’re finding yourself struggling with clusters, indexes, indexers or something else with your Splunk installation, or you’re just starting to explore Splunk for your IT environment, August Schell can help your IT and security team. Partnering with a Splunk consultant such as the engineering team at August Schell will ensure your Splunk clusters are stable and aim to make your environment work as smoothly and efficiently as possible.

If your agency needs assistance with your Splunk environment, get in touch with an August Schell specialist, or call us at (301)-838-9470.

Need more help?

Our team of engineers can address a multitude of IT challenges many CIOs face today while keeping security and compliance at the forefront of every mission. Check out our engineering services!